[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:00:55.071298   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:40595
I20260709 13:00:55.073355   431 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:00:55.074641   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:55.099520   438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:55.102603   437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:55.100728   440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:55.102658   431 server_base.cc:1061] running on GCE node
I20260709 13:00:56.540874   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:56.541155   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:56.541323   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602056541307 us; error 0 us; skew 500 ppm
I20260709 13:00:56.542287   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:56.549278   431 webserver.cc:533] Webserver started at http://127.0.107.254:42779/ using document root <none> and password file <none>
I20260709 13:00:56.550511   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:56.550817   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:56.551379   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:56.558444   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602054858684-431-0/minicluster-data/master-0-root/instance:
uuid: "a665c24edc1b4674a44b154f854abed7"
format_stamp: "Formatted at 2026-07-09 13:00:56 on dist-test-slave-mxd3"
I20260709 13:00:56.571170   431 fs_manager.cc:696] Time spent creating directory manager: real 0.011s	user 0.010s	sys 0.001s
I20260709 13:00:56.580664   446 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:56.583165   431 fs_manager.cc:730] Time spent opening block manager: real 0.008s	user 0.004s	sys 0.001s
I20260709 13:00:56.584448   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602054858684-431-0/minicluster-data/master-0-root
uuid: "a665c24edc1b4674a44b154f854abed7"
format_stamp: "Formatted at 2026-07-09 13:00:56 on dist-test-slave-mxd3"
I20260709 13:00:56.584939   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602054858684-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602054858684-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602054858684-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:56.664907   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:56.667025   431 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:00:56.667821   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:56.789016   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:40595
I20260709 13:00:56.789376   497 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:40595 every 8 connection(s)
I20260709 13:00:56.805390   498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:56.850858   498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7: Bootstrap starting.
I20260709 13:00:56.865024   498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:56.871876   498 log.cc:826] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7: Log is configured to *not* fsync() on all Append() calls
I20260709 13:00:56.886516   498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7: No bootstrap required, opened a new log
I20260709 13:00:56.924273   498 raft_consensus.cc:359] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a665c24edc1b4674a44b154f854abed7" member_type: VOTER }
I20260709 13:00:56.925782   498 raft_consensus.cc:385] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:56.926103   498 raft_consensus.cc:740] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a665c24edc1b4674a44b154f854abed7, State: Initialized, Role: FOLLOWER
I20260709 13:00:56.928206   498 consensus_queue.cc:260] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a665c24edc1b4674a44b154f854abed7" member_type: VOTER }
I20260709 13:00:56.929131   498 raft_consensus.cc:399] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:00:56.929764   498 raft_consensus.cc:493] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:00:56.930240   498 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:56.937819   498 raft_consensus.cc:515] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a665c24edc1b4674a44b154f854abed7" member_type: VOTER }
I20260709 13:00:56.938701   498 leader_election.cc:304] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a665c24edc1b4674a44b154f854abed7; no voters: 
I20260709 13:00:56.943068   498 leader_election.cc:290] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:00:56.943219   501 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:56.948765   501 raft_consensus.cc:697] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7 [term 1 LEADER]: Becoming Leader. State: Replica: a665c24edc1b4674a44b154f854abed7, State: Running, Role: LEADER
I20260709 13:00:56.950099   501 consensus_queue.cc:237] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a665c24edc1b4674a44b154f854abed7" member_type: VOTER }
I20260709 13:00:56.952872   498 sys_catalog.cc:565] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:00:56.972704   503 sys_catalog.cc:455] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a665c24edc1b4674a44b154f854abed7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a665c24edc1b4674a44b154f854abed7" member_type: VOTER } }
I20260709 13:00:56.972945   502 sys_catalog.cc:455] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a665c24edc1b4674a44b154f854abed7. Latest consensus state: current_term: 1 leader_uuid: "a665c24edc1b4674a44b154f854abed7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a665c24edc1b4674a44b154f854abed7" member_type: VOTER } }
I20260709 13:00:56.975154   503 sys_catalog.cc:458] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7 [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:56.975332   502 sys_catalog.cc:458] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7 [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:56.980209   511 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:00:56.998684   511 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:00:57.019232   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:00:57.031121   511 catalog_manager.cc:1383] Generated new cluster ID: 28e220f44f4348178b3dba19007f6395
I20260709 13:00:57.031549   511 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:00:57.070755   511 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:00:57.073056   511 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:00:57.093586   511 catalog_manager.cc:6070] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7: Generated new TSK 0
I20260709 13:00:57.095963   511 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:00:57.151245   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:57.161293   520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:57.167146   521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:57.172808   523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:57.174223   431 server_base.cc:1061] running on GCE node
I20260709 13:00:57.175155   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:57.175423   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:57.175644   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602057175624 us; error 0 us; skew 500 ppm
I20260709 13:00:57.176506   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:57.182526   431 webserver.cc:533] Webserver started at http://127.0.107.193:37713/ using document root <none> and password file <none>
I20260709 13:00:57.183565   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:57.183843   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:57.184206   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:57.186409   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602054858684-431-0/minicluster-data/ts-0-root/instance:
uuid: "da373aac3d614f56b081496dd9145b01"
format_stamp: "Formatted at 2026-07-09 13:00:57 on dist-test-slave-mxd3"
I20260709 13:00:57.196352   431 fs_manager.cc:696] Time spent creating directory manager: real 0.009s	user 0.002s	sys 0.007s
I20260709 13:00:57.203096   528 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:57.204118   431 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.000s	sys 0.005s
I20260709 13:00:57.204548   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602054858684-431-0/minicluster-data/ts-0-root
uuid: "da373aac3d614f56b081496dd9145b01"
format_stamp: "Formatted at 2026-07-09 13:00:57 on dist-test-slave-mxd3"
I20260709 13:00:57.204877   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602054858684-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602054858684-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602054858684-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:57.237151   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:57.238692   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:57.242611   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:57.249778   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:57.250041   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:57.250401   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:57.250569   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:57.409233   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:33951
I20260709 13:00:57.409343   590 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:33951 every 8 connection(s)
I20260709 13:00:57.419160   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:57.430913   595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:57.434955   596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:57.443811   431 server_base.cc:1061] running on GCE node
W20260709 13:00:57.444938   598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:57.446409   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:57.446691   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:57.446931   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602057446888 us; error 0 us; skew 500 ppm
I20260709 13:00:57.448076   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:57.453027   431 webserver.cc:533] Webserver started at http://127.0.107.194:33427/ using document root <none> and password file <none>
I20260709 13:00:57.453706   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:57.453819   591 heartbeater.cc:344] Connected to a master server at 127.0.107.254:40595
I20260709 13:00:57.454001   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:57.454646   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:57.454833   591 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:57.456521   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602054858684-431-0/minicluster-data/ts-1-root/instance:
uuid: "658e7299505c46d797b4ad2be4129399"
format_stamp: "Formatted at 2026-07-09 13:00:57 on dist-test-slave-mxd3"
I20260709 13:00:57.456871   591 heartbeater.cc:507] Master 127.0.107.254:40595 requested a full tablet report, sending...
I20260709 13:00:57.465945   463 ts_manager.cc:194] Registered new tserver with Master: da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951)
I20260709 13:00:57.466207   431 fs_manager.cc:696] Time spent creating directory manager: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:00:57.468448   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48828
I20260709 13:00:57.472337   603 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:57.473708   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:00:57.474153   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602054858684-431-0/minicluster-data/ts-1-root
uuid: "658e7299505c46d797b4ad2be4129399"
format_stamp: "Formatted at 2026-07-09 13:00:57 on dist-test-slave-mxd3"
I20260709 13:00:57.474581   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602054858684-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602054858684-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602054858684-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:57.503031   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:57.505016   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:57.508095   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:57.514014   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:57.514334   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:57.514856   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:57.515333   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:57.646231   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:44721
I20260709 13:00:57.646782   665 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:44721 every 8 connection(s)
I20260709 13:00:57.655774   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:57.672063   669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:57.677974   670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:57.682055   672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:57.685887   666 heartbeater.cc:344] Connected to a master server at 127.0.107.254:40595
I20260709 13:00:57.686370   666 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:57.687282   666 heartbeater.cc:507] Master 127.0.107.254:40595 requested a full tablet report, sending...
I20260709 13:00:57.690412   463 ts_manager.cc:194] Registered new tserver with Master: 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721)
I20260709 13:00:57.691299   431 server_base.cc:1061] running on GCE node
I20260709 13:00:57.692729   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:00:57.692740   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48838
W20260709 13:00:57.693107   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:57.693341   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602057693317 us; error 0 us; skew 500 ppm
I20260709 13:00:57.694217   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:57.701318   431 webserver.cc:533] Webserver started at http://127.0.107.195:42981/ using document root <none> and password file <none>
I20260709 13:00:57.701997   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:57.702245   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:57.702576   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:57.703841   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602054858684-431-0/minicluster-data/ts-2-root/instance:
uuid: "a92d6df25f4a43e0872ecbb6451058d5"
format_stamp: "Formatted at 2026-07-09 13:00:57 on dist-test-slave-mxd3"
I20260709 13:00:57.710230   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:00:57.715512   677 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:57.716650   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.001s
I20260709 13:00:57.717159   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602054858684-431-0/minicluster-data/ts-2-root
uuid: "a92d6df25f4a43e0872ecbb6451058d5"
format_stamp: "Formatted at 2026-07-09 13:00:57 on dist-test-slave-mxd3"
I20260709 13:00:57.717527   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602054858684-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602054858684-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602054858684-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:57.745401   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:57.747154   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:57.751109   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:57.755873   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:57.756225   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:57.756513   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:57.756707   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:57.951728   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:41111
I20260709 13:00:57.952796   739 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:41111 every 8 connection(s)
I20260709 13:00:57.972237   740 heartbeater.cc:344] Connected to a master server at 127.0.107.254:40595
I20260709 13:00:57.972752   740 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:57.973773   740 heartbeater.cc:507] Master 127.0.107.254:40595 requested a full tablet report, sending...
I20260709 13:00:57.976425   463 ts_manager.cc:194] Registered new tserver with Master: a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:57.976835   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017673855s
I20260709 13:00:57.978626   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48850
I20260709 13:00:58.049989   463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48858:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:00:58.054579   463 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:00:58.233758   629 tablet_service.cc:1511] Processing CreateTablet for tablet 8e4f664fa01d44aab86d4911e7fd45c7 (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:00:58.236629   630 tablet_service.cc:1511] Processing CreateTablet for tablet b6b6b05c36a64999b35a8273917bdcbc (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:00:58.233359   631 tablet_service.cc:1511] Processing CreateTablet for tablet d500279905444ef8b7d48b790d14d240 (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:00:58.252956   629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e4f664fa01d44aab86d4911e7fd45c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.249818   628 tablet_service.cc:1511] Processing CreateTablet for tablet 98c04e9dd15b431d91ea29bcb7152983 (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:00:58.254508   627 tablet_service.cc:1511] Processing CreateTablet for tablet d6ef1006746945cf9c09bed1220d384a (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:00:58.264218   631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d500279905444ef8b7d48b790d14d240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.257998   554 tablet_service.cc:1511] Processing CreateTablet for tablet 8e4f664fa01d44aab86d4911e7fd45c7 (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:00:58.259256   556 tablet_service.cc:1511] Processing CreateTablet for tablet d500279905444ef8b7d48b790d14d240 (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:00:58.255309   555 tablet_service.cc:1511] Processing CreateTablet for tablet b6b6b05c36a64999b35a8273917bdcbc (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:00:58.259590   704 tablet_service.cc:1511] Processing CreateTablet for tablet b6b6b05c36a64999b35a8273917bdcbc (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:00:58.258770   705 tablet_service.cc:1511] Processing CreateTablet for tablet d500279905444ef8b7d48b790d14d240 (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:00:58.275890   554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e4f664fa01d44aab86d4911e7fd45c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.261278   703 tablet_service.cc:1511] Processing CreateTablet for tablet 8e4f664fa01d44aab86d4911e7fd45c7 (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:00:58.276649   630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6b6b05c36a64999b35a8273917bdcbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.281579   627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6ef1006746945cf9c09bed1220d384a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.281237   704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6b6b05c36a64999b35a8273917bdcbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.273006   702 tablet_service.cc:1511] Processing CreateTablet for tablet 98c04e9dd15b431d91ea29bcb7152983 (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:00:58.277228   551 tablet_service.cc:1511] Processing CreateTablet for tablet ba58747e7ae04d4f81c4ffc2a323f570 (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:00:58.281318   556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d500279905444ef8b7d48b790d14d240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.290584   700 tablet_service.cc:1511] Processing CreateTablet for tablet ba58747e7ae04d4f81c4ffc2a323f570 (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:00:58.287932   701 tablet_service.cc:1511] Processing CreateTablet for tablet d6ef1006746945cf9c09bed1220d384a (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:00:58.295346   552 tablet_service.cc:1511] Processing CreateTablet for tablet d6ef1006746945cf9c09bed1220d384a (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:00:58.296358   626 tablet_service.cc:1511] Processing CreateTablet for tablet ba58747e7ae04d4f81c4ffc2a323f570 (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:00:58.298280   703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e4f664fa01d44aab86d4911e7fd45c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.285710   553 tablet_service.cc:1511] Processing CreateTablet for tablet 98c04e9dd15b431d91ea29bcb7152983 (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:00:58.306254   555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6b6b05c36a64999b35a8273917bdcbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.306041   550 tablet_service.cc:1511] Processing CreateTablet for tablet 78af706a10f146fe8c6a5c64d70c6894 (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:00:58.307337   705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d500279905444ef8b7d48b790d14d240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.310207   552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6ef1006746945cf9c09bed1220d384a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.307694   628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98c04e9dd15b431d91ea29bcb7152983. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.311923   553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98c04e9dd15b431d91ea29bcb7152983. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.315088   551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba58747e7ae04d4f81c4ffc2a323f570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.317246   626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba58747e7ae04d4f81c4ffc2a323f570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.316681   701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6ef1006746945cf9c09bed1220d384a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.318434   550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78af706a10f146fe8c6a5c64d70c6894. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.323160   700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba58747e7ae04d4f81c4ffc2a323f570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.344666   702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98c04e9dd15b431d91ea29bcb7152983. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.350477   625 tablet_service.cc:1511] Processing CreateTablet for tablet 78af706a10f146fe8c6a5c64d70c6894 (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:00:58.353605   625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78af706a10f146fe8c6a5c64d70c6894. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.397761   549 tablet_service.cc:1511] Processing CreateTablet for tablet 8af84729cf07482cb0244953a91a3a68 (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:00:58.400776   549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8af84729cf07482cb0244953a91a3a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.394461   699 tablet_service.cc:1511] Processing CreateTablet for tablet 78af706a10f146fe8c6a5c64d70c6894 (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:00:58.406215   699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78af706a10f146fe8c6a5c64d70c6894. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.465139   698 tablet_service.cc:1511] Processing CreateTablet for tablet 8af84729cf07482cb0244953a91a3a68 (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:00:58.467975   698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8af84729cf07482cb0244953a91a3a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.473984   591 heartbeater.cc:499] Master 127.0.107.254:40595 was elected leader, sending a full tablet report...
I20260709 13:00:58.477799   624 tablet_service.cc:1511] Processing CreateTablet for tablet 8af84729cf07482cb0244953a91a3a68 (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:00:58.484016   624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8af84729cf07482cb0244953a91a3a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.493741   548 tablet_service.cc:1511] Processing CreateTablet for tablet b236fa7371f141608adb61630a8c90ec (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:00:58.498121   548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b236fa7371f141608adb61630a8c90ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.550895   623 tablet_service.cc:1511] Processing CreateTablet for tablet b236fa7371f141608adb61630a8c90ec (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:00:58.552824   623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b236fa7371f141608adb61630a8c90ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.558159   758 tablet_bootstrap.cc:492] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01: Bootstrap starting.
I20260709 13:00:58.556258   697 tablet_service.cc:1511] Processing CreateTablet for tablet b236fa7371f141608adb61630a8c90ec (DEFAULT_TABLE table=test-workload [id=a132226f1d7a406380ff8886230cde4b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:00:58.571282   697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b236fa7371f141608adb61630a8c90ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.579329   760 tablet_bootstrap.cc:492] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5: Bootstrap starting.
I20260709 13:00:58.616323   759 tablet_bootstrap.cc:492] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399: Bootstrap starting.
I20260709 13:00:58.670044   760 tablet_bootstrap.cc:654] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:58.670044   758 tablet_bootstrap.cc:654] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:58.692332   760 tablet_bootstrap.cc:492] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5: No bootstrap required, opened a new log
I20260709 13:00:58.696666   759 tablet_bootstrap.cc:654] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:58.697332   666 heartbeater.cc:499] Master 127.0.107.254:40595 was elected leader, sending a full tablet report...
I20260709 13:00:58.698922   760 ts_tablet_manager.cc:1403] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5: Time spent bootstrapping tablet: real 0.123s	user 0.009s	sys 0.019s
I20260709 13:00:58.706815   760 raft_consensus.cc:359] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.707561   760 raft_consensus.cc:385] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:58.707862   760 raft_consensus.cc:740] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a92d6df25f4a43e0872ecbb6451058d5, State: Initialized, Role: FOLLOWER
I20260709 13:00:58.708796   760 consensus_queue.cc:260] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.714601   759 tablet_bootstrap.cc:492] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399: No bootstrap required, opened a new log
I20260709 13:00:58.715667   759 ts_tablet_manager.cc:1403] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399: Time spent bootstrapping tablet: real 0.136s	user 0.008s	sys 0.013s
I20260709 13:00:58.720059   759 raft_consensus.cc:359] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.720944   759 raft_consensus.cc:385] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:58.721282   759 raft_consensus.cc:740] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Initialized, Role: FOLLOWER
I20260709 13:00:58.723179   759 consensus_queue.cc:260] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.724402   758 tablet_bootstrap.cc:492] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01: No bootstrap required, opened a new log
I20260709 13:00:58.725051   758 ts_tablet_manager.cc:1403] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01: Time spent bootstrapping tablet: real 0.168s	user 0.005s	sys 0.029s
I20260709 13:00:58.730242   758 raft_consensus.cc:359] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.731524   758 raft_consensus.cc:385] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:58.732299   758 raft_consensus.cc:740] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da373aac3d614f56b081496dd9145b01, State: Initialized, Role: FOLLOWER
I20260709 13:00:58.739851   759 ts_tablet_manager.cc:1434] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399: Time spent starting tablet: real 0.024s	user 0.016s	sys 0.009s
I20260709 13:00:58.741113   759 tablet_bootstrap.cc:492] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399: Bootstrap starting.
I20260709 13:00:58.741593   740 heartbeater.cc:499] Master 127.0.107.254:40595 was elected leader, sending a full tablet report...
I20260709 13:00:58.741853   760 ts_tablet_manager.cc:1434] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5: Time spent starting tablet: real 0.043s	user 0.009s	sys 0.026s
I20260709 13:00:58.733623   758 consensus_queue.cc:260] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.746923   760 tablet_bootstrap.cc:492] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5: Bootstrap starting.
I20260709 13:00:58.755904   758 ts_tablet_manager.cc:1434] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01: Time spent starting tablet: real 0.030s	user 0.016s	sys 0.000s
I20260709 13:00:58.757064   758 tablet_bootstrap.cc:492] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01: Bootstrap starting.
I20260709 13:00:58.758379   759 tablet_bootstrap.cc:654] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:58.765156   758 tablet_bootstrap.cc:654] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:58.777295   759 tablet_bootstrap.cc:492] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399: No bootstrap required, opened a new log
I20260709 13:00:58.777966   759 ts_tablet_manager.cc:1403] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.016s
I20260709 13:00:58.780993   760 tablet_bootstrap.cc:654] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:58.781827   759 raft_consensus.cc:359] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.782792   759 raft_consensus.cc:385] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:58.783182   759 raft_consensus.cc:740] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Initialized, Role: FOLLOWER
I20260709 13:00:58.784570   759 consensus_queue.cc:260] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.787163   759 ts_tablet_manager.cc:1434] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:00:58.788239   759 tablet_bootstrap.cc:492] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399: Bootstrap starting.
I20260709 13:00:58.798300   758 tablet_bootstrap.cc:492] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01: No bootstrap required, opened a new log
I20260709 13:00:58.798944   758 ts_tablet_manager.cc:1403] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01: Time spent bootstrapping tablet: real 0.042s	user 0.026s	sys 0.011s
I20260709 13:00:58.799876   759 tablet_bootstrap.cc:654] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:58.802523   758 raft_consensus.cc:359] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.803382   758 raft_consensus.cc:385] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:58.803692   758 raft_consensus.cc:740] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da373aac3d614f56b081496dd9145b01, State: Initialized, Role: FOLLOWER
I20260709 13:00:58.804068   760 tablet_bootstrap.cc:492] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5: No bootstrap required, opened a new log
I20260709 13:00:58.804647   760 ts_tablet_manager.cc:1403] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5: Time spent bootstrapping tablet: real 0.058s	user 0.017s	sys 0.000s
I20260709 13:00:58.804527   758 consensus_queue.cc:260] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.807812   758 ts_tablet_manager.cc:1434] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.006s
I20260709 13:00:58.808935   758 tablet_bootstrap.cc:492] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01: Bootstrap starting.
I20260709 13:00:58.811494   760 raft_consensus.cc:359] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.812321   760 raft_consensus.cc:385] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:58.812602   760 raft_consensus.cc:740] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a92d6df25f4a43e0872ecbb6451058d5, State: Initialized, Role: FOLLOWER
I20260709 13:00:58.814240   760 consensus_queue.cc:260] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.820698   758 tablet_bootstrap.cc:654] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:58.823890   759 tablet_bootstrap.cc:492] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399: No bootstrap required, opened a new log
I20260709 13:00:58.824513   759 ts_tablet_manager.cc:1403] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399: Time spent bootstrapping tablet: real 0.036s	user 0.006s	sys 0.010s
I20260709 13:00:58.826520   760 ts_tablet_manager.cc:1434] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.001s
I20260709 13:00:58.827673   759 raft_consensus.cc:359] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.828502   759 raft_consensus.cc:385] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:58.828843   759 raft_consensus.cc:740] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Initialized, Role: FOLLOWER
I20260709 13:00:58.830099   759 consensus_queue.cc:260] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.832639   759 ts_tablet_manager.cc:1434] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:00:58.833670   759 tablet_bootstrap.cc:492] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399: Bootstrap starting.
I20260709 13:00:58.834470   758 tablet_bootstrap.cc:492] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01: No bootstrap required, opened a new log
I20260709 13:00:58.835012   758 ts_tablet_manager.cc:1403] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.011s
I20260709 13:00:58.839327   758 raft_consensus.cc:359] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.840611   758 raft_consensus.cc:385] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:58.840932   758 raft_consensus.cc:740] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da373aac3d614f56b081496dd9145b01, State: Initialized, Role: FOLLOWER
I20260709 13:00:58.842520   759 tablet_bootstrap.cc:654] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:58.841768   758 consensus_queue.cc:260] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.845963   758 ts_tablet_manager.cc:1434] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.002s
I20260709 13:00:58.847926   758 tablet_bootstrap.cc:492] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01: Bootstrap starting.
I20260709 13:00:58.849257   766 raft_consensus.cc:493] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.850291   766 raft_consensus.cc:515] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.851333   760 tablet_bootstrap.cc:492] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5: Bootstrap starting.
I20260709 13:00:58.866107   758 tablet_bootstrap.cc:654] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:58.878247   766 leader_election.cc:290] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:58.884655   760 tablet_bootstrap.cc:654] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:58.891970   759 tablet_bootstrap.cc:492] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399: No bootstrap required, opened a new log
I20260709 13:00:58.892714   759 ts_tablet_manager.cc:1403] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399: Time spent bootstrapping tablet: real 0.059s	user 0.022s	sys 0.018s
I20260709 13:00:58.904264   759 raft_consensus.cc:359] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.905579   759 raft_consensus.cc:385] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:00:58.915992   667 tablet.cc:2406] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:00:58.917666   758 tablet_bootstrap.cc:492] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01: No bootstrap required, opened a new log
I20260709 13:00:58.918252   758 ts_tablet_manager.cc:1403] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01: Time spent bootstrapping tablet: real 0.071s	user 0.014s	sys 0.016s
I20260709 13:00:58.907228   759 raft_consensus.cc:740] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Initialized, Role: FOLLOWER
I20260709 13:00:58.930200   760 tablet_bootstrap.cc:492] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5: No bootstrap required, opened a new log
I20260709 13:00:58.930725   760 ts_tablet_manager.cc:1403] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5: Time spent bootstrapping tablet: real 0.080s	user 0.020s	sys 0.012s
I20260709 13:00:58.925451   759 consensus_queue.cc:260] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.931209   766 raft_consensus.cc:493] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.931768   766 raft_consensus.cc:515] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.933020   759 ts_tablet_manager.cc:1434] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.005s
I20260709 13:00:58.929931   758 raft_consensus.cc:359] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.934320   766 leader_election.cc:290] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:58.935168   759 tablet_bootstrap.cc:492] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399: Bootstrap starting.
I20260709 13:00:58.944203   759 tablet_bootstrap.cc:654] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:58.934083   760 raft_consensus.cc:359] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.948021   760 raft_consensus.cc:385] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:58.934762   758 raft_consensus.cc:385] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:58.948683   760 raft_consensus.cc:740] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a92d6df25f4a43e0872ecbb6451058d5, State: Initialized, Role: FOLLOWER
I20260709 13:00:58.948274   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78af706a10f146fe8c6a5c64d70c6894" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5" is_pre_election: true
I20260709 13:00:58.948983   758 raft_consensus.cc:740] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da373aac3d614f56b081496dd9145b01, State: Initialized, Role: FOLLOWER
I20260709 13:00:58.949442   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500279905444ef8b7d48b790d14d240" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5" is_pre_election: true
I20260709 13:00:58.949911   760 consensus_queue.cc:260] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.949957   758 consensus_queue.cc:260] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.948541   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78af706a10f146fe8c6a5c64d70c6894" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399" is_pre_election: true
I20260709 13:00:58.950693   715 raft_consensus.cc:2468] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 0.
I20260709 13:00:58.951287   640 raft_consensus.cc:2468] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 0.
I20260709 13:00:58.948065   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500279905444ef8b7d48b790d14d240" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399" is_pre_election: true
I20260709 13:00:58.952191   641 raft_consensus.cc:2468] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 0.
I20260709 13:00:58.953032   758 ts_tablet_manager.cc:1434] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01: Time spent starting tablet: real 0.034s	user 0.011s	sys 0.000s
I20260709 13:00:58.953480   760 ts_tablet_manager.cc:1434] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.003s
W20260709 13:00:58.952991   531 leader_election.cc:343] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:58.954595   760 tablet_bootstrap.cc:492] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5: Bootstrap starting.
I20260709 13:00:58.954644   531 leader_election.cc:304] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a92d6df25f4a43e0872ecbb6451058d5, da373aac3d614f56b081496dd9145b01; no voters: 
I20260709 13:00:58.956683   529 leader_election.cc:304] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01; no voters: a92d6df25f4a43e0872ecbb6451058d5
I20260709 13:00:58.958280   766 raft_consensus.cc:2804] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:58.959161   766 raft_consensus.cc:493] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:58.959630   766 raft_consensus.cc:3060] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:58.962983   758 tablet_bootstrap.cc:492] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01: Bootstrap starting.
I20260709 13:00:58.974138   760 tablet_bootstrap.cc:654] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:58.972810   779 raft_consensus.cc:2804] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:58.976234   779 raft_consensus.cc:493] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:58.976665   779 raft_consensus.cc:3060] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:58.984573   510 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:58.988425   766 raft_consensus.cc:515] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:58.991925   766 leader_election.cc:290] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 election: Requested vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:58.992697   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500279905444ef8b7d48b790d14d240" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399"
I20260709 13:00:58.993314   641 raft_consensus.cc:3060] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:58.994253   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500279905444ef8b7d48b790d14d240" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:00:58.994976   715 raft_consensus.cc:3060] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:58.995793   759 tablet_bootstrap.cc:492] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399: No bootstrap required, opened a new log
I20260709 13:00:58.996760   759 ts_tablet_manager.cc:1403] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399: Time spent bootstrapping tablet: real 0.062s	user 0.007s	sys 0.015s
I20260709 13:00:59.004901   641 raft_consensus.cc:2468] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 1.
I20260709 13:00:59.004901   715 raft_consensus.cc:2468] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 1.
I20260709 13:00:59.007298   529 leader_election.cc:304] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01; no voters: 
I20260709 13:00:59.008159   766 raft_consensus.cc:2804] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:59.007944   759 raft_consensus.cc:359] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.009606   759 raft_consensus.cc:385] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.014456   759 raft_consensus.cc:740] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.015405   759 consensus_queue.cc:260] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.017733   758 tablet_bootstrap.cc:654] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.011727   766 raft_consensus.cc:697] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Becoming Leader. State: Replica: da373aac3d614f56b081496dd9145b01, State: Running, Role: LEADER
I20260709 13:00:59.019052   766 consensus_queue.cc:237] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.023831   759 ts_tablet_manager.cc:1434] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.007s
I20260709 13:00:59.024922   759 tablet_bootstrap.cc:492] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399: Bootstrap starting.
I20260709 13:00:59.028982   779 raft_consensus.cc:515] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.032320   758 tablet_bootstrap.cc:492] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01: No bootstrap required, opened a new log
I20260709 13:00:59.032557   779 leader_election.cc:290] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 election: Requested vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:59.032915   758 ts_tablet_manager.cc:1403] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01: Time spent bootstrapping tablet: real 0.075s	user 0.014s	sys 0.013s
I20260709 13:00:59.033236   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78af706a10f146fe8c6a5c64d70c6894" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
W20260709 13:00:59.035230   531 leader_election.cc:343] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:59.035351   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78af706a10f146fe8c6a5c64d70c6894" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399"
I20260709 13:00:59.036033   641 raft_consensus.cc:3060] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.036448   758 raft_consensus.cc:359] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.037247   758 raft_consensus.cc:385] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.037549   758 raft_consensus.cc:740] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da373aac3d614f56b081496dd9145b01, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.038481   758 consensus_queue.cc:260] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.040993   758 ts_tablet_manager.cc:1434] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:00:59.046562   760 tablet_bootstrap.cc:492] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5: No bootstrap required, opened a new log
I20260709 13:00:59.047727   641 raft_consensus.cc:2468] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 1.
I20260709 13:00:59.047798   760 ts_tablet_manager.cc:1403] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5: Time spent bootstrapping tablet: real 0.093s	user 0.015s	sys 0.009s
I20260709 13:00:59.049912   529 leader_election.cc:304] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01; no voters: a92d6df25f4a43e0872ecbb6451058d5
I20260709 13:00:59.051493   766 raft_consensus.cc:2804] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:59.052650   766 raft_consensus.cc:697] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Becoming Leader. State: Replica: da373aac3d614f56b081496dd9145b01, State: Running, Role: LEADER
I20260709 13:00:59.053534   766 consensus_queue.cc:237] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.055377   758 tablet_bootstrap.cc:492] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01: Bootstrap starting.
I20260709 13:00:59.057435   461 catalog_manager.cc:5697] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to da373aac3d614f56b081496dd9145b01 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "da373aac3d614f56b081496dd9145b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:59.064174   760 raft_consensus.cc:359] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.065243   760 raft_consensus.cc:385] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.065845   760 raft_consensus.cc:740] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a92d6df25f4a43e0872ecbb6451058d5, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.066991   760 consensus_queue.cc:260] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.079491   760 ts_tablet_manager.cc:1434] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.003s
I20260709 13:00:59.080829   759 tablet_bootstrap.cc:654] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.080962   760 tablet_bootstrap.cc:492] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5: Bootstrap starting.
I20260709 13:00:59.077395   758 tablet_bootstrap.cc:654] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.097883   758 tablet_bootstrap.cc:492] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01: No bootstrap required, opened a new log
I20260709 13:00:59.098456   758 ts_tablet_manager.cc:1403] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01: Time spent bootstrapping tablet: real 0.043s	user 0.008s	sys 0.010s
I20260709 13:00:59.098609   759 tablet_bootstrap.cc:492] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399: No bootstrap required, opened a new log
I20260709 13:00:59.099558   759 ts_tablet_manager.cc:1403] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399: Time spent bootstrapping tablet: real 0.075s	user 0.009s	sys 0.012s
I20260709 13:00:59.098807   760 tablet_bootstrap.cc:654] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.105695   758 raft_consensus.cc:359] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.107108   758 raft_consensus.cc:385] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.107486   758 raft_consensus.cc:740] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da373aac3d614f56b081496dd9145b01, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.109220   758 consensus_queue.cc:260] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.110952   759 raft_consensus.cc:359] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.112237   759 raft_consensus.cc:385] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.117810   759 raft_consensus.cc:740] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.116559   758 ts_tablet_manager.cc:1434] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.000s
I20260709 13:00:59.118789   759 consensus_queue.cc:260] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.119558   758 tablet_bootstrap.cc:492] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01: Bootstrap starting.
I20260709 13:00:59.122699   759 ts_tablet_manager.cc:1434] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.003s
I20260709 13:00:59.123822   759 tablet_bootstrap.cc:492] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399: Bootstrap starting.
I20260709 13:00:59.123759   461 catalog_manager.cc:5697] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to da373aac3d614f56b081496dd9145b01 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "da373aac3d614f56b081496dd9145b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:59.131454   760 tablet_bootstrap.cc:492] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5: No bootstrap required, opened a new log
I20260709 13:00:59.132066   760 ts_tablet_manager.cc:1403] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5: Time spent bootstrapping tablet: real 0.051s	user 0.012s	sys 0.019s
I20260709 13:00:59.135494   758 tablet_bootstrap.cc:654] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.146363   759 tablet_bootstrap.cc:654] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.152299   760 raft_consensus.cc:359] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.153141   760 raft_consensus.cc:385] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.153470   760 raft_consensus.cc:740] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a92d6df25f4a43e0872ecbb6451058d5, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.158257   760 consensus_queue.cc:260] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.160719   758 tablet_bootstrap.cc:492] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01: No bootstrap required, opened a new log
I20260709 13:00:59.161279   758 ts_tablet_manager.cc:1403] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01: Time spent bootstrapping tablet: real 0.042s	user 0.019s	sys 0.010s
I20260709 13:00:59.164487   758 raft_consensus.cc:359] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.166831   758 raft_consensus.cc:385] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.167193   758 raft_consensus.cc:740] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da373aac3d614f56b081496dd9145b01, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.168005   758 consensus_queue.cc:260] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.168980   760 ts_tablet_manager.cc:1434] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5: Time spent starting tablet: real 0.037s	user 0.003s	sys 0.006s
I20260709 13:00:59.170091   760 tablet_bootstrap.cc:492] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5: Bootstrap starting.
I20260709 13:00:59.171031   758 ts_tablet_manager.cc:1434] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01: Time spent starting tablet: real 0.009s	user 0.011s	sys 0.000s
I20260709 13:00:59.172276   758 tablet_bootstrap.cc:492] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01: Bootstrap starting.
I20260709 13:00:59.176690   759 tablet_bootstrap.cc:492] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399: No bootstrap required, opened a new log
I20260709 13:00:59.178751   764 raft_consensus.cc:493] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.179378   764 raft_consensus.cc:515] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.181874   758 tablet_bootstrap.cc:654] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.186673   764 leader_election.cc:290] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721)
I20260709 13:00:59.187647   759 ts_tablet_manager.cc:1403] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399: Time spent bootstrapping tablet: real 0.064s	user 0.012s	sys 0.009s
I20260709 13:00:59.195354   766 raft_consensus.cc:493] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.198923   766 raft_consensus.cc:515] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.202342   766 leader_election.cc:290] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:59.203845   765 raft_consensus.cc:493] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.204649   765 raft_consensus.cc:515] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.205327   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c04e9dd15b431d91ea29bcb7152983" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399" is_pre_election: true
I20260709 13:00:59.224378   765 leader_election.cc:290] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:59.228360   759 raft_consensus.cc:359] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.229177   759 raft_consensus.cc:385] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.228852   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c04e9dd15b431d91ea29bcb7152983" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5" is_pre_election: true
I20260709 13:00:59.229971   715 raft_consensus.cc:2468] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 0.
W20260709 13:00:59.229241   529 leader_election.cc:343] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:59.231551   531 leader_election.cc:304] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a92d6df25f4a43e0872ecbb6451058d5, da373aac3d614f56b081496dd9145b01; no voters: 658e7299505c46d797b4ad2be4129399
I20260709 13:00:59.232550   766 raft_consensus.cc:2804] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:59.233106   766 raft_consensus.cc:493] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.233497   766 raft_consensus.cc:3060] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.238663   760 tablet_bootstrap.cc:654] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.239231   759 raft_consensus.cc:740] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.244252   759 consensus_queue.cc:260] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.249490   766 raft_consensus.cc:515] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.251585   766 leader_election.cc:290] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 election: Requested vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:59.252346   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c04e9dd15b431d91ea29bcb7152983" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399"
I20260709 13:00:59.252964   641 raft_consensus.cc:3060] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.259065   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c04e9dd15b431d91ea29bcb7152983" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:00:59.259811   715 raft_consensus.cc:3060] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.261255   758 tablet_bootstrap.cc:492] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01: No bootstrap required, opened a new log
I20260709 13:00:59.261765   758 ts_tablet_manager.cc:1403] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01: Time spent bootstrapping tablet: real 0.090s	user 0.016s	sys 0.013s
I20260709 13:00:59.266722   765 raft_consensus.cc:493] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.268855   765 raft_consensus.cc:515] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.269600   759 ts_tablet_manager.cc:1434] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399: Time spent starting tablet: real 0.081s	user 0.010s	sys 0.000s
I20260709 13:00:59.271338   765 leader_election.cc:290] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:59.271165   759 tablet_bootstrap.cc:492] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399: Bootstrap starting.
I20260709 13:00:59.276394   758 raft_consensus.cc:359] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.277161   758 raft_consensus.cc:385] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.277421   758 raft_consensus.cc:740] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da373aac3d614f56b081496dd9145b01, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.275270   765 raft_consensus.cc:493] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.283840   758 consensus_queue.cc:260] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.281172   765 raft_consensus.cc:515] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.288491   641 raft_consensus.cc:2468] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 1.
I20260709 13:00:59.291256   529 leader_election.cc:304] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01; no voters: 
I20260709 13:00:59.292047   758 ts_tablet_manager.cc:1434] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.001s
I20260709 13:00:59.292310   766 raft_consensus.cc:2804] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:59.292754   766 raft_consensus.cc:697] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Becoming Leader. State: Replica: da373aac3d614f56b081496dd9145b01, State: Running, Role: LEADER
I20260709 13:00:59.293198   758 tablet_bootstrap.cc:492] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01: Bootstrap starting.
I20260709 13:00:59.293715   766 consensus_queue.cc:237] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.302205   764 raft_consensus.cc:493] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.302836   764 raft_consensus.cc:515] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.290640   765 leader_election.cc:290] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:59.303843   765 raft_consensus.cc:493] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.304378   765 raft_consensus.cc:515] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.305310   715 raft_consensus.cc:2468] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 1.
I20260709 13:00:59.305341   764 leader_election.cc:290] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721)
I20260709 13:00:59.325309   779 raft_consensus.cc:493] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.326686   779 raft_consensus.cc:515] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.327977   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c04e9dd15b431d91ea29bcb7152983" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399" is_pre_election: true
I20260709 13:00:59.329825   641 raft_consensus.cc:2393] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a92d6df25f4a43e0872ecbb6451058d5 in current term 1: Already voted for candidate da373aac3d614f56b081496dd9145b01 in this term.
I20260709 13:00:59.331446   779 leader_election.cc:290] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:59.333425   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af84729cf07482cb0244953a91a3a68" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399" is_pre_election: true
I20260709 13:00:59.334196   640 raft_consensus.cc:2468] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 0.
I20260709 13:00:59.343122   765 leader_election.cc:290] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:59.344235   760 tablet_bootstrap.cc:492] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5: No bootstrap required, opened a new log
I20260709 13:00:59.351327   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af84729cf07482cb0244953a91a3a68" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5" is_pre_election: true
I20260709 13:00:59.343842   779 raft_consensus.cc:493] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.349998   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4f664fa01d44aab86d4911e7fd45c7" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399" is_pre_election: true
I20260709 13:00:59.357468   641 raft_consensus.cc:2468] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a92d6df25f4a43e0872ecbb6451058d5 in term 0.
I20260709 13:00:59.356045   760 ts_tablet_manager.cc:1403] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5: Time spent bootstrapping tablet: real 0.186s	user 0.022s	sys 0.008s
I20260709 13:00:59.348016   529 leader_election.cc:304] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01; no voters: 
W20260709 13:00:59.361959   531 leader_election.cc:343] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:59.360345   678 leader_election.cc:304] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, a92d6df25f4a43e0872ecbb6451058d5; no voters: 
I20260709 13:00:59.363217   759 tablet_bootstrap.cc:654] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.364035   766 raft_consensus.cc:493] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.365752   764 raft_consensus.cc:2804] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:59.366267   764 raft_consensus.cc:493] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.366645   764 raft_consensus.cc:3060] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.355548   779 raft_consensus.cc:515] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.364145   782 raft_consensus.cc:2804] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:59.369681   766 raft_consensus.cc:515] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.370764   782 raft_consensus.cc:493] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.373907   782 raft_consensus.cc:3060] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.386313   760 raft_consensus.cc:359] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.386118   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4f664fa01d44aab86d4911e7fd45c7" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399" is_pre_election: true
I20260709 13:00:59.387073   760 raft_consensus.cc:385] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.402608   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399" is_pre_election: true
I20260709 13:00:59.403460   640 raft_consensus.cc:2468] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 0.
I20260709 13:00:59.404983   529 leader_election.cc:304] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01; no voters: 
I20260709 13:00:59.398175   641 raft_consensus.cc:2468] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 0.
I20260709 13:00:59.406277   766 leader_election.cc:290] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:59.409662   766 raft_consensus.cc:493] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.400403   760 raft_consensus.cc:740] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a92d6df25f4a43e0872ecbb6451058d5, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.407277   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4f664fa01d44aab86d4911e7fd45c7" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5" is_pre_election: true
I20260709 13:00:59.410141   766 raft_consensus.cc:515] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.413856   460 catalog_manager.cc:5697] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to da373aac3d614f56b081496dd9145b01 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "da373aac3d614f56b081496dd9145b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:59.450606   779 leader_election.cc:290] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:59.440277   760 consensus_queue.cc:260] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.417466   529 leader_election.cc:304] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01; no voters: 
I20260709 13:00:59.476804   758 tablet_bootstrap.cc:654] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.495461   759 tablet_bootstrap.cc:492] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399: No bootstrap required, opened a new log
I20260709 13:00:59.496174   759 ts_tablet_manager.cc:1403] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399: Time spent bootstrapping tablet: real 0.226s	user 0.020s	sys 0.029s
I20260709 13:00:59.504820   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5" is_pre_election: true
I20260709 13:00:59.491490   779 raft_consensus.cc:2804] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:59.515259   760 ts_tablet_manager.cc:1434] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5: Time spent starting tablet: real 0.151s	user 0.010s	sys 0.008s
I20260709 13:00:59.515111   711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5" is_pre_election: true
I20260709 13:00:59.515901   712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5" is_pre_election: true
I20260709 13:00:59.516593   760 tablet_bootstrap.cc:492] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5: Bootstrap starting.
I20260709 13:00:59.515136   710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af84729cf07482cb0244953a91a3a68" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5" is_pre_election: true
I20260709 13:00:59.516778   712 raft_consensus.cc:2468] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 658e7299505c46d797b4ad2be4129399 in term 0.
I20260709 13:00:59.511857   779 raft_consensus.cc:493] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.509562   766 leader_election.cc:290] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:59.521513   779 raft_consensus.cc:3060] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.542147   806 raft_consensus.cc:493] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:59.532614   606 leader_election.cc:343] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:59.538142   766 raft_consensus.cc:2804] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:00:59.545743   529 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44721, user_credentials={real_user=slave}} blocked reactor thread for 123951us
I20260709 13:00:59.548858   759 raft_consensus.cc:359] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.553331   759 raft_consensus.cc:385] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.536227   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4f664fa01d44aab86d4911e7fd45c7" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5" is_pre_election: true
I20260709 13:00:59.553962   759 raft_consensus.cc:740] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.555806   759 consensus_queue.cc:260] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.523094   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c04e9dd15b431d91ea29bcb7152983" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da373aac3d614f56b081496dd9145b01" is_pre_election: true
I20260709 13:00:59.545939   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4f664fa01d44aab86d4911e7fd45c7" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da373aac3d614f56b081496dd9145b01" is_pre_election: true
I20260709 13:00:59.550545   766 raft_consensus.cc:493] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.559515   759 ts_tablet_manager.cc:1434] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.004s
I20260709 13:00:59.568341   679 leader_election.cc:304] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a92d6df25f4a43e0872ecbb6451058d5; no voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01
I20260709 13:00:59.562252   766 raft_consensus.cc:3060] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.574357   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5" is_pre_election: true
I20260709 13:00:59.576979   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399" is_pre_election: true
I20260709 13:00:59.577420   713 raft_consensus.cc:2468] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 0.
I20260709 13:00:59.578027   641 raft_consensus.cc:2468] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 0.
I20260709 13:00:59.579456   529 leader_election.cc:304] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01; no voters: 
I20260709 13:00:59.554754   806 raft_consensus.cc:515] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.568419   606 leader_election.cc:304] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, a92d6df25f4a43e0872ecbb6451058d5; no voters: 
I20260709 13:00:59.586680   808 raft_consensus.cc:493] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.586576   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4f664fa01d44aab86d4911e7fd45c7" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da373aac3d614f56b081496dd9145b01" is_pre_election: true
I20260709 13:00:59.593844   765 raft_consensus.cc:2804] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:59.589159   808 raft_consensus.cc:515] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
W20260709 13:00:59.590307   531 leader_election.cc:343] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:59.596062   765 raft_consensus.cc:493] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.615123   764 raft_consensus.cc:515] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.625342   715 raft_consensus.cc:2393] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da373aac3d614f56b081496dd9145b01 in current term 1: Already voted for candidate a92d6df25f4a43e0872ecbb6451058d5 in this term.
I20260709 13:00:59.614614   765 raft_consensus.cc:3060] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:59.599674   606 leader_election.cc:343] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:59.590796   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da373aac3d614f56b081496dd9145b01" is_pre_election: true
I20260709 13:00:59.588868   563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da373aac3d614f56b081496dd9145b01" is_pre_election: true
I20260709 13:00:59.619269   782 raft_consensus.cc:515] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
W20260709 13:00:59.611768   679 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:33951, user_credentials={real_user=slave}} blocked reactor thread for 45732.9us
W20260709 13:00:59.645622   606 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41111, user_credentials={real_user=slave}} blocked reactor thread for 46636.4us
I20260709 13:00:59.646917   713 raft_consensus.cc:3060] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.599478   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af84729cf07482cb0244953a91a3a68" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da373aac3d614f56b081496dd9145b01" is_pre_election: true
I20260709 13:00:59.677205   811 raft_consensus.cc:2804] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:59.675271   807 raft_consensus.cc:493] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:59.675513   531 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:41111, user_credentials={real_user=slave}} blocked reactor thread for 85975us
I20260709 13:00:59.686542   812 raft_consensus.cc:2749] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:59.684180   562 raft_consensus.cc:2393] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 658e7299505c46d797b4ad2be4129399 in current term 1: Already voted for candidate da373aac3d614f56b081496dd9145b01 in this term.
I20260709 13:00:59.685920   811 raft_consensus.cc:493] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.688294   811 raft_consensus.cc:3060] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.683800   606 leader_election.cc:304] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 658e7299505c46d797b4ad2be4129399; no voters: a92d6df25f4a43e0872ecbb6451058d5, da373aac3d614f56b081496dd9145b01
I20260709 13:00:59.694146   802 raft_consensus.cc:2749] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:59.688180   807 raft_consensus.cc:515] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.703476   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba58747e7ae04d4f81c4ffc2a323f570" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399" is_pre_election: true
I20260709 13:00:59.706636   640 raft_consensus.cc:2468] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a92d6df25f4a43e0872ecbb6451058d5 in term 0.
I20260709 13:00:59.703266   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af84729cf07482cb0244953a91a3a68" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399"
I20260709 13:00:59.692844   808 leader_election.cc:290] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721)
I20260709 13:00:59.710414   641 raft_consensus.cc:3060] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.702710   806 leader_election.cc:290] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721)
I20260709 13:00:59.708015   782 leader_election.cc:290] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 election: Requested vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:59.715322   605 leader_election.cc:304] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 658e7299505c46d797b4ad2be4129399; no voters: a92d6df25f4a43e0872ecbb6451058d5, da373aac3d614f56b081496dd9145b01
I20260709 13:00:59.710232   779 raft_consensus.cc:515] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.716938   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78af706a10f146fe8c6a5c64d70c6894" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399" is_pre_election: true
I20260709 13:00:59.712757   561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba58747e7ae04d4f81c4ffc2a323f570" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da373aac3d614f56b081496dd9145b01" is_pre_election: true
I20260709 13:00:59.717475   678 leader_election.cc:304] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, a92d6df25f4a43e0872ecbb6451058d5; no voters: 
I20260709 13:00:59.722182   563 raft_consensus.cc:2393] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 658e7299505c46d797b4ad2be4129399 in current term 1: Already voted for candidate da373aac3d614f56b081496dd9145b01 in this term.
I20260709 13:00:59.717617   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78af706a10f146fe8c6a5c64d70c6894" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da373aac3d614f56b081496dd9145b01" is_pre_election: true
I20260709 13:00:59.720605   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4f664fa01d44aab86d4911e7fd45c7" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da373aac3d614f56b081496dd9145b01"
I20260709 13:00:59.721258   813 consensus_queue.cc:1048] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:00:59.728170   710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af84729cf07482cb0244953a91a3a68" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:00:59.721760   807 leader_election.cc:290] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:59.726969   806 raft_consensus.cc:2804] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:59.730881   758 tablet_bootstrap.cc:492] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01: No bootstrap required, opened a new log
I20260709 13:00:59.731134   806 raft_consensus.cc:493] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.731410   758 ts_tablet_manager.cc:1403] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01: Time spent bootstrapping tablet: real 0.438s	user 0.019s	sys 0.017s
I20260709 13:00:59.731555   806 raft_consensus.cc:3060] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.728804   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba58747e7ae04d4f81c4ffc2a323f570" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5" is_pre_election: true
I20260709 13:00:59.729928   640 raft_consensus.cc:2393] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a92d6df25f4a43e0872ecbb6451058d5 in current term 1: Already voted for candidate da373aac3d614f56b081496dd9145b01 in this term.
I20260709 13:00:59.722473   561 raft_consensus.cc:2468] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a92d6df25f4a43e0872ecbb6451058d5 in term 0.
I20260709 13:00:59.731302   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba58747e7ae04d4f81c4ffc2a323f570" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399" is_pre_election: true
I20260709 13:00:59.740036   779 leader_election.cc:290] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 election: Requested vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:59.737557   639 raft_consensus.cc:2468] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 0.
I20260709 13:00:59.741099   679 leader_election.cc:304] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a92d6df25f4a43e0872ecbb6451058d5; no voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01
I20260709 13:00:59.742513   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399"
I20260709 13:00:59.743294   640 raft_consensus.cc:3060] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:59.738282   531 leader_election.cc:343] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:59.733889   766 raft_consensus.cc:515] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.753597   765 raft_consensus.cc:515] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.756778   712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:00:59.752230   779 consensus_queue.cc:1048] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:00:59.762141   565 raft_consensus.cc:2393] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a92d6df25f4a43e0872ecbb6451058d5 in current term 1: Already voted for candidate da373aac3d614f56b081496dd9145b01 in this term.
I20260709 13:00:59.762586   766 leader_election.cc:290] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 election: Requested vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:59.766629   758 raft_consensus.cc:359] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.767704   758 raft_consensus.cc:385] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.757030   529 leader_election.cc:304] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01; no voters: 
I20260709 13:00:59.768050   758 raft_consensus.cc:740] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da373aac3d614f56b081496dd9145b01, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.769989   758 consensus_queue.cc:260] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
W20260709 13:00:59.766816   531 leader_election.cc:343] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:59.773592   758 ts_tablet_manager.cc:1434] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01: Time spent starting tablet: real 0.042s	user 0.003s	sys 0.006s
I20260709 13:00:59.771744   779 raft_consensus.cc:2804] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:59.793231   806 raft_consensus.cc:515] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.798384   779 raft_consensus.cc:493] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.798826   779 raft_consensus.cc:3060] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.800532   714 raft_consensus.cc:2393] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da373aac3d614f56b081496dd9145b01 in current term 1: Already voted for candidate a92d6df25f4a43e0872ecbb6451058d5 in this term.
I20260709 13:00:59.800726   808 raft_consensus.cc:2749] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:59.801196   641 raft_consensus.cc:2468] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 1.
I20260709 13:00:59.801775   802 raft_consensus.cc:2749] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:59.773612   712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4f664fa01d44aab86d4911e7fd45c7" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:00:59.774097   765 leader_election.cc:290] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 1 election: Requested vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:59.776407   711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:00:59.808030   711 raft_consensus.cc:3060] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.808357   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba58747e7ae04d4f81c4ffc2a323f570" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da373aac3d614f56b081496dd9145b01"
I20260709 13:00:59.809237   760 tablet_bootstrap.cc:654] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:59.771747   811 raft_consensus.cc:515] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.809423   529 leader_election.cc:304] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01; no voters: a92d6df25f4a43e0872ecbb6451058d5
I20260709 13:00:59.815459   782 raft_consensus.cc:2804] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:59.815986   782 raft_consensus.cc:697] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Becoming Leader. State: Replica: da373aac3d614f56b081496dd9145b01, State: Running, Role: LEADER
I20260709 13:00:59.806479   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4f664fa01d44aab86d4911e7fd45c7" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399"
I20260709 13:00:59.817142   639 raft_consensus.cc:3060] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.817107   782 consensus_queue.cc:237] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.819175   564 raft_consensus.cc:2393] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 658e7299505c46d797b4ad2be4129399 in current term 1: Already voted for candidate da373aac3d614f56b081496dd9145b01 in this term.
I20260709 13:00:59.827682   811 leader_election.cc:290] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 election: Requested vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:59.829173   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399"
I20260709 13:00:59.835961   641 raft_consensus.cc:2393] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate da373aac3d614f56b081496dd9145b01 in current term 1: Already voted for candidate 658e7299505c46d797b4ad2be4129399 in this term.
I20260709 13:00:59.841513   779 raft_consensus.cc:515] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.844373   566 raft_consensus.cc:2393] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a92d6df25f4a43e0872ecbb6451058d5 in current term 1: Already voted for candidate da373aac3d614f56b081496dd9145b01 in this term.
I20260709 13:00:59.849004   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba58747e7ae04d4f81c4ffc2a323f570" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399"
I20260709 13:00:59.849952   641 raft_consensus.cc:3060] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:59.838214   640 raft_consensus.cc:2468] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 1.
I20260709 13:00:59.859862   782 raft_consensus.cc:493] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.848349   779 leader_election.cc:290] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 election: Requested vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:59.862027   529 leader_election.cc:304] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01; no voters: a92d6df25f4a43e0872ecbb6451058d5
I20260709 13:00:59.862121   782 raft_consensus.cc:515] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.868580   807 raft_consensus.cc:2804] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:59.872752   807 raft_consensus.cc:697] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Becoming Leader. State: Replica: da373aac3d614f56b081496dd9145b01, State: Running, Role: LEADER
I20260709 13:00:59.870658   782 leader_election.cc:290] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:00:59.879917   711 raft_consensus.cc:2468] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658e7299505c46d797b4ad2be4129399 in term 1.
I20260709 13:00:59.880985   807 consensus_queue.cc:237] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.888166   606 leader_election.cc:304] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, a92d6df25f4a43e0872ecbb6451058d5; no voters: 
I20260709 13:00:59.888861   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b236fa7371f141608adb61630a8c90ec" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399" is_pre_election: true
I20260709 13:00:59.889938   765 raft_consensus.cc:2804] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:59.893728   765 raft_consensus.cc:697] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 1 LEADER]: Becoming Leader. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Running, Role: LEADER
W20260709 13:00:59.895605   529 leader_election.cc:343] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:59.895519   765 consensus_queue.cc:237] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.899793   641 raft_consensus.cc:2468] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 1.
I20260709 13:00:59.917325   760 tablet_bootstrap.cc:492] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5: No bootstrap required, opened a new log
I20260709 13:00:59.919078   760 ts_tablet_manager.cc:1403] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5: Time spent bootstrapping tablet: real 0.403s	user 0.026s	sys 0.039s
I20260709 13:00:59.919049   529 leader_election.cc:304] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01; no voters: 
I20260709 13:00:59.920862   759 tablet_bootstrap.cc:492] T b236fa7371f141608adb61630a8c90ec P 658e7299505c46d797b4ad2be4129399: Bootstrap starting.
I20260709 13:00:59.924314   822 raft_consensus.cc:2804] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:59.924616   760 raft_consensus.cc:359] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.926115   760 raft_consensus.cc:385] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:59.928916   760 raft_consensus.cc:740] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a92d6df25f4a43e0872ecbb6451058d5, State: Initialized, Role: FOLLOWER
I20260709 13:00:59.931041   760 consensus_queue.cc:260] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:00:59.939481   760 ts_tablet_manager.cc:1434] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.007s
I20260709 13:00:59.939718   639 raft_consensus.cc:2468] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 1.
I20260709 13:00:59.941466   809 consensus_queue.cc:1048] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:00:59.945057   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:00:59.945999   715 raft_consensus.cc:2393] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate da373aac3d614f56b081496dd9145b01 in current term 1: Already voted for candidate 658e7299505c46d797b4ad2be4129399 in this term.
I20260709 13:00:59.947511   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba58747e7ae04d4f81c4ffc2a323f570" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:00:59.948352   715 raft_consensus.cc:2393] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate da373aac3d614f56b081496dd9145b01 in current term 1: Already voted for candidate a92d6df25f4a43e0872ecbb6451058d5 in this term.
I20260709 13:00:59.949544   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b236fa7371f141608adb61630a8c90ec" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5" is_pre_election: true
I20260709 13:00:59.954731   531 leader_election.cc:304] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da373aac3d614f56b081496dd9145b01; no voters: 658e7299505c46d797b4ad2be4129399, a92d6df25f4a43e0872ecbb6451058d5
I20260709 13:00:59.955459   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba58747e7ae04d4f81c4ffc2a323f570" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399"
I20260709 13:00:59.955562   764 leader_election.cc:290] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 1 election: Requested vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721)
I20260709 13:00:59.956393   809 raft_consensus.cc:2749] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:59.956781   640 raft_consensus.cc:2393] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a92d6df25f4a43e0872ecbb6451058d5 in current term 1: Already voted for candidate da373aac3d614f56b081496dd9145b01 in this term.
W20260709 13:00:59.960376   531 leader_election.cc:343] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:59.960932   531 leader_election.cc:304] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da373aac3d614f56b081496dd9145b01; no voters: 658e7299505c46d797b4ad2be4129399, a92d6df25f4a43e0872ecbb6451058d5
I20260709 13:00:59.961437   561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da373aac3d614f56b081496dd9145b01"
I20260709 13:00:59.962430   561 raft_consensus.cc:2393] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 658e7299505c46d797b4ad2be4129399 in current term 1: Already voted for candidate da373aac3d614f56b081496dd9145b01 in this term.
I20260709 13:00:59.963271   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4f664fa01d44aab86d4911e7fd45c7" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399"
I20260709 13:00:59.964097   640 raft_consensus.cc:2393] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a92d6df25f4a43e0872ecbb6451058d5 in current term 1: Already voted for candidate da373aac3d614f56b081496dd9145b01 in this term.
I20260709 13:00:59.925573   822 raft_consensus.cc:697] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Becoming Leader. State: Replica: da373aac3d614f56b081496dd9145b01, State: Running, Role: LEADER
I20260709 13:00:59.940867   760 tablet_bootstrap.cc:492] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5: Bootstrap starting.
I20260709 13:00:59.966553   679 leader_election.cc:304] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a92d6df25f4a43e0872ecbb6451058d5; no voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01
I20260709 13:00:59.972855   605 leader_election.cc:304] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 658e7299505c46d797b4ad2be4129399; no voters: a92d6df25f4a43e0872ecbb6451058d5, da373aac3d614f56b081496dd9145b01
I20260709 13:00:59.974457   802 raft_consensus.cc:2749] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:59.977411   764 raft_consensus.cc:2749] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:59.981447   806 leader_election.cc:290] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 1 election: Requested vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721)
I20260709 13:00:59.981449   809 raft_consensus.cc:2749] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:59.986562   529 leader_election.cc:304] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01; no voters: a92d6df25f4a43e0872ecbb6451058d5
I20260709 13:00:59.987509   809 raft_consensus.cc:2804] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:59.990149   810 consensus_queue.cc:1048] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:00:59.988355   809 raft_consensus.cc:697] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Becoming Leader. State: Replica: da373aac3d614f56b081496dd9145b01, State: Running, Role: LEADER
I20260709 13:00:59.993546   766 consensus_queue.cc:1048] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:00:59.998433   678 leader_election.cc:304] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a92d6df25f4a43e0872ecbb6451058d5; no voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01
I20260709 13:01:00.005699   822 consensus_queue.cc:237] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
W20260709 13:01:00.013492   510 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:00.017623   809 consensus_queue.cc:237] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:00.025085   766 consensus_queue.cc:1048] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:00.031867   759 tablet_bootstrap.cc:654] T b236fa7371f141608adb61630a8c90ec P 658e7299505c46d797b4ad2be4129399: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:00.026571   806 raft_consensus.cc:2749] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:01:00.036854   463 catalog_manager.cc:5697] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 reported cstate change: term changed from 0 to 1, leader changed from <none> to 658e7299505c46d797b4ad2be4129399 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "658e7299505c46d797b4ad2be4129399" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.050338   760 tablet_bootstrap.cc:654] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:00.120223   806 raft_consensus.cc:493] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.120750   806 raft_consensus.cc:515] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:00.122918   806 leader_election.cc:290] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721)
I20260709 13:01:00.124641   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da373aac3d614f56b081496dd9145b01" is_pre_election: true
I20260709 13:01:00.138070   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399" is_pre_election: true
I20260709 13:01:00.139057   641 raft_consensus.cc:2393] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a92d6df25f4a43e0872ecbb6451058d5 in current term 1: Already voted for candidate da373aac3d614f56b081496dd9145b01 in this term.
I20260709 13:01:00.140440   760 tablet_bootstrap.cc:492] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5: No bootstrap required, opened a new log
I20260709 13:01:00.140985   760 ts_tablet_manager.cc:1403] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5: Time spent bootstrapping tablet: real 0.200s	user 0.012s	sys 0.028s
I20260709 13:01:00.140187   678 leader_election.cc:304] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a92d6df25f4a43e0872ecbb6451058d5; no voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01
I20260709 13:01:00.143293   806 raft_consensus.cc:3060] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.144321   760 raft_consensus.cc:359] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:00.145133   760 raft_consensus.cc:385] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:00.145444   760 raft_consensus.cc:740] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a92d6df25f4a43e0872ecbb6451058d5, State: Initialized, Role: FOLLOWER
I20260709 13:01:00.146207   760 consensus_queue.cc:260] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:00.148412   760 ts_tablet_manager.cc:1434] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:01:00.149380   760 tablet_bootstrap.cc:492] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5: Bootstrap starting.
I20260709 13:01:00.153875   806 raft_consensus.cc:2749] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:00.155916   760 tablet_bootstrap.cc:654] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:00.156870   460 catalog_manager.cc:5697] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to da373aac3d614f56b081496dd9145b01 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "da373aac3d614f56b081496dd9145b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.156751   759 tablet_bootstrap.cc:492] T b236fa7371f141608adb61630a8c90ec P 658e7299505c46d797b4ad2be4129399: No bootstrap required, opened a new log
I20260709 13:01:00.159600   759 ts_tablet_manager.cc:1403] T b236fa7371f141608adb61630a8c90ec P 658e7299505c46d797b4ad2be4129399: Time spent bootstrapping tablet: real 0.239s	user 0.009s	sys 0.026s
I20260709 13:01:00.159217   460 catalog_manager.cc:5697] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to da373aac3d614f56b081496dd9145b01 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "da373aac3d614f56b081496dd9145b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.161782   460 catalog_manager.cc:5697] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to da373aac3d614f56b081496dd9145b01 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "da373aac3d614f56b081496dd9145b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.163221   759 raft_consensus.cc:359] T b236fa7371f141608adb61630a8c90ec P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:00.163975   759 raft_consensus.cc:385] T b236fa7371f141608adb61630a8c90ec P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:00.164297   759 raft_consensus.cc:740] T b236fa7371f141608adb61630a8c90ec P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Initialized, Role: FOLLOWER
I20260709 13:01:00.163797   460 catalog_manager.cc:5697] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to da373aac3d614f56b081496dd9145b01 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "da373aac3d614f56b081496dd9145b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.165442   759 consensus_queue.cc:260] T b236fa7371f141608adb61630a8c90ec P 658e7299505c46d797b4ad2be4129399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:00.186475   759 ts_tablet_manager.cc:1434] T b236fa7371f141608adb61630a8c90ec P 658e7299505c46d797b4ad2be4129399: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 13:01:00.189277   760 tablet_bootstrap.cc:492] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5: No bootstrap required, opened a new log
I20260709 13:01:00.189826   760 ts_tablet_manager.cc:1403] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.016s
I20260709 13:01:00.192610   760 raft_consensus.cc:359] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:00.193322   760 raft_consensus.cc:385] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:00.193667   760 raft_consensus.cc:740] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a92d6df25f4a43e0872ecbb6451058d5, State: Initialized, Role: FOLLOWER
I20260709 13:01:00.194430   760 consensus_queue.cc:260] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:00.196552   760 ts_tablet_manager.cc:1434] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:01:00.294788   822 consensus_queue.cc:1048] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.312256   822 consensus_queue.cc:1048] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:00.403173   711 raft_consensus.cc:3060] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.414431   806 raft_consensus.cc:493] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.415164   806 raft_consensus.cc:515] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:00.415431   766 consensus_queue.cc:1048] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:00.416471   822 consensus_queue.cc:1048] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.421018   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af84729cf07482cb0244953a91a3a68" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da373aac3d614f56b081496dd9145b01" is_pre_election: true
I20260709 13:01:00.424374   806 leader_election.cc:290] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721)
I20260709 13:01:00.425024   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af84729cf07482cb0244953a91a3a68" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399" is_pre_election: true
I20260709 13:01:00.428009   678 leader_election.cc:304] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a92d6df25f4a43e0872ecbb6451058d5; no voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01
I20260709 13:01:00.432904   806 raft_consensus.cc:2749] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:00.437081   802 consensus_queue.cc:1048] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.445254   806 raft_consensus.cc:493] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.445839   806 raft_consensus.cc:515] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:00.449420   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b236fa7371f141608adb61630a8c90ec" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da373aac3d614f56b081496dd9145b01" is_pre_election: true
I20260709 13:01:00.450206   565 raft_consensus.cc:2468] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a92d6df25f4a43e0872ecbb6451058d5 in term 0.
I20260709 13:01:00.451777   679 leader_election.cc:304] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a92d6df25f4a43e0872ecbb6451058d5, da373aac3d614f56b081496dd9145b01; no voters: 
I20260709 13:01:00.452688   764 raft_consensus.cc:2804] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:00.453102   764 raft_consensus.cc:493] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.453681   764 raft_consensus.cc:3060] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.461898   806 leader_election.cc:290] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721)
I20260709 13:01:00.464242   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b236fa7371f141608adb61630a8c90ec" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399" is_pre_election: true
I20260709 13:01:00.465153   641 raft_consensus.cc:2468] T b236fa7371f141608adb61630a8c90ec P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a92d6df25f4a43e0872ecbb6451058d5 in term 0.
I20260709 13:01:00.465278   822 consensus_queue.cc:1048] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:01:00.469399   764 raft_consensus.cc:515] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:00.475379   839 consensus_queue.cc:1048] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:00.477192   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b236fa7371f141608adb61630a8c90ec" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da373aac3d614f56b081496dd9145b01"
I20260709 13:01:00.477975   565 raft_consensus.cc:3060] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:00.487716   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b236fa7371f141608adb61630a8c90ec" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658e7299505c46d797b4ad2be4129399"
I20260709 13:01:00.488415   640 raft_consensus.cc:3060] T b236fa7371f141608adb61630a8c90ec P 658e7299505c46d797b4ad2be4129399 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:00.488832   605 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33951, user_credentials={real_user=slave}} blocked reactor thread for 54560.8us
I20260709 13:01:00.520961   565 raft_consensus.cc:2468] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a92d6df25f4a43e0872ecbb6451058d5 in term 1.
I20260709 13:01:00.522975   640 raft_consensus.cc:2468] T b236fa7371f141608adb61630a8c90ec P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a92d6df25f4a43e0872ecbb6451058d5 in term 1.
I20260709 13:01:00.526669   679 leader_election.cc:304] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a92d6df25f4a43e0872ecbb6451058d5, da373aac3d614f56b081496dd9145b01; no voters: 
I20260709 13:01:00.523609   764 leader_election.cc:290] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 1 election: Requested vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721)
I20260709 13:01:00.527648   806 raft_consensus.cc:2804] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:00.532889   806 raft_consensus.cc:697] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [term 1 LEADER]: Becoming Leader. State: Replica: a92d6df25f4a43e0872ecbb6451058d5, State: Running, Role: LEADER
I20260709 13:01:00.533861   806 consensus_queue.cc:237] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:00.542068   809 consensus_queue.cc:1048] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:00.561080   813 consensus_queue.cc:1048] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:00.561546   463 catalog_manager.cc:5697] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a92d6df25f4a43e0872ecbb6451058d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:00.579557   813 consensus_queue.cc:1048] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:00.744189   641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc"
dest_uuid: "658e7299505c46d797b4ad2be4129399"
mode: GRACEFUL
new_leader_uuid: "da373aac3d614f56b081496dd9145b01"
 from {username='slave'} at 127.0.0.1:35100
I20260709 13:01:00.745146   641 raft_consensus.cc:606] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 1 LEADER]: Received request to transfer leadership to TS da373aac3d614f56b081496dd9145b01
I20260709 13:01:00.777663   565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a"
dest_uuid: "da373aac3d614f56b081496dd9145b01"
mode: GRACEFUL
new_leader_uuid: "658e7299505c46d797b4ad2be4129399"
 from {username='slave'} at 127.0.0.1:47664
I20260709 13:01:00.778484   565 raft_consensus.cc:606] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Received request to transfer leadership to TS 658e7299505c46d797b4ad2be4129399
I20260709 13:01:00.787698   565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba58747e7ae04d4f81c4ffc2a323f570"
dest_uuid: "da373aac3d614f56b081496dd9145b01"
mode: GRACEFUL
new_leader_uuid: "658e7299505c46d797b4ad2be4129399"
 from {username='slave'} at 127.0.0.1:47664
I20260709 13:01:00.788709   565 raft_consensus.cc:606] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Received request to transfer leadership to TS 658e7299505c46d797b4ad2be4129399
I20260709 13:01:00.798810   565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78af706a10f146fe8c6a5c64d70c6894"
dest_uuid: "da373aac3d614f56b081496dd9145b01"
mode: GRACEFUL
new_leader_uuid: "658e7299505c46d797b4ad2be4129399"
 from {username='slave'} at 127.0.0.1:47664
I20260709 13:01:00.799530   565 raft_consensus.cc:606] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Received request to transfer leadership to TS 658e7299505c46d797b4ad2be4129399
I20260709 13:01:00.807267   565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8af84729cf07482cb0244953a91a3a68"
dest_uuid: "da373aac3d614f56b081496dd9145b01"
mode: GRACEFUL
new_leader_uuid: "658e7299505c46d797b4ad2be4129399"
 from {username='slave'} at 127.0.0.1:47664
I20260709 13:01:00.808879   565 raft_consensus.cc:606] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Received request to transfer leadership to TS 658e7299505c46d797b4ad2be4129399
I20260709 13:01:00.823776   809 raft_consensus.cc:993] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01: : Instructing follower 658e7299505c46d797b4ad2be4129399 to start an election
I20260709 13:01:00.824370   822 raft_consensus.cc:1081] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Signalling peer 658e7299505c46d797b4ad2be4129399 to start an election
I20260709 13:01:00.826748   641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a"
dest_uuid: "658e7299505c46d797b4ad2be4129399"
 from {username='slave'} at 127.0.0.1:35086
I20260709 13:01:00.827397   641 raft_consensus.cc:493] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:00.827718   641 raft_consensus.cc:3060] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.835069   641 raft_consensus.cc:515] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:00.837509   641 leader_election.cc:290] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 2 election: Requested vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:01:00.838835   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da373aac3d614f56b081496dd9145b01"
I20260709 13:01:00.839485   565 raft_consensus.cc:3055] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:00.839365   711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:01:00.839913   565 raft_consensus.cc:740] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da373aac3d614f56b081496dd9145b01, State: Running, Role: LEADER
I20260709 13:01:00.840202   711 raft_consensus.cc:3060] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.840925   565 consensus_queue.cc:260] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:00.843024   565 raft_consensus.cc:3060] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.848093   711 raft_consensus.cc:2468] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658e7299505c46d797b4ad2be4129399 in term 2.
I20260709 13:01:00.849900   606 leader_election.cc:304] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, a92d6df25f4a43e0872ecbb6451058d5; no voters: 
I20260709 13:01:00.851002   840 raft_consensus.cc:2804] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:00.851060   565 raft_consensus.cc:2468] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658e7299505c46d797b4ad2be4129399 in term 2.
I20260709 13:01:00.851755   840 raft_consensus.cc:697] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Becoming Leader. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Running, Role: LEADER
I20260709 13:01:00.853583   840 consensus_queue.cc:237] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:00.864006   460 catalog_manager.cc:5697] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 reported cstate change: term changed from 1 to 2, leader changed from da373aac3d614f56b081496dd9145b01 (127.0.107.193) to 658e7299505c46d797b4ad2be4129399 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "658e7299505c46d797b4ad2be4129399" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.908125   809 raft_consensus.cc:993] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01: : Instructing follower 658e7299505c46d797b4ad2be4129399 to start an election
I20260709 13:01:00.908852   822 raft_consensus.cc:1081] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Signalling peer 658e7299505c46d797b4ad2be4129399 to start an election
I20260709 13:01:00.910936   641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8af84729cf07482cb0244953a91a3a68"
dest_uuid: "658e7299505c46d797b4ad2be4129399"
 from {username='slave'} at 127.0.0.1:35086
I20260709 13:01:00.911566   641 raft_consensus.cc:493] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:00.911916   641 raft_consensus.cc:3060] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.919313   641 raft_consensus.cc:515] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:00.922067   641 leader_election.cc:290] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 2 election: Requested vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:01:00.923110   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af84729cf07482cb0244953a91a3a68" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da373aac3d614f56b081496dd9145b01"
I20260709 13:01:00.923357   711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af84729cf07482cb0244953a91a3a68" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:01:00.923787   565 raft_consensus.cc:3055] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:00.924052   711 raft_consensus.cc:3060] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.924220   565 raft_consensus.cc:740] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da373aac3d614f56b081496dd9145b01, State: Running, Role: LEADER
I20260709 13:01:00.925187   565 consensus_queue.cc:260] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:00.926469   565 raft_consensus.cc:3060] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:00.932655   711 raft_consensus.cc:2468] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658e7299505c46d797b4ad2be4129399 in term 2.
I20260709 13:01:00.934759   565 raft_consensus.cc:2468] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658e7299505c46d797b4ad2be4129399 in term 2.
I20260709 13:01:00.935355   606 leader_election.cc:304] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, a92d6df25f4a43e0872ecbb6451058d5; no voters: 
I20260709 13:01:00.936465   840 raft_consensus.cc:2804] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:00.937075   840 raft_consensus.cc:697] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Becoming Leader. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Running, Role: LEADER
I20260709 13:01:00.938071   840 consensus_queue.cc:237] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:00.947316   460 catalog_manager.cc:5697] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 reported cstate change: term changed from 1 to 2, leader changed from da373aac3d614f56b081496dd9145b01 (127.0.107.193) to 658e7299505c46d797b4ad2be4129399 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "658e7299505c46d797b4ad2be4129399" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.962651   806 consensus_queue.cc:1048] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:00.979437   764 consensus_queue.cc:1048] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:01.041421   802 raft_consensus.cc:993] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399: : Instructing follower da373aac3d614f56b081496dd9145b01 to start an election
I20260709 13:01:01.042011   840 raft_consensus.cc:1081] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 1 LEADER]: Signalling peer da373aac3d614f56b081496dd9145b01 to start an election
I20260709 13:01:01.043928   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc"
dest_uuid: "da373aac3d614f56b081496dd9145b01"
 from {username='slave'} at 127.0.0.1:47660
I20260709 13:01:01.044687   565 raft_consensus.cc:493] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:01.045189   565 raft_consensus.cc:3060] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:01.052827   565 raft_consensus.cc:515] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:01.055904   565 leader_election.cc:290] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 2 election: Requested vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:01:01.057384   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "658e7299505c46d797b4ad2be4129399"
I20260709 13:01:01.058163   641 raft_consensus.cc:3055] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:01.058578   641 raft_consensus.cc:740] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Running, Role: LEADER
I20260709 13:01:01.059122   711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:01:01.059855   711 raft_consensus.cc:3060] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:01.059908   641 consensus_queue.cc:260] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:01.061180   641 raft_consensus.cc:3060] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:01.066661   809 raft_consensus.cc:993] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01: : Instructing follower 658e7299505c46d797b4ad2be4129399 to start an election
I20260709 13:01:01.068688   809 raft_consensus.cc:1081] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Signalling peer 658e7299505c46d797b4ad2be4129399 to start an election
I20260709 13:01:01.069139   711 raft_consensus.cc:2468] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 2.
I20260709 13:01:01.072846   641 raft_consensus.cc:2468] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 2.
I20260709 13:01:01.073446   640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ba58747e7ae04d4f81c4ffc2a323f570"
dest_uuid: "658e7299505c46d797b4ad2be4129399"
 from {username='slave'} at 127.0.0.1:35086
I20260709 13:01:01.074777   640 raft_consensus.cc:493] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:01.074968   529 leader_election.cc:304] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01; no voters: 
I20260709 13:01:01.075390   640 raft_consensus.cc:3060] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:01.075927   807 raft_consensus.cc:2804] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:01.076550   807 raft_consensus.cc:697] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 2 LEADER]: Becoming Leader. State: Replica: da373aac3d614f56b081496dd9145b01, State: Running, Role: LEADER
I20260709 13:01:01.077474   807 consensus_queue.cc:237] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:01.091938   640 raft_consensus.cc:515] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:01.094913   462 catalog_manager.cc:5697] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 reported cstate change: term changed from 1 to 2, leader changed from 658e7299505c46d797b4ad2be4129399 (127.0.107.194) to da373aac3d614f56b081496dd9145b01 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "da373aac3d614f56b081496dd9145b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:01.097447   640 leader_election.cc:290] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 2 election: Requested vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:01:01.097214   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba58747e7ae04d4f81c4ffc2a323f570" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da373aac3d614f56b081496dd9145b01"
I20260709 13:01:01.097349   711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba58747e7ae04d4f81c4ffc2a323f570" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:01:01.098420   565 raft_consensus.cc:3055] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:01.098827   565 raft_consensus.cc:740] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da373aac3d614f56b081496dd9145b01, State: Running, Role: LEADER
I20260709 13:01:01.100163   565 consensus_queue.cc:260] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:01.101430   565 raft_consensus.cc:3060] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:01.102044   711 raft_consensus.cc:3060] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:01.112534   711 raft_consensus.cc:2468] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658e7299505c46d797b4ad2be4129399 in term 2.
I20260709 13:01:01.114601   606 leader_election.cc:304] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, a92d6df25f4a43e0872ecbb6451058d5; no voters: 
I20260709 13:01:01.116219   840 raft_consensus.cc:2804] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:01.116825   840 raft_consensus.cc:697] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Becoming Leader. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Running, Role: LEADER
I20260709 13:01:01.117221   565 raft_consensus.cc:2468] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658e7299505c46d797b4ad2be4129399 in term 2.
I20260709 13:01:01.117892   840 consensus_queue.cc:237] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:01.127517   462 catalog_manager.cc:5697] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 reported cstate change: term changed from 1 to 2, leader changed from da373aac3d614f56b081496dd9145b01 (127.0.107.193) to 658e7299505c46d797b4ad2be4129399 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "658e7299505c46d797b4ad2be4129399" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:01.230952   807 raft_consensus.cc:993] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01: : Instructing follower 658e7299505c46d797b4ad2be4129399 to start an election
I20260709 13:01:01.231519   807 raft_consensus.cc:1081] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Signalling peer 658e7299505c46d797b4ad2be4129399 to start an election
I20260709 13:01:01.233599   640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "78af706a10f146fe8c6a5c64d70c6894"
dest_uuid: "658e7299505c46d797b4ad2be4129399"
 from {username='slave'} at 127.0.0.1:35086
I20260709 13:01:01.234611   640 raft_consensus.cc:493] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:01.235247   640 raft_consensus.cc:3060] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:01.243680   640 raft_consensus.cc:515] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:01.247493   711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78af706a10f146fe8c6a5c64d70c6894" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:01:01.245913   640 leader_election.cc:290] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 2 election: Requested vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:01:01.248142   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78af706a10f146fe8c6a5c64d70c6894" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da373aac3d614f56b081496dd9145b01"
I20260709 13:01:01.248888   565 raft_consensus.cc:3055] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:01.249395   565 raft_consensus.cc:740] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da373aac3d614f56b081496dd9145b01, State: Running, Role: LEADER
I20260709 13:01:01.248175   711 raft_consensus.cc:3060] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:01.251398   565 consensus_queue.cc:260] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:01.252707   565 raft_consensus.cc:3060] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:01.257431   711 raft_consensus.cc:2468] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658e7299505c46d797b4ad2be4129399 in term 2.
I20260709 13:01:01.259025   606 leader_election.cc:304] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, a92d6df25f4a43e0872ecbb6451058d5; no voters: 
I20260709 13:01:01.260058   840 raft_consensus.cc:2804] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:01.261009   565 raft_consensus.cc:2468] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658e7299505c46d797b4ad2be4129399 in term 2.
I20260709 13:01:01.261236   840 raft_consensus.cc:697] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Becoming Leader. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Running, Role: LEADER
I20260709 13:01:01.262738   840 consensus_queue.cc:237] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:01.275487   711 raft_consensus.cc:1275] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Refusing update from remote peer 658e7299505c46d797b4ad2be4129399: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:01.278280   840 consensus_queue.cc:1048] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:01.277801   462 catalog_manager.cc:5697] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 reported cstate change: term changed from 1 to 2, leader changed from da373aac3d614f56b081496dd9145b01 (127.0.107.193) to 658e7299505c46d797b4ad2be4129399 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "658e7299505c46d797b4ad2be4129399" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:01.290894   565 raft_consensus.cc:1275] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Refusing update from remote peer 658e7299505c46d797b4ad2be4129399: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:01.294785   855 consensus_queue.cc:1048] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:01.449267   711 raft_consensus.cc:1275] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Refusing update from remote peer 658e7299505c46d797b4ad2be4129399: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:01.451023   840 consensus_queue.cc:1048] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:01.463546   565 raft_consensus.cc:1275] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Refusing update from remote peer 658e7299505c46d797b4ad2be4129399: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:01.465754   802 consensus_queue.cc:1048] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:01.476950   640 raft_consensus.cc:1275] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Refusing update from remote peer da373aac3d614f56b081496dd9145b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:01.479717   807 consensus_queue.cc:1048] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:01.495467   711 raft_consensus.cc:1275] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Refusing update from remote peer da373aac3d614f56b081496dd9145b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:01.498955   807 consensus_queue.cc:1048] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:01.591251   565 raft_consensus.cc:1275] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Refusing update from remote peer 658e7299505c46d797b4ad2be4129399: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:01.593566   802 consensus_queue.cc:1048] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:01.603255   711 raft_consensus.cc:1275] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Refusing update from remote peer 658e7299505c46d797b4ad2be4129399: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:01.605592   840 consensus_queue.cc:1048] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:01.682665   565 raft_consensus.cc:1275] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Refusing update from remote peer 658e7299505c46d797b4ad2be4129399: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:01.684461   840 consensus_queue.cc:1048] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:01.698518   715 raft_consensus.cc:1275] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Refusing update from remote peer 658e7299505c46d797b4ad2be4129399: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:01.703125   840 consensus_queue.cc:1048] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:03.853372   431 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:01:03.854465   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:03.885710   563 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc"
dest_uuid: "da373aac3d614f56b081496dd9145b01"
mode: GRACEFUL
new_leader_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
 from {username='slave'} at 127.0.0.1:47670
I20260709 13:01:03.886662   563 raft_consensus.cc:606] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 2 LEADER]: Received request to transfer leadership to TS a92d6df25f4a43e0872ecbb6451058d5
I20260709 13:01:03.904450   641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a"
dest_uuid: "658e7299505c46d797b4ad2be4129399"
mode: GRACEFUL
new_leader_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:01:03.905444   641 raft_consensus.cc:606] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Received request to transfer leadership to TS a92d6df25f4a43e0872ecbb6451058d5
I20260709 13:01:03.907348   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:01:03.908071   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:03.967940   807 raft_consensus.cc:993] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01: : Instructing follower a92d6df25f4a43e0872ecbb6451058d5 to start an election
I20260709 13:01:03.968389   807 raft_consensus.cc:1081] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 2 LEADER]: Signalling peer a92d6df25f4a43e0872ecbb6451058d5 to start an election
I20260709 13:01:03.970430   715 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc"
dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
 from {username='slave'} at 127.0.0.1:41710
I20260709 13:01:03.971082   715 raft_consensus.cc:493] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:03.971382   715 raft_consensus.cc:3060] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:03.979739   715 raft_consensus.cc:515] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:03.984449   715 leader_election.cc:290] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 3 election: Requested vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721)
I20260709 13:01:03.984858   563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "da373aac3d614f56b081496dd9145b01"
I20260709 13:01:03.985842   563 raft_consensus.cc:3055] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:03.986322   563 raft_consensus.cc:740] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: da373aac3d614f56b081496dd9145b01, State: Running, Role: LEADER
I20260709 13:01:03.985706   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "658e7299505c46d797b4ad2be4129399"
I20260709 13:01:03.987072   641 raft_consensus.cc:3060] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:03.987111   563 consensus_queue.cc:260] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:03.988613   563 raft_consensus.cc:3060] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:03.993110   641 raft_consensus.cc:2468] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a92d6df25f4a43e0872ecbb6451058d5 in term 3.
I20260709 13:01:03.995158   678 leader_election.cc:304] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, a92d6df25f4a43e0872ecbb6451058d5; no voters: 
I20260709 13:01:03.998551   563 raft_consensus.cc:2468] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a92d6df25f4a43e0872ecbb6451058d5 in term 3.
I20260709 13:01:03.998696   864 raft_consensus.cc:2804] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:04.001096   864 raft_consensus.cc:697] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 3 LEADER]: Becoming Leader. State: Replica: a92d6df25f4a43e0872ecbb6451058d5, State: Running, Role: LEADER
I20260709 13:01:04.002285   864 consensus_queue.cc:237] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:04.011663   462 catalog_manager.cc:5697] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 reported cstate change: term changed from 2 to 3, leader changed from da373aac3d614f56b081496dd9145b01 (127.0.107.193) to a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "a92d6df25f4a43e0872ecbb6451058d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:04.060945   840 raft_consensus.cc:993] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399: : Instructing follower a92d6df25f4a43e0872ecbb6451058d5 to start an election
I20260709 13:01:04.061812   840 raft_consensus.cc:1081] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Signalling peer a92d6df25f4a43e0872ecbb6451058d5 to start an election
I20260709 13:01:04.065088   715 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a"
dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
 from {username='slave'} at 127.0.0.1:41722
I20260709 13:01:04.065997   715 raft_consensus.cc:493] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:04.066680   715 raft_consensus.cc:3060] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.073338   715 raft_consensus.cc:515] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:04.075472   715 leader_election.cc:290] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 3 election: Requested vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721)
I20260709 13:01:04.076479   563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "da373aac3d614f56b081496dd9145b01"
I20260709 13:01:04.077242   563 raft_consensus.cc:3060] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.081807   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "658e7299505c46d797b4ad2be4129399"
I20260709 13:01:04.085083   563 raft_consensus.cc:2468] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a92d6df25f4a43e0872ecbb6451058d5 in term 3.
I20260709 13:01:04.084502   641 raft_consensus.cc:3055] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:04.085907   641 raft_consensus.cc:740] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Running, Role: LEADER
I20260709 13:01:04.086645   679 leader_election.cc:304] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a92d6df25f4a43e0872ecbb6451058d5, da373aac3d614f56b081496dd9145b01; no voters: 
I20260709 13:01:04.086973   641 consensus_queue.cc:260] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:04.087486   864 raft_consensus.cc:2804] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:04.088019   864 raft_consensus.cc:697] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 3 LEADER]: Becoming Leader. State: Replica: a92d6df25f4a43e0872ecbb6451058d5, State: Running, Role: LEADER
I20260709 13:01:04.088239   641 raft_consensus.cc:3060] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:04.088919   864 consensus_queue.cc:237] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:04.094045   641 raft_consensus.cc:2468] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a92d6df25f4a43e0872ecbb6451058d5 in term 3.
I20260709 13:01:04.103531   462 catalog_manager.cc:5697] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 reported cstate change: term changed from 2 to 3, leader changed from 658e7299505c46d797b4ad2be4129399 (127.0.107.194) to a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "a92d6df25f4a43e0872ecbb6451058d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:04.517153   641 raft_consensus.cc:1275] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 3 FOLLOWER]: Refusing update from remote peer a92d6df25f4a43e0872ecbb6451058d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:04.518730   864 consensus_queue.cc:1048] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:04.529714   563 raft_consensus.cc:1275] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 3 FOLLOWER]: Refusing update from remote peer a92d6df25f4a43e0872ecbb6451058d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:04.532490   864 consensus_queue.cc:1048] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:04.542802   563 raft_consensus.cc:1275] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 3 FOLLOWER]: Refusing update from remote peer a92d6df25f4a43e0872ecbb6451058d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:04.547470   864 consensus_queue.cc:1048] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:04.557988   641 raft_consensus.cc:1275] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 3 FOLLOWER]: Refusing update from remote peer a92d6df25f4a43e0872ecbb6451058d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:04.560071   864 consensus_queue.cc:1048] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:04.909149   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:04.922228   563 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d500279905444ef8b7d48b790d14d240"
dest_uuid: "da373aac3d614f56b081496dd9145b01"
mode: GRACEFUL
new_leader_uuid: "658e7299505c46d797b4ad2be4129399"
 from {username='slave'} at 127.0.0.1:47664
I20260709 13:01:04.922827   563 raft_consensus.cc:606] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Received request to transfer leadership to TS 658e7299505c46d797b4ad2be4129399
I20260709 13:01:04.940912   711 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc"
dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
mode: GRACEFUL
new_leader_uuid: "658e7299505c46d797b4ad2be4129399"
 from {username='slave'} at 127.0.0.1:41738
I20260709 13:01:04.941519   711 raft_consensus.cc:606] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 3 LEADER]: Received request to transfer leadership to TS 658e7299505c46d797b4ad2be4129399
I20260709 13:01:04.949040   563 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e4f664fa01d44aab86d4911e7fd45c7"
dest_uuid: "da373aac3d614f56b081496dd9145b01"
mode: GRACEFUL
new_leader_uuid: "658e7299505c46d797b4ad2be4129399"
 from {username='slave'} at 127.0.0.1:47664
I20260709 13:01:04.949625   563 raft_consensus.cc:606] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Received request to transfer leadership to TS 658e7299505c46d797b4ad2be4129399
I20260709 13:01:04.955915   563 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98c04e9dd15b431d91ea29bcb7152983"
dest_uuid: "da373aac3d614f56b081496dd9145b01"
mode: GRACEFUL
new_leader_uuid: "658e7299505c46d797b4ad2be4129399"
 from {username='slave'} at 127.0.0.1:47664
I20260709 13:01:04.956900   563 raft_consensus.cc:606] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Received request to transfer leadership to TS 658e7299505c46d797b4ad2be4129399
I20260709 13:01:04.966836   711 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a"
dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
mode: GRACEFUL
new_leader_uuid: "658e7299505c46d797b4ad2be4129399"
 from {username='slave'} at 127.0.0.1:41738
I20260709 13:01:04.967459   711 raft_consensus.cc:606] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 3 LEADER]: Received request to transfer leadership to TS 658e7299505c46d797b4ad2be4129399
I20260709 13:01:04.984323   883 raft_consensus.cc:993] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5: : Instructing follower 658e7299505c46d797b4ad2be4129399 to start an election
I20260709 13:01:04.984877   883 raft_consensus.cc:1081] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 3 LEADER]: Signalling peer 658e7299505c46d797b4ad2be4129399 to start an election
I20260709 13:01:04.987224   641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc"
dest_uuid: "658e7299505c46d797b4ad2be4129399"
 from {username='slave'} at 127.0.0.1:35090
I20260709 13:01:04.987771   641 raft_consensus.cc:493] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:04.988082   641 raft_consensus.cc:3060] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:04.993566   641 raft_consensus.cc:515] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:04.996987   563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "da373aac3d614f56b081496dd9145b01"
I20260709 13:01:04.997754   563 raft_consensus.cc:3060] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:04.997822   711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:01:04.998059   641 leader_election.cc:290] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 4 election: Requested vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:01:04.999017   711 raft_consensus.cc:3055] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:04.999454   711 raft_consensus.cc:740] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a92d6df25f4a43e0872ecbb6451058d5, State: Running, Role: LEADER
I20260709 13:01:05.000479   711 consensus_queue.cc:260] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:05.001981   711 raft_consensus.cc:3060] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:05.007027   563 raft_consensus.cc:2468] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658e7299505c46d797b4ad2be4129399 in term 4.
I20260709 13:01:05.009032   605 leader_election.cc:304] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01; no voters: 
I20260709 13:01:05.010246   840 raft_consensus.cc:2804] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:05.010546   711 raft_consensus.cc:2468] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658e7299505c46d797b4ad2be4129399 in term 4.
I20260709 13:01:05.010952   840 raft_consensus.cc:697] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 4 LEADER]: Becoming Leader. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Running, Role: LEADER
I20260709 13:01:05.012096   840 consensus_queue.cc:237] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:05.019843   460 catalog_manager.cc:5697] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 reported cstate change: term changed from 3 to 4, leader changed from a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195) to 658e7299505c46d797b4ad2be4129399 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "658e7299505c46d797b4ad2be4129399" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:05.050894   883 raft_consensus.cc:993] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5: : Instructing follower 658e7299505c46d797b4ad2be4129399 to start an election
I20260709 13:01:05.051409   883 raft_consensus.cc:1081] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 3 LEADER]: Signalling peer 658e7299505c46d797b4ad2be4129399 to start an election
I20260709 13:01:05.053448   641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a"
dest_uuid: "658e7299505c46d797b4ad2be4129399"
 from {username='slave'} at 127.0.0.1:35090
I20260709 13:01:05.054176   641 raft_consensus.cc:493] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:05.054548   641 raft_consensus.cc:3060] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:05.061010   641 raft_consensus.cc:515] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:05.063092   641 leader_election.cc:290] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 4 election: Requested vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:01:05.064112   563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "da373aac3d614f56b081496dd9145b01"
I20260709 13:01:05.064733   563 raft_consensus.cc:3060] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:05.064570   711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:01:05.066007   711 raft_consensus.cc:3055] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:05.066355   711 raft_consensus.cc:740] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a92d6df25f4a43e0872ecbb6451058d5, State: Running, Role: LEADER
I20260709 13:01:05.068228   807 raft_consensus.cc:993] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01: : Instructing follower 658e7299505c46d797b4ad2be4129399 to start an election
I20260709 13:01:05.069046   891 raft_consensus.cc:1081] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Signalling peer 658e7299505c46d797b4ad2be4129399 to start an election
I20260709 13:01:05.069087   711 consensus_queue.cc:260] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:05.070838   711 raft_consensus.cc:3060] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:05.071528   640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d500279905444ef8b7d48b790d14d240"
dest_uuid: "658e7299505c46d797b4ad2be4129399"
 from {username='slave'} at 127.0.0.1:35086
I20260709 13:01:05.072276   640 raft_consensus.cc:493] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:05.072717   640 raft_consensus.cc:3060] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.074311   563 raft_consensus.cc:2468] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658e7299505c46d797b4ad2be4129399 in term 4.
I20260709 13:01:05.076678   605 leader_election.cc:304] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, da373aac3d614f56b081496dd9145b01; no voters: 
I20260709 13:01:05.077945   840 raft_consensus.cc:2804] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:05.078845   840 raft_consensus.cc:697] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 4 LEADER]: Becoming Leader. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Running, Role: LEADER
I20260709 13:01:05.080242   711 raft_consensus.cc:2468] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658e7299505c46d797b4ad2be4129399 in term 4.
I20260709 13:01:05.080114   840 consensus_queue.cc:237] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:05.080606   640 raft_consensus.cc:515] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:05.085263   640 leader_election.cc:290] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 2 election: Requested vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:01:05.086402   711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500279905444ef8b7d48b790d14d240" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:01:05.087527   711 raft_consensus.cc:3060] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.088604   563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500279905444ef8b7d48b790d14d240" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da373aac3d614f56b081496dd9145b01"
I20260709 13:01:05.089406   563 raft_consensus.cc:3055] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:05.089996   563 raft_consensus.cc:740] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da373aac3d614f56b081496dd9145b01, State: Running, Role: LEADER
I20260709 13:01:05.090487   460 catalog_manager.cc:5697] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 reported cstate change: term changed from 3 to 4, leader changed from a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195) to 658e7299505c46d797b4ad2be4129399 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "658e7299505c46d797b4ad2be4129399" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:05.091276   563 consensus_queue.cc:260] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:05.092944   563 raft_consensus.cc:3060] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.102415   711 raft_consensus.cc:2468] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658e7299505c46d797b4ad2be4129399 in term 2.
I20260709 13:01:05.102603   563 raft_consensus.cc:2468] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658e7299505c46d797b4ad2be4129399 in term 2.
I20260709 13:01:05.105310   606 leader_election.cc:304] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, a92d6df25f4a43e0872ecbb6451058d5; no voters: 
I20260709 13:01:05.106356   840 raft_consensus.cc:2804] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:05.107028   840 raft_consensus.cc:697] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Becoming Leader. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Running, Role: LEADER
I20260709 13:01:05.107954   840 consensus_queue.cc:237] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:05.173084   460 catalog_manager.cc:5697] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 reported cstate change: term changed from 1 to 2, leader changed from da373aac3d614f56b081496dd9145b01 (127.0.107.193) to 658e7299505c46d797b4ad2be4129399 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "658e7299505c46d797b4ad2be4129399" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:05.306185   807 raft_consensus.cc:993] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01: : Instructing follower 658e7299505c46d797b4ad2be4129399 to start an election
I20260709 13:01:05.306867   807 raft_consensus.cc:1081] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Signalling peer 658e7299505c46d797b4ad2be4129399 to start an election
I20260709 13:01:05.309816   640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e4f664fa01d44aab86d4911e7fd45c7"
dest_uuid: "658e7299505c46d797b4ad2be4129399"
 from {username='slave'} at 127.0.0.1:35086
I20260709 13:01:05.311048   640 raft_consensus.cc:493] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:05.311611   640 raft_consensus.cc:3060] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.318320   891 raft_consensus.cc:993] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01: : Instructing follower 658e7299505c46d797b4ad2be4129399 to start an election
I20260709 13:01:05.319007   891 raft_consensus.cc:1081] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Signalling peer 658e7299505c46d797b4ad2be4129399 to start an election
I20260709 13:01:05.320993   641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "98c04e9dd15b431d91ea29bcb7152983"
dest_uuid: "658e7299505c46d797b4ad2be4129399"
 from {username='slave'} at 127.0.0.1:35086
I20260709 13:01:05.321488   641 raft_consensus.cc:493] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:05.321452   640 raft_consensus.cc:515] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:05.322422   641 raft_consensus.cc:3060] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.325366   640 leader_election.cc:290] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 2 election: Requested vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:01:05.326006   711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4f664fa01d44aab86d4911e7fd45c7" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:01:05.325690   563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4f664fa01d44aab86d4911e7fd45c7" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da373aac3d614f56b081496dd9145b01"
I20260709 13:01:05.326910   711 raft_consensus.cc:3060] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.327368   563 raft_consensus.cc:3055] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:05.327751   563 raft_consensus.cc:740] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da373aac3d614f56b081496dd9145b01, State: Running, Role: LEADER
I20260709 13:01:05.328727   563 consensus_queue.cc:260] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:05.329528   641 raft_consensus.cc:515] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:05.330706   563 raft_consensus.cc:3060] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.332877   641 leader_election.cc:290] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 2 election: Requested vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:01:05.333185   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c04e9dd15b431d91ea29bcb7152983" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:01:05.334076   715 raft_consensus.cc:3060] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.333185   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c04e9dd15b431d91ea29bcb7152983" candidate_uuid: "658e7299505c46d797b4ad2be4129399" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da373aac3d614f56b081496dd9145b01"
I20260709 13:01:05.334635   565 raft_consensus.cc:3055] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:05.334985   565 raft_consensus.cc:740] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da373aac3d614f56b081496dd9145b01, State: Running, Role: LEADER
I20260709 13:01:05.335749   711 raft_consensus.cc:2468] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658e7299505c46d797b4ad2be4129399 in term 2.
I20260709 13:01:05.336174   565 consensus_queue.cc:260] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:05.336989   606 leader_election.cc:304] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, a92d6df25f4a43e0872ecbb6451058d5; no voters: 
I20260709 13:01:05.337997   840 raft_consensus.cc:2804] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:05.337841   565 raft_consensus.cc:3060] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:05.338519   840 raft_consensus.cc:697] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Becoming Leader. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Running, Role: LEADER
I20260709 13:01:05.339612   840 consensus_queue.cc:237] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:05.348240   715 raft_consensus.cc:2468] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658e7299505c46d797b4ad2be4129399 in term 2.
I20260709 13:01:05.350025   606 leader_election.cc:304] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658e7299505c46d797b4ad2be4129399, a92d6df25f4a43e0872ecbb6451058d5; no voters: 
I20260709 13:01:05.351141   840 raft_consensus.cc:2804] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:05.351686   565 raft_consensus.cc:2468] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658e7299505c46d797b4ad2be4129399 in term 2.
I20260709 13:01:05.351917   840 raft_consensus.cc:697] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Becoming Leader. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Running, Role: LEADER
I20260709 13:01:05.351773   460 catalog_manager.cc:5697] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 reported cstate change: term changed from 1 to 2, leader changed from da373aac3d614f56b081496dd9145b01 (127.0.107.193) to 658e7299505c46d797b4ad2be4129399 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "658e7299505c46d797b4ad2be4129399" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:05.353168   840 consensus_queue.cc:237] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:05.355446   563 raft_consensus.cc:2468] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658e7299505c46d797b4ad2be4129399 in term 2.
I20260709 13:01:05.372184   460 catalog_manager.cc:5697] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399 reported cstate change: term changed from 1 to 2, leader changed from da373aac3d614f56b081496dd9145b01 (127.0.107.193) to 658e7299505c46d797b4ad2be4129399 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "658e7299505c46d797b4ad2be4129399" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:05.468175   715 raft_consensus.cc:1275] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 4 FOLLOWER]: Refusing update from remote peer 658e7299505c46d797b4ad2be4129399: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:05.470120   885 consensus_queue.cc:1048] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:05.491369   563 raft_consensus.cc:1275] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 4 FOLLOWER]: Refusing update from remote peer 658e7299505c46d797b4ad2be4129399: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:05.493105   885 consensus_queue.cc:1048] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:05.550588   563 raft_consensus.cc:1275] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Refusing update from remote peer 658e7299505c46d797b4ad2be4129399: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:05.552635   885 consensus_queue.cc:1048] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:05.566552   715 raft_consensus.cc:1275] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Refusing update from remote peer 658e7299505c46d797b4ad2be4129399: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:05.570509   898 consensus_queue.cc:1048] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:05.589071   711 raft_consensus.cc:1275] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 4 FOLLOWER]: Refusing update from remote peer 658e7299505c46d797b4ad2be4129399: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:05.590956   898 consensus_queue.cc:1048] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:05.611209   563 raft_consensus.cc:1275] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 4 FOLLOWER]: Refusing update from remote peer 658e7299505c46d797b4ad2be4129399: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:05.613838   901 consensus_queue.cc:1048] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:05.821365   711 raft_consensus.cc:1275] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Refusing update from remote peer 658e7299505c46d797b4ad2be4129399: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:05.823069   901 consensus_queue.cc:1048] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:05.833889   563 raft_consensus.cc:1275] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Refusing update from remote peer 658e7299505c46d797b4ad2be4129399: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:05.836408   840 consensus_queue.cc:1048] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:05.847754   563 raft_consensus.cc:1275] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Refusing update from remote peer 658e7299505c46d797b4ad2be4129399: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:05.850157   901 consensus_queue.cc:1048] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:05.859622   711 raft_consensus.cc:1275] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Refusing update from remote peer 658e7299505c46d797b4ad2be4129399: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:05.861370   898 consensus_queue.cc:1048] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:07.987527   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:07.996258   641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78af706a10f146fe8c6a5c64d70c6894"
dest_uuid: "658e7299505c46d797b4ad2be4129399"
mode: GRACEFUL
new_leader_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:01:07.997342   641 raft_consensus.cc:606] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Received request to transfer leadership to TS a92d6df25f4a43e0872ecbb6451058d5
I20260709 13:01:08.000967   641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8af84729cf07482cb0244953a91a3a68"
dest_uuid: "658e7299505c46d797b4ad2be4129399"
mode: GRACEFUL
new_leader_uuid: "da373aac3d614f56b081496dd9145b01"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:01:08.001547   641 raft_consensus.cc:606] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Received request to transfer leadership to TS da373aac3d614f56b081496dd9145b01
I20260709 13:01:08.004436   641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc"
dest_uuid: "658e7299505c46d797b4ad2be4129399"
mode: GRACEFUL
new_leader_uuid: "da373aac3d614f56b081496dd9145b01"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:01:08.004983   641 raft_consensus.cc:606] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 4 LEADER]: Received request to transfer leadership to TS da373aac3d614f56b081496dd9145b01
I20260709 13:01:08.008419   641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d500279905444ef8b7d48b790d14d240"
dest_uuid: "658e7299505c46d797b4ad2be4129399"
mode: GRACEFUL
new_leader_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:01:08.009518   641 raft_consensus.cc:606] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Received request to transfer leadership to TS a92d6df25f4a43e0872ecbb6451058d5
I20260709 13:01:08.012844   641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a"
dest_uuid: "658e7299505c46d797b4ad2be4129399"
mode: GRACEFUL
new_leader_uuid: "da373aac3d614f56b081496dd9145b01"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:01:08.013465   641 raft_consensus.cc:606] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 4 LEADER]: Received request to transfer leadership to TS da373aac3d614f56b081496dd9145b01
I20260709 13:01:08.014989   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:01:08.015671   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:08.074178   898 raft_consensus.cc:993] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399: : Instructing follower a92d6df25f4a43e0872ecbb6451058d5 to start an election
I20260709 13:01:08.074600   898 raft_consensus.cc:1081] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Signalling peer a92d6df25f4a43e0872ecbb6451058d5 to start an election
I20260709 13:01:08.077535   711 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "78af706a10f146fe8c6a5c64d70c6894"
dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
 from {username='slave'} at 127.0.0.1:41722
I20260709 13:01:08.078217   711 raft_consensus.cc:493] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:08.078574   711 raft_consensus.cc:3060] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.084344   711 raft_consensus.cc:515] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:08.086606   711 leader_election.cc:290] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 3 election: Requested vote from peers da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951), 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721)
I20260709 13:01:08.088641   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78af706a10f146fe8c6a5c64d70c6894" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "658e7299505c46d797b4ad2be4129399"
I20260709 13:01:08.089212   641 raft_consensus.cc:3055] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:08.089184   563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78af706a10f146fe8c6a5c64d70c6894" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "da373aac3d614f56b081496dd9145b01"
I20260709 13:01:08.089855   641 raft_consensus.cc:740] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Running, Role: LEADER
I20260709 13:01:08.090157   563 raft_consensus.cc:3060] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.090713   641 consensus_queue.cc:260] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:08.091745   641 raft_consensus.cc:3060] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.096129   563 raft_consensus.cc:2468] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a92d6df25f4a43e0872ecbb6451058d5 in term 3.
I20260709 13:01:08.097926   641 raft_consensus.cc:2468] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a92d6df25f4a43e0872ecbb6451058d5 in term 3.
I20260709 13:01:08.098174   679 leader_election.cc:304] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a92d6df25f4a43e0872ecbb6451058d5, da373aac3d614f56b081496dd9145b01; no voters: 
I20260709 13:01:08.099447   869 raft_consensus.cc:2804] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:08.100550   869 raft_consensus.cc:697] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [term 3 LEADER]: Becoming Leader. State: Replica: a92d6df25f4a43e0872ecbb6451058d5, State: Running, Role: LEADER
I20260709 13:01:08.102281   869 consensus_queue.cc:237] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:08.113425   463 catalog_manager.cc:5697] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 reported cstate change: term changed from 2 to 3, leader changed from 658e7299505c46d797b4ad2be4129399 (127.0.107.194) to a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "a92d6df25f4a43e0872ecbb6451058d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:08.355482   909 raft_consensus.cc:993] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399: : Instructing follower a92d6df25f4a43e0872ecbb6451058d5 to start an election
I20260709 13:01:08.356046   898 raft_consensus.cc:1081] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Signalling peer a92d6df25f4a43e0872ecbb6451058d5 to start an election
I20260709 13:01:08.357985   711 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d500279905444ef8b7d48b790d14d240"
dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
 from {username='slave'} at 127.0.0.1:41722
I20260709 13:01:08.358588   711 raft_consensus.cc:493] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:08.358909   711 raft_consensus.cc:3060] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.363828   711 raft_consensus.cc:515] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:08.365736   711 leader_election.cc:290] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 3 election: Requested vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951)
I20260709 13:01:08.367143   563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500279905444ef8b7d48b790d14d240" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "da373aac3d614f56b081496dd9145b01"
I20260709 13:01:08.367143   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d500279905444ef8b7d48b790d14d240" candidate_uuid: "a92d6df25f4a43e0872ecbb6451058d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "658e7299505c46d797b4ad2be4129399"
I20260709 13:01:08.368124   563 raft_consensus.cc:3060] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.368137   641 raft_consensus.cc:3055] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:08.368738   641 raft_consensus.cc:740] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Running, Role: LEADER
I20260709 13:01:08.369727   641 consensus_queue.cc:260] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:08.371135   641 raft_consensus.cc:3060] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.376538   563 raft_consensus.cc:2468] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a92d6df25f4a43e0872ecbb6451058d5 in term 3.
I20260709 13:01:08.378750   679 leader_election.cc:304] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a92d6df25f4a43e0872ecbb6451058d5, da373aac3d614f56b081496dd9145b01; no voters: 
I20260709 13:01:08.379942   869 raft_consensus.cc:2804] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:08.379931   641 raft_consensus.cc:2468] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a92d6df25f4a43e0872ecbb6451058d5 in term 3.
I20260709 13:01:08.380677   869 raft_consensus.cc:697] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [term 3 LEADER]: Becoming Leader. State: Replica: a92d6df25f4a43e0872ecbb6451058d5, State: Running, Role: LEADER
I20260709 13:01:08.382015   869 consensus_queue.cc:237] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:08.384455   898 raft_consensus.cc:993] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399: : Instructing follower da373aac3d614f56b081496dd9145b01 to start an election
I20260709 13:01:08.385545   909 raft_consensus.cc:1081] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Signalling peer da373aac3d614f56b081496dd9145b01 to start an election
I20260709 13:01:08.388219   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8af84729cf07482cb0244953a91a3a68"
dest_uuid: "da373aac3d614f56b081496dd9145b01"
 from {username='slave'} at 127.0.0.1:47660
I20260709 13:01:08.388856   565 raft_consensus.cc:493] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:08.389308   565 raft_consensus.cc:3060] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.394282   463 catalog_manager.cc:5697] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 reported cstate change: term changed from 2 to 3, leader changed from 658e7299505c46d797b4ad2be4129399 (127.0.107.194) to a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "a92d6df25f4a43e0872ecbb6451058d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:08.397562   565 raft_consensus.cc:515] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:08.399909   565 leader_election.cc:290] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 3 election: Requested vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:01:08.401769   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af84729cf07482cb0244953a91a3a68" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "658e7299505c46d797b4ad2be4129399"
I20260709 13:01:08.401866   711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af84729cf07482cb0244953a91a3a68" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:01:08.402361   641 raft_consensus.cc:3055] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:08.402773   641 raft_consensus.cc:740] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Running, Role: LEADER
I20260709 13:01:08.402920   711 raft_consensus.cc:3060] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.403499   641 consensus_queue.cc:260] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:08.404986   641 raft_consensus.cc:3060] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.410540   711 raft_consensus.cc:2468] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 3.
I20260709 13:01:08.411731   641 raft_consensus.cc:2468] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 3.
I20260709 13:01:08.412362   531 leader_election.cc:304] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a92d6df25f4a43e0872ecbb6451058d5, da373aac3d614f56b081496dd9145b01; no voters: 
I20260709 13:01:08.414631   918 raft_consensus.cc:2804] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:08.416723   918 raft_consensus.cc:697] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 3 LEADER]: Becoming Leader. State: Replica: da373aac3d614f56b081496dd9145b01, State: Running, Role: LEADER
I20260709 13:01:08.417893   918 consensus_queue.cc:237] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:08.427850   463 catalog_manager.cc:5697] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 reported cstate change: term changed from 2 to 3, leader changed from 658e7299505c46d797b4ad2be4129399 (127.0.107.194) to da373aac3d614f56b081496dd9145b01 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "da373aac3d614f56b081496dd9145b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:08.511652   916 raft_consensus.cc:993] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399: : Instructing follower da373aac3d614f56b081496dd9145b01 to start an election
I20260709 13:01:08.512203   916 raft_consensus.cc:1081] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 4 LEADER]: Signalling peer da373aac3d614f56b081496dd9145b01 to start an election
I20260709 13:01:08.513775   916 raft_consensus.cc:993] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399: : Instructing follower da373aac3d614f56b081496dd9145b01 to start an election
I20260709 13:01:08.514338   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc"
dest_uuid: "da373aac3d614f56b081496dd9145b01"
 from {username='slave'} at 127.0.0.1:47660
I20260709 13:01:08.514351   916 raft_consensus.cc:1081] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 4 LEADER]: Signalling peer da373aac3d614f56b081496dd9145b01 to start an election
I20260709 13:01:08.515051   565 raft_consensus.cc:493] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:08.515530   565 raft_consensus.cc:3060] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:08.517967   563 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a"
dest_uuid: "da373aac3d614f56b081496dd9145b01"
 from {username='slave'} at 127.0.0.1:47660
I20260709 13:01:08.518766   563 raft_consensus.cc:493] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:08.519157   563 raft_consensus.cc:3060] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:08.522035   565 raft_consensus.cc:515] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:08.523975   565 leader_election.cc:290] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 5 election: Requested vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:01:08.525177   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:01:08.525534   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b6b05c36a64999b35a8273917bdcbc" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "658e7299505c46d797b4ad2be4129399"
I20260709 13:01:08.525837   715 raft_consensus.cc:3060] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:08.526046   641 raft_consensus.cc:3055] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:08.526388   641 raft_consensus.cc:740] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Running, Role: LEADER
I20260709 13:01:08.526365   563 raft_consensus.cc:515] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:08.527287   641 consensus_queue.cc:260] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:08.528746   563 leader_election.cc:290] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 5 election: Requested vote from peers 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721), a92d6df25f4a43e0872ecbb6451058d5 (127.0.107.195:41111)
I20260709 13:01:08.529680   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "658e7299505c46d797b4ad2be4129399"
I20260709 13:01:08.529927   711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ef1006746945cf9c09bed1220d384a" candidate_uuid: "da373aac3d614f56b081496dd9145b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "a92d6df25f4a43e0872ecbb6451058d5"
I20260709 13:01:08.530285   640 raft_consensus.cc:3055] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:08.530388   711 raft_consensus.cc:3060] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:08.530678   640 raft_consensus.cc:740] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 658e7299505c46d797b4ad2be4129399, State: Running, Role: LEADER
I20260709 13:01:08.531489   640 consensus_queue.cc:260] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:08.532694   640 raft_consensus.cc:3060] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:08.528537   641 raft_consensus.cc:3060] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:08.534083   715 raft_consensus.cc:2468] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 5.
I20260709 13:01:08.535620   531 leader_election.cc:304] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a92d6df25f4a43e0872ecbb6451058d5, da373aac3d614f56b081496dd9145b01; no voters: 
I20260709 13:01:08.536870   918 raft_consensus.cc:2804] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:08.537999   918 raft_consensus.cc:697] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 5 LEADER]: Becoming Leader. State: Replica: da373aac3d614f56b081496dd9145b01, State: Running, Role: LEADER
I20260709 13:01:08.537533   711 raft_consensus.cc:2468] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 5.
I20260709 13:01:08.539223   918 consensus_queue.cc:237] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:08.539695   640 raft_consensus.cc:2468] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 5.
I20260709 13:01:08.539880   531 leader_election.cc:304] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a92d6df25f4a43e0872ecbb6451058d5, da373aac3d614f56b081496dd9145b01; no voters: 
I20260709 13:01:08.540753   919 raft_consensus.cc:2804] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:08.542284   641 raft_consensus.cc:2468] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da373aac3d614f56b081496dd9145b01 in term 5.
I20260709 13:01:08.542465   919 raft_consensus.cc:697] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 5 LEADER]: Becoming Leader. State: Replica: da373aac3d614f56b081496dd9145b01, State: Running, Role: LEADER
I20260709 13:01:08.543540   919 consensus_queue.cc:237] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } }
I20260709 13:01:08.556197   463 catalog_manager.cc:5697] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 reported cstate change: term changed from 4 to 5, leader changed from 658e7299505c46d797b4ad2be4129399 (127.0.107.194) to da373aac3d614f56b081496dd9145b01 (127.0.107.193). New cstate: current_term: 5 leader_uuid: "da373aac3d614f56b081496dd9145b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:08.573582   463 catalog_manager.cc:5697] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 reported cstate change: term changed from 4 to 5, leader changed from 658e7299505c46d797b4ad2be4129399 (127.0.107.194) to da373aac3d614f56b081496dd9145b01 (127.0.107.193). New cstate: current_term: 5 leader_uuid: "da373aac3d614f56b081496dd9145b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:08.595088   563 raft_consensus.cc:1275] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 3 FOLLOWER]: Refusing update from remote peer a92d6df25f4a43e0872ecbb6451058d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:08.596973   869 consensus_queue.cc:1048] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:08.612510   641 raft_consensus.cc:1275] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 3 FOLLOWER]: Refusing update from remote peer a92d6df25f4a43e0872ecbb6451058d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:08.614308   869 consensus_queue.cc:1048] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:08.860208   563 raft_consensus.cc:1275] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 3 FOLLOWER]: Refusing update from remote peer a92d6df25f4a43e0872ecbb6451058d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:08.862218   869 consensus_queue.cc:1048] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da373aac3d614f56b081496dd9145b01" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:08.872117   641 raft_consensus.cc:1275] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [term 3 FOLLOWER]: Refusing update from remote peer a92d6df25f4a43e0872ecbb6451058d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:08.873718   910 consensus_queue.cc:1048] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:08.951097   711 raft_consensus.cc:1275] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5 [term 3 FOLLOWER]: Refusing update from remote peer da373aac3d614f56b081496dd9145b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:08.952567   918 consensus_queue.cc:1048] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:08.957921   641 raft_consensus.cc:1275] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 5 FOLLOWER]: Refusing update from remote peer da373aac3d614f56b081496dd9145b01: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:01:08.959767   918 consensus_queue.cc:1048] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:08.964514   640 raft_consensus.cc:1275] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 3 FOLLOWER]: Refusing update from remote peer da373aac3d614f56b081496dd9145b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:08.966827   921 consensus_queue.cc:1048] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:08.978305   711 raft_consensus.cc:1275] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 5 FOLLOWER]: Refusing update from remote peer da373aac3d614f56b081496dd9145b01: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:01:08.982888   921 consensus_queue.cc:1048] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:08.992947   640 raft_consensus.cc:1275] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 5 FOLLOWER]: Refusing update from remote peer da373aac3d614f56b081496dd9145b01: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:01:08.996031   933 consensus_queue.cc:1048] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "658e7299505c46d797b4ad2be4129399" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:09.012058   711 raft_consensus.cc:1275] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 5 FOLLOWER]: Refusing update from remote peer da373aac3d614f56b081496dd9145b01: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:01:09.014261   919 consensus_queue.cc:1048] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92d6df25f4a43e0872ecbb6451058d5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:09.018196   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:09.028767   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:01:09.083815   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:09.084722   431 tablet_replica.cc:333] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01: stopping tablet replica
I20260709 13:01:09.085541   431 raft_consensus.cc:2243] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:01:09.086565   431 raft_consensus.cc:2272] T b6b6b05c36a64999b35a8273917bdcbc P da373aac3d614f56b081496dd9145b01 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.091234   431 tablet_replica.cc:333] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01: stopping tablet replica
I20260709 13:01:09.091928   431 raft_consensus.cc:2243] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:09.092730   431 raft_consensus.cc:2272] T 8af84729cf07482cb0244953a91a3a68 P da373aac3d614f56b081496dd9145b01 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.095548   431 tablet_replica.cc:333] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01: stopping tablet replica
I20260709 13:01:09.096136   431 raft_consensus.cc:2243] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:09.096642   431 raft_consensus.cc:2272] T 98c04e9dd15b431d91ea29bcb7152983 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.098320   431 tablet_replica.cc:333] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01: stopping tablet replica
I20260709 13:01:09.098835   431 raft_consensus.cc:2243] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:09.099254   431 raft_consensus.cc:2272] T ba58747e7ae04d4f81c4ffc2a323f570 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.100971   431 tablet_replica.cc:333] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01: stopping tablet replica
I20260709 13:01:09.101465   431 raft_consensus.cc:2243] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:09.101888   431 raft_consensus.cc:2272] T 8e4f664fa01d44aab86d4911e7fd45c7 P da373aac3d614f56b081496dd9145b01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.104048   431 tablet_replica.cc:333] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01: stopping tablet replica
I20260709 13:01:09.104759   431 raft_consensus.cc:2243] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:09.105285   431 raft_consensus.cc:2272] T b236fa7371f141608adb61630a8c90ec P da373aac3d614f56b081496dd9145b01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.107161   431 tablet_replica.cc:333] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01: stopping tablet replica
I20260709 13:01:09.107813   431 raft_consensus.cc:2243] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:01:09.108582   431 raft_consensus.cc:2272] T d6ef1006746945cf9c09bed1220d384a P da373aac3d614f56b081496dd9145b01 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.111083   431 tablet_replica.cc:333] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01: stopping tablet replica
I20260709 13:01:09.111665   431 raft_consensus.cc:2243] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:09.112175   431 raft_consensus.cc:2272] T d500279905444ef8b7d48b790d14d240 P da373aac3d614f56b081496dd9145b01 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.115141   431 tablet_replica.cc:333] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01: stopping tablet replica
I20260709 13:01:09.115715   431 raft_consensus.cc:2243] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:09.116205   431 raft_consensus.cc:2272] T 78af706a10f146fe8c6a5c64d70c6894 P da373aac3d614f56b081496dd9145b01 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:01:09.138268   605 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33951: connect: Connection refused (error 111)
W20260709 13:01:09.282230   605 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33951, user_credentials={real_user=slave}} blocked reactor thread for 144291us
W20260709 13:01:09.285458   605 consensus_peers.cc:597] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 -> Peer da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951): Couldn't send request to peer da373aac3d614f56b081496dd9145b01. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33951: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:09.288194   679 consensus_peers.cc:597] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 -> Peer da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951): Couldn't send request to peer da373aac3d614f56b081496dd9145b01. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33951: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:09.288923   605 consensus_peers.cc:597] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 -> Peer da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951): Couldn't send request to peer da373aac3d614f56b081496dd9145b01. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33951: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:09.295287   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 13:01:09.314662   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:01:09.334095   679 consensus_peers.cc:597] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 -> Peer da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951): Couldn't send request to peer da373aac3d614f56b081496dd9145b01. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33951: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:09.347745   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:09.348869   431 tablet_replica.cc:333] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399: stopping tablet replica
I20260709 13:01:09.349536   431 raft_consensus.cc:2243] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:01:09.350346   431 raft_consensus.cc:2272] T 98c04e9dd15b431d91ea29bcb7152983 P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.352497   431 tablet_replica.cc:333] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399: stopping tablet replica
I20260709 13:01:09.353235   431 raft_consensus.cc:2243] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:09.353932   431 raft_consensus.cc:2272] T 8af84729cf07482cb0244953a91a3a68 P 658e7299505c46d797b4ad2be4129399 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.356875   431 tablet_replica.cc:333] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399: stopping tablet replica
I20260709 13:01:09.357476   431 raft_consensus.cc:2243] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:09.358031   431 raft_consensus.cc:2272] T d500279905444ef8b7d48b790d14d240 P 658e7299505c46d797b4ad2be4129399 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.361907   431 tablet_replica.cc:333] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399: stopping tablet replica
I20260709 13:01:09.362596   431 raft_consensus.cc:2243] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:09.363191   431 raft_consensus.cc:2272] T 78af706a10f146fe8c6a5c64d70c6894 P 658e7299505c46d797b4ad2be4129399 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.366091   431 tablet_replica.cc:333] T b236fa7371f141608adb61630a8c90ec P 658e7299505c46d797b4ad2be4129399: stopping tablet replica
I20260709 13:01:09.366710   431 raft_consensus.cc:2243] T b236fa7371f141608adb61630a8c90ec P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:09.367195   431 raft_consensus.cc:2272] T b236fa7371f141608adb61630a8c90ec P 658e7299505c46d797b4ad2be4129399 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.369050   431 tablet_replica.cc:333] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399: stopping tablet replica
I20260709 13:01:09.369603   431 raft_consensus.cc:2243] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:09.370143   431 raft_consensus.cc:2272] T d6ef1006746945cf9c09bed1220d384a P 658e7299505c46d797b4ad2be4129399 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.373066   431 tablet_replica.cc:333] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399: stopping tablet replica
I20260709 13:01:09.374055   431 raft_consensus.cc:2243] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:09.374835   431 raft_consensus.cc:2272] T b6b6b05c36a64999b35a8273917bdcbc P 658e7299505c46d797b4ad2be4129399 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.378115   431 tablet_replica.cc:333] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399: stopping tablet replica
I20260709 13:01:09.379165   431 raft_consensus.cc:2243] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:01:09.380071   431 raft_consensus.cc:2272] T ba58747e7ae04d4f81c4ffc2a323f570 P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.382088   431 tablet_replica.cc:333] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399: stopping tablet replica
I20260709 13:01:09.382601   431 raft_consensus.cc:2243] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:01:09.383432   431 raft_consensus.cc:2272] T 8e4f664fa01d44aab86d4911e7fd45c7 P 658e7299505c46d797b4ad2be4129399 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.415956   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 13:01:09.441268   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:01:09.481000   678 consensus_peers.cc:597] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 -> Peer 658e7299505c46d797b4ad2be4129399 (127.0.107.194:44721): Couldn't send request to peer 658e7299505c46d797b4ad2be4129399. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44721: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:09.494720   679 consensus_peers.cc:597] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 -> Peer da373aac3d614f56b081496dd9145b01 (127.0.107.193:33951): Couldn't send request to peer da373aac3d614f56b081496dd9145b01. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33951: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:09.500823   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:09.501829   431 tablet_replica.cc:333] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5: stopping tablet replica
I20260709 13:01:09.502429   431 raft_consensus.cc:2243] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:01:09.503217   431 raft_consensus.cc:2272] T b236fa7371f141608adb61630a8c90ec P a92d6df25f4a43e0872ecbb6451058d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.505117   431 tablet_replica.cc:333] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5: stopping tablet replica
I20260709 13:01:09.505702   431 raft_consensus.cc:2243] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:09.506114   431 raft_consensus.cc:2272] T d6ef1006746945cf9c09bed1220d384a P a92d6df25f4a43e0872ecbb6451058d5 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.508985   431 tablet_replica.cc:333] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5: stopping tablet replica
I20260709 13:01:09.509579   431 raft_consensus.cc:2243] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:09.510146   431 raft_consensus.cc:2272] T b6b6b05c36a64999b35a8273917bdcbc P a92d6df25f4a43e0872ecbb6451058d5 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.512194   431 tablet_replica.cc:333] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5: stopping tablet replica
I20260709 13:01:09.512702   431 raft_consensus.cc:2243] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:09.513429   431 raft_consensus.cc:2272] T d500279905444ef8b7d48b790d14d240 P a92d6df25f4a43e0872ecbb6451058d5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.515717   431 tablet_replica.cc:333] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5: stopping tablet replica
I20260709 13:01:09.516240   431 raft_consensus.cc:2243] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:09.516983   431 raft_consensus.cc:2272] T 78af706a10f146fe8c6a5c64d70c6894 P a92d6df25f4a43e0872ecbb6451058d5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.519342   431 tablet_replica.cc:333] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5: stopping tablet replica
I20260709 13:01:09.519965   431 raft_consensus.cc:2243] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:09.520396   431 raft_consensus.cc:2272] T ba58747e7ae04d4f81c4ffc2a323f570 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.522181   431 tablet_replica.cc:333] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5: stopping tablet replica
I20260709 13:01:09.522698   431 raft_consensus.cc:2243] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:09.523090   431 raft_consensus.cc:2272] T 8e4f664fa01d44aab86d4911e7fd45c7 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.524531   431 tablet_replica.cc:333] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5: stopping tablet replica
I20260709 13:01:09.524992   431 raft_consensus.cc:2243] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:09.525394   431 raft_consensus.cc:2272] T 8af84729cf07482cb0244953a91a3a68 P a92d6df25f4a43e0872ecbb6451058d5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.527520   431 tablet_replica.cc:333] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5: stopping tablet replica
I20260709 13:01:09.527976   431 raft_consensus.cc:2243] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:09.528359   431 raft_consensus.cc:2272] T 98c04e9dd15b431d91ea29bcb7152983 P a92d6df25f4a43e0872ecbb6451058d5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.562269   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:01:09.590916   431 master.cc:562] Master@127.0.107.254:40595 shutting down...
I20260709 13:01:09.726871   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:01:09.727434   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:09.727772   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P a665c24edc1b4674a44b154f854abed7: stopping tablet replica
I20260709 13:01:09.749130   431 master.cc:584] Master@127.0.107.254:40595 shutdown complete.
658e7299505c46d797b4ad2be4129399  4
a92d6df25f4a43e0872ecbb6451058d5  1
da373aac3d614f56b081496dd9145b01  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (14729 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:01:09.797425   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:36129
I20260709 13:01:09.799381   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:09.807811   945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:09.808316   948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:09.807804   946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:09.809512   431 server_base.cc:1061] running on GCE node
I20260709 13:01:09.810716   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:09.810977   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:09.811148   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602069811130 us; error 0 us; skew 500 ppm
I20260709 13:01:09.811712   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:09.815582   431 webserver.cc:533] Webserver started at http://127.0.107.254:38155/ using document root <none> and password file <none>
I20260709 13:01:09.816160   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:09.816380   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:09.816663   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:09.817920   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602054858684-431-0/minicluster-data/master-0-root/instance:
uuid: "6a62ac7243d846fdaf49f6af6831b8e2"
format_stamp: "Formatted at 2026-07-09 13:01:09 on dist-test-slave-mxd3"
I20260709 13:01:09.822439   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:01:09.826359   953 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:09.827415   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:01:09.827744   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602054858684-431-0/minicluster-data/master-0-root
uuid: "6a62ac7243d846fdaf49f6af6831b8e2"
format_stamp: "Formatted at 2026-07-09 13:01:09 on dist-test-slave-mxd3"
I20260709 13:01:09.828043   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602054858684-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602054858684-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602054858684-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:09.882663   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:09.884402   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:09.977962   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:36129
I20260709 13:01:09.978221  1004 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:36129 every 8 connection(s)
I20260709 13:01:09.984385  1005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:09.995756  1005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2: Bootstrap starting.
I20260709 13:01:10.001637  1005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:10.009224  1005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2: No bootstrap required, opened a new log
I20260709 13:01:10.012063  1005 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a62ac7243d846fdaf49f6af6831b8e2" member_type: VOTER }
I20260709 13:01:10.012499  1005 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:10.012682  1005 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a62ac7243d846fdaf49f6af6831b8e2, State: Initialized, Role: FOLLOWER
I20260709 13:01:10.013243  1005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a62ac7243d846fdaf49f6af6831b8e2" member_type: VOTER }
I20260709 13:01:10.013834  1005 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:01:10.014070  1005 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:01:10.014293  1005 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:10.030157  1005 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a62ac7243d846fdaf49f6af6831b8e2" member_type: VOTER }
I20260709 13:01:10.031016  1005 leader_election.cc:304] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a62ac7243d846fdaf49f6af6831b8e2; no voters: 
I20260709 13:01:10.032832  1005 leader_election.cc:290] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:01:10.033363  1008 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:10.035142  1008 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2 [term 1 LEADER]: Becoming Leader. State: Replica: 6a62ac7243d846fdaf49f6af6831b8e2, State: Running, Role: LEADER
I20260709 13:01:10.035930  1008 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a62ac7243d846fdaf49f6af6831b8e2" member_type: VOTER }
I20260709 13:01:10.036214  1005 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:01:10.041895  1010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a62ac7243d846fdaf49f6af6831b8e2. Latest consensus state: current_term: 1 leader_uuid: "6a62ac7243d846fdaf49f6af6831b8e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a62ac7243d846fdaf49f6af6831b8e2" member_type: VOTER } }
I20260709 13:01:10.042677  1010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2 [sys.catalog]: This master's current role is: LEADER
I20260709 13:01:10.043360  1009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a62ac7243d846fdaf49f6af6831b8e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a62ac7243d846fdaf49f6af6831b8e2" member_type: VOTER } }
I20260709 13:01:10.044422  1009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2 [sys.catalog]: This master's current role is: LEADER
I20260709 13:01:10.048642  1017 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:01:10.057603  1017 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:01:10.058831   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:01:10.068879  1017 catalog_manager.cc:1383] Generated new cluster ID: 1bb8963ea4f1412f9f77c9bc1fae6bd8
I20260709 13:01:10.069308  1017 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:01:10.088937  1017 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:01:10.090873  1017 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:01:10.112589  1017 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2: Generated new TSK 0
I20260709 13:01:10.113269  1017 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:01:10.125854   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:10.133365  1026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:10.138703  1027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:10.142396  1029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:10.143743   431 server_base.cc:1061] running on GCE node
I20260709 13:01:10.145186   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:10.145556   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:10.145794   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602070145775 us; error 0 us; skew 500 ppm
I20260709 13:01:10.146481   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:10.149806   431 webserver.cc:533] Webserver started at http://127.0.107.193:36563/ using document root <none> and password file <none>
I20260709 13:01:10.150437   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:10.150697   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:10.151016   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:10.152465   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602054858684-431-0/minicluster-data/ts-0-root/instance:
uuid: "591e55d6b5724c309aeb00994e320bd0"
format_stamp: "Formatted at 2026-07-09 13:01:10 on dist-test-slave-mxd3"
I20260709 13:01:10.158998   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.003s	sys 0.005s
I20260709 13:01:10.164335  1034 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:10.165457   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:01:10.166193   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602054858684-431-0/minicluster-data/ts-0-root
uuid: "591e55d6b5724c309aeb00994e320bd0"
format_stamp: "Formatted at 2026-07-09 13:01:10 on dist-test-slave-mxd3"
I20260709 13:01:10.166913   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602054858684-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602054858684-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602054858684-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:10.206802   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:10.208369   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:10.210983   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:10.214694   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:10.215608   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20260709 13:01:10.216154   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:10.216442   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:10.289587   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:36343
I20260709 13:01:10.290014  1096 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:36343 every 8 connection(s)
I20260709 13:01:10.300347   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:10.312158  1101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:10.314357  1102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:10.317013   431 server_base.cc:1061] running on GCE node
W20260709 13:01:10.319350  1105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:10.321067   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:10.321365   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:10.321506   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602070321494 us; error 0 us; skew 500 ppm
I20260709 13:01:10.322934   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:10.327394  1097 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36129
I20260709 13:01:10.327863  1097 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:10.328696  1097 heartbeater.cc:507] Master 127.0.107.254:36129 requested a full tablet report, sending...
I20260709 13:01:10.329793   431 webserver.cc:533] Webserver started at http://127.0.107.194:33475/ using document root <none> and password file <none>
I20260709 13:01:10.330761   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:10.331089   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:10.331485   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:10.331444   970 ts_manager.cc:194] Registered new tserver with Master: 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343)
I20260709 13:01:10.333532   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602054858684-431-0/minicluster-data/ts-1-root/instance:
uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
format_stamp: "Formatted at 2026-07-09 13:01:10 on dist-test-slave-mxd3"
I20260709 13:01:10.334524   970 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45032
I20260709 13:01:10.341348   431 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:01:10.349191  1109 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:10.350427   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:01:10.350862   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602054858684-431-0/minicluster-data/ts-1-root
uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
format_stamp: "Formatted at 2026-07-09 13:01:10 on dist-test-slave-mxd3"
I20260709 13:01:10.351362   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602054858684-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602054858684-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602054858684-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:10.421381   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:10.423226   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:10.425082   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:10.428885   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:10.429189   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:10.429489   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:10.429723   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.002s	sys 0.000s
I20260709 13:01:10.505932   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:44299
I20260709 13:01:10.505949  1171 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:44299 every 8 connection(s)
I20260709 13:01:10.512497   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:10.523598  1175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:10.525025  1176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:10.531329  1172 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36129
I20260709 13:01:10.532428  1172 heartbeater.cc:461] Registering TS with master...
W20260709 13:01:10.532998  1178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:10.533038   431 server_base.cc:1061] running on GCE node
I20260709 13:01:10.533715  1172 heartbeater.cc:507] Master 127.0.107.254:36129 requested a full tablet report, sending...
I20260709 13:01:10.534391   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:10.534664   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:10.534890   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602070534869 us; error 0 us; skew 500 ppm
I20260709 13:01:10.535635   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:10.536813   970 ts_manager.cc:194] Registered new tserver with Master: 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299)
I20260709 13:01:10.539172   431 webserver.cc:533] Webserver started at http://127.0.107.195:46177/ using document root <none> and password file <none>
I20260709 13:01:10.540227   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:10.540469   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:10.539872   970 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45040
I20260709 13:01:10.540799   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:10.542495   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602054858684-431-0/minicluster-data/ts-2-root/instance:
uuid: "406a72268a484d05920a71f05d71691e"
format_stamp: "Formatted at 2026-07-09 13:01:10 on dist-test-slave-mxd3"
I20260709 13:01:10.548310   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:01:10.552467  1183 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:10.553558   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:01:10.553983   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602054858684-431-0/minicluster-data/ts-2-root
uuid: "406a72268a484d05920a71f05d71691e"
format_stamp: "Formatted at 2026-07-09 13:01:10 on dist-test-slave-mxd3"
I20260709 13:01:10.554255   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602054858684-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602054858684-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602054858684-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:10.577126   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:10.578670   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:10.580804   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:10.583792   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:10.584101   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:10.584403   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:10.584591   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:10.647037   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:36731
I20260709 13:01:10.647194  1245 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:36731 every 8 connection(s)
I20260709 13:01:10.665964  1246 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36129
I20260709 13:01:10.666913  1246 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:10.668529  1246 heartbeater.cc:507] Master 127.0.107.254:36129 requested a full tablet report, sending...
I20260709 13:01:10.672194   970 ts_manager.cc:194] Registered new tserver with Master: 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:10.672328   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.021413237s
I20260709 13:01:10.674551   970 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45042
I20260709 13:01:10.715308   970 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45048:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:01:10.719080   970 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:01:10.835544  1137 tablet_service.cc:1511] Processing CreateTablet for tablet 7d4f631913d84fa2b0ec6feece991332 (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:01:10.837178  1137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d4f631913d84fa2b0ec6feece991332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.838610  1133 tablet_service.cc:1511] Processing CreateTablet for tablet 8d86aeab58674fd2b92beda6f5fc0d7a (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:01:10.840194  1133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d86aeab58674fd2b92beda6f5fc0d7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.841912  1211 tablet_service.cc:1511] Processing CreateTablet for tablet 7d4f631913d84fa2b0ec6feece991332 (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:01:10.842484  1210 tablet_service.cc:1511] Processing CreateTablet for tablet 1df16b6ed89d4d1eb05931a646d3db26 (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:01:10.843695  1211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d4f631913d84fa2b0ec6feece991332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.843096  1134 tablet_service.cc:1511] Processing CreateTablet for tablet 98633dc5342842c294d5ff93c4301c9d (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:01:10.843870  1209 tablet_service.cc:1511] Processing CreateTablet for tablet 1c850d5de21d4c02b851d9925ef9c881 (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:01:10.844508  1210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1df16b6ed89d4d1eb05931a646d3db26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.845216  1134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98633dc5342842c294d5ff93c4301c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.845508  1209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c850d5de21d4c02b851d9925ef9c881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.846920  1135 tablet_service.cc:1511] Processing CreateTablet for tablet 1c850d5de21d4c02b851d9925ef9c881 (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:01:10.847098  1208 tablet_service.cc:1511] Processing CreateTablet for tablet 98633dc5342842c294d5ff93c4301c9d (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:01:10.849040  1208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98633dc5342842c294d5ff93c4301c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.849426  1136 tablet_service.cc:1511] Processing CreateTablet for tablet 1df16b6ed89d4d1eb05931a646d3db26 (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:01:10.851480  1207 tablet_service.cc:1511] Processing CreateTablet for tablet 8d86aeab58674fd2b92beda6f5fc0d7a (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:01:10.854022  1135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c850d5de21d4c02b851d9925ef9c881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.856740  1206 tablet_service.cc:1511] Processing CreateTablet for tablet e0f4d34e991c4787b1c336afee69d01e (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:01:10.858695  1206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0f4d34e991c4787b1c336afee69d01e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.864908  1207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d86aeab58674fd2b92beda6f5fc0d7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.870146  1136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1df16b6ed89d4d1eb05931a646d3db26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.882310  1132 tablet_service.cc:1511] Processing CreateTablet for tablet e0f4d34e991c4787b1c336afee69d01e (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:01:10.884258  1132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0f4d34e991c4787b1c336afee69d01e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.898726  1062 tablet_service.cc:1511] Processing CreateTablet for tablet 7d4f631913d84fa2b0ec6feece991332 (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:01:10.901288  1062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d4f631913d84fa2b0ec6feece991332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.907091  1211 tablet_service.cc:1511] Processing CreateTablet for tablet ec42386aeae44255bf83eae6530c4f97 (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:01:10.908960  1211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec42386aeae44255bf83eae6530c4f97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.915119  1131 tablet_service.cc:1511] Processing CreateTablet for tablet ec42386aeae44255bf83eae6530c4f97 (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:01:10.916731  1131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec42386aeae44255bf83eae6530c4f97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.918470  1061 tablet_service.cc:1511] Processing CreateTablet for tablet 1df16b6ed89d4d1eb05931a646d3db26 (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:01:10.919931  1061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1df16b6ed89d4d1eb05931a646d3db26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.936721  1059 tablet_service.cc:1511] Processing CreateTablet for tablet 98633dc5342842c294d5ff93c4301c9d (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:01:10.937582  1056 tablet_service.cc:1511] Processing CreateTablet for tablet ec42386aeae44255bf83eae6530c4f97 (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:01:10.937798  1057 tablet_service.cc:1511] Processing CreateTablet for tablet e0f4d34e991c4787b1c336afee69d01e (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:01:10.935734  1264 tablet_bootstrap.cc:492] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Bootstrap starting.
I20260709 13:01:10.945318  1265 tablet_bootstrap.cc:492] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e: Bootstrap starting.
I20260709 13:01:10.939121  1059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98633dc5342842c294d5ff93c4301c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.939870  1058 tablet_service.cc:1511] Processing CreateTablet for tablet 8d86aeab58674fd2b92beda6f5fc0d7a (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:01:10.946456  1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec42386aeae44255bf83eae6530c4f97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.946703  1060 tablet_service.cc:1511] Processing CreateTablet for tablet 1c850d5de21d4c02b851d9925ef9c881 (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:01:10.948537  1060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c850d5de21d4c02b851d9925ef9c881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.949610  1057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0f4d34e991c4787b1c336afee69d01e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.950702  1058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d86aeab58674fd2b92beda6f5fc0d7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.988433  1130 tablet_service.cc:1511] Processing CreateTablet for tablet 74a921ab3514481087dcb4ae5bfd554a (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:01:10.990259  1130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74a921ab3514481087dcb4ae5bfd554a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.032867  1264 tablet_bootstrap.cc:654] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.035364  1055 tablet_service.cc:1511] Processing CreateTablet for tablet 74a921ab3514481087dcb4ae5bfd554a (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:01:11.037237  1055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74a921ab3514481087dcb4ae5bfd554a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.041383  1209 tablet_service.cc:1511] Processing CreateTablet for tablet 74a921ab3514481087dcb4ae5bfd554a (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:01:11.043042  1209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74a921ab3514481087dcb4ae5bfd554a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:11.051574  1018 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:11.051047  1054 tablet_service.cc:1511] Processing CreateTablet for tablet 0d8166a5fa5a4e298eaf4b3c7cc1c098 (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:01:11.054415  1054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d8166a5fa5a4e298eaf4b3c7cc1c098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.057791  1265 tablet_bootstrap.cc:654] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.070195  1135 tablet_service.cc:1511] Processing CreateTablet for tablet 0d8166a5fa5a4e298eaf4b3c7cc1c098 (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:01:11.072054  1135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d8166a5fa5a4e298eaf4b3c7cc1c098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.077853  1207 tablet_service.cc:1511] Processing CreateTablet for tablet 0d8166a5fa5a4e298eaf4b3c7cc1c098 (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:01:11.079751  1207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d8166a5fa5a4e298eaf4b3c7cc1c098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.096485  1053 tablet_service.cc:1511] Processing CreateTablet for tablet a3c20fcfe7b04c0fa8ca55c75fb57879 (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:01:11.100740  1136 tablet_service.cc:1511] Processing CreateTablet for tablet a3c20fcfe7b04c0fa8ca55c75fb57879 (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:01:11.102568  1136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3c20fcfe7b04c0fa8ca55c75fb57879. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.104739  1053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3c20fcfe7b04c0fa8ca55c75fb57879. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.118692  1210 tablet_service.cc:1511] Processing CreateTablet for tablet a3c20fcfe7b04c0fa8ca55c75fb57879 (DEFAULT_TABLE table=test-workload [id=6e8d402618dc4331b70094f193701108]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:01:11.120115  1210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3c20fcfe7b04c0fa8ca55c75fb57879. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:11.125123  1265 tablet_bootstrap.cc:492] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e: No bootstrap required, opened a new log
I20260709 13:01:11.125766  1265 ts_tablet_manager.cc:1403] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e: Time spent bootstrapping tablet: real 0.181s	user 0.020s	sys 0.012s
I20260709 13:01:11.128775  1265 raft_consensus.cc:359] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.141912  1265 raft_consensus.cc:385] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.142307  1265 raft_consensus.cc:740] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 406a72268a484d05920a71f05d71691e, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.143282  1265 consensus_queue.cc:260] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.173305  1264 tablet_bootstrap.cc:492] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa: No bootstrap required, opened a new log
I20260709 13:01:11.174211  1264 ts_tablet_manager.cc:1403] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Time spent bootstrapping tablet: real 0.239s	user 0.026s	sys 0.028s
I20260709 13:01:11.183828  1264 raft_consensus.cc:359] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } }
I20260709 13:01:11.188242  1264 raft_consensus.cc:385] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.188313  1268 tablet_bootstrap.cc:492] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0: Bootstrap starting.
I20260709 13:01:11.188781  1264 raft_consensus.cc:740] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.193173  1264 consensus_queue.cc:260] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } }
I20260709 13:01:11.197777  1268 tablet_bootstrap.cc:654] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.206063  1246 heartbeater.cc:499] Master 127.0.107.254:36129 was elected leader, sending a full tablet report...
I20260709 13:01:11.208755  1270 raft_consensus.cc:493] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.209985  1270 raft_consensus.cc:515] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.212321  1265 ts_tablet_manager.cc:1434] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e: Time spent starting tablet: real 0.086s	user 0.014s	sys 0.003s
I20260709 13:01:11.214123  1265 tablet_bootstrap.cc:492] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e: Bootstrap starting.
I20260709 13:01:11.223476  1264 ts_tablet_manager.cc:1434] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Time spent starting tablet: real 0.049s	user 0.012s	sys 0.012s
I20260709 13:01:11.224779  1172 heartbeater.cc:499] Master 127.0.107.254:36129 was elected leader, sending a full tablet report...
I20260709 13:01:11.230455  1270 leader_election.cc:290] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299)
I20260709 13:01:11.230921  1268 tablet_bootstrap.cc:492] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0: No bootstrap required, opened a new log
I20260709 13:01:11.231475  1268 ts_tablet_manager.cc:1403] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0: Time spent bootstrapping tablet: real 0.043s	user 0.024s	sys 0.004s
I20260709 13:01:11.232955  1265 tablet_bootstrap.cc:654] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.235108  1264 tablet_bootstrap.cc:492] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Bootstrap starting.
I20260709 13:01:11.234807  1268 raft_consensus.cc:359] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.237039  1268 raft_consensus.cc:385] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.238000  1268 raft_consensus.cc:740] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.239141  1268 consensus_queue.cc:260] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.244539  1265 tablet_bootstrap.cc:492] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e: No bootstrap required, opened a new log
I20260709 13:01:11.245441  1265 ts_tablet_manager.cc:1403] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.000s
I20260709 13:01:11.253461  1268 ts_tablet_manager.cc:1434] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0: Time spent starting tablet: real 0.022s	user 0.012s	sys 0.005s
I20260709 13:01:11.254500  1097 heartbeater.cc:499] Master 127.0.107.254:36129 was elected leader, sending a full tablet report...
I20260709 13:01:11.254931  1268 tablet_bootstrap.cc:492] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0: Bootstrap starting.
I20260709 13:01:11.254402  1265 raft_consensus.cc:359] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } }
I20260709 13:01:11.258435  1265 raft_consensus.cc:385] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.263291  1265 raft_consensus.cc:740] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 406a72268a484d05920a71f05d71691e, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.264937  1275 raft_consensus.cc:493] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.265519  1275 raft_consensus.cc:515] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.272993  1264 tablet_bootstrap.cc:654] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.273608  1265 consensus_queue.cc:260] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } }
I20260709 13:01:11.275885  1268 tablet_bootstrap.cc:654] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.281946  1072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98633dc5342842c294d5ff93c4301c9d" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0" is_pre_election: true
I20260709 13:01:11.290043  1265 ts_tablet_manager.cc:1434] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.003s
I20260709 13:01:11.292121  1265 tablet_bootstrap.cc:492] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e: Bootstrap starting.
I20260709 13:01:11.294245  1275 leader_election.cc:290] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:11.312532  1147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98633dc5342842c294d5ff93c4301c9d" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" is_pre_election: true
W20260709 13:01:11.318181  1186 leader_election.cc:343] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.318938  1265 tablet_bootstrap.cc:654] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:11.330094  1186 leader_election.cc:343] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.330677  1186 leader_election.cc:304] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 406a72268a484d05920a71f05d71691e; no voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 591e55d6b5724c309aeb00994e320bd0
I20260709 13:01:11.333949  1147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec42386aeae44255bf83eae6530c4f97" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" is_pre_election: true
I20260709 13:01:11.334520  1270 raft_consensus.cc:2749] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:01:11.337802  1037 leader_election.cc:343] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.346226  1272 raft_consensus.cc:493] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.346966  1272 raft_consensus.cc:515] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } }
I20260709 13:01:11.349946  1264 tablet_bootstrap.cc:492] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa: No bootstrap required, opened a new log
I20260709 13:01:11.350862  1264 ts_tablet_manager.cc:1403] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Time spent bootstrapping tablet: real 0.116s	user 0.014s	sys 0.015s
I20260709 13:01:11.357797  1268 tablet_bootstrap.cc:492] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0: No bootstrap required, opened a new log
I20260709 13:01:11.362421  1268 ts_tablet_manager.cc:1403] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0: Time spent bootstrapping tablet: real 0.108s	user 0.034s	sys 0.007s
I20260709 13:01:11.360894  1272 leader_election.cc:290] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 406a72268a484d05920a71f05d71691e (127.0.107.195:36731), 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343)
I20260709 13:01:11.378261  1265 tablet_bootstrap.cc:492] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e: No bootstrap required, opened a new log
I20260709 13:01:11.378964  1265 ts_tablet_manager.cc:1403] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e: Time spent bootstrapping tablet: real 0.087s	user 0.014s	sys 0.005s
I20260709 13:01:11.383468  1264 raft_consensus.cc:359] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.388002  1264 raft_consensus.cc:385] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.388679  1264 raft_consensus.cc:740] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.387849  1268 raft_consensus.cc:359] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.389304  1268 raft_consensus.cc:385] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.389910  1268 raft_consensus.cc:740] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.393033  1221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec42386aeae44255bf83eae6530c4f97" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "406a72268a484d05920a71f05d71691e" is_pre_election: true
I20260709 13:01:11.394654  1265 raft_consensus.cc:359] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
W20260709 13:01:11.395116  1036 leader_election.cc:343] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 406a72268a484d05920a71f05d71691e (127.0.107.195:36731): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.395601  1265 raft_consensus.cc:385] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.395720  1036 leader_election.cc:304] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 591e55d6b5724c309aeb00994e320bd0; no voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 406a72268a484d05920a71f05d71691e
I20260709 13:01:11.396055  1265 raft_consensus.cc:740] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 406a72268a484d05920a71f05d71691e, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.396943  1265 consensus_queue.cc:260] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.390312  1264 consensus_queue.cc:260] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.397749  1275 raft_consensus.cc:2749] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:11.400362  1265 ts_tablet_manager.cc:1434] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:01:11.402283  1268 consensus_queue.cc:260] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.405288  1268 ts_tablet_manager.cc:1434] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.000s
I20260709 13:01:11.406378  1268 tablet_bootstrap.cc:492] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0: Bootstrap starting.
I20260709 13:01:11.407363  1264 ts_tablet_manager.cc:1434] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Time spent starting tablet: real 0.056s	user 0.014s	sys 0.002s
I20260709 13:01:11.408727  1264 tablet_bootstrap.cc:492] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Bootstrap starting.
I20260709 13:01:11.413802  1072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4f631913d84fa2b0ec6feece991332" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0" is_pre_election: true
I20260709 13:01:11.414553  1265 tablet_bootstrap.cc:492] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e: Bootstrap starting.
W20260709 13:01:11.416062  1112 leader_election.cc:343] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.418738  1268 tablet_bootstrap.cc:654] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.419044  1264 tablet_bootstrap.cc:654] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.423013  1265 tablet_bootstrap.cc:654] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.432885  1264 tablet_bootstrap.cc:492] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa: No bootstrap required, opened a new log
I20260709 13:01:11.433477  1264 ts_tablet_manager.cc:1403] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.011s
I20260709 13:01:11.439697  1264 raft_consensus.cc:359] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.441080  1264 raft_consensus.cc:385] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.441437  1264 raft_consensus.cc:740] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.442917  1264 consensus_queue.cc:260] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.447312  1264 ts_tablet_manager.cc:1434] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Time spent starting tablet: real 0.013s	user 0.012s	sys 0.000s
I20260709 13:01:11.448669  1264 tablet_bootstrap.cc:492] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa: Bootstrap starting.
I20260709 13:01:11.461861  1270 raft_consensus.cc:493] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.466993  1270 raft_consensus.cc:515] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.468727  1264 tablet_bootstrap.cc:654] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.475327  1072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0f4d34e991c4787b1c336afee69d01e" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0" is_pre_election: true
I20260709 13:01:11.476589  1072 raft_consensus.cc:2468] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 0.
I20260709 13:01:11.482270  1186 leader_election.cc:304] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 406a72268a484d05920a71f05d71691e, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:11.486730  1269 raft_consensus.cc:2804] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:11.487838  1269 raft_consensus.cc:493] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.488818  1269 raft_consensus.cc:3060] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.492200  1268 tablet_bootstrap.cc:492] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0: No bootstrap required, opened a new log
I20260709 13:01:11.493155  1268 ts_tablet_manager.cc:1403] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0: Time spent bootstrapping tablet: real 0.087s	user 0.007s	sys 0.014s
I20260709 13:01:11.493181  1264 tablet_bootstrap.cc:492] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa: No bootstrap required, opened a new log
I20260709 13:01:11.494172  1264 ts_tablet_manager.cc:1403] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa: Time spent bootstrapping tablet: real 0.046s	user 0.015s	sys 0.015s
I20260709 13:01:11.494318  1221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4f631913d84fa2b0ec6feece991332" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "406a72268a484d05920a71f05d71691e" is_pre_election: true
I20260709 13:01:11.495241  1221 raft_consensus.cc:2468] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 0.
I20260709 13:01:11.498039  1111 leader_election.cc:304] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 406a72268a484d05920a71f05d71691e; no voters: 591e55d6b5724c309aeb00994e320bd0
I20260709 13:01:11.499843  1272 raft_consensus.cc:2804] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:11.500468  1272 raft_consensus.cc:493] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.501199  1272 raft_consensus.cc:3060] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.504052  1270 leader_election.cc:290] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299)
I20260709 13:01:11.504443  1264 raft_consensus.cc:359] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.505383  1264 raft_consensus.cc:385] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.506167  1264 raft_consensus.cc:740] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.506547  1147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0f4d34e991c4787b1c336afee69d01e" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" is_pre_election: true
I20260709 13:01:11.507267  1268 raft_consensus.cc:359] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.508477  1268 raft_consensus.cc:385] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.507793  1264 consensus_queue.cc:260] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
W20260709 13:01:11.508770  1186 leader_election.cc:343] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.508965  1268 raft_consensus.cc:740] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.513370  1265 tablet_bootstrap.cc:492] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e: No bootstrap required, opened a new log
I20260709 13:01:11.513976  1265 ts_tablet_manager.cc:1403] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e: Time spent bootstrapping tablet: real 0.100s	user 0.036s	sys 0.061s
I20260709 13:01:11.517167  1265 raft_consensus.cc:359] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.518105  1265 raft_consensus.cc:385] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.518512  1265 raft_consensus.cc:740] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 406a72268a484d05920a71f05d71691e, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.518780  1269 raft_consensus.cc:515] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.519357  1265 consensus_queue.cc:260] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.511070  1268 consensus_queue.cc:260] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.521627  1265 ts_tablet_manager.cc:1434] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.001s
I20260709 13:01:11.524871  1265 tablet_bootstrap.cc:492] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e: Bootstrap starting.
I20260709 13:01:11.523839  1268 ts_tablet_manager.cc:1434] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.002s
I20260709 13:01:11.526916  1268 tablet_bootstrap.cc:492] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0: Bootstrap starting.
I20260709 13:01:11.530134  1272 raft_consensus.cc:515] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } }
I20260709 13:01:11.533506  1268 tablet_bootstrap.cc:654] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.535370  1221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4f631913d84fa2b0ec6feece991332" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:11.536852  1221 raft_consensus.cc:3060] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.542171  1269 leader_election.cc:290] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 election: Requested vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299)
I20260709 13:01:11.543123  1265 tablet_bootstrap.cc:654] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.546552  1072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0f4d34e991c4787b1c336afee69d01e" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
I20260709 13:01:11.547271  1072 raft_consensus.cc:3060] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.554435  1147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0f4d34e991c4787b1c336afee69d01e" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
I20260709 13:01:11.556023  1264 ts_tablet_manager.cc:1434] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa: Time spent starting tablet: real 0.061s	user 0.021s	sys 0.029s
W20260709 13:01:11.564028  1186 leader_election.cc:343] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.576046  1264 tablet_bootstrap.cc:492] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Bootstrap starting.
I20260709 13:01:11.594316  1264 tablet_bootstrap.cc:654] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.598956  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4f631913d84fa2b0ec6feece991332" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
I20260709 13:01:11.604108  1291 raft_consensus.cc:493] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.606051  1291 raft_consensus.cc:515] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.608429  1294 raft_consensus.cc:493] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.609495  1291 leader_election.cc:290] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
W20260709 13:01:11.611250  1112 leader_election.cc:343] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.615074  1070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c850d5de21d4c02b851d9925ef9c881" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0" is_pre_election: true
I20260709 13:01:11.614539  1294 raft_consensus.cc:515] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.622972  1220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c850d5de21d4c02b851d9925ef9c881" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "406a72268a484d05920a71f05d71691e" is_pre_election: true
W20260709 13:01:11.623674  1112 leader_election.cc:343] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.623878  1220 raft_consensus.cc:2468] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 0.
I20260709 13:01:11.624920  1070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16b6ed89d4d1eb05931a646d3db26" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0" is_pre_election: true
I20260709 13:01:11.625516  1294 leader_election.cc:290] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:11.625680  1111 leader_election.cc:304] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 406a72268a484d05920a71f05d71691e; no voters: 591e55d6b5724c309aeb00994e320bd0
I20260709 13:01:11.627071  1264 tablet_bootstrap.cc:492] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa: No bootstrap required, opened a new log
I20260709 13:01:11.627223  1070 raft_consensus.cc:2468] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 0.
I20260709 13:01:11.627684  1291 raft_consensus.cc:2804] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:11.628121  1291 raft_consensus.cc:493] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.628484  1291 raft_consensus.cc:3060] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.628661  1112 leader_election.cc:304] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:11.629621  1220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16b6ed89d4d1eb05931a646d3db26" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "406a72268a484d05920a71f05d71691e" is_pre_election: true
W20260709 13:01:11.633762  1111 leader_election.cc:343] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 406a72268a484d05920a71f05d71691e (127.0.107.195:36731): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.635143  1294 raft_consensus.cc:2804] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:11.636751  1294 raft_consensus.cc:493] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.637259  1294 raft_consensus.cc:3060] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.645597  1272 leader_election.cc:290] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 election: Requested vote from peers 406a72268a484d05920a71f05d71691e (127.0.107.195:36731), 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343)
I20260709 13:01:11.646647  1264 ts_tablet_manager.cc:1403] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Time spent bootstrapping tablet: real 0.071s	user 0.028s	sys 0.004s
I20260709 13:01:11.653882  1264 raft_consensus.cc:359] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.654870  1264 raft_consensus.cc:385] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.655239  1264 raft_consensus.cc:740] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.657902  1072 raft_consensus.cc:2468] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 1.
I20260709 13:01:11.660166  1221 raft_consensus.cc:2468] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 1.
I20260709 13:01:11.666446  1186 leader_election.cc:304] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 406a72268a484d05920a71f05d71691e, 591e55d6b5724c309aeb00994e320bd0; no voters: 24f31d4f10ad41eab5902ec7bb8ba5aa
I20260709 13:01:11.668562  1269 raft_consensus.cc:2804] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:11.668365  1111 leader_election.cc:304] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 406a72268a484d05920a71f05d71691e; no voters: 591e55d6b5724c309aeb00994e320bd0
I20260709 13:01:11.660634  1291 raft_consensus.cc:515] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.656585  1264 consensus_queue.cc:260] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.670635  1269 raft_consensus.cc:697] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 1 LEADER]: Becoming Leader. State: Replica: 406a72268a484d05920a71f05d71691e, State: Running, Role: LEADER
I20260709 13:01:11.672597  1294 raft_consensus.cc:515] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.674130  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c850d5de21d4c02b851d9925ef9c881" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
W20260709 13:01:11.676648  1112 leader_election.cc:343] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.677454  1291 leader_election.cc:290] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 election: Requested vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:11.677343  1269 consensus_queue.cc:237] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.680271  1265 tablet_bootstrap.cc:492] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e: No bootstrap required, opened a new log
I20260709 13:01:11.686655  1265 ts_tablet_manager.cc:1403] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e: Time spent bootstrapping tablet: real 0.162s	user 0.041s	sys 0.054s
I20260709 13:01:11.679355  1220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16b6ed89d4d1eb05931a646d3db26" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:11.690342  1265 raft_consensus.cc:359] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
W20260709 13:01:11.691018  1111 leader_election.cc:343] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 406a72268a484d05920a71f05d71691e (127.0.107.195:36731): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.680179  1221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c850d5de21d4c02b851d9925ef9c881" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:11.691045  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16b6ed89d4d1eb05931a646d3db26" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
I20260709 13:01:11.691845  1221 raft_consensus.cc:3060] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.692301  1071 raft_consensus.cc:3060] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.670656  1272 raft_consensus.cc:2804] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:11.691188  1265 raft_consensus.cc:385] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.694448  1265 raft_consensus.cc:740] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 406a72268a484d05920a71f05d71691e, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.694897  1272 raft_consensus.cc:697] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Becoming Leader. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Running, Role: LEADER
I20260709 13:01:11.695420  1265 consensus_queue.cc:260] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.697587  1272 consensus_queue.cc:237] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } }
I20260709 13:01:11.698518  1265 ts_tablet_manager.cc:1434] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.005s
I20260709 13:01:11.699955  1221 raft_consensus.cc:2468] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 1.
I20260709 13:01:11.678337  1264 ts_tablet_manager.cc:1434] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Time spent starting tablet: real 0.030s	user 0.010s	sys 0.002s
I20260709 13:01:11.708630  1265 tablet_bootstrap.cc:492] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e: Bootstrap starting.
I20260709 13:01:11.709542  1264 tablet_bootstrap.cc:492] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa: Bootstrap starting.
I20260709 13:01:11.677349  1294 leader_election.cc:290] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 election: Requested vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:11.706198  1268 tablet_bootstrap.cc:492] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0: No bootstrap required, opened a new log
I20260709 13:01:11.714176  1268 ts_tablet_manager.cc:1403] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0: Time spent bootstrapping tablet: real 0.187s	user 0.035s	sys 0.082s
I20260709 13:01:11.717151  1265 tablet_bootstrap.cc:654] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.717837  1268 raft_consensus.cc:359] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } }
I20260709 13:01:11.718621  1268 raft_consensus.cc:385] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.718993  1268 raft_consensus.cc:740] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.720007  1268 consensus_queue.cc:260] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } }
I20260709 13:01:11.725931  1268 ts_tablet_manager.cc:1434] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.005s
I20260709 13:01:11.727010  1268 tablet_bootstrap.cc:492] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0: Bootstrap starting.
I20260709 13:01:11.728016  1071 raft_consensus.cc:2468] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 1.
I20260709 13:01:11.729862  1112 leader_election.cc:304] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 591e55d6b5724c309aeb00994e320bd0; no voters: 406a72268a484d05920a71f05d71691e
I20260709 13:01:11.729403   970 catalog_manager.cc:5697] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194). New cstate: current_term: 1 leader_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:11.730880  1272 raft_consensus.cc:2804] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:11.731472  1272 raft_consensus.cc:697] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Becoming Leader. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Running, Role: LEADER
I20260709 13:01:11.732419  1272 consensus_queue.cc:237] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.733565  1268 tablet_bootstrap.cc:654] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.741367  1264 tablet_bootstrap.cc:654] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.743368  1111 leader_election.cc:304] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 406a72268a484d05920a71f05d71691e; no voters: 591e55d6b5724c309aeb00994e320bd0
I20260709 13:01:11.744704  1294 raft_consensus.cc:2804] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:11.745194  1294 raft_consensus.cc:697] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Becoming Leader. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Running, Role: LEADER
I20260709 13:01:11.746286  1294 consensus_queue.cc:237] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.751020  1265 tablet_bootstrap.cc:492] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e: No bootstrap required, opened a new log
I20260709 13:01:11.751814  1265 ts_tablet_manager.cc:1403] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e: Time spent bootstrapping tablet: real 0.043s	user 0.017s	sys 0.000s
I20260709 13:01:11.757577  1265 raft_consensus.cc:359] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.758733  1265 raft_consensus.cc:385] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.759088  1265 raft_consensus.cc:740] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 406a72268a484d05920a71f05d71691e, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.762527  1268 tablet_bootstrap.cc:492] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0: No bootstrap required, opened a new log
I20260709 13:01:11.763334  1268 ts_tablet_manager.cc:1403] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0: Time spent bootstrapping tablet: real 0.037s	user 0.009s	sys 0.009s
I20260709 13:01:11.760775  1265 consensus_queue.cc:260] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.764727  1264 tablet_bootstrap.cc:492] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa: No bootstrap required, opened a new log
I20260709 13:01:11.767446  1268 raft_consensus.cc:359] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.768746  1268 raft_consensus.cc:385] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.769143  1268 raft_consensus.cc:740] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.770407  1268 consensus_queue.cc:260] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.768769  1264 ts_tablet_manager.cc:1403] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa: Time spent bootstrapping tablet: real 0.059s	user 0.020s	sys 0.009s
I20260709 13:01:11.773572  1268 ts_tablet_manager.cc:1434] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0: Time spent starting tablet: real 0.010s	user 0.000s	sys 0.008s
I20260709 13:01:11.775007  1268 tablet_bootstrap.cc:492] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0: Bootstrap starting.
I20260709 13:01:11.776358  1264 raft_consensus.cc:359] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.777169  1264 raft_consensus.cc:385] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.777493  1264 raft_consensus.cc:740] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.779141  1265 ts_tablet_manager.cc:1434] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e: Time spent starting tablet: real 0.027s	user 0.012s	sys 0.004s
I20260709 13:01:11.780658  1265 tablet_bootstrap.cc:492] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e: Bootstrap starting.
I20260709 13:01:11.783300  1268 tablet_bootstrap.cc:654] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.791829  1265 tablet_bootstrap.cc:654] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.794777  1264 consensus_queue.cc:260] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.799001  1264 ts_tablet_manager.cc:1434] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 13:01:11.794675   966 catalog_manager.cc:5697] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e reported cstate change: term changed from 0 to 1, leader changed from <none> to 406a72268a484d05920a71f05d71691e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "406a72268a484d05920a71f05d71691e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:11.801962   970 catalog_manager.cc:5697] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194). New cstate: current_term: 1 leader_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:11.800693  1264 tablet_bootstrap.cc:492] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa: Bootstrap starting.
I20260709 13:01:11.806010   970 catalog_manager.cc:5697] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194). New cstate: current_term: 1 leader_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:11.816473  1269 raft_consensus.cc:493] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.817219  1269 raft_consensus.cc:515] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.819684  1269 leader_election.cc:290] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299)
I20260709 13:01:11.820640  1264 tablet_bootstrap.cc:654] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.822926  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d86aeab58674fd2b92beda6f5fc0d7a" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0" is_pre_election: true
I20260709 13:01:11.823427  1147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d86aeab58674fd2b92beda6f5fc0d7a" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" is_pre_election: true
I20260709 13:01:11.824262  1147 raft_consensus.cc:2468] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 0.
W20260709 13:01:11.825150  1186 leader_election.cc:343] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.826136  1186 leader_election.cc:304] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 406a72268a484d05920a71f05d71691e; no voters: 591e55d6b5724c309aeb00994e320bd0
I20260709 13:01:11.827308  1269 raft_consensus.cc:2804] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:11.827965  1269 raft_consensus.cc:493] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.828424  1269 raft_consensus.cc:3060] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.835759  1268 tablet_bootstrap.cc:492] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0: No bootstrap required, opened a new log
I20260709 13:01:11.836251  1268 ts_tablet_manager.cc:1403] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0: Time spent bootstrapping tablet: real 0.062s	user 0.012s	sys 0.046s
I20260709 13:01:11.838899  1268 raft_consensus.cc:359] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.839684  1268 raft_consensus.cc:385] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.839972  1268 raft_consensus.cc:740] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.840804  1268 consensus_queue.cc:260] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.845805  1275 raft_consensus.cc:493] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.845521  1269 raft_consensus.cc:515] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.846274  1275 raft_consensus.cc:515] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.850261  1275 leader_election.cc:290] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:11.851024  1269 leader_election.cc:290] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 election: Requested vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299)
I20260709 13:01:11.851135  1147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8166a5fa5a4e298eaf4b3c7cc1c098" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" is_pre_election: true
I20260709 13:01:11.852031  1268 ts_tablet_manager.cc:1434] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:01:11.846875  1264 tablet_bootstrap.cc:492] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa: No bootstrap required, opened a new log
W20260709 13:01:11.853029  1037 leader_election.cc:343] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.853466  1264 ts_tablet_manager.cc:1403] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa: Time spent bootstrapping tablet: real 0.053s	user 0.014s	sys 0.016s
I20260709 13:01:11.853953  1221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8166a5fa5a4e298eaf4b3c7cc1c098" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "406a72268a484d05920a71f05d71691e" is_pre_election: true
I20260709 13:01:11.854048  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d86aeab58674fd2b92beda6f5fc0d7a" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
I20260709 13:01:11.854408  1294 raft_consensus.cc:493] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.854825  1071 raft_consensus.cc:3060] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.855036  1294 raft_consensus.cc:515] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.856465  1264 raft_consensus.cc:359] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.857244  1264 raft_consensus.cc:385] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.857788  1264 raft_consensus.cc:740] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Initialized, Role: FOLLOWER
W20260709 13:01:11.858292  1036 leader_election.cc:343] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 406a72268a484d05920a71f05d71691e (127.0.107.195:36731): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.858839  1036 leader_election.cc:304] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 591e55d6b5724c309aeb00994e320bd0; no voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 406a72268a484d05920a71f05d71691e
I20260709 13:01:11.858603  1264 consensus_queue.cc:260] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.847909  1265 tablet_bootstrap.cc:492] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e: No bootstrap required, opened a new log
I20260709 13:01:11.860906  1268 tablet_bootstrap.cc:492] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0: Bootstrap starting.
I20260709 13:01:11.861521  1147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d86aeab58674fd2b92beda6f5fc0d7a" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
I20260709 13:01:11.870184  1265 ts_tablet_manager.cc:1403] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e: Time spent bootstrapping tablet: real 0.090s	user 0.015s	sys 0.020s
I20260709 13:01:11.870275  1147 raft_consensus.cc:3060] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.860615  1275 raft_consensus.cc:2749] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:11.863318  1294 leader_election.cc:290] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:11.863708  1072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98633dc5342842c294d5ff93c4301c9d" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0" is_pre_election: true
I20260709 13:01:11.873771  1265 raft_consensus.cc:359] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.874924  1265 raft_consensus.cc:385] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.875320  1265 raft_consensus.cc:740] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 406a72268a484d05920a71f05d71691e, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.866871  1221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98633dc5342842c294d5ff93c4301c9d" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "406a72268a484d05920a71f05d71691e" is_pre_election: true
W20260709 13:01:11.877019  1112 leader_election.cc:343] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.877184  1221 raft_consensus.cc:2468] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 0.
I20260709 13:01:11.878286  1071 raft_consensus.cc:2468] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 1.
I20260709 13:01:11.879366  1265 consensus_queue.cc:260] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.879946  1111 leader_election.cc:304] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 406a72268a484d05920a71f05d71691e; no voters: 591e55d6b5724c309aeb00994e320bd0
I20260709 13:01:11.881946  1294 raft_consensus.cc:2804] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:11.882107  1265 ts_tablet_manager.cc:1434] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.005s
I20260709 13:01:11.878414  1264 ts_tablet_manager.cc:1434] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa: Time spent starting tablet: real 0.025s	user 0.003s	sys 0.005s
I20260709 13:01:11.879951  1186 leader_election.cc:304] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 406a72268a484d05920a71f05d71691e, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:11.884881  1270 raft_consensus.cc:2804] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:11.882459  1294 raft_consensus.cc:493] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.885910  1294 raft_consensus.cc:3060] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.885413  1265 tablet_bootstrap.cc:492] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e: Bootstrap starting.
I20260709 13:01:11.888159  1147 raft_consensus.cc:2468] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 1.
I20260709 13:01:11.890636  1264 tablet_bootstrap.cc:492] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa: Bootstrap starting.
I20260709 13:01:11.894418  1270 raft_consensus.cc:697] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 1 LEADER]: Becoming Leader. State: Replica: 406a72268a484d05920a71f05d71691e, State: Running, Role: LEADER
I20260709 13:01:11.895749  1270 consensus_queue.cc:237] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.906868  1264 tablet_bootstrap.cc:654] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.906868  1268 tablet_bootstrap.cc:654] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.910481  1265 tablet_bootstrap.cc:654] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.919902  1294 raft_consensus.cc:515] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.921833  1268 tablet_bootstrap.cc:492] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0: No bootstrap required, opened a new log
I20260709 13:01:11.922562  1268 ts_tablet_manager.cc:1403] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0: Time spent bootstrapping tablet: real 0.070s	user 0.018s	sys 0.000s
I20260709 13:01:11.923864  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98633dc5342842c294d5ff93c4301c9d" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
I20260709 13:01:11.924274  1294 leader_election.cc:290] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 election: Requested vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
W20260709 13:01:11.925529  1112 leader_election.cc:343] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:11.926188  1221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98633dc5342842c294d5ff93c4301c9d" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:11.926959  1221 raft_consensus.cc:3060] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:11.927351  1268 raft_consensus.cc:359] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.928171  1268 raft_consensus.cc:385] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.928799  1268 raft_consensus.cc:740] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.930505  1264 tablet_bootstrap.cc:492] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa: No bootstrap required, opened a new log
I20260709 13:01:11.929894  1268 consensus_queue.cc:260] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.931279  1264 ts_tablet_manager.cc:1403] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa: Time spent bootstrapping tablet: real 0.041s	user 0.012s	sys 0.007s
I20260709 13:01:11.933600   966 catalog_manager.cc:5697] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e reported cstate change: term changed from 0 to 1, leader changed from <none> to 406a72268a484d05920a71f05d71691e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "406a72268a484d05920a71f05d71691e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:11.934899  1264 raft_consensus.cc:359] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.935760  1264 raft_consensus.cc:385] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.936056  1264 raft_consensus.cc:740] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.937170  1265 tablet_bootstrap.cc:492] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e: No bootstrap required, opened a new log
I20260709 13:01:11.937310  1264 consensus_queue.cc:260] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.937863  1265 ts_tablet_manager.cc:1403] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e: Time spent bootstrapping tablet: real 0.053s	user 0.010s	sys 0.026s
I20260709 13:01:11.939782  1264 ts_tablet_manager.cc:1434] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:01:11.940958  1264 tablet_bootstrap.cc:492] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Bootstrap starting.
I20260709 13:01:11.944242  1221 raft_consensus.cc:2468] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 1.
I20260709 13:01:11.946547  1111 leader_election.cc:304] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 406a72268a484d05920a71f05d71691e; no voters: 591e55d6b5724c309aeb00994e320bd0
I20260709 13:01:11.948158  1264 tablet_bootstrap.cc:654] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.948091  1294 raft_consensus.cc:2804] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:11.941990  1268 ts_tablet_manager.cc:1434] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:01:11.948882  1294 raft_consensus.cc:697] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Becoming Leader. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Running, Role: LEADER
I20260709 13:01:11.949887  1294 consensus_queue.cc:237] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.956176  1268 tablet_bootstrap.cc:492] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0: Bootstrap starting.
I20260709 13:01:11.962041  1265 raft_consensus.cc:359] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.963261  1265 raft_consensus.cc:385] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.963671  1265 raft_consensus.cc:740] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 406a72268a484d05920a71f05d71691e, State: Initialized, Role: FOLLOWER
I20260709 13:01:11.964937  1265 consensus_queue.cc:260] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.969522  1265 ts_tablet_manager.cc:1434] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.005s
I20260709 13:01:11.971349  1265 tablet_bootstrap.cc:492] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e: Bootstrap starting.
I20260709 13:01:11.980218  1265 tablet_bootstrap.cc:654] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.981207  1268 tablet_bootstrap.cc:654] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:11.986541  1270 raft_consensus.cc:493] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.987308  1270 raft_consensus.cc:515] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.990399  1264 tablet_bootstrap.cc:492] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa: No bootstrap required, opened a new log
I20260709 13:01:11.991045  1264 ts_tablet_manager.cc:1403] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Time spent bootstrapping tablet: real 0.050s	user 0.025s	sys 0.015s
I20260709 13:01:11.991632  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec42386aeae44255bf83eae6530c4f97" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0" is_pre_election: true
I20260709 13:01:11.993230  1071 raft_consensus.cc:2468] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 0.
I20260709 13:01:11.994484  1264 raft_consensus.cc:359] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:11.996836  1270 leader_election.cc:290] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299)
I20260709 13:01:11.997125  1186 leader_election.cc:304] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 406a72268a484d05920a71f05d71691e, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:11.997155  1264 raft_consensus.cc:385] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:11.993544   968 catalog_manager.cc:5697] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194). New cstate: current_term: 1 leader_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:11.999063  1264 raft_consensus.cc:740] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.000109  1310 raft_consensus.cc:2804] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:11.999953  1264 consensus_queue.cc:260] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.001820  1147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec42386aeae44255bf83eae6530c4f97" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" is_pre_election: true
I20260709 13:01:12.002488  1264 ts_tablet_manager.cc:1434] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.003s
I20260709 13:01:12.002528  1147 raft_consensus.cc:2468] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 0.
I20260709 13:01:12.003552  1264 tablet_bootstrap.cc:492] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Bootstrap starting.
I20260709 13:01:12.000528  1310 raft_consensus.cc:493] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.004482  1310 raft_consensus.cc:3060] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.015436  1270 raft_consensus.cc:493] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.017369  1264 tablet_bootstrap.cc:654] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.016693  1270 raft_consensus.cc:515] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.021536  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16b6ed89d4d1eb05931a646d3db26" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0" is_pre_election: true
I20260709 13:01:12.022452  1270 leader_election.cc:290] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299)
I20260709 13:01:12.022765  1071 raft_consensus.cc:2393] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 406a72268a484d05920a71f05d71691e in current term 1: Already voted for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in this term.
I20260709 13:01:12.023461  1147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16b6ed89d4d1eb05931a646d3db26" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" is_pre_election: true
I20260709 13:01:12.025312  1186 leader_election.cc:304] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 406a72268a484d05920a71f05d71691e; no voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 591e55d6b5724c309aeb00994e320bd0
I20260709 13:01:12.026928  1270 raft_consensus.cc:3060] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.039913  1265 tablet_bootstrap.cc:492] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e: No bootstrap required, opened a new log
I20260709 13:01:12.040642  1265 ts_tablet_manager.cc:1403] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e: Time spent bootstrapping tablet: real 0.070s	user 0.013s	sys 0.018s
I20260709 13:01:12.043210  1270 raft_consensus.cc:2749] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:12.043318  1269 raft_consensus.cc:493] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.043807  1265 raft_consensus.cc:359] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.043964  1269 raft_consensus.cc:515] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.044721  1265 raft_consensus.cc:385] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.044942  1265 raft_consensus.cc:740] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 406a72268a484d05920a71f05d71691e, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.045567  1265 consensus_queue.cc:260] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.046708  1269 leader_election.cc:290] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299)
I20260709 13:01:12.047940  1265 ts_tablet_manager.cc:1434] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:01:12.049470  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74a921ab3514481087dcb4ae5bfd554a" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0" is_pre_election: true
I20260709 13:01:12.049845  1147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74a921ab3514481087dcb4ae5bfd554a" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" is_pre_election: true
I20260709 13:01:12.050742  1071 raft_consensus.cc:2468] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 0.
I20260709 13:01:12.052706  1186 leader_election.cc:304] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 406a72268a484d05920a71f05d71691e, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:12.050915  1147 raft_consensus.cc:2468] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 0.
I20260709 13:01:12.061151  1275 raft_consensus.cc:493] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.086823  1275 raft_consensus.cc:515] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } }
I20260709 13:01:12.090001  1275 leader_election.cc:290] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:12.080945  1264 tablet_bootstrap.cc:492] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa: No bootstrap required, opened a new log
I20260709 13:01:12.091244  1264 ts_tablet_manager.cc:1403] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Time spent bootstrapping tablet: real 0.088s	user 0.023s	sys 0.043s
I20260709 13:01:12.072526  1268 tablet_bootstrap.cc:492] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0: No bootstrap required, opened a new log
I20260709 13:01:12.092628  1268 ts_tablet_manager.cc:1403] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0: Time spent bootstrapping tablet: real 0.137s	user 0.041s	sys 0.016s
I20260709 13:01:12.096303  1147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4f631913d84fa2b0ec6feece991332" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" is_pre_election: true
I20260709 13:01:12.096263  1221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4f631913d84fa2b0ec6feece991332" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "406a72268a484d05920a71f05d71691e" is_pre_election: true
I20260709 13:01:12.097167  1264 raft_consensus.cc:359] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.097783  1221 raft_consensus.cc:2393] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 591e55d6b5724c309aeb00994e320bd0 in current term 1: Already voted for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in this term.
I20260709 13:01:12.098084  1264 raft_consensus.cc:385] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.098526  1264 raft_consensus.cc:740] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.097491  1268 raft_consensus.cc:359] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.099450  1268 raft_consensus.cc:385] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.099988  1268 raft_consensus.cc:740] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.101994  1268 consensus_queue.cc:260] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.054847  1265 tablet_bootstrap.cc:492] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 406a72268a484d05920a71f05d71691e: Bootstrap starting.
I20260709 13:01:12.106396  1268 ts_tablet_manager.cc:1434] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.002s
I20260709 13:01:12.106174  1037 leader_election.cc:304] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 591e55d6b5724c309aeb00994e320bd0; no voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 406a72268a484d05920a71f05d71691e
I20260709 13:01:12.095366  1270 raft_consensus.cc:2804] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:12.108225  1270 raft_consensus.cc:493] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.108613  1268 tablet_bootstrap.cc:492] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 591e55d6b5724c309aeb00994e320bd0: Bootstrap starting.
I20260709 13:01:12.109009  1270 raft_consensus.cc:3060] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.052893  1310 raft_consensus.cc:515] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.099836  1264 consensus_queue.cc:260] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.116596  1268 tablet_bootstrap.cc:654] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 591e55d6b5724c309aeb00994e320bd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.117570  1264 ts_tablet_manager.cc:1434] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa: Time spent starting tablet: real 0.026s	user 0.013s	sys 0.001s
I20260709 13:01:12.117810  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec42386aeae44255bf83eae6530c4f97" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
I20260709 13:01:12.118613  1071 raft_consensus.cc:3060] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:12.083989  1018 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:12.123898  1310 leader_election.cc:290] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 election: Requested vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299)
I20260709 13:01:12.123592  1275 raft_consensus.cc:3060] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.118698  1265 tablet_bootstrap.cc:654] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 406a72268a484d05920a71f05d71691e: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.129894  1147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec42386aeae44255bf83eae6530c4f97" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
I20260709 13:01:12.130551  1147 raft_consensus.cc:3060] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.134200  1275 raft_consensus.cc:2749] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:12.134580  1294 consensus_queue.cc:1048] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:12.138034  1265 tablet_bootstrap.cc:492] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 406a72268a484d05920a71f05d71691e: No bootstrap required, opened a new log
I20260709 13:01:12.138067  1147 raft_consensus.cc:2468] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 1.
I20260709 13:01:12.138756  1265 ts_tablet_manager.cc:1403] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 406a72268a484d05920a71f05d71691e: Time spent bootstrapping tablet: real 0.084s	user 0.019s	sys 0.013s
I20260709 13:01:12.139716  1186 leader_election.cc:304] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 406a72268a484d05920a71f05d71691e; no voters: 
I20260709 13:01:12.140964  1269 raft_consensus.cc:2804] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:12.141472  1269 raft_consensus.cc:697] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 1 LEADER]: Becoming Leader. State: Replica: 406a72268a484d05920a71f05d71691e, State: Running, Role: LEADER
I20260709 13:01:12.142334  1269 consensus_queue.cc:237] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.148212  1265 raft_consensus.cc:359] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.149077  1265 raft_consensus.cc:385] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.149374  1265 raft_consensus.cc:740] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 406a72268a484d05920a71f05d71691e, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.150490  1265 consensus_queue.cc:260] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 406a72268a484d05920a71f05d71691e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.153105  1265 ts_tablet_manager.cc:1434] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 406a72268a484d05920a71f05d71691e: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:01:12.142352  1071 raft_consensus.cc:2468] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 1.
I20260709 13:01:12.172346  1071 raft_consensus.cc:3060] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.173592   968 catalog_manager.cc:5697] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e reported cstate change: term changed from 0 to 1, leader changed from <none> to 406a72268a484d05920a71f05d71691e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "406a72268a484d05920a71f05d71691e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:12.183077  1296 consensus_queue.cc:1048] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:12.196251  1270 raft_consensus.cc:515] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.197249  1268 tablet_bootstrap.cc:492] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 591e55d6b5724c309aeb00994e320bd0: No bootstrap required, opened a new log
I20260709 13:01:12.198092  1268 ts_tablet_manager.cc:1403] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 591e55d6b5724c309aeb00994e320bd0: Time spent bootstrapping tablet: real 0.090s	user 0.041s	sys 0.018s
I20260709 13:01:12.199127  1270 leader_election.cc:290] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 election: Requested vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299)
I20260709 13:01:12.200482  1294 consensus_queue.cc:1048] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:12.201217  1268 raft_consensus.cc:359] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.202045  1268 raft_consensus.cc:385] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.202136  1072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74a921ab3514481087dcb4ae5bfd554a" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
I20260709 13:01:12.202364  1268 raft_consensus.cc:740] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.203097  1072 raft_consensus.cc:3060] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.203888  1147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74a921ab3514481087dcb4ae5bfd554a" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
I20260709 13:01:12.204766  1147 raft_consensus.cc:3060] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.213824  1268 consensus_queue.cc:260] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 591e55d6b5724c309aeb00994e320bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.215744  1147 raft_consensus.cc:2468] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 1.
I20260709 13:01:12.216212  1268 ts_tablet_manager.cc:1434] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 591e55d6b5724c309aeb00994e320bd0: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:01:12.217389  1268 tablet_bootstrap.cc:492] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0: Bootstrap starting.
I20260709 13:01:12.217279  1186 leader_election.cc:304] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 406a72268a484d05920a71f05d71691e; no voters: 
I20260709 13:01:12.218832  1270 raft_consensus.cc:2804] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:12.218979  1072 raft_consensus.cc:2468] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 1.
I20260709 13:01:12.219695  1270 raft_consensus.cc:697] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 1 LEADER]: Becoming Leader. State: Replica: 406a72268a484d05920a71f05d71691e, State: Running, Role: LEADER
I20260709 13:01:12.222231  1270 consensus_queue.cc:237] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.225085  1268 tablet_bootstrap.cc:654] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:12.230314  1294 consensus_queue.cc:1048] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:12.236879  1269 consensus_queue.cc:1048] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:12.244565  1272 raft_consensus.cc:493] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.245223  1272 raft_consensus.cc:515] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.248698  1072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c20fcfe7b04c0fa8ca55c75fb57879" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0" is_pre_election: true
I20260709 13:01:12.249346  1072 raft_consensus.cc:2468] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 0.
I20260709 13:01:12.250906  1112 leader_election.cc:304] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:12.250934  1272 leader_election.cc:290] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:12.252259  1272 raft_consensus.cc:2804] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:12.252619  1272 raft_consensus.cc:493] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.252944  1272 raft_consensus.cc:3060] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.253087  1268 tablet_bootstrap.cc:492] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0: No bootstrap required, opened a new log
I20260709 13:01:12.253674  1268 ts_tablet_manager.cc:1403] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.009s
I20260709 13:01:12.254518  1221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c20fcfe7b04c0fa8ca55c75fb57879" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "406a72268a484d05920a71f05d71691e" is_pre_election: true
I20260709 13:01:12.255839  1221 raft_consensus.cc:2468] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 0.
I20260709 13:01:12.256242  1268 raft_consensus.cc:359] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.257000  1268 raft_consensus.cc:385] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:12.257289  1268 raft_consensus.cc:740] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Initialized, Role: FOLLOWER
I20260709 13:01:12.257979  1268 consensus_queue.cc:260] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.261227  1268 ts_tablet_manager.cc:1434] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:01:12.262888  1272 raft_consensus.cc:515] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.267009  1272 leader_election.cc:290] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 election: Requested vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:12.267930  1147 raft_consensus.cc:3060] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.270943  1221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c20fcfe7b04c0fa8ca55c75fb57879" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:12.271858  1221 raft_consensus.cc:3060] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.283005   968 catalog_manager.cc:5697] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e reported cstate change: term changed from 0 to 1, leader changed from <none> to 406a72268a484d05920a71f05d71691e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "406a72268a484d05920a71f05d71691e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:12.285167  1310 consensus_queue.cc:1048] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:12.286365  1069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c20fcfe7b04c0fa8ca55c75fb57879" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
I20260709 13:01:12.287190  1069 raft_consensus.cc:3060] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.293322  1221 raft_consensus.cc:2468] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 1.
I20260709 13:01:12.295118  1111 leader_election.cc:304] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 406a72268a484d05920a71f05d71691e; no voters: 
I20260709 13:01:12.297071  1330 raft_consensus.cc:2804] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:12.297537  1069 raft_consensus.cc:2468] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 1.
I20260709 13:01:12.297739  1330 raft_consensus.cc:697] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Becoming Leader. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Running, Role: LEADER
I20260709 13:01:12.298840  1330 consensus_queue.cc:237] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.306986  1294 consensus_queue.cc:1048] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:12.324962  1330 consensus_queue.cc:1048] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:12.339471   970 catalog_manager.cc:5697] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194). New cstate: current_term: 1 leader_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:12.358455  1269 consensus_queue.cc:1048] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:12.369062  1270 consensus_queue.cc:1048] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:12.470314  1069 raft_consensus.cc:3060] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.477962  1330 raft_consensus.cc:493] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.478502  1330 raft_consensus.cc:515] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.480638  1330 leader_election.cc:290] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:12.481415  1294 consensus_queue.cc:1048] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:12.481712  1069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8166a5fa5a4e298eaf4b3c7cc1c098" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0" is_pre_election: true
I20260709 13:01:12.482000  1221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8166a5fa5a4e298eaf4b3c7cc1c098" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "406a72268a484d05920a71f05d71691e" is_pre_election: true
I20260709 13:01:12.482610  1221 raft_consensus.cc:2468] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 0.
I20260709 13:01:12.482610  1069 raft_consensus.cc:2468] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 0.
I20260709 13:01:12.484460  1111 leader_election.cc:304] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 406a72268a484d05920a71f05d71691e; no voters: 
I20260709 13:01:12.485535  1294 raft_consensus.cc:2804] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:12.486073  1294 raft_consensus.cc:493] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.486655  1294 raft_consensus.cc:3060] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.494433  1294 raft_consensus.cc:515] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.498381  1069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8166a5fa5a4e298eaf4b3c7cc1c098" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
I20260709 13:01:12.499280  1069 raft_consensus.cc:3060] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.500036  1296 consensus_queue.cc:1048] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:12.508191  1294 leader_election.cc:290] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 election: Requested vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:12.508673  1220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8166a5fa5a4e298eaf4b3c7cc1c098" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:12.509541  1220 raft_consensus.cc:3060] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 406a72268a484d05920a71f05d71691e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:12.512713  1069 raft_consensus.cc:2468] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 1.
I20260709 13:01:12.515014  1112 leader_election.cc:304] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:12.516896  1330 raft_consensus.cc:2804] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:12.517773  1330 raft_consensus.cc:697] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Becoming Leader. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Running, Role: LEADER
I20260709 13:01:12.518864  1330 consensus_queue.cc:237] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.521122  1220 raft_consensus.cc:2468] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 1.
I20260709 13:01:12.529475   966 catalog_manager.cc:5697] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194). New cstate: current_term: 1 leader_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:12.582950  1270 consensus_queue.cc:1048] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:12.597823  1327 consensus_queue.cc:1048] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:12.625885  1327 consensus_queue.cc:1048] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:12.636313  1147 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d4f631913d84fa2b0ec6feece991332"
dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
mode: GRACEFUL
new_leader_uuid: "591e55d6b5724c309aeb00994e320bd0"
 from {username='slave'} at 127.0.0.1:44698
I20260709 13:01:12.636956  1147 raft_consensus.cc:606] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Received request to transfer leadership to TS 591e55d6b5724c309aeb00994e320bd0
I20260709 13:01:12.646565  1147 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1df16b6ed89d4d1eb05931a646d3db26"
dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
mode: GRACEFUL
new_leader_uuid: "591e55d6b5724c309aeb00994e320bd0"
 from {username='slave'} at 127.0.0.1:44698
I20260709 13:01:12.647169  1147 raft_consensus.cc:606] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Received request to transfer leadership to TS 591e55d6b5724c309aeb00994e320bd0
I20260709 13:01:12.653777  1146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c850d5de21d4c02b851d9925ef9c881"
dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
mode: GRACEFUL
new_leader_uuid: "591e55d6b5724c309aeb00994e320bd0"
 from {username='slave'} at 127.0.0.1:44698
I20260709 13:01:12.654462  1146 raft_consensus.cc:606] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Received request to transfer leadership to TS 591e55d6b5724c309aeb00994e320bd0
I20260709 13:01:12.655560  1270 consensus_queue.cc:1048] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:12.664445  1147 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98633dc5342842c294d5ff93c4301c9d"
dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
mode: GRACEFUL
new_leader_uuid: "591e55d6b5724c309aeb00994e320bd0"
 from {username='slave'} at 127.0.0.1:44698
I20260709 13:01:12.665124  1147 raft_consensus.cc:606] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Received request to transfer leadership to TS 591e55d6b5724c309aeb00994e320bd0
I20260709 13:01:12.687222  1220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d86aeab58674fd2b92beda6f5fc0d7a"
dest_uuid: "406a72268a484d05920a71f05d71691e"
mode: GRACEFUL
new_leader_uuid: "591e55d6b5724c309aeb00994e320bd0"
 from {username='slave'} at 127.0.0.1:58862
I20260709 13:01:12.687841  1220 raft_consensus.cc:606] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 1 LEADER]: Received request to transfer leadership to TS 591e55d6b5724c309aeb00994e320bd0
I20260709 13:01:12.697800  1220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0f4d34e991c4787b1c336afee69d01e"
dest_uuid: "406a72268a484d05920a71f05d71691e"
mode: GRACEFUL
new_leader_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
 from {username='slave'} at 127.0.0.1:58862
I20260709 13:01:12.698536  1220 raft_consensus.cc:606] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 1 LEADER]: Received request to transfer leadership to TS 24f31d4f10ad41eab5902ec7bb8ba5aa
I20260709 13:01:12.701056  1327 raft_consensus.cc:993] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e: : Instructing follower 24f31d4f10ad41eab5902ec7bb8ba5aa to start an election
I20260709 13:01:12.701781  1269 raft_consensus.cc:1081] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 1 LEADER]: Signalling peer 24f31d4f10ad41eab5902ec7bb8ba5aa to start an election
I20260709 13:01:12.703877  1146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e0f4d34e991c4787b1c336afee69d01e"
dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
 from {username='slave'} at 127.0.0.1:44678
I20260709 13:01:12.705164  1146 raft_consensus.cc:493] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:12.706732  1146 raft_consensus.cc:3060] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.709250  1221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec42386aeae44255bf83eae6530c4f97"
dest_uuid: "406a72268a484d05920a71f05d71691e"
mode: GRACEFUL
new_leader_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
 from {username='slave'} at 127.0.0.1:58862
I20260709 13:01:12.710125  1221 raft_consensus.cc:606] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 1 LEADER]: Received request to transfer leadership to TS 24f31d4f10ad41eab5902ec7bb8ba5aa
I20260709 13:01:12.714368  1330 consensus_queue.cc:1048] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:12.716917  1221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "74a921ab3514481087dcb4ae5bfd554a"
dest_uuid: "406a72268a484d05920a71f05d71691e"
mode: GRACEFUL
new_leader_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
 from {username='slave'} at 127.0.0.1:58862
I20260709 13:01:12.717700  1221 raft_consensus.cc:606] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 1 LEADER]: Received request to transfer leadership to TS 24f31d4f10ad41eab5902ec7bb8ba5aa
I20260709 13:01:12.719220  1146 raft_consensus.cc:515] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.722723  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0f4d34e991c4787b1c336afee69d01e" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
I20260709 13:01:12.723121  1146 leader_election.cc:290] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 2 election: Requested vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:12.723210  1221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0f4d34e991c4787b1c336afee69d01e" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:12.724323  1221 raft_consensus.cc:3055] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:12.723351  1071 raft_consensus.cc:3060] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.724759  1221 raft_consensus.cc:740] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 406a72268a484d05920a71f05d71691e, State: Running, Role: LEADER
I20260709 13:01:12.726485  1221 consensus_queue.cc:260] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.728068  1221 raft_consensus.cc:3060] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.741360  1146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3c20fcfe7b04c0fa8ca55c75fb57879"
dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
mode: GRACEFUL
new_leader_uuid: "406a72268a484d05920a71f05d71691e"
 from {username='slave'} at 127.0.0.1:44698
I20260709 13:01:12.742089  1146 raft_consensus.cc:606] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Received request to transfer leadership to TS 406a72268a484d05920a71f05d71691e
I20260709 13:01:12.742777  1343 consensus_queue.cc:1048] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:12.736006  1071 raft_consensus.cc:2468] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 2.
I20260709 13:01:12.744813  1112 leader_election.cc:304] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:12.752764  1343 raft_consensus.cc:2804] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:12.753531  1343 raft_consensus.cc:697] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 LEADER]: Becoming Leader. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Running, Role: LEADER
I20260709 13:01:12.755501  1343 consensus_queue.cc:237] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.758423  1221 raft_consensus.cc:2468] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 2.
I20260709 13:01:12.760609  1272 raft_consensus.cc:993] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa: : Instructing follower 406a72268a484d05920a71f05d71691e to start an election
I20260709 13:01:12.761147  1272 raft_consensus.cc:1081] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Signalling peer 406a72268a484d05920a71f05d71691e to start an election
I20260709 13:01:12.770134  1221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3c20fcfe7b04c0fa8ca55c75fb57879"
dest_uuid: "406a72268a484d05920a71f05d71691e"
 from {username='slave'} at 127.0.0.1:58852
I20260709 13:01:12.771572  1221 raft_consensus.cc:493] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:12.772262  1221 raft_consensus.cc:3060] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.771953   970 catalog_manager.cc:5697] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa reported cstate change: term changed from 1 to 2, leader changed from 406a72268a484d05920a71f05d71691e (127.0.107.195) to 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194). New cstate: current_term: 2 leader_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:12.788056  1221 raft_consensus.cc:515] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.791128  1221 leader_election.cc:290] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 2 election: Requested vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299)
I20260709 13:01:12.793346  1146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c20fcfe7b04c0fa8ca55c75fb57879" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
I20260709 13:01:12.794384  1146 raft_consensus.cc:3055] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:12.794737  1146 raft_consensus.cc:740] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Running, Role: LEADER
I20260709 13:01:12.795363  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c20fcfe7b04c0fa8ca55c75fb57879" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
I20260709 13:01:12.796485  1294 raft_consensus.cc:993] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa: : Instructing follower 591e55d6b5724c309aeb00994e320bd0 to start an election
I20260709 13:01:12.796131  1071 raft_consensus.cc:3060] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.796106  1146 consensus_queue.cc:260] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.797178  1294 raft_consensus.cc:1081] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Signalling peer 591e55d6b5724c309aeb00994e320bd0 to start an election
I20260709 13:01:12.798877  1146 raft_consensus.cc:3060] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.799787  1069 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d4f631913d84fa2b0ec6feece991332"
dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
 from {username='slave'} at 127.0.0.1:48870
I20260709 13:01:12.800485  1069 raft_consensus.cc:493] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:12.800917  1069 raft_consensus.cc:3060] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.805397  1071 raft_consensus.cc:2468] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 2.
I20260709 13:01:12.809289  1069 raft_consensus.cc:515] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } }
I20260709 13:01:12.810679  1146 raft_consensus.cc:2468] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 2.
I20260709 13:01:12.811884  1186 leader_election.cc:304] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 406a72268a484d05920a71f05d71691e, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:12.814733  1221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4f631913d84fa2b0ec6feece991332" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:12.815766  1327 raft_consensus.cc:2804] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:12.816593  1069 leader_election.cc:290] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 2 election: Requested vote from peers 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:12.816892  1327 raft_consensus.cc:697] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e [term 2 LEADER]: Becoming Leader. State: Replica: 406a72268a484d05920a71f05d71691e, State: Running, Role: LEADER
I20260709 13:01:12.818202  1327 consensus_queue.cc:237] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.815816  1221 raft_consensus.cc:3060] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.822340  1146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4f631913d84fa2b0ec6feece991332" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
I20260709 13:01:12.823021  1146 raft_consensus.cc:3055] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:12.823400  1146 raft_consensus.cc:740] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Running, Role: LEADER
I20260709 13:01:12.824190  1146 consensus_queue.cc:260] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } }
I20260709 13:01:12.825134  1146 raft_consensus.cc:3060] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.832314  1146 raft_consensus.cc:2468] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e55d6b5724c309aeb00994e320bd0 in term 2.
I20260709 13:01:12.832695  1221 raft_consensus.cc:2468] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e55d6b5724c309aeb00994e320bd0 in term 2.
I20260709 13:01:12.833995   970 catalog_manager.cc:5697] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e reported cstate change: term changed from 1 to 2, leader changed from 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194) to 406a72268a484d05920a71f05d71691e (127.0.107.195). New cstate: current_term: 2 leader_uuid: "406a72268a484d05920a71f05d71691e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:12.835532  1036 leader_election.cc:304] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 406a72268a484d05920a71f05d71691e, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:12.840340  1320 raft_consensus.cc:2804] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:12.842557  1320 raft_consensus.cc:697] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Becoming Leader. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Running, Role: LEADER
I20260709 13:01:12.843580  1320 consensus_queue.cc:237] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } }
I20260709 13:01:12.847477  1327 raft_consensus.cc:993] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e: : Instructing follower 591e55d6b5724c309aeb00994e320bd0 to start an election
I20260709 13:01:12.848477  1327 raft_consensus.cc:1081] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 1 LEADER]: Signalling peer 591e55d6b5724c309aeb00994e320bd0 to start an election
I20260709 13:01:12.850869  1069 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d86aeab58674fd2b92beda6f5fc0d7a"
dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
 from {username='slave'} at 127.0.0.1:48858
I20260709 13:01:12.851640  1069 raft_consensus.cc:493] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:12.851985  1069 raft_consensus.cc:3060] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.853926  1343 raft_consensus.cc:993] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa: : Instructing follower 591e55d6b5724c309aeb00994e320bd0 to start an election
I20260709 13:01:12.854638  1294 raft_consensus.cc:1081] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Signalling peer 591e55d6b5724c309aeb00994e320bd0 to start an election
I20260709 13:01:12.856963  1071 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1df16b6ed89d4d1eb05931a646d3db26"
dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
 from {username='slave'} at 127.0.0.1:48870
I20260709 13:01:12.857928  1071 raft_consensus.cc:493] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:12.858369  1071 raft_consensus.cc:3060] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.857280   970 catalog_manager.cc:5697] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 reported cstate change: term changed from 1 to 2, leader changed from 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194) to 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "591e55d6b5724c309aeb00994e320bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:12.860752  1069 raft_consensus.cc:515] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.864035  1069 leader_election.cc:290] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 2 election: Requested vote from peers 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:12.864887  1146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d86aeab58674fd2b92beda6f5fc0d7a" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
I20260709 13:01:12.865428  1221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d86aeab58674fd2b92beda6f5fc0d7a" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:12.866356  1146 raft_consensus.cc:3060] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.866760  1221 raft_consensus.cc:3055] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:12.867586  1221 raft_consensus.cc:740] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 406a72268a484d05920a71f05d71691e, State: Running, Role: LEADER
I20260709 13:01:12.869095  1221 consensus_queue.cc:260] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.870808  1071 raft_consensus.cc:515] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.872767  1221 raft_consensus.cc:3060] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.875120  1147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16b6ed89d4d1eb05931a646d3db26" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
I20260709 13:01:12.874837  1071 leader_election.cc:290] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 2 election: Requested vote from peers 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:12.876161  1147 raft_consensus.cc:3055] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:12.876590  1147 raft_consensus.cc:740] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Running, Role: LEADER
I20260709 13:01:12.875555  1220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16b6ed89d4d1eb05931a646d3db26" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:12.877596  1147 consensus_queue.cc:260] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.878211  1220 raft_consensus.cc:3060] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.879223  1147 raft_consensus.cc:3060] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.881567  1146 raft_consensus.cc:2468] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e55d6b5724c309aeb00994e320bd0 in term 2.
I20260709 13:01:12.883307  1037 leader_election.cc:304] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:12.884413  1320 raft_consensus.cc:2804] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:12.884971  1320 raft_consensus.cc:697] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Becoming Leader. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Running, Role: LEADER
I20260709 13:01:12.885988  1320 consensus_queue.cc:237] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.886775  1221 raft_consensus.cc:2468] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e55d6b5724c309aeb00994e320bd0 in term 2.
I20260709 13:01:12.886801  1220 raft_consensus.cc:2468] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e55d6b5724c309aeb00994e320bd0 in term 2.
I20260709 13:01:12.888438  1036 leader_election.cc:304] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 406a72268a484d05920a71f05d71691e, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:12.889325  1355 raft_consensus.cc:2804] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:12.890730  1355 raft_consensus.cc:697] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Becoming Leader. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Running, Role: LEADER
I20260709 13:01:12.890955  1147 raft_consensus.cc:2468] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e55d6b5724c309aeb00994e320bd0 in term 2.
I20260709 13:01:12.891799  1355 consensus_queue.cc:237] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.897903   970 catalog_manager.cc:5697] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 reported cstate change: term changed from 1 to 2, leader changed from 406a72268a484d05920a71f05d71691e (127.0.107.195) to 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "591e55d6b5724c309aeb00994e320bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:12.913234   970 catalog_manager.cc:5697] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 reported cstate change: term changed from 1 to 2, leader changed from 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194) to 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "591e55d6b5724c309aeb00994e320bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:12.934113  1294 consensus_queue.cc:1048] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:12.944783  1343 consensus_queue.cc:1048] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:12.961555  1294 raft_consensus.cc:993] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa: : Instructing follower 591e55d6b5724c309aeb00994e320bd0 to start an election
I20260709 13:01:12.962514  1343 raft_consensus.cc:1081] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Signalling peer 591e55d6b5724c309aeb00994e320bd0 to start an election
I20260709 13:01:12.964532  1071 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "98633dc5342842c294d5ff93c4301c9d"
dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
 from {username='slave'} at 127.0.0.1:48870
I20260709 13:01:12.965150  1071 raft_consensus.cc:493] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:12.965597  1071 raft_consensus.cc:3060] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.972620  1071 raft_consensus.cc:515] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.976657  1071 leader_election.cc:290] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 2 election: Requested vote from peers 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:12.982355  1221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98633dc5342842c294d5ff93c4301c9d" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:12.982355  1147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98633dc5342842c294d5ff93c4301c9d" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
I20260709 13:01:12.983472  1221 raft_consensus.cc:3060] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.983472  1147 raft_consensus.cc:3055] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:12.984509  1147 raft_consensus.cc:740] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Running, Role: LEADER
I20260709 13:01:12.985479  1147 consensus_queue.cc:260] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:12.986821  1147 raft_consensus.cc:3060] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.991818  1221 raft_consensus.cc:2468] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e55d6b5724c309aeb00994e320bd0 in term 2.
I20260709 13:01:12.994526  1147 raft_consensus.cc:2468] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e55d6b5724c309aeb00994e320bd0 in term 2.
I20260709 13:01:12.994604  1036 leader_election.cc:304] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 406a72268a484d05920a71f05d71691e, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:12.996336  1355 raft_consensus.cc:2804] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:12.997579  1355 raft_consensus.cc:697] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Becoming Leader. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Running, Role: LEADER
I20260709 13:01:12.998900  1355 consensus_queue.cc:237] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:13.009027   968 catalog_manager.cc:5697] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 reported cstate change: term changed from 1 to 2, leader changed from 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194) to 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "591e55d6b5724c309aeb00994e320bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:13.053767  1294 raft_consensus.cc:993] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa: : Instructing follower 591e55d6b5724c309aeb00994e320bd0 to start an election
I20260709 13:01:13.054355  1343 raft_consensus.cc:1081] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Signalling peer 591e55d6b5724c309aeb00994e320bd0 to start an election
I20260709 13:01:13.056114  1071 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c850d5de21d4c02b851d9925ef9c881"
dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
 from {username='slave'} at 127.0.0.1:48870
I20260709 13:01:13.056780  1071 raft_consensus.cc:493] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:13.057168  1071 raft_consensus.cc:3060] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.063511  1071 raft_consensus.cc:515] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:13.066406  1071 leader_election.cc:290] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 2 election: Requested vote from peers 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:13.067188  1147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c850d5de21d4c02b851d9925ef9c881" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
I20260709 13:01:13.067894  1147 raft_consensus.cc:3055] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:13.068465  1147 raft_consensus.cc:740] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Running, Role: LEADER
I20260709 13:01:13.068655  1221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c850d5de21d4c02b851d9925ef9c881" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:13.069240  1221 raft_consensus.cc:3060] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.069471  1147 consensus_queue.cc:260] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:13.071132  1147 raft_consensus.cc:3060] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.078634  1221 raft_consensus.cc:2468] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e55d6b5724c309aeb00994e320bd0 in term 2.
I20260709 13:01:13.079814  1147 raft_consensus.cc:2468] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e55d6b5724c309aeb00994e320bd0 in term 2.
I20260709 13:01:13.081030  1037 leader_election.cc:304] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:13.081810  1355 raft_consensus.cc:2804] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:13.082270  1355 raft_consensus.cc:697] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Becoming Leader. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Running, Role: LEADER
I20260709 13:01:13.083132  1355 consensus_queue.cc:237] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:13.091645   968 catalog_manager.cc:5697] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 reported cstate change: term changed from 1 to 2, leader changed from 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194) to 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "591e55d6b5724c309aeb00994e320bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:13.181478  1327 raft_consensus.cc:993] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e: : Instructing follower 24f31d4f10ad41eab5902ec7bb8ba5aa to start an election
I20260709 13:01:13.182149  1327 raft_consensus.cc:1081] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 1 LEADER]: Signalling peer 24f31d4f10ad41eab5902ec7bb8ba5aa to start an election
I20260709 13:01:13.184396  1147 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "74a921ab3514481087dcb4ae5bfd554a"
dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
 from {username='slave'} at 127.0.0.1:44678
I20260709 13:01:13.185053  1147 raft_consensus.cc:493] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:13.185487  1147 raft_consensus.cc:3060] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.195050  1147 raft_consensus.cc:515] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:13.195864  1269 raft_consensus.cc:993] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e: : Instructing follower 24f31d4f10ad41eab5902ec7bb8ba5aa to start an election
I20260709 13:01:13.196805  1269 raft_consensus.cc:1081] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 1 LEADER]: Signalling peer 24f31d4f10ad41eab5902ec7bb8ba5aa to start an election
I20260709 13:01:13.198041  1147 leader_election.cc:290] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 2 election: Requested vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:13.198274  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74a921ab3514481087dcb4ae5bfd554a" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
I20260709 13:01:13.198865  1071 raft_consensus.cc:3060] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.200387  1147 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec42386aeae44255bf83eae6530c4f97"
dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
 from {username='slave'} at 127.0.0.1:44678
I20260709 13:01:13.200356  1221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74a921ab3514481087dcb4ae5bfd554a" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:13.201392  1147 raft_consensus.cc:493] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:13.201913  1147 raft_consensus.cc:3060] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.201960  1221 raft_consensus.cc:3055] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:13.203001  1221 raft_consensus.cc:740] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 406a72268a484d05920a71f05d71691e, State: Running, Role: LEADER
I20260709 13:01:13.204913  1221 consensus_queue.cc:260] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:13.206202  1221 raft_consensus.cc:3060] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.207259  1071 raft_consensus.cc:2468] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 2.
I20260709 13:01:13.208755  1112 leader_election.cc:304] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:13.209954  1343 raft_consensus.cc:2804] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:13.209921  1147 raft_consensus.cc:515] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:13.210829  1343 raft_consensus.cc:697] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 LEADER]: Becoming Leader. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Running, Role: LEADER
I20260709 13:01:13.211694  1343 consensus_queue.cc:237] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:13.213287  1147 leader_election.cc:290] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 2 election: Requested vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:13.214107  1220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec42386aeae44255bf83eae6530c4f97" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:13.214840  1220 raft_consensus.cc:3055] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:13.215704  1220 raft_consensus.cc:740] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 406a72268a484d05920a71f05d71691e, State: Running, Role: LEADER
I20260709 13:01:13.216552  1071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec42386aeae44255bf83eae6530c4f97" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
I20260709 13:01:13.217258  1071 raft_consensus.cc:3060] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.216949  1220 consensus_queue.cc:260] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:13.218920  1220 raft_consensus.cc:3060] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.222177  1221 raft_consensus.cc:2468] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 2.
I20260709 13:01:13.224994  1071 raft_consensus.cc:2468] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 2.
I20260709 13:01:13.223167   968 catalog_manager.cc:5697] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa reported cstate change: term changed from 1 to 2, leader changed from 406a72268a484d05920a71f05d71691e (127.0.107.195) to 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194). New cstate: current_term: 2 leader_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:13.226102  1112 leader_election.cc:304] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:13.226794  1343 raft_consensus.cc:2804] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:13.227272  1343 raft_consensus.cc:697] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 LEADER]: Becoming Leader. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Running, Role: LEADER
I20260709 13:01:13.228223  1343 consensus_queue.cc:237] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:13.231647  1220 raft_consensus.cc:2468] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 2.
I20260709 13:01:13.245402   968 catalog_manager.cc:5697] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa reported cstate change: term changed from 1 to 2, leader changed from 406a72268a484d05920a71f05d71691e (127.0.107.195) to 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194). New cstate: current_term: 2 leader_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:13.246094  1071 raft_consensus.cc:1275] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Refusing update from remote peer 24f31d4f10ad41eab5902ec7bb8ba5aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:13.248386  1343 consensus_queue.cc:1048] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:13.259083  1221 raft_consensus.cc:1275] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Refusing update from remote peer 24f31d4f10ad41eab5902ec7bb8ba5aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:13.260221  1220 raft_consensus.cc:1275] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Refusing update from remote peer 591e55d6b5724c309aeb00994e320bd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:13.262423  1355 consensus_queue.cc:1048] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:13.263547  1294 consensus_queue.cc:1048] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:13.275128  1147 raft_consensus.cc:1275] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Refusing update from remote peer 591e55d6b5724c309aeb00994e320bd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:13.277835  1356 consensus_queue.cc:1048] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:13.289853  1147 raft_consensus.cc:1275] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Refusing update from remote peer 406a72268a484d05920a71f05d71691e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:13.291596  1269 consensus_queue.cc:1048] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:13.298540  1071 raft_consensus.cc:1275] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Refusing update from remote peer 406a72268a484d05920a71f05d71691e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:13.300424  1269 consensus_queue.cc:1048] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:13.321218  1221 raft_consensus.cc:1275] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Refusing update from remote peer 591e55d6b5724c309aeb00994e320bd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:13.322924  1356 consensus_queue.cc:1048] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:13.329412  1147 raft_consensus.cc:1275] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Refusing update from remote peer 591e55d6b5724c309aeb00994e320bd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:13.330886  1320 consensus_queue.cc:1048] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:13.335991  1146 raft_consensus.cc:1275] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Refusing update from remote peer 591e55d6b5724c309aeb00994e320bd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:13.337939  1361 consensus_queue.cc:1048] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:13.350477  1221 raft_consensus.cc:1275] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Refusing update from remote peer 591e55d6b5724c309aeb00994e320bd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:13.355751  1320 consensus_queue.cc:1048] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:13.506649  1221 raft_consensus.cc:1275] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Refusing update from remote peer 591e55d6b5724c309aeb00994e320bd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:13.507826  1220 raft_consensus.cc:1275] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Refusing update from remote peer 591e55d6b5724c309aeb00994e320bd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:13.508797  1361 consensus_queue.cc:1048] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:13.509845  1320 consensus_queue.cc:1048] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:13.523344  1147 raft_consensus.cc:1275] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Refusing update from remote peer 591e55d6b5724c309aeb00994e320bd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:13.524124  1146 raft_consensus.cc:1275] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Refusing update from remote peer 591e55d6b5724c309aeb00994e320bd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:13.527417  1356 consensus_queue.cc:1048] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:13.528497  1361 consensus_queue.cc:1048] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:13.642270  1071 raft_consensus.cc:1275] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Refusing update from remote peer 24f31d4f10ad41eab5902ec7bb8ba5aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:13.644425  1272 consensus_queue.cc:1048] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:13.655963  1220 raft_consensus.cc:1275] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Refusing update from remote peer 24f31d4f10ad41eab5902ec7bb8ba5aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:13.659473  1294 consensus_queue.cc:1048] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:13.679188  1071 raft_consensus.cc:1275] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Refusing update from remote peer 24f31d4f10ad41eab5902ec7bb8ba5aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:13.681532  1294 consensus_queue.cc:1048] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:13.700855  1220 raft_consensus.cc:1275] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Refusing update from remote peer 24f31d4f10ad41eab5902ec7bb8ba5aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:13.703537  1294 consensus_queue.cc:1048] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:15.815454   431 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:01:15.816962   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:15.849133  1069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c850d5de21d4c02b851d9925ef9c881"
dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
mode: GRACEFUL
new_leader_uuid: "406a72268a484d05920a71f05d71691e"
 from {username='slave'} at 127.0.0.1:48878
I20260709 13:01:15.849969  1069 raft_consensus.cc:606] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Received request to transfer leadership to TS 406a72268a484d05920a71f05d71691e
I20260709 13:01:15.854166  1069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d4f631913d84fa2b0ec6feece991332"
dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
mode: GRACEFUL
new_leader_uuid: "406a72268a484d05920a71f05d71691e"
 from {username='slave'} at 127.0.0.1:48878
I20260709 13:01:15.854801  1069 raft_consensus.cc:606] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Received request to transfer leadership to TS 406a72268a484d05920a71f05d71691e
I20260709 13:01:15.856451   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:01:15.857218   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:16.092077  1361 raft_consensus.cc:993] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0: : Instructing follower 406a72268a484d05920a71f05d71691e to start an election
I20260709 13:01:16.092859  1373 raft_consensus.cc:1081] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Signalling peer 406a72268a484d05920a71f05d71691e to start an election
I20260709 13:01:16.094961  1221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d4f631913d84fa2b0ec6feece991332"
dest_uuid: "406a72268a484d05920a71f05d71691e"
 from {username='slave'} at 127.0.0.1:58836
I20260709 13:01:16.095583  1221 raft_consensus.cc:493] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:16.095902  1221 raft_consensus.cc:3060] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:16.112565  1361 raft_consensus.cc:993] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0: : Instructing follower 406a72268a484d05920a71f05d71691e to start an election
I20260709 13:01:16.113173  1361 raft_consensus.cc:1081] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Signalling peer 406a72268a484d05920a71f05d71691e to start an election
I20260709 13:01:16.115622  1220 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c850d5de21d4c02b851d9925ef9c881"
dest_uuid: "406a72268a484d05920a71f05d71691e"
 from {username='slave'} at 127.0.0.1:58836
I20260709 13:01:16.116278  1220 raft_consensus.cc:493] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:16.116707  1220 raft_consensus.cc:3060] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:16.186766  1221 raft_consensus.cc:515] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } }
I20260709 13:01:16.191471  1146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4f631913d84fa2b0ec6feece991332" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
I20260709 13:01:16.192828  1146 raft_consensus.cc:3060] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:16.194816  1221 leader_election.cc:290] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 3 election: Requested vote from peers 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299), 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343)
I20260709 13:01:16.196374  1069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4f631913d84fa2b0ec6feece991332" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
I20260709 13:01:16.197438  1069 raft_consensus.cc:3055] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:16.197866  1069 raft_consensus.cc:740] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Running, Role: LEADER
I20260709 13:01:16.199247  1069 consensus_queue.cc:260] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } }
I20260709 13:01:16.204281  1069 raft_consensus.cc:3060] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:16.290057  1146 raft_consensus.cc:2468] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 3.
I20260709 13:01:16.290319  1069 raft_consensus.cc:2468] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 3.
I20260709 13:01:16.290122  1220 raft_consensus.cc:515] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:16.293016  1186 leader_election.cc:304] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 406a72268a484d05920a71f05d71691e; no voters: 
I20260709 13:01:16.295061  1359 raft_consensus.cc:2804] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:16.295429  1220 leader_election.cc:290] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 3 election: Requested vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299)
I20260709 13:01:16.297678  1069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c850d5de21d4c02b851d9925ef9c881" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
I20260709 13:01:16.297897  1146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c850d5de21d4c02b851d9925ef9c881" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
I20260709 13:01:16.298355  1069 raft_consensus.cc:3055] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:16.299118  1359 raft_consensus.cc:697] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 3 LEADER]: Becoming Leader. State: Replica: 406a72268a484d05920a71f05d71691e, State: Running, Role: LEADER
I20260709 13:01:16.299093  1146 raft_consensus.cc:3060] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:16.300544  1069 raft_consensus.cc:740] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Running, Role: LEADER
I20260709 13:01:16.302731  1359 consensus_queue.cc:237] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } }
I20260709 13:01:16.305194  1069 consensus_queue.cc:260] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:16.307178  1069 raft_consensus.cc:3060] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:16.322288   968 catalog_manager.cc:5697] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e reported cstate change: term changed from 2 to 3, leader changed from 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193) to 406a72268a484d05920a71f05d71691e (127.0.107.195). New cstate: current_term: 3 leader_uuid: "406a72268a484d05920a71f05d71691e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:16.389532  1069 raft_consensus.cc:2468] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 3.
I20260709 13:01:16.389714  1146 raft_consensus.cc:2468] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 3.
I20260709 13:01:16.391238  1186 leader_election.cc:304] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 406a72268a484d05920a71f05d71691e, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:16.392635  1359 raft_consensus.cc:2804] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:16.393545  1359 raft_consensus.cc:697] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 3 LEADER]: Becoming Leader. State: Replica: 406a72268a484d05920a71f05d71691e, State: Running, Role: LEADER
I20260709 13:01:16.394811  1359 consensus_queue.cc:237] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:16.406275   968 catalog_manager.cc:5697] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e reported cstate change: term changed from 2 to 3, leader changed from 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193) to 406a72268a484d05920a71f05d71691e (127.0.107.195). New cstate: current_term: 3 leader_uuid: "406a72268a484d05920a71f05d71691e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:16.719272  1146 raft_consensus.cc:1275] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Refusing update from remote peer 406a72268a484d05920a71f05d71691e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:16.721704  1359 consensus_queue.cc:1048] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:16.762351  1069 raft_consensus.cc:1275] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Refusing update from remote peer 406a72268a484d05920a71f05d71691e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:16.765734  1376 consensus_queue.cc:1048] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:16.858261   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:16.872781  1221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d4f631913d84fa2b0ec6feece991332"
dest_uuid: "406a72268a484d05920a71f05d71691e"
mode: GRACEFUL
new_leader_uuid: "591e55d6b5724c309aeb00994e320bd0"
 from {username='slave'} at 127.0.0.1:58862
I20260709 13:01:16.873327  1221 raft_consensus.cc:606] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 3 LEADER]: Received request to transfer leadership to TS 591e55d6b5724c309aeb00994e320bd0
I20260709 13:01:16.889065  1221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c850d5de21d4c02b851d9925ef9c881"
dest_uuid: "406a72268a484d05920a71f05d71691e"
mode: GRACEFUL
new_leader_uuid: "591e55d6b5724c309aeb00994e320bd0"
 from {username='slave'} at 127.0.0.1:58862
I20260709 13:01:16.890010  1221 raft_consensus.cc:606] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 3 LEADER]: Received request to transfer leadership to TS 591e55d6b5724c309aeb00994e320bd0
I20260709 13:01:16.891669  1069 raft_consensus.cc:1275] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Refusing update from remote peer 406a72268a484d05920a71f05d71691e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:16.894129  1376 consensus_queue.cc:1048] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:16.919934  1386 raft_consensus.cc:993] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e: : Instructing follower 591e55d6b5724c309aeb00994e320bd0 to start an election
I20260709 13:01:16.921447  1359 raft_consensus.cc:1081] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 3 LEADER]: Signalling peer 591e55d6b5724c309aeb00994e320bd0 to start an election
I20260709 13:01:16.924616  1071 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c850d5de21d4c02b851d9925ef9c881"
dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
 from {username='slave'} at 127.0.0.1:48858
I20260709 13:01:16.925221  1071 raft_consensus.cc:493] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:16.925804  1071 raft_consensus.cc:3060] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:16.931954  1146 raft_consensus.cc:1275] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Refusing update from remote peer 406a72268a484d05920a71f05d71691e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:16.933763  1147 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0f4d34e991c4787b1c336afee69d01e"
dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
mode: GRACEFUL
new_leader_uuid: "591e55d6b5724c309aeb00994e320bd0"
 from {username='slave'} at 127.0.0.1:44698
I20260709 13:01:16.934414  1147 raft_consensus.cc:606] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 LEADER]: Received request to transfer leadership to TS 591e55d6b5724c309aeb00994e320bd0
I20260709 13:01:16.935868  1376 consensus_queue.cc:1048] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:16.950822  1146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec42386aeae44255bf83eae6530c4f97"
dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
mode: GRACEFUL
new_leader_uuid: "591e55d6b5724c309aeb00994e320bd0"
 from {username='slave'} at 127.0.0.1:44698
I20260709 13:01:16.951784  1146 raft_consensus.cc:606] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 LEADER]: Received request to transfer leadership to TS 591e55d6b5724c309aeb00994e320bd0
I20260709 13:01:16.954102  1071 raft_consensus.cc:515] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:16.959669  1146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "74a921ab3514481087dcb4ae5bfd554a"
dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
mode: GRACEFUL
new_leader_uuid: "591e55d6b5724c309aeb00994e320bd0"
 from {username='slave'} at 127.0.0.1:44698
I20260709 13:01:16.960754  1146 raft_consensus.cc:606] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 LEADER]: Received request to transfer leadership to TS 591e55d6b5724c309aeb00994e320bd0
I20260709 13:01:16.969302  1071 leader_election.cc:290] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 4 election: Requested vote from peers 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:16.970554  1220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c850d5de21d4c02b851d9925ef9c881" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:16.971223  1146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c850d5de21d4c02b851d9925ef9c881" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
I20260709 13:01:16.971743  1220 raft_consensus.cc:3055] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:16.971894  1146 raft_consensus.cc:3060] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:16.972335  1220 raft_consensus.cc:740] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 406a72268a484d05920a71f05d71691e, State: Running, Role: LEADER
I20260709 13:01:16.973368  1220 consensus_queue.cc:260] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:16.974864  1220 raft_consensus.cc:3060] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:17.063424  1330 raft_consensus.cc:993] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa: : Instructing follower 591e55d6b5724c309aeb00994e320bd0 to start an election
I20260709 13:01:17.064343  1382 raft_consensus.cc:1081] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 LEADER]: Signalling peer 591e55d6b5724c309aeb00994e320bd0 to start an election
I20260709 13:01:17.066730  1071 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec42386aeae44255bf83eae6530c4f97"
dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
 from {username='slave'} at 127.0.0.1:48870
I20260709 13:01:17.067472  1071 raft_consensus.cc:493] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:17.067667  1146 raft_consensus.cc:2468] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e55d6b5724c309aeb00994e320bd0 in term 4.
I20260709 13:01:17.067871  1071 raft_consensus.cc:3060] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:17.068730  1220 raft_consensus.cc:2468] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e55d6b5724c309aeb00994e320bd0 in term 4.
I20260709 13:01:17.070569  1037 leader_election.cc:304] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:17.072618  1361 raft_consensus.cc:2804] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:17.077251  1361 raft_consensus.cc:697] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 4 LEADER]: Becoming Leader. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Running, Role: LEADER
I20260709 13:01:17.078944  1361 consensus_queue.cc:237] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:17.090895   966 catalog_manager.cc:5697] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 reported cstate change: term changed from 3 to 4, leader changed from 406a72268a484d05920a71f05d71691e (127.0.107.195) to 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "591e55d6b5724c309aeb00994e320bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:17.122238  1330 raft_consensus.cc:993] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa: : Instructing follower 591e55d6b5724c309aeb00994e320bd0 to start an election
I20260709 13:01:17.122958  1382 raft_consensus.cc:1081] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 LEADER]: Signalling peer 591e55d6b5724c309aeb00994e320bd0 to start an election
I20260709 13:01:17.125411  1069 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "74a921ab3514481087dcb4ae5bfd554a"
dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
 from {username='slave'} at 127.0.0.1:48870
I20260709 13:01:17.126927  1069 raft_consensus.cc:493] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:17.127338  1069 raft_consensus.cc:3060] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:17.151415  1359 raft_consensus.cc:993] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e: : Instructing follower 591e55d6b5724c309aeb00994e320bd0 to start an election
I20260709 13:01:17.152427  1359 raft_consensus.cc:1081] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 3 LEADER]: Signalling peer 591e55d6b5724c309aeb00994e320bd0 to start an election
I20260709 13:01:17.156982  1070 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d4f631913d84fa2b0ec6feece991332"
dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
 from {username='slave'} at 127.0.0.1:48858
I20260709 13:01:17.157830  1070 raft_consensus.cc:493] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:17.158239  1070 raft_consensus.cc:3060] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:17.166465  1071 raft_consensus.cc:515] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:17.171919  1071 leader_election.cc:290] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 3 election: Requested vote from peers 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:17.173820  1146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec42386aeae44255bf83eae6530c4f97" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
I20260709 13:01:17.174678  1146 raft_consensus.cc:3055] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:17.176723  1146 raft_consensus.cc:740] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Running, Role: LEADER
I20260709 13:01:17.178328  1220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec42386aeae44255bf83eae6530c4f97" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:17.178946  1220 raft_consensus.cc:3060] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:17.182809  1146 consensus_queue.cc:260] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:17.184711  1146 raft_consensus.cc:3060] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:17.234385  1069 raft_consensus.cc:515] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:17.234385  1070 raft_consensus.cc:515] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } }
I20260709 13:01:17.239416  1146 raft_consensus.cc:2468] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e55d6b5724c309aeb00994e320bd0 in term 3.
I20260709 13:01:17.241899  1221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4f631913d84fa2b0ec6feece991332" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:17.242553  1221 raft_consensus.cc:3055] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:17.239594  1070 leader_election.cc:290] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 4 election: Requested vote from peers 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:17.242870  1219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74a921ab3514481087dcb4ae5bfd554a" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:17.239679  1147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4f631913d84fa2b0ec6feece991332" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
I20260709 13:01:17.243835  1147 raft_consensus.cc:3060] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:17.243649  1219 raft_consensus.cc:3060] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:17.244108  1037 leader_election.cc:304] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:17.241170  1220 raft_consensus.cc:2468] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e55d6b5724c309aeb00994e320bd0 in term 3.
I20260709 13:01:17.245096  1361 raft_consensus.cc:2804] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:17.239995  1069 leader_election.cc:290] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 3 election: Requested vote from peers 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:17.245675  1361 raft_consensus.cc:697] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 3 LEADER]: Becoming Leader. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Running, Role: LEADER
I20260709 13:01:17.240239  1145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74a921ab3514481087dcb4ae5bfd554a" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
I20260709 13:01:17.247018  1361 consensus_queue.cc:237] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:17.247884  1145 raft_consensus.cc:3055] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:17.248226  1145 raft_consensus.cc:740] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Running, Role: LEADER
I20260709 13:01:17.242977  1221 raft_consensus.cc:740] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 406a72268a484d05920a71f05d71691e, State: Running, Role: LEADER
I20260709 13:01:17.249125  1145 consensus_queue.cc:260] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:17.253737  1145 raft_consensus.cc:3060] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:17.250272  1221 consensus_queue.cc:260] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } }
I20260709 13:01:17.255849  1219 raft_consensus.cc:2468] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e55d6b5724c309aeb00994e320bd0 in term 3.
I20260709 13:01:17.258387  1221 raft_consensus.cc:3060] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:17.262799  1147 raft_consensus.cc:2468] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e55d6b5724c309aeb00994e320bd0 in term 4.
I20260709 13:01:17.260327  1036 leader_election.cc:304] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 406a72268a484d05920a71f05d71691e, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:17.264940  1392 raft_consensus.cc:2804] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:17.265265  1037 leader_election.cc:304] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:17.265519  1392 raft_consensus.cc:697] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [term 3 LEADER]: Becoming Leader. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Running, Role: LEADER
I20260709 13:01:17.267308  1393 raft_consensus.cc:2804] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:17.275847  1392 consensus_queue.cc:237] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:17.279508  1221 raft_consensus.cc:2468] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e55d6b5724c309aeb00994e320bd0 in term 4.
I20260709 13:01:17.282728  1393 raft_consensus.cc:697] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 4 LEADER]: Becoming Leader. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Running, Role: LEADER
I20260709 13:01:17.284072  1393 consensus_queue.cc:237] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } }
I20260709 13:01:17.310423  1145 raft_consensus.cc:2468] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e55d6b5724c309aeb00994e320bd0 in term 3.
I20260709 13:01:17.314201   966 catalog_manager.cc:5697] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 reported cstate change: term changed from 2 to 3, leader changed from 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194) to 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "591e55d6b5724c309aeb00994e320bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:17.331012  1382 raft_consensus.cc:993] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa: : Instructing follower 591e55d6b5724c309aeb00994e320bd0 to start an election
I20260709 13:01:17.332126  1397 raft_consensus.cc:1081] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 LEADER]: Signalling peer 591e55d6b5724c309aeb00994e320bd0 to start an election
I20260709 13:01:17.334264  1069 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e0f4d34e991c4787b1c336afee69d01e"
dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
 from {username='slave'} at 127.0.0.1:48870
I20260709 13:01:17.334836  1069 raft_consensus.cc:493] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:17.335229  1069 raft_consensus.cc:3060] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:17.342058  1069 raft_consensus.cc:515] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:17.345191  1221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0f4d34e991c4787b1c336afee69d01e" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:17.344732  1069 leader_election.cc:290] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 3 election: Requested vote from peers 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:17.345957  1221 raft_consensus.cc:3060] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:17.341747   966 catalog_manager.cc:5697] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 reported cstate change: term changed from 2 to 3, leader changed from 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194) to 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "591e55d6b5724c309aeb00994e320bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:17.345767  1145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0f4d34e991c4787b1c336afee69d01e" candidate_uuid: "591e55d6b5724c309aeb00994e320bd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
I20260709 13:01:17.346804  1145 raft_consensus.cc:3055] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:17.347321  1145 raft_consensus.cc:740] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Running, Role: LEADER
I20260709 13:01:17.348910  1145 consensus_queue.cc:260] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:17.349480   966 catalog_manager.cc:5697] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 reported cstate change: term changed from 3 to 4, leader changed from 406a72268a484d05920a71f05d71691e (127.0.107.195) to 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "591e55d6b5724c309aeb00994e320bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:17.350657  1145 raft_consensus.cc:3060] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:17.351933  1221 raft_consensus.cc:2468] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e55d6b5724c309aeb00994e320bd0 in term 3.
I20260709 13:01:17.353333  1036 leader_election.cc:304] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 406a72268a484d05920a71f05d71691e, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:17.354622  1392 raft_consensus.cc:2804] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:17.355268  1392 raft_consensus.cc:697] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 3 LEADER]: Becoming Leader. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Running, Role: LEADER
I20260709 13:01:17.356748  1392 consensus_queue.cc:237] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:17.367272  1145 raft_consensus.cc:2468] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e55d6b5724c309aeb00994e320bd0 in term 3.
I20260709 13:01:17.381222   966 catalog_manager.cc:5697] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 reported cstate change: term changed from 2 to 3, leader changed from 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194) to 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "591e55d6b5724c309aeb00994e320bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:17.470806  1145 raft_consensus.cc:1275] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 4 FOLLOWER]: Refusing update from remote peer 591e55d6b5724c309aeb00994e320bd0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:17.472638  1392 consensus_queue.cc:1048] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:17.487380  1221 raft_consensus.cc:1275] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 4 FOLLOWER]: Refusing update from remote peer 591e55d6b5724c309aeb00994e320bd0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:17.489126  1392 consensus_queue.cc:1048] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:17.636178  1145 raft_consensus.cc:1275] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Refusing update from remote peer 591e55d6b5724c309aeb00994e320bd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:17.637979  1392 consensus_queue.cc:1048] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:17.649580  1221 raft_consensus.cc:1275] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Refusing update from remote peer 591e55d6b5724c309aeb00994e320bd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:17.652210  1393 consensus_queue.cc:1048] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:17.687676  1221 raft_consensus.cc:1275] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Refusing update from remote peer 591e55d6b5724c309aeb00994e320bd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:17.689731  1393 consensus_queue.cc:1048] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:17.707775  1145 raft_consensus.cc:1275] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Refusing update from remote peer 591e55d6b5724c309aeb00994e320bd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:17.710562  1393 consensus_queue.cc:1048] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:17.800582  1221 raft_consensus.cc:1275] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Refusing update from remote peer 591e55d6b5724c309aeb00994e320bd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:17.803052  1393 consensus_queue.cc:1048] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:17.815397  1145 raft_consensus.cc:1275] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Refusing update from remote peer 591e55d6b5724c309aeb00994e320bd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:17.817699  1361 consensus_queue.cc:1048] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:17.825178  1147 raft_consensus.cc:1275] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 4 FOLLOWER]: Refusing update from remote peer 591e55d6b5724c309aeb00994e320bd0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:17.827078  1361 consensus_queue.cc:1048] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:17.839767  1221 raft_consensus.cc:1275] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 4 FOLLOWER]: Refusing update from remote peer 591e55d6b5724c309aeb00994e320bd0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:17.841470  1393 consensus_queue.cc:1048] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:19.992441   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:20.002190  1069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d86aeab58674fd2b92beda6f5fc0d7a"
dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
mode: GRACEFUL
new_leader_uuid: "406a72268a484d05920a71f05d71691e"
 from {username='slave'} at 127.0.0.1:48878
I20260709 13:01:20.002856  1069 raft_consensus.cc:606] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Received request to transfer leadership to TS 406a72268a484d05920a71f05d71691e
I20260709 13:01:20.006191  1069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98633dc5342842c294d5ff93c4301c9d"
dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
mode: GRACEFUL
new_leader_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
 from {username='slave'} at 127.0.0.1:48878
I20260709 13:01:20.007026  1069 raft_consensus.cc:606] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Received request to transfer leadership to TS 24f31d4f10ad41eab5902ec7bb8ba5aa
I20260709 13:01:20.010768  1069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0f4d34e991c4787b1c336afee69d01e"
dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
mode: GRACEFUL
new_leader_uuid: "406a72268a484d05920a71f05d71691e"
 from {username='slave'} at 127.0.0.1:48878
I20260709 13:01:20.011739  1069 raft_consensus.cc:606] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 3 LEADER]: Received request to transfer leadership to TS 406a72268a484d05920a71f05d71691e
I20260709 13:01:20.015529  1069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec42386aeae44255bf83eae6530c4f97"
dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
mode: GRACEFUL
new_leader_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
 from {username='slave'} at 127.0.0.1:48878
I20260709 13:01:20.016279  1069 raft_consensus.cc:606] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 3 LEADER]: Received request to transfer leadership to TS 24f31d4f10ad41eab5902ec7bb8ba5aa
I20260709 13:01:20.017733   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:01:20.018370   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:20.207288  1416 raft_consensus.cc:993] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0: : Instructing follower 24f31d4f10ad41eab5902ec7bb8ba5aa to start an election
I20260709 13:01:20.207876  1416 raft_consensus.cc:1081] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Signalling peer 24f31d4f10ad41eab5902ec7bb8ba5aa to start an election
I20260709 13:01:20.209985  1145 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "98633dc5342842c294d5ff93c4301c9d"
dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
 from {username='slave'} at 127.0.0.1:44688
I20260709 13:01:20.210573  1145 raft_consensus.cc:493] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:20.211035  1145 raft_consensus.cc:3060] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:20.219729  1145 raft_consensus.cc:515] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:20.221836  1145 leader_election.cc:290] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 3 election: Requested vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:20.222694  1069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98633dc5342842c294d5ff93c4301c9d" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
I20260709 13:01:20.223302  1069 raft_consensus.cc:3055] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:20.223588  1069 raft_consensus.cc:740] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Running, Role: LEADER
I20260709 13:01:20.223057  1221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98633dc5342842c294d5ff93c4301c9d" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:20.224069  1221 raft_consensus.cc:3060] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:20.224556  1069 consensus_queue.cc:260] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:20.225772  1069 raft_consensus.cc:3060] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:20.229554  1416 raft_consensus.cc:993] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0: : Instructing follower 406a72268a484d05920a71f05d71691e to start an election
I20260709 13:01:20.230120  1416 raft_consensus.cc:1081] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Signalling peer 406a72268a484d05920a71f05d71691e to start an election
I20260709 13:01:20.231839  1069 raft_consensus.cc:2468] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 3.
I20260709 13:01:20.232213  1219 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d86aeab58674fd2b92beda6f5fc0d7a"
dest_uuid: "406a72268a484d05920a71f05d71691e"
 from {username='slave'} at 127.0.0.1:58836
I20260709 13:01:20.232746  1219 raft_consensus.cc:493] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:20.233062  1219 raft_consensus.cc:3060] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:20.233819  1221 raft_consensus.cc:2468] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 3.
I20260709 13:01:20.234030  1112 leader_election.cc:304] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:20.235019  1397 raft_consensus.cc:2804] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:20.236192  1397 raft_consensus.cc:697] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 LEADER]: Becoming Leader. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Running, Role: LEADER
I20260709 13:01:20.237391  1397 consensus_queue.cc:237] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:20.242122  1219 raft_consensus.cc:515] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:20.244592  1219 leader_election.cc:290] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 3 election: Requested vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299)
I20260709 13:01:20.246389  1145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d86aeab58674fd2b92beda6f5fc0d7a" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
I20260709 13:01:20.246981  1145 raft_consensus.cc:3060] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:20.247038  1069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d86aeab58674fd2b92beda6f5fc0d7a" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
I20260709 13:01:20.247907  1069 raft_consensus.cc:3055] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:20.247473   966 catalog_manager.cc:5697] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa reported cstate change: term changed from 2 to 3, leader changed from 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193) to 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194). New cstate: current_term: 3 leader_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:20.248291  1069 raft_consensus.cc:740] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Running, Role: LEADER
I20260709 13:01:20.249578  1069 consensus_queue.cc:260] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:20.251338  1069 raft_consensus.cc:3060] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:20.256395  1145 raft_consensus.cc:2468] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 3.
I20260709 13:01:20.258862  1186 leader_election.cc:304] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 406a72268a484d05920a71f05d71691e; no voters: 
I20260709 13:01:20.260743  1415 raft_consensus.cc:2804] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:20.261968  1069 raft_consensus.cc:2468] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 3.
I20260709 13:01:20.266644  1415 raft_consensus.cc:697] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 3 LEADER]: Becoming Leader. State: Replica: 406a72268a484d05920a71f05d71691e, State: Running, Role: LEADER
I20260709 13:01:20.268003  1415 consensus_queue.cc:237] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:20.286064   968 catalog_manager.cc:5697] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e reported cstate change: term changed from 2 to 3, leader changed from 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193) to 406a72268a484d05920a71f05d71691e (127.0.107.195). New cstate: current_term: 3 leader_uuid: "406a72268a484d05920a71f05d71691e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:20.310148  1416 raft_consensus.cc:993] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0: : Instructing follower 24f31d4f10ad41eab5902ec7bb8ba5aa to start an election
I20260709 13:01:20.310640  1361 raft_consensus.cc:1081] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 3 LEADER]: Signalling peer 24f31d4f10ad41eab5902ec7bb8ba5aa to start an election
I20260709 13:01:20.312729  1145 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec42386aeae44255bf83eae6530c4f97"
dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
 from {username='slave'} at 127.0.0.1:44688
I20260709 13:01:20.313252  1145 raft_consensus.cc:493] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:20.314000  1145 raft_consensus.cc:3060] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:20.320711  1145 raft_consensus.cc:515] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:20.323423  1145 leader_election.cc:290] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 4 election: Requested vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 406a72268a484d05920a71f05d71691e (127.0.107.195:36731)
I20260709 13:01:20.324155  1069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec42386aeae44255bf83eae6530c4f97" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
I20260709 13:01:20.324469  1219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec42386aeae44255bf83eae6530c4f97" candidate_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "406a72268a484d05920a71f05d71691e"
I20260709 13:01:20.324759  1069 raft_consensus.cc:3055] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:20.325008  1219 raft_consensus.cc:3060] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:20.325076  1069 raft_consensus.cc:740] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Running, Role: LEADER
I20260709 13:01:20.326148  1069 consensus_queue.cc:260] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:20.327610  1069 raft_consensus.cc:3060] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:20.334872  1219 raft_consensus.cc:2468] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 4.
I20260709 13:01:20.335260  1069 raft_consensus.cc:2468] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f31d4f10ad41eab5902ec7bb8ba5aa in term 4.
I20260709 13:01:20.336840  1112 leader_election.cc:304] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 591e55d6b5724c309aeb00994e320bd0; no voters: 
I20260709 13:01:20.338132  1397 raft_consensus.cc:2804] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:20.338948  1397 raft_consensus.cc:697] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 4 LEADER]: Becoming Leader. State: Replica: 24f31d4f10ad41eab5902ec7bb8ba5aa, State: Running, Role: LEADER
I20260709 13:01:20.339915  1397 consensus_queue.cc:237] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:20.349205   968 catalog_manager.cc:5697] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa reported cstate change: term changed from 3 to 4, leader changed from 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193) to 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194). New cstate: current_term: 4 leader_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:20.375811  1416 raft_consensus.cc:993] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0: : Instructing follower 406a72268a484d05920a71f05d71691e to start an election
I20260709 13:01:20.376334  1361 raft_consensus.cc:1081] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 3 LEADER]: Signalling peer 406a72268a484d05920a71f05d71691e to start an election
I20260709 13:01:20.379372  1221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e0f4d34e991c4787b1c336afee69d01e"
dest_uuid: "406a72268a484d05920a71f05d71691e"
 from {username='slave'} at 127.0.0.1:58836
I20260709 13:01:20.380103  1221 raft_consensus.cc:493] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:20.380641  1221 raft_consensus.cc:3060] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:20.389181  1221 raft_consensus.cc:515] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:20.391467  1221 leader_election.cc:290] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 4 election: Requested vote from peers 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343), 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299)
I20260709 13:01:20.393240  1145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0f4d34e991c4787b1c336afee69d01e" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa"
I20260709 13:01:20.393402  1069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0f4d34e991c4787b1c336afee69d01e" candidate_uuid: "406a72268a484d05920a71f05d71691e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "591e55d6b5724c309aeb00994e320bd0"
I20260709 13:01:20.393854  1145 raft_consensus.cc:3060] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:20.394163  1069 raft_consensus.cc:3055] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:20.394982  1069 raft_consensus.cc:740] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 591e55d6b5724c309aeb00994e320bd0, State: Running, Role: LEADER
I20260709 13:01:20.395956  1069 consensus_queue.cc:260] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:20.397035  1069 raft_consensus.cc:3060] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:20.399974  1145 raft_consensus.cc:2468] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 4.
I20260709 13:01:20.401707  1186 leader_election.cc:304] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f31d4f10ad41eab5902ec7bb8ba5aa, 406a72268a484d05920a71f05d71691e; no voters: 
I20260709 13:01:20.402519  1415 raft_consensus.cc:2804] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:20.402937  1415 raft_consensus.cc:697] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 4 LEADER]: Becoming Leader. State: Replica: 406a72268a484d05920a71f05d71691e, State: Running, Role: LEADER
I20260709 13:01:20.403980  1415 consensus_queue.cc:237] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } }
I20260709 13:01:20.405109  1069 raft_consensus.cc:2468] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 406a72268a484d05920a71f05d71691e in term 4.
I20260709 13:01:20.413214   968 catalog_manager.cc:5697] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e reported cstate change: term changed from 3 to 4, leader changed from 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193) to 406a72268a484d05920a71f05d71691e (127.0.107.195). New cstate: current_term: 4 leader_uuid: "406a72268a484d05920a71f05d71691e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:20.639871  1069 raft_consensus.cc:1275] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Refusing update from remote peer 24f31d4f10ad41eab5902ec7bb8ba5aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:20.641476  1397 consensus_queue.cc:1048] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:20.656508  1221 raft_consensus.cc:1275] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Refusing update from remote peer 24f31d4f10ad41eab5902ec7bb8ba5aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:20.658447  1417 consensus_queue.cc:1048] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:20.812095  1145 raft_consensus.cc:1275] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 4 FOLLOWER]: Refusing update from remote peer 406a72268a484d05920a71f05d71691e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:20.813818  1420 consensus_queue.cc:1048] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:20.827844  1069 raft_consensus.cc:1275] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 4 FOLLOWER]: Refusing update from remote peer 406a72268a484d05920a71f05d71691e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:20.829988  1415 consensus_queue.cc:1048] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:01:20.832415  1069 raft_consensus.cc:1275] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Refusing update from remote peer 406a72268a484d05920a71f05d71691e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:20.835839  1415 consensus_queue.cc:1048] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:20.847739  1145 raft_consensus.cc:1275] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Refusing update from remote peer 406a72268a484d05920a71f05d71691e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:20.850945  1221 raft_consensus.cc:1275] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 4 FOLLOWER]: Refusing update from remote peer 24f31d4f10ad41eab5902ec7bb8ba5aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:20.850822  1415 consensus_queue.cc:1048] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f31d4f10ad41eab5902ec7bb8ba5aa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:20.852725  1417 consensus_queue.cc:1048] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "406a72268a484d05920a71f05d71691e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:20.868641  1069 raft_consensus.cc:1275] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 4 FOLLOWER]: Refusing update from remote peer 24f31d4f10ad41eab5902ec7bb8ba5aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:20.871002  1417 consensus_queue.cc:1048] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e55d6b5724c309aeb00994e320bd0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:21.019403   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:21.025185   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:01:21.055308   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:21.056140   431 tablet_replica.cc:333] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 591e55d6b5724c309aeb00994e320bd0: stopping tablet replica
I20260709 13:01:21.056915   431 raft_consensus.cc:2243] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.057403   431 raft_consensus.cc:2272] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.059381   431 tablet_replica.cc:333] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0: stopping tablet replica
I20260709 13:01:21.059969   431 raft_consensus.cc:2243] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:21.060734   431 raft_consensus.cc:2272] T 74a921ab3514481087dcb4ae5bfd554a P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.062631   431 tablet_replica.cc:333] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0: stopping tablet replica
I20260709 13:01:21.063390   431 raft_consensus.cc:2243] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:01:21.064188   431 raft_consensus.cc:2272] T 1c850d5de21d4c02b851d9925ef9c881 P 591e55d6b5724c309aeb00994e320bd0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.066157   431 tablet_replica.cc:333] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0: stopping tablet replica
I20260709 13:01:21.066763   431 raft_consensus.cc:2243] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.067203   431 raft_consensus.cc:2272] T 8d86aeab58674fd2b92beda6f5fc0d7a P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.070945   431 tablet_replica.cc:333] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0: stopping tablet replica
I20260709 13:01:21.071643   431 raft_consensus.cc:2243] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.072096   431 raft_consensus.cc:2272] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 591e55d6b5724c309aeb00994e320bd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.074232   431 tablet_replica.cc:333] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0: stopping tablet replica
I20260709 13:01:21.074765   431 raft_consensus.cc:2243] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:01:21.075505   431 raft_consensus.cc:2272] T 7d4f631913d84fa2b0ec6feece991332 P 591e55d6b5724c309aeb00994e320bd0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.077350   431 tablet_replica.cc:333] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0: stopping tablet replica
I20260709 13:01:21.078336   431 raft_consensus.cc:2243] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.078800   431 raft_consensus.cc:2272] T 98633dc5342842c294d5ff93c4301c9d P 591e55d6b5724c309aeb00994e320bd0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.081792   431 tablet_replica.cc:333] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0: stopping tablet replica
I20260709 13:01:21.082451   431 raft_consensus.cc:2243] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.082980   431 raft_consensus.cc:2272] T ec42386aeae44255bf83eae6530c4f97 P 591e55d6b5724c309aeb00994e320bd0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.086148   431 tablet_replica.cc:333] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0: stopping tablet replica
I20260709 13:01:21.086757   431 raft_consensus.cc:2243] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:01:21.087663   431 raft_consensus.cc:2272] T 1df16b6ed89d4d1eb05931a646d3db26 P 591e55d6b5724c309aeb00994e320bd0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.089743   431 tablet_replica.cc:333] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0: stopping tablet replica
I20260709 13:01:21.090276   431 raft_consensus.cc:2243] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.090751   431 raft_consensus.cc:2272] T e0f4d34e991c4787b1c336afee69d01e P 591e55d6b5724c309aeb00994e320bd0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.126065   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 13:01:21.158766   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:01:21.202275  1112 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36343: connect: Connection refused (error 111) [suppressed 5 similar messages]
I20260709 13:01:21.205379   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:21.206359   431 tablet_replica.cc:333] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa: stopping tablet replica
I20260709 13:01:21.207723   431 raft_consensus.cc:2243] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:01:21.208082  1112 consensus_peers.cc:597] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa -> Peer 591e55d6b5724c309aeb00994e320bd0 (127.0.107.193:36343): Couldn't send request to peer 591e55d6b5724c309aeb00994e320bd0. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36343: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:21.208797   431 raft_consensus.cc:2272] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.211159   431 tablet_replica.cc:333] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa: stopping tablet replica
I20260709 13:01:21.211752   431 raft_consensus.cc:2243] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.212256   431 raft_consensus.cc:2272] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.214226   431 tablet_replica.cc:333] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa: stopping tablet replica
I20260709 13:01:21.214741   431 raft_consensus.cc:2243] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.215174   431 raft_consensus.cc:2272] T 74a921ab3514481087dcb4ae5bfd554a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.216813   431 tablet_replica.cc:333] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa: stopping tablet replica
I20260709 13:01:21.217351   431 raft_consensus.cc:2243] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.217840   431 raft_consensus.cc:2272] T e0f4d34e991c4787b1c336afee69d01e P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.220168   431 tablet_replica.cc:333] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa: stopping tablet replica
I20260709 13:01:21.220767   431 raft_consensus.cc:2243] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:21.221469   431 raft_consensus.cc:2272] T 98633dc5342842c294d5ff93c4301c9d P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.223913   431 tablet_replica.cc:333] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa: stopping tablet replica
I20260709 13:01:21.224550   431 raft_consensus.cc:2243] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.224987   431 raft_consensus.cc:2272] T 1df16b6ed89d4d1eb05931a646d3db26 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.226711   431 tablet_replica.cc:333] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa: stopping tablet replica
I20260709 13:01:21.227182   431 raft_consensus.cc:2243] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:01:21.227921   431 raft_consensus.cc:2272] T ec42386aeae44255bf83eae6530c4f97 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.230099   431 tablet_replica.cc:333] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa: stopping tablet replica
I20260709 13:01:21.230623   431 raft_consensus.cc:2243] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.231025   431 raft_consensus.cc:2272] T 8d86aeab58674fd2b92beda6f5fc0d7a P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.233072   431 tablet_replica.cc:333] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa: stopping tablet replica
I20260709 13:01:21.233531   431 raft_consensus.cc:2243] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.234025   431 raft_consensus.cc:2272] T 1c850d5de21d4c02b851d9925ef9c881 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.235529   431 tablet_replica.cc:333] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa: stopping tablet replica
I20260709 13:01:21.235956   431 raft_consensus.cc:2243] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.236363   431 raft_consensus.cc:2272] T 7d4f631913d84fa2b0ec6feece991332 P 24f31d4f10ad41eab5902ec7bb8ba5aa [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.267864   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:01:21.294816  1186 consensus_peers.cc:597] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e -> Peer 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299): Couldn't send request to peer 24f31d4f10ad41eab5902ec7bb8ba5aa. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44299: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:21.296429   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:01:21.306094  1186 consensus_peers.cc:597] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e -> Peer 24f31d4f10ad41eab5902ec7bb8ba5aa (127.0.107.194:44299): Couldn't send request to peer 24f31d4f10ad41eab5902ec7bb8ba5aa. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44299: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:21.335568   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:21.336535   431 tablet_replica.cc:333] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 406a72268a484d05920a71f05d71691e: stopping tablet replica
I20260709 13:01:21.337249   431 raft_consensus.cc:2243] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.337894   431 raft_consensus.cc:2272] T 0d8166a5fa5a4e298eaf4b3c7cc1c098 P 406a72268a484d05920a71f05d71691e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.341177   431 tablet_replica.cc:333] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e: stopping tablet replica
I20260709 13:01:21.342657   431 raft_consensus.cc:2243] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:01:21.343462   431 raft_consensus.cc:2272] T a3c20fcfe7b04c0fa8ca55c75fb57879 P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.345181   431 tablet_replica.cc:333] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e: stopping tablet replica
I20260709 13:01:21.345764   431 raft_consensus.cc:2243] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.346189   431 raft_consensus.cc:2272] T 74a921ab3514481087dcb4ae5bfd554a P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.348177   431 tablet_replica.cc:333] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e: stopping tablet replica
I20260709 13:01:21.349051   431 raft_consensus.cc:2243] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.349910   431 raft_consensus.cc:2272] T ec42386aeae44255bf83eae6530c4f97 P 406a72268a484d05920a71f05d71691e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.355122   431 tablet_replica.cc:333] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e: stopping tablet replica
I20260709 13:01:21.356403   431 raft_consensus.cc:2243] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.357362   431 raft_consensus.cc:2272] T 98633dc5342842c294d5ff93c4301c9d P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.361801   431 tablet_replica.cc:333] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e: stopping tablet replica
I20260709 13:01:21.362509   431 raft_consensus.cc:2243] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.363075   431 raft_consensus.cc:2272] T 1df16b6ed89d4d1eb05931a646d3db26 P 406a72268a484d05920a71f05d71691e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.365913   431 tablet_replica.cc:333] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e: stopping tablet replica
I20260709 13:01:21.366619   431 raft_consensus.cc:2243] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:21.367800   431 raft_consensus.cc:2272] T 8d86aeab58674fd2b92beda6f5fc0d7a P 406a72268a484d05920a71f05d71691e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.371739   431 tablet_replica.cc:333] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e: stopping tablet replica
I20260709 13:01:21.372411   431 raft_consensus.cc:2243] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.373894   431 raft_consensus.cc:2272] T 1c850d5de21d4c02b851d9925ef9c881 P 406a72268a484d05920a71f05d71691e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.376369   431 tablet_replica.cc:333] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e: stopping tablet replica
I20260709 13:01:21.377071   431 raft_consensus.cc:2243] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:01:21.378223   431 raft_consensus.cc:2272] T e0f4d34e991c4787b1c336afee69d01e P 406a72268a484d05920a71f05d71691e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.381417   431 tablet_replica.cc:333] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e: stopping tablet replica
I20260709 13:01:21.382182   431 raft_consensus.cc:2243] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.382741   431 raft_consensus.cc:2272] T 7d4f631913d84fa2b0ec6feece991332 P 406a72268a484d05920a71f05d71691e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.422463   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:01:21.449136   431 master.cc:562] Master@127.0.107.254:36129 shutting down...
I20260709 13:01:21.493824   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:01:21.494431   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.494819   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6a62ac7243d846fdaf49f6af6831b8e2: stopping tablet replica
I20260709 13:01:21.517364   431 master.cc:584] Master@127.0.107.254:36129 shutdown complete.
24f31d4f10ad41eab5902ec7bb8ba5aa  4
406a72268a484d05920a71f05d71691e  1
591e55d6b5724c309aeb00994e320bd0  5
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (11769 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:01:21.566823   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:38281
I20260709 13:01:21.568080   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:21.578229  1447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:21.578282  1449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:21.578420  1446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:21.578496   431 server_base.cc:1061] running on GCE node
I20260709 13:01:21.581295   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:21.581617   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:21.581864   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602081581843 us; error 0 us; skew 500 ppm
I20260709 13:01:21.582737   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:21.587957   431 webserver.cc:533] Webserver started at http://127.0.107.254:36409/ using document root <none> and password file <none>
I20260709 13:01:21.588716   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:21.588963   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:21.589283   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:21.590721   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/master-0-root/instance:
uuid: "220fdfc07a3c4ce28ca7333445633466"
format_stamp: "Formatted at 2026-07-09 13:01:21 on dist-test-slave-mxd3"
I20260709 13:01:21.599160   431 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.002s	sys 0.007s
I20260709 13:01:21.605801  1454 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:21.608151   431 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.003s	sys 0.001s
I20260709 13:01:21.610031   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/master-0-root
uuid: "220fdfc07a3c4ce28ca7333445633466"
format_stamp: "Formatted at 2026-07-09 13:01:21 on dist-test-slave-mxd3"
I20260709 13:01:21.610493   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:21.651448   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:21.653218   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:21.737721   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:38281
I20260709 13:01:21.737890  1505 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:38281 every 8 connection(s)
I20260709 13:01:21.744355  1506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:21.755203  1506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Bootstrap starting.
I20260709 13:01:21.761456  1506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:21.768419  1506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: No bootstrap required, opened a new log
I20260709 13:01:21.771209  1506 raft_consensus.cc:359] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220fdfc07a3c4ce28ca7333445633466" member_type: VOTER }
I20260709 13:01:21.771698  1506 raft_consensus.cc:385] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:21.772042  1506 raft_consensus.cc:740] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 220fdfc07a3c4ce28ca7333445633466, State: Initialized, Role: FOLLOWER
I20260709 13:01:21.772864  1506 consensus_queue.cc:260] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220fdfc07a3c4ce28ca7333445633466" member_type: VOTER }
I20260709 13:01:21.773411  1506 raft_consensus.cc:399] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:01:21.773808  1506 raft_consensus.cc:493] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:01:21.774147  1506 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:21.779340  1506 raft_consensus.cc:515] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220fdfc07a3c4ce28ca7333445633466" member_type: VOTER }
I20260709 13:01:21.780014  1506 leader_election.cc:304] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 220fdfc07a3c4ce28ca7333445633466; no voters: 
I20260709 13:01:21.781723  1506 leader_election.cc:290] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:01:21.782090  1509 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:21.783982  1509 raft_consensus.cc:697] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466 [term 1 LEADER]: Becoming Leader. State: Replica: 220fdfc07a3c4ce28ca7333445633466, State: Running, Role: LEADER
I20260709 13:01:21.784830  1509 consensus_queue.cc:237] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220fdfc07a3c4ce28ca7333445633466" member_type: VOTER }
I20260709 13:01:21.785522  1506 sys_catalog.cc:565] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:01:21.790822  1510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "220fdfc07a3c4ce28ca7333445633466" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220fdfc07a3c4ce28ca7333445633466" member_type: VOTER } }
I20260709 13:01:21.791862  1511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 220fdfc07a3c4ce28ca7333445633466. Latest consensus state: current_term: 1 leader_uuid: "220fdfc07a3c4ce28ca7333445633466" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220fdfc07a3c4ce28ca7333445633466" member_type: VOTER } }
I20260709 13:01:21.792878  1510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466 [sys.catalog]: This master's current role is: LEADER
I20260709 13:01:21.792886  1511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466 [sys.catalog]: This master's current role is: LEADER
I20260709 13:01:21.795743  1518 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:01:21.804121  1518 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:01:21.805207   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:01:21.815346  1518 catalog_manager.cc:1383] Generated new cluster ID: 2d988443e9334bf5b5b0484c2801ae89
I20260709 13:01:21.815701  1518 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:01:21.857582  1518 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:01:21.859228  1518 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:01:21.882577  1518 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Generated new TSK 0
I20260709 13:01:21.883335  1518 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:01:21.937350   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:21.946105  1527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:21.948064  1528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:21.949802  1530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:21.950734   431 server_base.cc:1061] running on GCE node
I20260709 13:01:21.951773   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:21.952016   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:21.952216   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602081952190 us; error 0 us; skew 500 ppm
I20260709 13:01:21.953258   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:21.956990   431 webserver.cc:533] Webserver started at http://127.0.107.193:40443/ using document root <none> and password file <none>
I20260709 13:01:21.958405   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:21.958659   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:21.960721   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:21.962219   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-0-root/instance:
uuid: "066ee0109aba4465ba17fa65e14ef504"
format_stamp: "Formatted at 2026-07-09 13:01:21 on dist-test-slave-mxd3"
I20260709 13:01:21.968294   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:01:21.975263  1535 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:21.976482   431 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.002s	sys 0.003s
I20260709 13:01:21.977106   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-0-root
uuid: "066ee0109aba4465ba17fa65e14ef504"
format_stamp: "Formatted at 2026-07-09 13:01:21 on dist-test-slave-mxd3"
I20260709 13:01:21.977533   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:21.993289   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:21.994855   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:21.997182   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:22.000913   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:22.001261   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:22.001565   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:22.001843   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:01:22.059986   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:39999
I20260709 13:01:22.060106  1597 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:39999 every 8 connection(s)
I20260709 13:01:22.067183   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:22.075786  1602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:22.077471  1603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:22.082039   431 server_base.cc:1061] running on GCE node
W20260709 13:01:22.083235  1605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:22.084558   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:22.084879   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:22.085068   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602082085046 us; error 0 us; skew 500 ppm
I20260709 13:01:22.085750   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:22.088711  1598 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38281
I20260709 13:01:22.089188  1598 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:22.089512   431 webserver.cc:533] Webserver started at http://127.0.107.194:37355/ using document root <none> and password file <none>
I20260709 13:01:22.090106  1598 heartbeater.cc:507] Master 127.0.107.254:38281 requested a full tablet report, sending...
I20260709 13:01:22.090219   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:22.090620   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:22.090965   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:22.092250   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-1-root/instance:
uuid: "f5f4b2707a9647e5bb99e065068dea2b"
format_stamp: "Formatted at 2026-07-09 13:01:22 on dist-test-slave-mxd3"
I20260709 13:01:22.092365  1471 ts_manager.cc:194] Registered new tserver with Master: 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:01:22.094370  1471 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38650
I20260709 13:01:22.099079   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.004s	sys 0.003s
I20260709 13:01:22.103471  1610 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:22.105093   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 13:01:22.105559   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-1-root
uuid: "f5f4b2707a9647e5bb99e065068dea2b"
format_stamp: "Formatted at 2026-07-09 13:01:22 on dist-test-slave-mxd3"
I20260709 13:01:22.106227   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:22.117933   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:22.119338   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:22.121416   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:22.124457   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:22.124724   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:22.125020   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:22.125202   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:22.183012   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:44293
I20260709 13:01:22.183188  1672 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:44293 every 8 connection(s)
I20260709 13:01:22.189046   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:22.197615  1676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:22.198959  1677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:22.206835   431 server_base.cc:1061] running on GCE node
W20260709 13:01:22.207849  1679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:22.208570  1673 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38281
I20260709 13:01:22.209040  1673 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:22.209502   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:01:22.210078  1673 heartbeater.cc:507] Master 127.0.107.254:38281 requested a full tablet report, sending...
W20260709 13:01:22.210214   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:22.210493   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602082210468 us; error 0 us; skew 500 ppm
I20260709 13:01:22.211467   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:22.213625  1471 ts_manager.cc:194] Registered new tserver with Master: f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:22.215374   431 webserver.cc:533] Webserver started at http://127.0.107.195:42951/ using document root <none> and password file <none>
I20260709 13:01:22.216919  1471 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38662
I20260709 13:01:22.217100   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:22.218127   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:22.218954   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:22.221115   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-2-root/instance:
uuid: "130bc5ff39b847b1b3204626f284f838"
format_stamp: "Formatted at 2026-07-09 13:01:22 on dist-test-slave-mxd3"
I20260709 13:01:22.227211   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.000s	sys 0.004s
I20260709 13:01:22.231906  1684 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:22.232993   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:01:22.233482   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-2-root
uuid: "130bc5ff39b847b1b3204626f284f838"
format_stamp: "Formatted at 2026-07-09 13:01:22 on dist-test-slave-mxd3"
I20260709 13:01:22.234180   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:22.268858   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:22.271349   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:22.273613   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:22.277943   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:22.278345   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:22.279057   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:22.279656   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:22.337989  1746 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:34553 every 8 connection(s)
I20260709 13:01:22.338014   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:34553
I20260709 13:01:22.356069  1747 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38281
I20260709 13:01:22.356824  1747 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:22.357686  1747 heartbeater.cc:507] Master 127.0.107.254:38281 requested a full tablet report, sending...
I20260709 13:01:22.360437  1471 ts_manager.cc:194] Registered new tserver with Master: 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:22.361616   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018362566s
I20260709 13:01:22.362360  1471 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38666
I20260709 13:01:22.398886  1471 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38676:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:01:22.405074  1471 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:01:22.640686  1638 tablet_service.cc:1511] Processing CreateTablet for tablet e89a11b4204e48568da409c196e33be4 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:01:22.641777  1636 tablet_service.cc:1511] Processing CreateTablet for tablet 6b98c1e123bd4894b2ab4096efb42cd8 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:01:22.644202  1561 tablet_service.cc:1511] Processing CreateTablet for tablet 6b98c1e123bd4894b2ab4096efb42cd8 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:01:22.642582  1638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e89a11b4204e48568da409c196e33be4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.645854  1561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b98c1e123bd4894b2ab4096efb42cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.646449  1635 tablet_service.cc:1511] Processing CreateTablet for tablet 7501558aa93542308c5fc758446387df (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:01:22.648037  1635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7501558aa93542308c5fc758446387df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.651010  1634 tablet_service.cc:1511] Processing CreateTablet for tablet dcc462cf3919460abb1ef504377a5382 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:01:22.654601  1557 tablet_service.cc:1511] Processing CreateTablet for tablet 380ceca851ad4a84b89cc2e55313d10b (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:01:22.656095  1557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 380ceca851ad4a84b89cc2e55313d10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.659194  1633 tablet_service.cc:1511] Processing CreateTablet for tablet eca30b43c26f420c965ee25f475581a0 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:01:22.644299  1559 tablet_service.cc:1511] Processing CreateTablet for tablet dcc462cf3919460abb1ef504377a5382 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:01:22.662752  1559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcc462cf3919460abb1ef504377a5382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.662920  1558 tablet_service.cc:1511] Processing CreateTablet for tablet eca30b43c26f420c965ee25f475581a0 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:01:22.664744  1558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eca30b43c26f420c965ee25f475581a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.667369  1631 tablet_service.cc:1511] Processing CreateTablet for tablet baa23f35bd3b4c2d99e3da0ff0f80e97 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:01:22.669808  1631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet baa23f35bd3b4c2d99e3da0ff0f80e97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.649576  1563 tablet_service.cc:1511] Processing CreateTablet for tablet e89a11b4204e48568da409c196e33be4 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:01:22.671634  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e89a11b4204e48568da409c196e33be4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.654637  1562 tablet_service.cc:1511] Processing CreateTablet for tablet ad4b6776963146fab5ee9487b6684e8c (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:01:22.667369  1556 tablet_service.cc:1511] Processing CreateTablet for tablet baa23f35bd3b4c2d99e3da0ff0f80e97 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:01:22.644481  1637 tablet_service.cc:1511] Processing CreateTablet for tablet ad4b6776963146fab5ee9487b6684e8c (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:01:22.677946  1637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad4b6776963146fab5ee9487b6684e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.678926  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet baa23f35bd3b4c2d99e3da0ff0f80e97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.669028  1632 tablet_service.cc:1511] Processing CreateTablet for tablet 380ceca851ad4a84b89cc2e55313d10b (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:01:22.680562  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad4b6776963146fab5ee9487b6684e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.656590  1560 tablet_service.cc:1511] Processing CreateTablet for tablet 7501558aa93542308c5fc758446387df (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:01:22.684845  1560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7501558aa93542308c5fc758446387df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.685048  1636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b98c1e123bd4894b2ab4096efb42cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.686013  1634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcc462cf3919460abb1ef504377a5382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.688156  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 380ceca851ad4a84b89cc2e55313d10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.691919  1633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eca30b43c26f420c965ee25f475581a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.729050  1765 tablet_bootstrap.cc:492] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:22.734342  1712 tablet_service.cc:1511] Processing CreateTablet for tablet e89a11b4204e48568da409c196e33be4 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:01:22.736238  1709 tablet_service.cc:1511] Processing CreateTablet for tablet 7501558aa93542308c5fc758446387df (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:01:22.734457  1710 tablet_service.cc:1511] Processing CreateTablet for tablet 6b98c1e123bd4894b2ab4096efb42cd8 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:01:22.734473  1711 tablet_service.cc:1511] Processing CreateTablet for tablet ad4b6776963146fab5ee9487b6684e8c (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:01:22.738993  1708 tablet_service.cc:1511] Processing CreateTablet for tablet dcc462cf3919460abb1ef504377a5382 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:01:22.743012  1707 tablet_service.cc:1511] Processing CreateTablet for tablet eca30b43c26f420c965ee25f475581a0 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:01:22.739421  1712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e89a11b4204e48568da409c196e33be4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.746729  1706 tablet_service.cc:1511] Processing CreateTablet for tablet 380ceca851ad4a84b89cc2e55313d10b (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:01:22.747308  1705 tablet_service.cc:1511] Processing CreateTablet for tablet baa23f35bd3b4c2d99e3da0ff0f80e97 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:01:22.748485  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 380ceca851ad4a84b89cc2e55313d10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.749219  1710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b98c1e123bd4894b2ab4096efb42cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.749974  1708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcc462cf3919460abb1ef504377a5382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.750809  1705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet baa23f35bd3b4c2d99e3da0ff0f80e97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.752173  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eca30b43c26f420c965ee25f475581a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.757057  1711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad4b6776963146fab5ee9487b6684e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.766866  1766 tablet_bootstrap.cc:492] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:22.769621  1709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7501558aa93542308c5fc758446387df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.790848  1634 tablet_service.cc:1511] Processing CreateTablet for tablet adc326ba3c1b4dc7aa89a3a071803a7e (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:01:22.792601  1634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adc326ba3c1b4dc7aa89a3a071803a7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.792339  1704 tablet_service.cc:1511] Processing CreateTablet for tablet adc326ba3c1b4dc7aa89a3a071803a7e (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:01:22.794024  1704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adc326ba3c1b4dc7aa89a3a071803a7e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:22.798683  1519 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:22.767453  1561 tablet_service.cc:1511] Processing CreateTablet for tablet adc326ba3c1b4dc7aa89a3a071803a7e (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:01:22.807402  1561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adc326ba3c1b4dc7aa89a3a071803a7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.815394  1765 tablet_bootstrap.cc:654] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:22.825031  1765 tablet_bootstrap.cc:492] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:22.831727  1766 tablet_bootstrap.cc:654] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:22.835084  1638 tablet_service.cc:1511] Processing CreateTablet for tablet f0ee72a7f576482aabe40ef837964708 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:01:22.825551  1765 ts_tablet_manager.cc:1403] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.097s	user 0.012s	sys 0.000s
I20260709 13:01:22.853500  1638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0ee72a7f576482aabe40ef837964708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.856359  1765 raft_consensus.cc:359] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:22.857328  1765 raft_consensus.cc:385] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:22.857755  1765 raft_consensus.cc:740] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:22.858057  1558 tablet_service.cc:1511] Processing CreateTablet for tablet f0ee72a7f576482aabe40ef837964708 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:01:22.838392  1768 tablet_bootstrap.cc:492] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:22.838096  1703 tablet_service.cc:1511] Processing CreateTablet for tablet f0ee72a7f576482aabe40ef837964708 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:01:22.844187  1560 tablet_service.cc:1511] Processing CreateTablet for tablet d2618e77160b44339d8499efcd1e8eca (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:01:22.858825  1765 consensus_queue.cc:260] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:22.862303  1703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0ee72a7f576482aabe40ef837964708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.859596  1558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0ee72a7f576482aabe40ef837964708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.904081  1560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2618e77160b44339d8499efcd1e8eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.905447  1635 tablet_service.cc:1511] Processing CreateTablet for tablet d2618e77160b44339d8499efcd1e8eca (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:01:22.949791  1556 tablet_service.cc:1511] Processing CreateTablet for tablet f50422a819a849c0a93a4a07c9af6ba0 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:01:22.949020  1712 tablet_service.cc:1511] Processing CreateTablet for tablet d2618e77160b44339d8499efcd1e8eca (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:01:22.952284  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f50422a819a849c0a93a4a07c9af6ba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.955534  1636 tablet_service.cc:1511] Processing CreateTablet for tablet f50422a819a849c0a93a4a07c9af6ba0 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:01:22.961050  1598 heartbeater.cc:499] Master 127.0.107.254:38281 was elected leader, sending a full tablet report...
I20260709 13:01:22.941246  1635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2618e77160b44339d8499efcd1e8eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.962500  1768 tablet_bootstrap.cc:654] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:22.962456  1636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f50422a819a849c0a93a4a07c9af6ba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.966966  1765 ts_tablet_manager.cc:1434] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.114s	user 0.019s	sys 0.011s
I20260709 13:01:22.956404  1712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2618e77160b44339d8499efcd1e8eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.983676  1765 tablet_bootstrap.cc:492] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:22.980044  1707 tablet_service.cc:1511] Processing CreateTablet for tablet f50422a819a849c0a93a4a07c9af6ba0 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:01:22.999801  1631 tablet_service.cc:1511] Processing CreateTablet for tablet e59deed37c414a929faeb85bdbaaa406 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:01:23.001821  1631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e59deed37c414a929faeb85bdbaaa406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:22.995008  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f50422a819a849c0a93a4a07c9af6ba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.023023  1766 tablet_bootstrap.cc:492] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:23.023546  1766 ts_tablet_manager.cc:1403] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.257s	user 0.012s	sys 0.005s
I20260709 13:01:23.026165  1709 tablet_service.cc:1511] Processing CreateTablet for tablet e59deed37c414a929faeb85bdbaaa406 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:01:23.027968  1632 tablet_service.cc:1511] Processing CreateTablet for tablet f81336f0bd194664ac95af1a897b4b18 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:01:23.029336  1709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e59deed37c414a929faeb85bdbaaa406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.029985  1708 tablet_service.cc:1511] Processing CreateTablet for tablet f81336f0bd194664ac95af1a897b4b18 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:01:23.031363  1708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f81336f0bd194664ac95af1a897b4b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.031982  1560 tablet_service.cc:1511] Processing CreateTablet for tablet 49c94b9e96df4808adbda99815b09c5f (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:01:23.033505  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f81336f0bd194664ac95af1a897b4b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.034057  1560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49c94b9e96df4808adbda99815b09c5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.036973  1765 tablet_bootstrap.cc:654] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:23.042356  1766 raft_consensus.cc:359] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.051410  1558 tablet_service.cc:1511] Processing CreateTablet for tablet f81336f0bd194664ac95af1a897b4b18 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:01:23.052451  1711 tablet_service.cc:1511] Processing CreateTablet for tablet 49c94b9e96df4808adbda99815b09c5f (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:01:23.053194  1558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f81336f0bd194664ac95af1a897b4b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.054353  1711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49c94b9e96df4808adbda99815b09c5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.053823  1563 tablet_service.cc:1511] Processing CreateTablet for tablet 1a01faf4c6c64b549e80adf03bba9f51 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:01:23.053984  1561 tablet_service.cc:1511] Processing CreateTablet for tablet e59deed37c414a929faeb85bdbaaa406 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:01:23.051410  1634 tablet_service.cc:1511] Processing CreateTablet for tablet 49c94b9e96df4808adbda99815b09c5f (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:01:23.046134  1766 raft_consensus.cc:385] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:23.069972  1637 tablet_service.cc:1511] Processing CreateTablet for tablet 1a01faf4c6c64b549e80adf03bba9f51 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:01:23.069933  1634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49c94b9e96df4808adbda99815b09c5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.074160  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a01faf4c6c64b549e80adf03bba9f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.080492  1561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e59deed37c414a929faeb85bdbaaa406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.082387  1703 tablet_service.cc:1511] Processing CreateTablet for tablet 1a01faf4c6c64b549e80adf03bba9f51 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:01:23.084029  1703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a01faf4c6c64b549e80adf03bba9f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.091799  1637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a01faf4c6c64b549e80adf03bba9f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.075385  1766 raft_consensus.cc:740] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:23.118088  1556 tablet_service.cc:1511] Processing CreateTablet for tablet 7e9b1a584a5f430a9a0e7e617064bc2c (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:01:23.119594  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e9b1a584a5f430a9a0e7e617064bc2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.123703  1766 consensus_queue.cc:260] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.131461  1768 tablet_bootstrap.cc:492] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:23.132004  1768 ts_tablet_manager.cc:1403] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.294s	user 0.017s	sys 0.018s
I20260709 13:01:23.134435  1635 tablet_service.cc:1511] Processing CreateTablet for tablet 7e9b1a584a5f430a9a0e7e617064bc2c (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:01:23.135766  1635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e9b1a584a5f430a9a0e7e617064bc2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.164579  1562 tablet_service.cc:1511] Processing CreateTablet for tablet 418152c2665a4b04be8a39c62a52f6ca (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:01:23.166658  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 418152c2665a4b04be8a39c62a52f6ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.167471  1708 tablet_service.cc:1511] Processing CreateTablet for tablet 7e9b1a584a5f430a9a0e7e617064bc2c (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:01:23.171427  1708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e9b1a584a5f430a9a0e7e617064bc2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.171448  1769 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.173026  1769 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.182927  1769 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:23.186830  1765 tablet_bootstrap.cc:492] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:23.187559  1765 ts_tablet_manager.cc:1403] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.204s	user 0.007s	sys 0.012s
I20260709 13:01:23.196146  1768 raft_consensus.cc:359] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.197190  1768 raft_consensus.cc:385] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:23.197600  1768 raft_consensus.cc:740] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:23.198514  1768 consensus_queue.cc:260] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.214196  1712 tablet_service.cc:1511] Processing CreateTablet for tablet 418152c2665a4b04be8a39c62a52f6ca (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:01:23.214625  1635 tablet_service.cc:1511] Processing CreateTablet for tablet 418152c2665a4b04be8a39c62a52f6ca (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:01:23.216325  1712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 418152c2665a4b04be8a39c62a52f6ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.216676  1635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 418152c2665a4b04be8a39c62a52f6ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.221964  1673 heartbeater.cc:499] Master 127.0.107.254:38281 was elected leader, sending a full tablet report...
I20260709 13:01:23.222497  1765 raft_consensus.cc:359] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.223356  1765 raft_consensus.cc:385] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:23.223673  1765 raft_consensus.cc:740] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:23.224537  1765 consensus_queue.cc:260] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.263626  1768 ts_tablet_manager.cc:1434] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.131s	user 0.015s	sys 0.002s
I20260709 13:01:23.265477  1768 tablet_bootstrap.cc:492] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:23.268024  1747 heartbeater.cc:499] Master 127.0.107.254:38281 was elected leader, sending a full tablet report...
I20260709 13:01:23.272166  1765 ts_tablet_manager.cc:1434] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.084s	user 0.002s	sys 0.005s
I20260709 13:01:23.273612  1768 tablet_bootstrap.cc:654] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:23.270375  1766 ts_tablet_manager.cc:1434] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.247s	user 0.013s	sys 0.031s
I20260709 13:01:23.277529  1766 tablet_bootstrap.cc:492] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:23.279052  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:01:23.291994  1538 leader_election.cc:343] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:23.286181  1765 tablet_bootstrap.cc:492] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:23.302084  1765 tablet_bootstrap.cc:654] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:23.312342  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:23.315127  1537 leader_election.cc:343] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:23.315822  1537 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:23.319175  1769 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:23.324100  1766 tablet_bootstrap.cc:654] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:23.335383  1637 tablet_service.cc:1511] Processing CreateTablet for tablet e7a5f5eedbd1406788cafbad6a95a4c7 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:01:23.337819  1712 tablet_service.cc:1511] Processing CreateTablet for tablet e7a5f5eedbd1406788cafbad6a95a4c7 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:01:23.338469  1637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7a5f5eedbd1406788cafbad6a95a4c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.339713  1712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7a5f5eedbd1406788cafbad6a95a4c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.341346  1773 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.342128  1773 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.343477  1558 tablet_service.cc:1511] Processing CreateTablet for tablet e7a5f5eedbd1406788cafbad6a95a4c7 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:01:23.345295  1558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7a5f5eedbd1406788cafbad6a95a4c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.349603  1563 tablet_service.cc:1511] Processing CreateTablet for tablet ca2582f1941a4c588f8ef51c03dfb492 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:01:23.351521  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca2582f1941a4c588f8ef51c03dfb492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.371951  1768 tablet_bootstrap.cc:492] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:23.390738  1768 ts_tablet_manager.cc:1403] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.125s	user 0.028s	sys 0.040s
I20260709 13:01:23.404107  1765 tablet_bootstrap.cc:492] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:23.405372  1765 ts_tablet_manager.cc:1403] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.119s	user 0.016s	sys 0.006s
I20260709 13:01:23.408268  1768 raft_consensus.cc:359] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.409051  1768 raft_consensus.cc:385] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:23.409468  1768 raft_consensus.cc:740] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:23.427624  1768 consensus_queue.cc:260] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.417891  1773 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:23.430605  1768 ts_tablet_manager.cc:1434] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.039s	user 0.010s	sys 0.000s
I20260709 13:01:23.418278  1765 raft_consensus.cc:359] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.431950  1765 raft_consensus.cc:385] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:23.432317  1765 raft_consensus.cc:740] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:23.434229  1765 consensus_queue.cc:260] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.435021  1632 tablet_service.cc:1511] Processing CreateTablet for tablet ca2582f1941a4c588f8ef51c03dfb492 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:01:23.442649  1712 tablet_service.cc:1511] Processing CreateTablet for tablet 08853c4c3f0b40ddb5a0c34f0c2ee95a (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:01:23.444193  1712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08853c4c3f0b40ddb5a0c34f0c2ee95a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.446514  1635 tablet_service.cc:1511] Processing CreateTablet for tablet 08853c4c3f0b40ddb5a0c34f0c2ee95a (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:01:23.447316  1765 ts_tablet_manager.cc:1434] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.041s	user 0.004s	sys 0.006s
I20260709 13:01:23.448559  1765 tablet_bootstrap.cc:492] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:23.448948  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca2582f1941a4c588f8ef51c03dfb492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.431710  1768 tablet_bootstrap.cc:492] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:23.436993  1562 tablet_service.cc:1511] Processing CreateTablet for tablet 08853c4c3f0b40ddb5a0c34f0c2ee95a (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:01:23.443758  1558 tablet_service.cc:1511] Processing CreateTablet for tablet ca833b0b78d74adc8ca86e91fc94f96f (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:01:23.450335  1631 tablet_service.cc:1511] Processing CreateTablet for tablet ca833b0b78d74adc8ca86e91fc94f96f (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:01:23.454888  1631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca833b0b78d74adc8ca86e91fc94f96f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.448948  1708 tablet_service.cc:1511] Processing CreateTablet for tablet ca2582f1941a4c588f8ef51c03dfb492 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:01:23.459074  1708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca2582f1941a4c588f8ef51c03dfb492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.460265  1635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08853c4c3f0b40ddb5a0c34f0c2ee95a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.464601  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:23.463856  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08853c4c3f0b40ddb5a0c34f0c2ee95a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:23.467554  1612 leader_election.cc:343] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:23.470219  1558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca833b0b78d74adc8ca86e91fc94f96f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.473515  1768 tablet_bootstrap.cc:654] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:23.483462  1769 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.485404  1769 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.492515  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:23.493759  1769 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:01:23.494436  1538 leader_election.cc:343] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:23.495209  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:23.497434  1537 leader_election.cc:343] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:23.498015  1537 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:23.500238  1703 tablet_service.cc:1511] Processing CreateTablet for tablet ca833b0b78d74adc8ca86e91fc94f96f (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:01:23.502242  1703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca833b0b78d74adc8ca86e91fc94f96f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.504447  1766 tablet_bootstrap.cc:492] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:23.505097  1766 ts_tablet_manager.cc:1403] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.228s	user 0.032s	sys 0.027s
I20260709 13:01:23.508769  1766 raft_consensus.cc:359] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.509719  1765 tablet_bootstrap.cc:654] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:23.505059  1769 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:23.509878  1766 raft_consensus.cc:385] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:23.510995  1766 raft_consensus.cc:740] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:23.511943  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:23.512197  1766 consensus_queue.cc:260] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:23.514501  1611 leader_election.cc:343] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:23.515159  1766 ts_tablet_manager.cc:1434] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:01:23.515064  1611 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:23.525224  1766 tablet_bootstrap.cc:492] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:23.528564  1558 tablet_service.cc:1511] Processing CreateTablet for tablet e5c79254e6ff49639cf09f663490dfd7 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:01:23.531946  1558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5c79254e6ff49639cf09f663490dfd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.538311  1711 tablet_service.cc:1511] Processing CreateTablet for tablet e5c79254e6ff49639cf09f663490dfd7 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:01:23.539779  1711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5c79254e6ff49639cf09f663490dfd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.540536  1561 tablet_service.cc:1511] Processing CreateTablet for tablet 56082a5ff84643f0ba368ceba0524623 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:01:23.546610  1766 tablet_bootstrap.cc:654] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:23.558969  1561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56082a5ff84643f0ba368ceba0524623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.569015  1773 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:01:23.581066  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 66795.6us
I20260709 13:01:23.599116  1631 tablet_service.cc:1511] Processing CreateTablet for tablet e5c79254e6ff49639cf09f663490dfd7 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:01:23.601852  1631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5c79254e6ff49639cf09f663490dfd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.599116  1637 tablet_service.cc:1511] Processing CreateTablet for tablet 56082a5ff84643f0ba368ceba0524623 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:01:23.612507  1766 tablet_bootstrap.cc:492] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:23.606688  1773 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.615485  1637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56082a5ff84643f0ba368ceba0524623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.619889  1768 tablet_bootstrap.cc:492] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:23.620750  1768 ts_tablet_manager.cc:1403] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.189s	user 0.023s	sys 0.018s
I20260709 13:01:23.625061  1768 raft_consensus.cc:359] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.627624  1768 raft_consensus.cc:385] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:23.628154  1768 raft_consensus.cc:740] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:23.620715  1773 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.634426  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:23.635078  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:23.636893  1612 leader_election.cc:343] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:23.638319  1777 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.638875  1777 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:23.643985  1611 leader_election.cc:343] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:23.644168  1777 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:23.646044  1563 tablet_service.cc:1511] Processing CreateTablet for tablet 5bd2b69727c743cfbdc52253e16279ce (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:01:23.647462  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bd2b69727c743cfbdc52253e16279ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.647840  1712 tablet_service.cc:1511] Processing CreateTablet for tablet 56082a5ff84643f0ba368ceba0524623 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:01:23.644516  1611 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:23.663621  1712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56082a5ff84643f0ba368ceba0524623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.664232  1562 tablet_service.cc:1511] Processing CreateTablet for tablet d747d0b326fa4be595dbd31d3638164d (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:01:23.661847  1773 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:23.665835  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d747d0b326fa4be595dbd31d3638164d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.673877  1784 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:23.676203  1635 tablet_service.cc:1511] Processing CreateTablet for tablet 5bd2b69727c743cfbdc52253e16279ce (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:01:23.678149  1635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bd2b69727c743cfbdc52253e16279ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.670034  1768 consensus_queue.cc:260] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.666571  1766 ts_tablet_manager.cc:1403] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.146s	user 0.014s	sys 0.014s
I20260709 13:01:23.684320  1765 tablet_bootstrap.cc:492] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:23.688627  1636 tablet_service.cc:1511] Processing CreateTablet for tablet d747d0b326fa4be595dbd31d3638164d (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:01:23.697144  1765 ts_tablet_manager.cc:1403] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.249s	user 0.017s	sys 0.020s
I20260709 13:01:23.697870  1636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d747d0b326fa4be595dbd31d3638164d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.698668  1560 tablet_service.cc:1511] Processing CreateTablet for tablet e87427e2b27d4f89b7953163626fd8e4 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:01:23.700680  1560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e87427e2b27d4f89b7953163626fd8e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.705961  1765 raft_consensus.cc:359] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.706766  1765 raft_consensus.cc:385] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:23.707561  1765 raft_consensus.cc:740] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:23.710506  1766 raft_consensus.cc:359] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.711477  1766 raft_consensus.cc:385] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:23.711808  1766 raft_consensus.cc:740] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:23.712684  1766 consensus_queue.cc:260] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.716255  1768 ts_tablet_manager.cc:1434] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.095s	user 0.012s	sys 0.000s
I20260709 13:01:23.717082  1634 tablet_service.cc:1511] Processing CreateTablet for tablet e87427e2b27d4f89b7953163626fd8e4 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:01:23.718712  1634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e87427e2b27d4f89b7953163626fd8e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.720849  1556 tablet_service.cc:1511] Processing CreateTablet for tablet 0df07602233941598bee7b19607b2e3b (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:01:23.722362  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df07602233941598bee7b19607b2e3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.708703  1765 consensus_queue.cc:260] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.731981  1765 ts_tablet_manager.cc:1434] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.034s	user 0.009s	sys 0.000s
I20260709 13:01:23.735620  1703 tablet_service.cc:1511] Processing CreateTablet for tablet 5bd2b69727c743cfbdc52253e16279ce (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:01:23.736594  1708 tablet_service.cc:1511] Processing CreateTablet for tablet e87427e2b27d4f89b7953163626fd8e4 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:01:23.737159  1703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bd2b69727c743cfbdc52253e16279ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.737926  1708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e87427e2b27d4f89b7953163626fd8e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.735620  1711 tablet_service.cc:1511] Processing CreateTablet for tablet d747d0b326fa4be595dbd31d3638164d (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:01:23.739506  1711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d747d0b326fa4be595dbd31d3638164d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.741554  1768 tablet_bootstrap.cc:492] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:23.745465  1777 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.754035  1765 tablet_bootstrap.cc:492] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:23.746675  1777 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.771932  1777 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:23.777211  1766 ts_tablet_manager.cc:1434] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.075s	user 0.011s	sys 0.002s
I20260709 13:01:23.789204  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:23.792424  1766 tablet_bootstrap.cc:492] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:23.791905  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:23.800455  1707 tablet_service.cc:1511] Processing CreateTablet for tablet 0df07602233941598bee7b19607b2e3b (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
W20260709 13:01:23.802471  1519 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:01:23.803397  1685 leader_election.cc:343] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:23.794147  1637 tablet_service.cc:1511] Processing CreateTablet for tablet 0df07602233941598bee7b19607b2e3b (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:01:23.809968  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df07602233941598bee7b19607b2e3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.811978  1631 tablet_service.cc:1511] Processing CreateTablet for tablet 48ee6d32e41946dc9cf53a3d62cdb9ae (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:01:23.813743  1631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48ee6d32e41946dc9cf53a3d62cdb9ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.794279  1572 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:23.815258  1685 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:23.816547  1777 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:23.817058  1777 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.817459  1777 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:23.822006  1766 tablet_bootstrap.cc:654] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:23.822269  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:23.807325  1562 tablet_service.cc:1511] Processing CreateTablet for tablet 48ee6d32e41946dc9cf53a3d62cdb9ae (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:01:23.825295  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:01:23.828202  1687 leader_election.cc:343] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:23.828445  1712 tablet_service.cc:1511] Processing CreateTablet for tablet 48ee6d32e41946dc9cf53a3d62cdb9ae (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:01:23.828629  1768 tablet_bootstrap.cc:654] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:23.829226  1687 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:23.829926  1712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48ee6d32e41946dc9cf53a3d62cdb9ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.830793  1637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df07602233941598bee7b19607b2e3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.832935  1765 tablet_bootstrap.cc:654] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:23.838189  1563 tablet_service.cc:1511] Processing CreateTablet for tablet bbc4aafd02974a29b0c9b332b55c978c (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:01:23.839788  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbc4aafd02974a29b0c9b332b55c978c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.842021  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48ee6d32e41946dc9cf53a3d62cdb9ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.852443  1777 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.854998  1635 tablet_service.cc:1511] Processing CreateTablet for tablet bbc4aafd02974a29b0c9b332b55c978c (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:01:23.859197  1635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbc4aafd02974a29b0c9b332b55c978c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.858706  1709 tablet_service.cc:1511] Processing CreateTablet for tablet bbc4aafd02974a29b0c9b332b55c978c (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:01:23.860858  1709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbc4aafd02974a29b0c9b332b55c978c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.870250  1777 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:23.871263  1777 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:23.876189  1636 tablet_service.cc:1511] Processing CreateTablet for tablet 45c609109d4a4c40a3e74ff0917c7958 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:01:23.866578  1556 tablet_service.cc:1511] Processing CreateTablet for tablet 45c609109d4a4c40a3e74ff0917c7958 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:01:23.877825  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:23.878604  1572 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:23.879858  1703 tablet_service.cc:1511] Processing CreateTablet for tablet 45c609109d4a4c40a3e74ff0917c7958 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:01:23.881968  1636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45c609109d4a4c40a3e74ff0917c7958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.886140  1703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45c609109d4a4c40a3e74ff0917c7958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.905185  1769 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.903631  1560 tablet_service.cc:1511] Processing CreateTablet for tablet 7ae86dda8d6f4caf8f08a1607327d9f9 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:01:23.908193  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45c609109d4a4c40a3e74ff0917c7958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.907974  1769 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.909446  1560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ae86dda8d6f4caf8f08a1607327d9f9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:23.904160  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 76153.6us
I20260709 13:01:23.910521  1768 tablet_bootstrap.cc:492] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:23.910704  1769 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:01:23.911140  1687 leader_election.cc:343] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:23.913262  1634 tablet_service.cc:1511] Processing CreateTablet for tablet 7ae86dda8d6f4caf8f08a1607327d9f9 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:01:23.914848  1634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ae86dda8d6f4caf8f08a1607327d9f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.914705  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:23.911167  1768 ts_tablet_manager.cc:1403] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.170s	user 0.008s	sys 0.020s
I20260709 13:01:23.917407  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:23.919049  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:01:23.920776  1538 leader_election.cc:343] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:23.910936  1766 tablet_bootstrap.cc:492] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:23.921860  1766 ts_tablet_manager.cc:1403] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.130s	user 0.011s	sys 0.005s
I20260709 13:01:23.923359  1711 tablet_service.cc:1511] Processing CreateTablet for tablet 7ae86dda8d6f4caf8f08a1607327d9f9 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:01:23.925518  1561 tablet_service.cc:1511] Processing CreateTablet for tablet 58b222526a1a4b52bdcda825e15c46f9 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
W20260709 13:01:23.927363  1537 leader_election.cc:343] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:23.928344  1561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58b222526a1a4b52bdcda825e15c46f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.928082  1537 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:23.929951  1766 raft_consensus.cc:359] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.930864  1766 raft_consensus.cc:385] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:23.930980  1769 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:23.931252  1766 raft_consensus.cc:740] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:23.921396  1765 tablet_bootstrap.cc:492] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:23.932221  1765 ts_tablet_manager.cc:1403] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.178s	user 0.013s	sys 0.009s
I20260709 13:01:23.932497  1766 consensus_queue.cc:260] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.934139  1711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ae86dda8d6f4caf8f08a1607327d9f9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:23.932499  1687 leader_election.cc:343] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:23.944092  1765 raft_consensus.cc:359] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.944932  1765 raft_consensus.cc:385] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:23.950495  1637 tablet_service.cc:1511] Processing CreateTablet for tablet 58b222526a1a4b52bdcda825e15c46f9 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:01:23.952239  1637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58b222526a1a4b52bdcda825e15c46f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.945453  1765 raft_consensus.cc:740] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:23.958962  1766 ts_tablet_manager.cc:1434] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 13:01:23.959476  1572 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:23.962484  1685 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:23.965020  1791 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:23.966799  1791 raft_consensus.cc:697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:01:23.967078  1712 tablet_service.cc:1511] Processing CreateTablet for tablet 58b222526a1a4b52bdcda825e15c46f9 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:01:23.967923  1791 consensus_queue.cc:237] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.968729  1712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58b222526a1a4b52bdcda825e15c46f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.970374  1766 tablet_bootstrap.cc:492] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:23.970888  1558 tablet_service.cc:1511] Processing CreateTablet for tablet c6485a5991c348019b9779be81656a45 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:01:23.973582  1558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6485a5991c348019b9779be81656a45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.976105  1768 raft_consensus.cc:359] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.978215  1765 consensus_queue.cc:260] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:23.994656  1766 tablet_bootstrap.cc:654] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:23.977501  1768 raft_consensus.cc:385] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:23.998152  1768 raft_consensus.cc:740] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:23.999414  1631 tablet_service.cc:1511] Processing CreateTablet for tablet c6485a5991c348019b9779be81656a45 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:01:24.001004  1631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6485a5991c348019b9779be81656a45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.004253  1765 ts_tablet_manager.cc:1434] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.072s	user 0.009s	sys 0.002s
I20260709 13:01:23.999089  1768 consensus_queue.cc:260] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.020449  1777 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.020960  1563 tablet_service.cc:1511] Processing CreateTablet for tablet d3ff300e7f45424ea3bcd7c816082956 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:01:24.022688  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3ff300e7f45424ea3bcd7c816082956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.023195  1712 tablet_service.cc:1511] Processing CreateTablet for tablet c6485a5991c348019b9779be81656a45 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:01:24.024263  1634 tablet_service.cc:1511] Processing CreateTablet for tablet d3ff300e7f45424ea3bcd7c816082956 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:01:24.025046  1712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6485a5991c348019b9779be81656a45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.025933  1634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3ff300e7f45424ea3bcd7c816082956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.026665  1766 tablet_bootstrap.cc:492] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:24.027174  1766 ts_tablet_manager.cc:1403] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.057s	user 0.009s	sys 0.004s
I20260709 13:01:24.021458  1777 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.032487  1777 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:24.032946  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:24.033098  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:24.033746  1572 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
W20260709 13:01:24.034916  1687 leader_election.cc:343] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:24.035732  1685 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:24.037137  1777 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:24.038947  1777 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.039906  1777 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:24.033994  1766 raft_consensus.cc:359] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.040591  1766 raft_consensus.cc:385] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:24.040925  1784 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.041003  1766 raft_consensus.cc:740] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:24.041523  1784 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.044234  1711 tablet_service.cc:1511] Processing CreateTablet for tablet d3ff300e7f45424ea3bcd7c816082956 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:01:24.045447  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:24.046056  1711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3ff300e7f45424ea3bcd7c816082956. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:24.047235  1612 leader_election.cc:343] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:24.042025  1766 consensus_queue.cc:260] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.049191  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:24.049950  1572 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:24.051353  1611 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:24.052372  1773 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:24.052922  1773 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.053303  1773 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:24.055162  1469 catalog_manager.cc:5697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:24.058194  1766 ts_tablet_manager.cc:1434] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.030s	user 0.003s	sys 0.006s
I20260709 13:01:24.062726  1784 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:24.069043  1784 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.069609  1784 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.070150  1766 tablet_bootstrap.cc:492] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:24.062610  1765 tablet_bootstrap.cc:492] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:24.064388  1768 ts_tablet_manager.cc:1434] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.148s	user 0.008s	sys 0.001s
I20260709 13:01:24.078899  1769 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.079804  1768 tablet_bootstrap.cc:492] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:24.079644  1769 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.085412  1791 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.087028  1791 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.109920  1556 tablet_service.cc:1511] Processing CreateTablet for tablet 27aa13d58e6d4987b2e34613996ef421 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:01:24.110240  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:24.110740  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:24.111193  1647 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:24.111778  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27aa13d58e6d4987b2e34613996ef421. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:24.112902  1537 leader_election.cc:343] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:24.123785  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:24.125341  1722 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:24.126547  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:24.126825  1612 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:24.113358  1773 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.128337  1796 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:01:24.128513  1685 leader_election.cc:343] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:24.128819  1796 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.129163  1712 tablet_service.cc:1511] Processing CreateTablet for tablet 27aa13d58e6d4987b2e34613996ef421 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:01:24.129479  1558 tablet_service.cc:1511] Processing CreateTablet for tablet 59d236b930214b1f91fb55ece6fecd68 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:01:24.130328  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:24.131213  1558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59d236b930214b1f91fb55ece6fecd68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.130592  1712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27aa13d58e6d4987b2e34613996ef421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.125007  1538 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:24.135917  1768 tablet_bootstrap.cc:654] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:24.109946  1769 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:24.138013  1769 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:24.138449  1769 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.138939  1769 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:24.140436  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:24.129194  1796 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:24.143013  1612 leader_election.cc:343] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:24.144394  1777 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.145381  1773 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:24.148011  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:24.148665  1572 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:24.149255  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:24.150398  1573 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:24.150746  1631 tablet_service.cc:1511] Processing CreateTablet for tablet 27aa13d58e6d4987b2e34613996ef421 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
W20260709 13:01:24.149348  1611 leader_election.cc:343] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:24.162746  1632 tablet_service.cc:1511] Processing CreateTablet for tablet 59d236b930214b1f91fb55ece6fecd68 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:01:24.164785  1791 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:24.142390  1784 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:24.169872  1777 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:24.170639  1572 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:24.171968  1685 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:24.172756  1766 tablet_bootstrap.cc:654] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:24.174348  1709 tablet_service.cc:1511] Processing CreateTablet for tablet 59d236b930214b1f91fb55ece6fecd68 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:01:24.175827  1709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59d236b930214b1f91fb55ece6fecd68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.180337  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:24.181349  1631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27aa13d58e6d4987b2e34613996ef421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.184309  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59d236b930214b1f91fb55ece6fecd68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.184839  1768 tablet_bootstrap.cc:492] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:24.180469  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:24.185449  1768 ts_tablet_manager.cc:1403] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.106s	user 0.004s	sys 0.008s
I20260709 13:01:24.178814  1561 tablet_service.cc:1511] Processing CreateTablet for tablet 1d9390db02bf4ea689cbc36b1b946a14 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:01:24.187162  1561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d9390db02bf4ea689cbc36b1b946a14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.187578  1765 tablet_bootstrap.cc:654] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:24.191336  1687 leader_election.cc:343] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:24.193271  1687 leader_election.cc:343] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:24.194067  1687 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:24.195343  1791 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:24.197218  1703 tablet_service.cc:1511] Processing CreateTablet for tablet 1d9390db02bf4ea689cbc36b1b946a14 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:01:24.198346  1635 tablet_service.cc:1511] Processing CreateTablet for tablet 1d9390db02bf4ea689cbc36b1b946a14 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:01:24.199849  1635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d9390db02bf4ea689cbc36b1b946a14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.205693  1562 tablet_service.cc:1511] Processing CreateTablet for tablet 12ac45e0e48449a98fb89704bae3de27 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
W20260709 13:01:24.178135  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 65257.9us
I20260709 13:01:24.207374  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12ac45e0e48449a98fb89704bae3de27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.207522  1768 raft_consensus.cc:359] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.208354  1768 raft_consensus.cc:385] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:24.211033  1777 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:24.213989  1703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d9390db02bf4ea689cbc36b1b946a14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.208679  1768 raft_consensus.cc:740] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:24.223654  1573 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:24.224221  1768 consensus_queue.cc:260] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.228829  1791 raft_consensus.cc:697] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:01:24.239161  1791 consensus_queue.cc:237] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.240041  1712 tablet_service.cc:1511] Processing CreateTablet for tablet 12ac45e0e48449a98fb89704bae3de27 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:01:24.241503  1712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12ac45e0e48449a98fb89704bae3de27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.240269  1563 tablet_service.cc:1511] Processing CreateTablet for tablet 9688a8a5839c4bb0b5a0196390670b14 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:01:24.247707  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9688a8a5839c4bb0b5a0196390670b14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.252646  1711 tablet_service.cc:1511] Processing CreateTablet for tablet 9688a8a5839c4bb0b5a0196390670b14 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:01:24.253067  1766 tablet_bootstrap.cc:492] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:24.256407  1766 ts_tablet_manager.cc:1403] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.186s	user 0.012s	sys 0.012s
I20260709 13:01:24.256422  1768 ts_tablet_manager.cc:1434] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.071s	user 0.008s	sys 0.001s
I20260709 13:01:24.258771  1707 tablet_service.cc:1511] Processing CreateTablet for tablet 15e07cb6db0d46a78611f24c17d04cc9 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:01:24.254555  1560 tablet_service.cc:1511] Processing CreateTablet for tablet 15e07cb6db0d46a78611f24c17d04cc9 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:01:24.260246  1768 tablet_bootstrap.cc:492] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:24.260829  1560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15e07cb6db0d46a78611f24c17d04cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.260550  1558 tablet_service.cc:1511] Processing CreateTablet for tablet 0764c6a89eca47f39e28feb56e2123c7 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:01:24.246033  1796 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.262121  1558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0764c6a89eca47f39e28feb56e2123c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.255885  1711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9688a8a5839c4bb0b5a0196390670b14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.266651  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15e07cb6db0d46a78611f24c17d04cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.268595  1766 raft_consensus.cc:359] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.269526  1766 raft_consensus.cc:385] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:24.269922  1766 raft_consensus.cc:740] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:24.270757  1766 consensus_queue.cc:260] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.272900  1769 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.274420  1766 ts_tablet_manager.cc:1434] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:01:24.233321  1611 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:24.277825  1796 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:24.279707  1708 tablet_service.cc:1511] Processing CreateTablet for tablet 0764c6a89eca47f39e28feb56e2123c7 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:01:24.281347  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:24.284720  1557 tablet_service.cc:1511] Processing CreateTablet for tablet f59ca0c8e4c0491480502ebf944ba503 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:01:24.279757  1765 tablet_bootstrap.cc:492] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:24.306335  1557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f59ca0c8e4c0491480502ebf944ba503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.308262  1559 tablet_service.cc:1511] Processing CreateTablet for tablet 3ee0e81a514641709317e983fcf96fe4 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:01:24.309749  1708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0764c6a89eca47f39e28feb56e2123c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.311123  1559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ee0e81a514641709317e983fcf96fe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.312214  1706 tablet_service.cc:1511] Processing CreateTablet for tablet f59ca0c8e4c0491480502ebf944ba503 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:01:24.295892  1722 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:24.314270  1784 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.315305  1784 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:24.318188  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 74324us
I20260709 13:01:24.320986  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:24.302899  1796 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:24.326593  1636 tablet_service.cc:1511] Processing CreateTablet for tablet 9688a8a5839c4bb0b5a0196390670b14 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:01:24.326961  1637 tablet_service.cc:1511] Processing CreateTablet for tablet 15e07cb6db0d46a78611f24c17d04cc9 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:01:24.327617  1796 raft_consensus.cc:697] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:01:24.328524  1636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9688a8a5839c4bb0b5a0196390670b14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.330338  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f59ca0c8e4c0491480502ebf944ba503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.331269  1637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15e07cb6db0d46a78611f24c17d04cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.332772  1769 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:24.334646  1784 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:24.334865  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:24.341627  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:24.342521  1648 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:24.343024  1638 tablet_service.cc:1511] Processing CreateTablet for tablet 0764c6a89eca47f39e28feb56e2123c7 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
W20260709 13:01:24.346369  1537 leader_election.cc:343] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:24.345367  1638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0764c6a89eca47f39e28feb56e2123c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.358566  1648 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:24.360208  1538 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:24.361488  1769 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:24.346453  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:24.328914  1796 consensus_queue.cc:237] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:24.363648  1612 leader_election.cc:343] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:24.363235  1769 raft_consensus.cc:697] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:24.365329  1769 consensus_queue.cc:237] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.375114  1631 tablet_service.cc:1511] Processing CreateTablet for tablet 12ac45e0e48449a98fb89704bae3de27 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:01:24.379235  1766 tablet_bootstrap.cc:492] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:24.348372  1765 ts_tablet_manager.cc:1403] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.276s	user 0.012s	sys 0.015s
I20260709 13:01:24.390116  1631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12ac45e0e48449a98fb89704bae3de27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.392879  1709 tablet_service.cc:1511] Processing CreateTablet for tablet 3ee0e81a514641709317e983fcf96fe4 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:01:24.394966  1709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ee0e81a514641709317e983fcf96fe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.396054  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:24.396716  1712 tablet_service.cc:1511] Processing CreateTablet for tablet a51c23e1c3ae4bdbbb5f21f1607da772 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:01:24.397985  1712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a51c23e1c3ae4bdbbb5f21f1607da772. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:24.398588  1611 leader_election.cc:343] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:24.399354  1611 leader_election.cc:343] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:24.400708  1632 tablet_service.cc:1511] Processing CreateTablet for tablet f59ca0c8e4c0491480502ebf944ba503 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
W20260709 13:01:24.407335  1455 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 66867.7us
I20260709 13:01:24.408387  1765 raft_consensus.cc:359] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.415400  1765 raft_consensus.cc:385] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:24.432972  1704 tablet_service.cc:1511] Processing CreateTablet for tablet 08cf0535992b4db395d567393667593b (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:01:24.435048  1704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08cf0535992b4db395d567393667593b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.433248  1561 tablet_service.cc:1511] Processing CreateTablet for tablet a51c23e1c3ae4bdbbb5f21f1607da772 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:01:24.411682  1611 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:24.438668  1561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a51c23e1c3ae4bdbbb5f21f1607da772. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.440171  1556 tablet_service.cc:1511] Processing CreateTablet for tablet 08cf0535992b4db395d567393667593b (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:01:24.428097  1765 raft_consensus.cc:740] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:24.484599  1765 consensus_queue.cc:260] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.450765  1634 tablet_service.cc:1511] Processing CreateTablet for tablet 3ee0e81a514641709317e983fcf96fe4 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:01:24.453812  1803 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.451689  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f59ca0c8e4c0491480502ebf944ba503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.486588  1803 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.489076  1803 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:24.458354  1633 tablet_service.cc:1511] Processing CreateTablet for tablet 08cf0535992b4db395d567393667593b (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:01:24.490681  1633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08cf0535992b4db395d567393667593b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.491997  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:24.459726  1796 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:01:24.459815  1687 consensus_peers.cc:597] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:24.493893  1573 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:24.494565  1634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ee0e81a514641709317e983fcf96fe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.460948  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08cf0535992b4db395d567393667593b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.496477  1685 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:24.497706  1803 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:24.498164  1803 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.498545  1803 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:24.464671  1766 tablet_bootstrap.cc:654] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:24.444940  1722 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:24.503328  1612 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:01:24.505244  1796 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:24.505851  1796 raft_consensus.cc:697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:01:24.462459  1635 tablet_service.cc:1511] Processing CreateTablet for tablet a51c23e1c3ae4bdbbb5f21f1607da772 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:01:24.497867  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:24.510241  1648 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:24.508278  1796 consensus_queue.cc:237] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.451665  1768 tablet_bootstrap.cc:700] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838: Time spent opening tablet: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:01:24.518657  1768 tablet_bootstrap.cc:654] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:24.522514  1635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a51c23e1c3ae4bdbbb5f21f1607da772. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:24.539944  1457 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 79297.3us
I20260709 13:01:24.544167  1791 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:24.509541  1469 catalog_manager.cc:5697] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:24.548161  1765 ts_tablet_manager.cc:1434] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.161s	user 0.007s	sys 0.002s
W20260709 13:01:24.555653  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 43561.5us
I20260709 13:01:24.565881  1765 tablet_bootstrap.cc:492] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:24.568799  1707 tablet_service.cc:1511] Processing CreateTablet for tablet 8f06439379e24c5ebedb229b007236c2 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:01:24.570740  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f06439379e24c5ebedb229b007236c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.580291  1630 tablet_service.cc:1511] Processing CreateTablet for tablet 97987f9622cb4f52a1b74b4ee6bfdfdb (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:01:24.582551  1766 tablet_bootstrap.cc:492] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:24.583488  1766 ts_tablet_manager.cc:1403] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.205s	user 0.021s	sys 0.000s
I20260709 13:01:24.569118  1636 tablet_service.cc:1511] Processing CreateTablet for tablet 8f06439379e24c5ebedb229b007236c2 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:01:24.582778  1630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97987f9622cb4f52a1b74b4ee6bfdfdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.584615  1562 tablet_service.cc:1511] Processing CreateTablet for tablet 97987f9622cb4f52a1b74b4ee6bfdfdb (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:01:24.588945  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97987f9622cb4f52a1b74b4ee6bfdfdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.574085  1563 tablet_service.cc:1511] Processing CreateTablet for tablet 8f06439379e24c5ebedb229b007236c2 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:01:24.588807  1636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f06439379e24c5ebedb229b007236c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.597894  1470 catalog_manager.cc:5697] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:24.606082  1471 catalog_manager.cc:5697] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:24.613615  1769 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.616688  1769 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.618239  1560 tablet_service.cc:1511] Processing CreateTablet for tablet bddd1ebc9af24fa282580ab028d643e0 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:01:24.620075  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:24.620833  1648 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:24.622453  1711 tablet_service.cc:1511] Processing CreateTablet for tablet 97987f9622cb4f52a1b74b4ee6bfdfdb (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:01:24.623935  1711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97987f9622cb4f52a1b74b4ee6bfdfdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.624257  1629 tablet_service.cc:1511] Processing CreateTablet for tablet bddd1ebc9af24fa282580ab028d643e0 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:01:24.627154  1558 tablet_service.cc:1511] Processing CreateTablet for tablet dbbf62b8237649f584432aa4a9d96ec6 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:01:24.628600  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:24.629873  1798 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:24.630261  1798 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.630597  1798 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:24.631567  1803 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.634287  1706 tablet_service.cc:1511] Processing CreateTablet for tablet dbbf62b8237649f584432aa4a9d96ec6 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:01:24.637482  1712 tablet_service.cc:1511] Processing CreateTablet for tablet bddd1ebc9af24fa282580ab028d643e0 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
W20260709 13:01:24.646778  1687 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:24.647426  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbbf62b8237649f584432aa4a9d96ec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.648540  1629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bddd1ebc9af24fa282580ab028d643e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.667026  1631 tablet_service.cc:1511] Processing CreateTablet for tablet dbbf62b8237649f584432aa4a9d96ec6 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:01:24.672129  1631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbbf62b8237649f584432aa4a9d96ec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.665508  1712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bddd1ebc9af24fa282580ab028d643e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.608145  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f06439379e24c5ebedb229b007236c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.679462  1555 tablet_service.cc:1511] Processing CreateTablet for tablet a8d17fad9f114b0e98d2af0a989c3185 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:01:24.700088  1637 tablet_service.cc:1511] Processing CreateTablet for tablet a8d17fad9f114b0e98d2af0a989c3185 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:01:24.701859  1637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8d17fad9f114b0e98d2af0a989c3185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.707756  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:24.713961  1537 leader_election.cc:343] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:24.719018  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:24.719731  1573 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:24.719842  1765 tablet_bootstrap.cc:654] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:24.704784  1769 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:24.742175  1631 tablet_service.cc:1511] Processing CreateTablet for tablet 18bdd841c8294b08857684489689676d (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:01:24.756122  1631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18bdd841c8294b08857684489689676d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.743544  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:24.759316  1648 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:24.746001  1555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8d17fad9f114b0e98d2af0a989c3185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.765466  1766 raft_consensus.cc:359] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.766386  1766 raft_consensus.cc:385] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:24.766882  1766 raft_consensus.cc:740] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:24.767832  1766 consensus_queue.cc:260] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.745033  1557 tablet_service.cc:1511] Processing CreateTablet for tablet 18bdd841c8294b08857684489689676d (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:01:24.771672  1557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18bdd841c8294b08857684489689676d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.775688  1470 catalog_manager.cc:5697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:24.776988  1612 consensus_peers.cc:597] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Couldn't send request to peer 130bc5ff39b847b1b3204626f284f838. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:24.783679  1796 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.784886  1796 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.786459  1711 tablet_service.cc:1511] Processing CreateTablet for tablet 18bdd841c8294b08857684489689676d (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:01:24.787925  1711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18bdd841c8294b08857684489689676d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.790091  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:24.790701  1768 tablet_bootstrap.cc:492] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:24.790462  1798 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.791647  1768 ts_tablet_manager.cc:1403] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.532s	user 0.028s	sys 0.034s
I20260709 13:01:24.794371  1798 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:24.795152  1768 raft_consensus.cc:359] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.736450  1803 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:24.790153  1796 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:24.796154  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:24.797775  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:24.799512  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:24.754213  1573 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:24.805423  1768 raft_consensus.cc:385] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:24.814072  1647 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:24.815755  1537 leader_election.cc:343] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:24.816849  1612 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:01:24.819675  1519 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:24.810765  1560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bddd1ebc9af24fa282580ab028d643e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.824898  1766 ts_tablet_manager.cc:1434] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.241s	user 0.005s	sys 0.007s
I20260709 13:01:24.828722  1632 tablet_service.cc:1511] Processing CreateTablet for tablet 0c6f11a07e734ec095c6185363fd6b52 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:01:24.831594  1768 raft_consensus.cc:740] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:24.843034  1704 tablet_service.cc:1511] Processing CreateTablet for tablet 0c6f11a07e734ec095c6185363fd6b52 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:01:24.834291  1648 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:24.863605  1687 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:24.864770  1803 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:24.865316  1803 raft_consensus.cc:697] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:01:24.866664  1803 consensus_queue.cc:237] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.868098  1766 tablet_bootstrap.cc:492] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:24.846853  1558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbbf62b8237649f584432aa4a9d96ec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.848096  1556 tablet_service.cc:1511] Processing CreateTablet for tablet 0c6f11a07e734ec095c6185363fd6b52 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:01:24.877969  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c6f11a07e734ec095c6185363fd6b52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.849814  1647 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:24.850920  1559 tablet_service.cc:1511] Processing CreateTablet for tablet b7b0d191c5dc46468a777fba0a666b32 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:01:24.881089  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:24.881524  1559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7b0d191c5dc46468a777fba0a666b32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.852037  1796 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:24.854880  1638 tablet_service.cc:1511] Processing CreateTablet for tablet b7b0d191c5dc46468a777fba0a666b32 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:01:24.883945  1798 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:24.884405  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c6f11a07e734ec095c6185363fd6b52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.884943  1798 raft_consensus.cc:697] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:24.885214  1638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7b0d191c5dc46468a777fba0a666b32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.855583  1765 tablet_bootstrap.cc:492] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:24.886045  1765 ts_tablet_manager.cc:1403] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.337s	user 0.022s	sys 0.017s
I20260709 13:01:24.885918  1798 consensus_queue.cc:237] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.866729  1711 tablet_service.cc:1511] Processing CreateTablet for tablet b7b0d191c5dc46468a777fba0a666b32 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:01:24.745349  1708 tablet_service.cc:1511] Processing CreateTablet for tablet a8d17fad9f114b0e98d2af0a989c3185 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:01:24.896056  1630 tablet_service.cc:1511] Processing CreateTablet for tablet 82cbfb23571048409c1bde90d4611c1a (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:01:24.897842  1630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82cbfb23571048409c1bde90d4611c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.897331  1765 raft_consensus.cc:359] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.898542  1765 raft_consensus.cc:385] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:24.898991  1765 raft_consensus.cc:740] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:24.900305  1765 consensus_queue.cc:260] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.883497  1704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c6f11a07e734ec095c6185363fd6b52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.904832  1765 ts_tablet_manager.cc:1434] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:01:24.909741  1791 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:01:24.919483  1765 tablet_bootstrap.cc:492] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:24.914501  1647 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:24.915038  1796 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:01:24.910192  1711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7b0d191c5dc46468a777fba0a666b32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.929204  1768 consensus_queue.cc:260] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:24.930058  1708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8d17fad9f114b0e98d2af0a989c3185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.931528  1768 ts_tablet_manager.cc:1434] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.139s	user 0.007s	sys 0.001s
I20260709 13:01:24.933423  1554 tablet_service.cc:1511] Processing CreateTablet for tablet 82cbfb23571048409c1bde90d4611c1a (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:01:24.934821  1554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82cbfb23571048409c1bde90d4611c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.937661  1768 tablet_bootstrap.cc:492] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:24.951974  1706 tablet_service.cc:1511] Processing CreateTablet for tablet 82cbfb23571048409c1bde90d4611c1a (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:01:24.954116  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82cbfb23571048409c1bde90d4611c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:24.963608  1471 catalog_manager.cc:5697] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:24.968652  1537 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 -> Peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Couldn't send request to peer 130bc5ff39b847b1b3204626f284f838. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:24.925024  1469 catalog_manager.cc:5697] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:24.992380  1766 tablet_bootstrap.cc:654] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:24.994856  1768 tablet_bootstrap.cc:654] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:24.994087  1636 tablet_service.cc:1511] Processing CreateTablet for tablet dbcb28b8d4104c6fae6682866cc0469d (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:01:24.996996  1636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbcb28b8d4104c6fae6682866cc0469d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:25.008013  1784 consensus_queue.cc:1048] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:25.035425  1765 tablet_bootstrap.cc:654] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:25.040669  1562 tablet_service.cc:1511] Processing CreateTablet for tablet dbcb28b8d4104c6fae6682866cc0469d (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:01:25.042152  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbcb28b8d4104c6fae6682866cc0469d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:25.043150  1706 tablet_service.cc:1511] Processing CreateTablet for tablet dbcb28b8d4104c6fae6682866cc0469d (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:01:25.044106  1766 tablet_bootstrap.cc:492] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:25.044507  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbcb28b8d4104c6fae6682866cc0469d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:25.044684  1766 ts_tablet_manager.cc:1403] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.177s	user 0.012s	sys 0.006s
I20260709 13:01:25.060907  1556 tablet_service.cc:1511] Processing CreateTablet for tablet aa55479d716d4bc4893f658afa382d73 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:01:25.063134  1632 tablet_service.cc:1511] Processing CreateTablet for tablet aa55479d716d4bc4893f658afa382d73 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:01:25.057427  1819 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:01:25.070749  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa55479d716d4bc4893f658afa382d73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:25.111035  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa55479d716d4bc4893f658afa382d73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:25.111181  1791 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.111848  1791 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.113866  1768 tablet_bootstrap.cc:492] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:25.115165  1768 ts_tablet_manager.cc:1403] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.178s	user 0.007s	sys 0.011s
I20260709 13:01:25.115389  1558 tablet_service.cc:1511] Processing CreateTablet for tablet baba78285345483d9cf6ead6bf20106b (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:01:25.119359  1563 tablet_service.cc:1511] Processing CreateTablet for tablet b82d3b49001741eda872554a7ecb2db3 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
W20260709 13:01:25.102526  1611 consensus_peers.cc:597] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:25.126955  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:25.128684  1629 tablet_service.cc:1511] Processing CreateTablet for tablet b82d3b49001741eda872554a7ecb2db3 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:01:25.131469  1637 tablet_service.cc:1511] Processing CreateTablet for tablet baba78285345483d9cf6ead6bf20106b (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:01:25.132251  1766 raft_consensus.cc:359] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.133054  1766 raft_consensus.cc:385] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:25.134620  1766 raft_consensus.cc:740] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:25.138849  1558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet baba78285345483d9cf6ead6bf20106b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:25.138592  1766 consensus_queue.cc:260] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.122352  1791 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:01:25.140936  1687 leader_election.cc:343] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:25.120357  1707 tablet_service.cc:1511] Processing CreateTablet for tablet b82d3b49001741eda872554a7ecb2db3 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:01:25.122505  1709 tablet_service.cc:1511] Processing CreateTablet for tablet baba78285345483d9cf6ead6bf20106b (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:01:25.143260  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b82d3b49001741eda872554a7ecb2db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:25.143389  1629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b82d3b49001741eda872554a7ecb2db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:25.145749  1637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet baba78285345483d9cf6ead6bf20106b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:25.154502  1711 tablet_service.cc:1511] Processing CreateTablet for tablet aa55479d716d4bc4893f658afa382d73 (DEFAULT_TABLE table=test-workload [id=3bdc78e1f00f46baa4d8126b52183327]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:01:25.167011  1784 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.167109  1768 raft_consensus.cc:359] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.168066  1768 raft_consensus.cc:385] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:25.168408  1768 raft_consensus.cc:740] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:25.168138  1784 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.170361  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b82d3b49001741eda872554a7ecb2db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:25.170779  1768 consensus_queue.cc:260] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.172755  1709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet baba78285345483d9cf6ead6bf20106b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:25.174031  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:25.174403  1768 ts_tablet_manager.cc:1434] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.005s
I20260709 13:01:25.175808  1791 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.176576  1711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa55479d716d4bc4893f658afa382d73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:25.176479  1791 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.180931  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:25.182102  1647 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 1: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:01:25.187088  1791 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:01:25.187796  1819 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.188508  1768 tablet_bootstrap.cc:492] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:25.188882  1819 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.195919  1765 tablet_bootstrap.cc:492] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:25.196487  1765 ts_tablet_manager.cc:1403] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.277s	user 0.019s	sys 0.006s
I20260709 13:01:25.198048  1766 ts_tablet_manager.cc:1434] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.153s	user 0.009s	sys 0.002s
I20260709 13:01:25.201597  1648 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:25.202993  1819 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:01:25.210747  1685 leader_election.cc:343] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:25.218240  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:25.212744  1685 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:25.220759  1791 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:25.223259  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:25.224105  1647 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:25.224799  1766 tablet_bootstrap.cc:492] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:25.225835  1538 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:25.228858  1819 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:25.230068  1819 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.230965  1819 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:25.233096  1784 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:25.234651  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:25.235067  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:25.230187  1765 raft_consensus.cc:359] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.242265  1765 raft_consensus.cc:385] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:25.242575  1765 raft_consensus.cc:740] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:25.242926  1612 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:25.243728  1766 tablet_bootstrap.cc:654] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:25.243801  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:25.243268  1765 consensus_queue.cc:260] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:25.239461  1537 leader_election.cc:343] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:25.246798  1765 ts_tablet_manager.cc:1434] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.000s
I20260709 13:01:25.248064  1765 tablet_bootstrap.cc:492] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:25.254222  1765 tablet_bootstrap.cc:654] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:25.265193  1768 tablet_bootstrap.cc:654] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:25.269904  1685 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:25.278528  1791 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:25.294039  1819 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.298664  1803 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:25.300283  1766 tablet_bootstrap.cc:492] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:25.300845  1766 ts_tablet_manager.cc:1403] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.076s	user 0.017s	sys 0.017s
I20260709 13:01:25.310976  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:25.311336  1766 raft_consensus.cc:359] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.312361  1646 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:25.312716  1766 raft_consensus.cc:385] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:25.313163  1766 raft_consensus.cc:740] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:25.313934  1766 consensus_queue.cc:260] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.315738  1766 ts_tablet_manager.cc:1434] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.004s
I20260709 13:01:25.315971  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:25.316680  1766 tablet_bootstrap.cc:492] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:25.318670  1769 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:25.319110  1537 leader_election.cc:343] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:25.319262  1769 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.324709  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:25.307583  1819 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:25.329478  1796 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:25.330060  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:25.331243  1791 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:25.332177  1537 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:25.337548  1765 tablet_bootstrap.cc:492] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:25.327030  1769 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:25.338341  1765 ts_tablet_manager.cc:1403] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.090s	user 0.030s	sys 0.025s
I20260709 13:01:25.338774  1819 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:25.342051  1765 raft_consensus.cc:359] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.342932  1765 raft_consensus.cc:385] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:25.343210  1765 raft_consensus.cc:740] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:25.344030  1765 consensus_queue.cc:260] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.346127  1765 ts_tablet_manager.cc:1434] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:01:25.347404  1646 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:25.349316  1538 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:25.350116  1765 tablet_bootstrap.cc:492] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:25.351023  1819 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:25.351555  1819 raft_consensus.cc:697] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:25.352730  1819 consensus_queue.cc:237] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.352757  1768 tablet_bootstrap.cc:492] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:25.357249  1765 tablet_bootstrap.cc:654] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:25.358040  1768 ts_tablet_manager.cc:1403] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.170s	user 0.017s	sys 0.006s
I20260709 13:01:25.358819  1777 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:25.365330  1768 raft_consensus.cc:359] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.366492  1768 raft_consensus.cc:385] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:25.367010  1768 raft_consensus.cc:740] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:25.368351  1768 consensus_queue.cc:260] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.344907  1766 tablet_bootstrap.cc:654] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:25.372625  1765 tablet_bootstrap.cc:492] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:25.373208  1768 ts_tablet_manager.cc:1434] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:01:25.373399  1765 ts_tablet_manager.cc:1403] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.006s
I20260709 13:01:25.374286  1768 tablet_bootstrap.cc:492] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:25.380442  1768 tablet_bootstrap.cc:654] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:25.433403  1769 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:01:25.440125  1798 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:25.440897  1766 tablet_bootstrap.cc:492] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:25.447201  1766 ts_tablet_manager.cc:1403] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.131s	user 0.008s	sys 0.011s
I20260709 13:01:25.453588  1766 raft_consensus.cc:359] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.462921  1766 raft_consensus.cc:385] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:25.474310  1766 raft_consensus.cc:740] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:25.380117  1765 raft_consensus.cc:359] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.478168  1766 consensus_queue.cc:260] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.479213  1765 raft_consensus.cc:385] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:25.479580  1765 raft_consensus.cc:740] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:25.480726  1765 consensus_queue.cc:260] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.484517  1766 ts_tablet_manager.cc:1434] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.008s
I20260709 13:01:25.489804  1765 ts_tablet_manager.cc:1434] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.116s	user 0.010s	sys 0.000s
I20260709 13:01:25.492663  1768 tablet_bootstrap.cc:492] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:25.493124  1766 tablet_bootstrap.cc:492] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:25.497535  1768 ts_tablet_manager.cc:1403] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.123s	user 0.033s	sys 0.026s
I20260709 13:01:25.519912  1721 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:25.495405  1765 tablet_bootstrap.cc:492] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:25.499747  1796 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.524241  1796 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.538546  1768 raft_consensus.cc:359] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.539533  1768 raft_consensus.cc:385] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:25.539896  1768 raft_consensus.cc:740] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:25.540834  1768 consensus_queue.cc:260] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.517518  1469 catalog_manager.cc:5697] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:25.544565  1765 tablet_bootstrap.cc:654] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:25.545413  1572 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:25.547811  1796 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:25.553256  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:25.554738  1720 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:25.557098  1612 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:25.557372  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:25.558133  1571 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:25.559073  1796 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:25.559535  1796 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.559921  1796 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:25.569080  1768 ts_tablet_manager.cc:1434] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.051s	user 0.009s	sys 0.004s
I20260709 13:01:25.575394  1765 tablet_bootstrap.cc:492] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:25.575076  1766 tablet_bootstrap.cc:654] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:25.576134  1765 ts_tablet_manager.cc:1403] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.081s	user 0.010s	sys 0.013s
I20260709 13:01:25.580470  1765 raft_consensus.cc:359] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.581925  1768 tablet_bootstrap.cc:492] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:25.581607  1765 raft_consensus.cc:385] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:25.585990  1765 raft_consensus.cc:740] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:25.587812  1796 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.587163  1765 consensus_queue.cc:260] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.594233  1768 tablet_bootstrap.cc:654] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:25.615267  1765 ts_tablet_manager.cc:1434] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.039s	user 0.014s	sys 0.014s
I20260709 13:01:25.617434  1765 tablet_bootstrap.cc:492] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:25.611711  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:25.612797  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:25.622789  1721 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:25.620271  1571 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:25.610790  1796 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:25.643730  1766 tablet_bootstrap.cc:492] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:25.644555  1766 ts_tablet_manager.cc:1403] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.157s	user 0.024s	sys 0.012s
I20260709 13:01:25.654831  1765 tablet_bootstrap.cc:654] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:25.649550  1766 raft_consensus.cc:359] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.655395  1766 raft_consensus.cc:385] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:25.655684  1766 raft_consensus.cc:740] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:25.657333  1766 consensus_queue.cc:260] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.659096  1721 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:25.659600  1766 ts_tablet_manager.cc:1434] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.004s
I20260709 13:01:25.661139  1766 tablet_bootstrap.cc:492] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:25.661028  1612 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:25.662549  1824 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:25.663086  1824 raft_consensus.cc:697] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:01:25.664189  1824 consensus_queue.cc:237] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.663060  1571 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:25.671586  1768 tablet_bootstrap.cc:492] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:25.672264  1768 ts_tablet_manager.cc:1403] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.091s	user 0.032s	sys 0.018s
I20260709 13:01:25.679252  1768 raft_consensus.cc:359] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.680050  1768 raft_consensus.cc:385] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:25.680392  1768 raft_consensus.cc:740] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:25.681216  1768 consensus_queue.cc:260] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.683635  1768 ts_tablet_manager.cc:1434] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 13:01:25.684681  1768 tablet_bootstrap.cc:492] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:25.699131  1766 tablet_bootstrap.cc:654] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:25.702456  1768 tablet_bootstrap.cc:654] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:25.714277  1798 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.714975  1798 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.717687  1765 tablet_bootstrap.cc:492] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:25.718335  1765 ts_tablet_manager.cc:1403] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.101s	user 0.024s	sys 0.009s
I20260709 13:01:25.719585  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:25.719794  1469 catalog_manager.cc:5697] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:25.720615  1648 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:25.721386  1765 raft_consensus.cc:359] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.722096  1765 raft_consensus.cc:385] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:25.722412  1765 raft_consensus.cc:740] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:25.722532  1538 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:25.723402  1769 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:25.723795  1769 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.724140  1769 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:25.726147  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:25.728006  1537 leader_election.cc:343] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:25.723889  1765 consensus_queue.cc:260] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.728634  1798 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:25.726397  1766 tablet_bootstrap.cc:492] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:25.729938  1766 ts_tablet_manager.cc:1403] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.069s	user 0.009s	sys 0.016s
I20260709 13:01:25.730085  1765 ts_tablet_manager.cc:1434] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:01:25.733062  1769 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.734143  1765 tablet_bootstrap.cc:492] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:25.735229  1768 tablet_bootstrap.cc:492] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:25.735657  1768 ts_tablet_manager.cc:1403] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.004s
I20260709 13:01:25.737093  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:25.737725  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:25.738317  1648 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:25.738860  1537 leader_election.cc:343] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:25.738497  1768 raft_consensus.cc:359] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.739465  1768 raft_consensus.cc:385] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:25.739615  1769 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:25.739792  1768 raft_consensus.cc:740] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:25.740466  1765 tablet_bootstrap.cc:654] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:25.740803  1768 consensus_queue.cc:260] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.733098  1766 raft_consensus.cc:359] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.744280  1766 raft_consensus.cc:385] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:25.744629  1766 raft_consensus.cc:740] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:25.745446  1766 consensus_queue.cc:260] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.747328  1648 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:25.748499  1766 ts_tablet_manager.cc:1434] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:01:25.749814  1766 tablet_bootstrap.cc:492] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:25.755722  1765 tablet_bootstrap.cc:492] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:25.756242  1765 ts_tablet_manager.cc:1403] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.007s
I20260709 13:01:25.742924  1768 ts_tablet_manager.cc:1434] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:01:25.757664  1768 tablet_bootstrap.cc:492] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:25.758107  1766 tablet_bootstrap.cc:654] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:25.758998  1765 raft_consensus.cc:359] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.759162  1538 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:25.760181  1798 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:25.760715  1798 raft_consensus.cc:697] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:25.761731  1798 consensus_queue.cc:237] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.766326  1768 tablet_bootstrap.cc:654] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:25.768810  1765 raft_consensus.cc:385] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:25.769162  1765 raft_consensus.cc:740] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:25.771456  1765 consensus_queue.cc:260] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.775475  1721 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:25.787784  1819 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.788798  1765 ts_tablet_manager.cc:1434] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.003s
I20260709 13:01:25.789245  1769 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.790292  1769 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.791859  1765 tablet_bootstrap.cc:492] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:25.788381  1819 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.794039  1769 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:25.795223  1819 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:25.796607  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:25.796864  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:25.797713  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:25.798411  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:25.799463  1537 leader_election.cc:343] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:25.799909  1538 leader_election.cc:343] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:25.800978  1538 leader_election.cc:343] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:25.801707  1538 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:25.802166  1470 catalog_manager.cc:5697] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:25.805792  1819 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:01:25.808358  1537 leader_election.cc:343] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:25.809029  1537 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:25.811250  1819 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:25.812775  1766 tablet_bootstrap.cc:492] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:25.813567  1766 ts_tablet_manager.cc:1403] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.064s	user 0.022s	sys 0.009s
I20260709 13:01:25.813934  1765 tablet_bootstrap.cc:654] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:25.818812  1766 raft_consensus.cc:359] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.819571  1766 raft_consensus.cc:385] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:25.819955  1768 tablet_bootstrap.cc:492] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:25.819904  1766 raft_consensus.cc:740] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:25.820434  1768 ts_tablet_manager.cc:1403] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.063s	user 0.017s	sys 0.021s
I20260709 13:01:25.820820  1766 consensus_queue.cc:260] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.823550  1766 ts_tablet_manager.cc:1434] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:01:25.824851  1766 tablet_bootstrap.cc:492] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:25.831144  1766 tablet_bootstrap.cc:654] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:25.842056  1824 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.842947  1824 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.846211  1766 tablet_bootstrap.cc:492] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:25.846714  1766 ts_tablet_manager.cc:1403] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.003s
I20260709 13:01:25.847241  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:25.849066  1796 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.849144  1766 raft_consensus.cc:359] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:25.850010  1611 leader_election.cc:343] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:25.850466  1766 raft_consensus.cc:385] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:25.850090  1796 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.850857  1766 raft_consensus.cc:740] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:25.851413  1824 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:25.851614  1766 consensus_queue.cc:260] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.852629  1796 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:25.853480  1766 ts_tablet_manager.cc:1434] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:01:25.853789  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:25.853942  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:25.853893  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:25.854666  1766 tablet_bootstrap.cc:492] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:25.854941  1571 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:25.856678  1611 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:01:25.858279  1612 leader_election.cc:343] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:25.859194  1612 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:25.860916  1796 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:25.861124  1766 tablet_bootstrap.cc:654] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:25.861495  1796 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.862049  1796 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:25.861887  1824 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:01:25.866863  1612 leader_election.cc:343] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:25.876417  1796 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.896838  1768 raft_consensus.cc:359] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.905282  1768 raft_consensus.cc:385] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:25.905792  1768 raft_consensus.cc:740] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:25.910794  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:25.910961  1824 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.911895  1571 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:25.912642  1824 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.915417  1819 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.916438  1791 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.916486  1819 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.917303  1791 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:25.923133  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:25.925581  1819 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:25.928551  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:25.940430  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:25.941357  1646 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:25.934520  1721 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:25.944615  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:25.941516  1572 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:25.945837  1646 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 1: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:01:25.947654  1611 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:25.950156  1687 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:25.936192  1769 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:01:25.957104  1791 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:25.972504  1538 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:25.981401  1798 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:25.981992  1798 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.982424  1798 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:25.982654  1796 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:25.978478  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:25.979048  1803 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:25.995683  1866 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:26.000849  1866 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.000717  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:25.973366  1571 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:26.002022  1866 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.003741  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:01:26.005939  1612 leader_election.cc:343] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:25.977802  1824 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:26.001044  1720 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:26.019346  1768 consensus_queue.cc:260] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.024080  1866 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.031848  1866 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:26.034878  1803 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.035609  1803 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.039088  1803 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:01:26.042291  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 94951.6us
I20260709 13:01:26.045805  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:26.047147  1796 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.047887  1796 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.050632  1796 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:01:26.051990  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 90853.6us
I20260709 13:01:26.059070  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:26.061244  1769 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:26.068500  1768 ts_tablet_manager.cc:1434] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.248s	user 0.010s	sys 0.003s
I20260709 13:01:26.070456  1766 tablet_bootstrap.cc:492] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:26.071197  1766 ts_tablet_manager.cc:1403] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.217s	user 0.074s	sys 0.112s
I20260709 13:01:26.074391  1768 tablet_bootstrap.cc:492] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:26.067293  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:26.063814  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:26.066249  1685 leader_election.cc:343] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:26.079402  1612 leader_election.cc:343] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:26.080229  1519 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:26.079587  1720 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.081146  1798 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.091488  1765 tablet_bootstrap.cc:492] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:26.092044  1765 ts_tablet_manager.cc:1403] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.301s	user 0.107s	sys 0.101s
I20260709 13:01:26.094018  1766 raft_consensus.cc:359] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.094744  1766 raft_consensus.cc:385] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.095022  1766 raft_consensus.cc:740] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.098198  1571 raft_consensus.cc:2393] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 1: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:01:26.100620  1765 raft_consensus.cc:359] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.101514  1765 raft_consensus.cc:385] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.102257  1765 raft_consensus.cc:740] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.102890  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:26.103857  1765 consensus_queue.cc:260] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.104554  1611 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:26.105482  1866 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:26.105993  1866 raft_consensus.cc:697] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:01:26.106808  1866 consensus_queue.cc:237] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:26.108220  1687 leader_election.cc:343] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:26.108709  1687 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:26.108996  1765 ts_tablet_manager.cc:1434] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.000s
I20260709 13:01:26.109513  1803 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:26.110457  1765 tablet_bootstrap.cc:492] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:26.114938  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:26.115671  1796 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:26.115942  1572 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:26.122337  1766 consensus_queue.cc:260] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.117604  1611 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:26.133992  1766 ts_tablet_manager.cc:1434] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.062s	user 0.009s	sys 0.001s
I20260709 13:01:26.135187  1766 tablet_bootstrap.cc:492] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:26.135411  1796 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:26.135988  1796 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.136495  1796 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.135073  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:26.136185  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:26.140080  1647 raft_consensus.cc:2393] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 1: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:01:26.132630  1798 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:26.144264  1768 tablet_bootstrap.cc:654] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.150899  1720 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:26.153232  1538 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:26.156673  1765 tablet_bootstrap.cc:654] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.158318  1798 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:01:26.170676  1766 tablet_bootstrap.cc:654] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.205103  1612 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:01:26.206568  1824 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:26.208693  1824 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:26.209285  1824 raft_consensus.cc:697] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:01:26.215723  1803 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.218518  1803 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.226365  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:26.227260  1647 raft_consensus.cc:2393] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 1: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:01:26.228480  1803 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:26.230065  1824 consensus_queue.cc:237] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.231312  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:26.237152  1685 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:26.239279  1803 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.209686  1796 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.259425  1765 tablet_bootstrap.cc:492] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:26.259960  1765 ts_tablet_manager.cc:1403] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.150s	user 0.027s	sys 0.026s
I20260709 13:01:26.262034  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:26.262843  1573 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.263208  1768 tablet_bootstrap.cc:492] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:26.263795  1768 ts_tablet_manager.cc:1403] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.190s	user 0.027s	sys 0.028s
I20260709 13:01:26.266672  1768 raft_consensus.cc:359] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.267422  1768 raft_consensus.cc:385] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.263146  1765 raft_consensus.cc:359] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.269330  1765 raft_consensus.cc:385] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.269603  1765 raft_consensus.cc:740] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.270324  1765 consensus_queue.cc:260] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.272468  1765 ts_tablet_manager.cc:1434] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.002s
I20260709 13:01:26.267728  1768 raft_consensus.cc:740] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.274245  1768 consensus_queue.cc:260] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.276511  1768 ts_tablet_manager.cc:1434] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:01:26.277820  1768 tablet_bootstrap.cc:492] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:26.273566  1765 tablet_bootstrap.cc:492] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:26.284696  1796 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:01:26.289409  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 81730.9us
I20260709 13:01:26.289948  1573 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:26.293062  1611 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:26.294008  1796 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:26.296103  1796 raft_consensus.cc:697] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:01:26.295840  1471 catalog_manager.cc:5697] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:26.297106  1766 tablet_bootstrap.cc:492] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:26.297303  1796 consensus_queue.cc:237] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.300161  1766 ts_tablet_manager.cc:1403] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.165s	user 0.029s	sys 0.023s
I20260709 13:01:26.303536  1803 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:26.303417  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:26.306834  1765 tablet_bootstrap.cc:654] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.306856  1768 tablet_bootstrap.cc:654] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.312935  1798 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:01:26.312466  1612 leader_election.cc:343] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:26.315614  1863 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.321015  1766 raft_consensus.cc:359] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.322554  1766 raft_consensus.cc:385] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.322826  1766 raft_consensus.cc:740] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.316228  1863 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.328590  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:01:26.330484  1611 leader_election.cc:343] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:26.335755  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:26.338375  1612 leader_election.cc:343] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:26.339100  1612 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:26.340247  1875 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:26.341595  1766 consensus_queue.cc:260] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.347276  1819 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:26.357038  1766 ts_tablet_manager.cc:1434] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.057s	user 0.006s	sys 0.002s
I20260709 13:01:26.358799  1766 tablet_bootstrap.cc:492] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:26.363394  1863 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:26.375288  1766 tablet_bootstrap.cc:654] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.376483  1768 tablet_bootstrap.cc:492] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:26.376467  1765 tablet_bootstrap.cc:492] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:26.377089  1768 ts_tablet_manager.cc:1403] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.100s	user 0.025s	sys 0.022s
I20260709 13:01:26.381490  1765 ts_tablet_manager.cc:1403] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.108s	user 0.027s	sys 0.017s
I20260709 13:01:26.384248  1768 raft_consensus.cc:359] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.384753  1765 raft_consensus.cc:359] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.385251  1768 raft_consensus.cc:385] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.385545  1765 raft_consensus.cc:385] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.385752  1768 raft_consensus.cc:740] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.385954  1765 raft_consensus.cc:740] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.386986  1765 consensus_queue.cc:260] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.387034  1768 consensus_queue.cc:260] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.390049  1768 ts_tablet_manager.cc:1434] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.012s	user 0.011s	sys 0.001s
I20260709 13:01:26.391098  1768 tablet_bootstrap.cc:492] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:26.391893  1766 tablet_bootstrap.cc:492] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:26.392373  1766 ts_tablet_manager.cc:1403] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.007s
I20260709 13:01:26.395133  1766 raft_consensus.cc:359] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.395820  1766 raft_consensus.cc:385] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.396093  1766 raft_consensus.cc:740] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.397127  1768 tablet_bootstrap.cc:654] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.396965  1766 consensus_queue.cc:260] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.399725  1766 ts_tablet_manager.cc:1434] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:01:26.401278  1766 tablet_bootstrap.cc:492] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:26.408175  1765 ts_tablet_manager.cc:1434] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 13:01:26.411069  1766 tablet_bootstrap.cc:654] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.423650  1470 catalog_manager.cc:5697] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:26.425796  1765 tablet_bootstrap.cc:492] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:26.428623  1766 tablet_bootstrap.cc:492] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:26.429343  1766 ts_tablet_manager.cc:1403] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.009s
I20260709 13:01:26.427932  1470 catalog_manager.cc:5697] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:26.434026  1766 raft_consensus.cc:359] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.435120  1766 raft_consensus.cc:385] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.435459  1766 raft_consensus.cc:740] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.437315  1765 tablet_bootstrap.cc:654] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.436904  1766 consensus_queue.cc:260] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.440068  1766 ts_tablet_manager.cc:1434] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:01:26.439327  1768 tablet_bootstrap.cc:492] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:26.441331  1766 tablet_bootstrap.cc:492] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:26.441457  1768 ts_tablet_manager.cc:1403] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.032s
I20260709 13:01:26.445801  1768 raft_consensus.cc:359] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.448066  1768 raft_consensus.cc:385] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.448722  1768 raft_consensus.cc:740] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.449193  1766 tablet_bootstrap.cc:654] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.449963  1768 consensus_queue.cc:260] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.455930  1765 tablet_bootstrap.cc:492] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:26.456786  1765 ts_tablet_manager.cc:1403] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.031s	user 0.021s	sys 0.003s
I20260709 13:01:26.458129  1768 ts_tablet_manager.cc:1434] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 13:01:26.460238  1768 tablet_bootstrap.cc:492] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:26.460073  1765 raft_consensus.cc:359] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.460826  1765 raft_consensus.cc:385] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.461148  1765 raft_consensus.cc:740] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.461850  1765 consensus_queue.cc:260] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.469411  1768 tablet_bootstrap.cc:654] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.470553  1766 tablet_bootstrap.cc:492] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:26.471122  1798 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.471647  1798 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.473752  1798 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:26.475365  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:26.475919  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:26.476295  1720 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:26.476776  1647 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:26.478025  1766 ts_tablet_manager.cc:1403] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.005s
I20260709 13:01:26.477756  1537 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:26.479635  1803 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.480130  1798 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:26.480180  1803 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.480824  1766 raft_consensus.cc:359] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.481484  1766 raft_consensus.cc:385] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.483101  1803 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:26.480636  1798 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.483498  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:26.483765  1798 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.484377  1573 raft_consensus.cc:2393] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 1: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:01:26.486091  1765 ts_tablet_manager.cc:1434] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.003s
I20260709 13:01:26.487526  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:26.488952  1765 tablet_bootstrap.cc:492] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:26.489454  1687 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:26.490803  1803 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.498571  1796 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:26.499528  1798 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.503100  1865 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.503916  1865 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.504748  1803 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:26.481755  1766 raft_consensus.cc:740] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.506434  1766 consensus_queue.cc:260] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.508363  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:26.508509  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:26.509042  1720 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:26.509368  1647 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.510759  1537 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:26.512351  1864 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:26.512885  1864 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.513284  1864 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.506784  1865 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:26.514170  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:26.514964  1766 ts_tablet_manager.cc:1434] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 13:01:26.515089  1646 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:26.517562  1765 tablet_bootstrap.cc:654] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.517782  1766 tablet_bootstrap.cc:492] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:26.531162  1768 tablet_bootstrap.cc:492] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:26.533581  1768 ts_tablet_manager.cc:1403] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.074s	user 0.027s	sys 0.000s
I20260709 13:01:26.535701  1864 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.537400  1766 tablet_bootstrap.cc:654] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.538546  1768 raft_consensus.cc:359] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.539376  1768 raft_consensus.cc:385] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.539667  1768 raft_consensus.cc:740] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.541429  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:26.542202  1720 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.541805  1768 consensus_queue.cc:260] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.545176  1768 ts_tablet_manager.cc:1434] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:01:26.554019  1766 tablet_bootstrap.cc:492] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:26.554239  1865 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.553550  1720 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:26.554633  1766 ts_tablet_manager.cc:1403] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.038s	user 0.018s	sys 0.005s
I20260709 13:01:26.554872  1865 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.555884  1537 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:26.556844  1769 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:26.557152  1865 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:26.557318  1769 raft_consensus.cc:697] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:26.558363  1769 consensus_queue.cc:237] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.559346  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:26.559726  1768 tablet_bootstrap.cc:492] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:26.560015  1646 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.563686  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:26.565546  1537 leader_election.cc:343] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:26.567008  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:26.567685  1766 raft_consensus.cc:359] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.568677  1766 raft_consensus.cc:385] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.568953  1766 raft_consensus.cc:740] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.568871  1648 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:26.569891  1766 consensus_queue.cc:260] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.553977  1864 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:26.573915  1646 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:26.577598  1766 ts_tablet_manager.cc:1434] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:01:26.579329  1766 tablet_bootstrap.cc:492] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:26.591687  1798 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:26.596474  1768 tablet_bootstrap.cc:654] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.596172  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:26.597846  1538 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:26.601441  1864 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:26.601819  1647 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:26.597216  1720 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.602164  1864 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.604180  1864 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.606746  1766 tablet_bootstrap.cc:654] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:26.611212  1612 consensus_peers.cc:597] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b -> Peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Couldn't send request to peer 130bc5ff39b847b1b3204626f284f838. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:26.611864  1538 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:26.615015  1471 catalog_manager.cc:5697] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:26.617468  1798 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:26.619068  1798 raft_consensus.cc:697] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:26.620253  1798 consensus_queue.cc:237] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.621358  1765 tablet_bootstrap.cc:492] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:26.621129  1720 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:26.621928  1765 ts_tablet_manager.cc:1403] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.133s	user 0.037s	sys 0.024s
I20260709 13:01:26.625496  1765 raft_consensus.cc:359] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.628001  1765 raft_consensus.cc:385] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.628328  1765 raft_consensus.cc:740] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.630250  1768 tablet_bootstrap.cc:492] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:26.630976  1768 ts_tablet_manager.cc:1403] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.071s	user 0.011s	sys 0.012s
I20260709 13:01:26.632332  1765 consensus_queue.cc:260] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.637002  1765 ts_tablet_manager.cc:1434] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:01:26.637557  1798 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.638715  1798 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.641804  1766 tablet_bootstrap.cc:492] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:26.642395  1766 ts_tablet_manager.cc:1403] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.063s	user 0.012s	sys 0.002s
I20260709 13:01:26.644012  1798 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:26.644814  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:26.645665  1766 raft_consensus.cc:359] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.646461  1766 raft_consensus.cc:385] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.646178  1768 raft_consensus.cc:359] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.646749  1766 raft_consensus.cc:740] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.646869  1768 raft_consensus.cc:385] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.647008  1765 tablet_bootstrap.cc:492] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:26.647531  1766 consensus_queue.cc:260] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.648279  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:26.649190  1647 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:26.650380  1766 ts_tablet_manager.cc:1434] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:01:26.650458  1768 raft_consensus.cc:740] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.653122  1766 tablet_bootstrap.cc:492] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:26.656641  1538 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:26.646186  1720 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:26.659548  1819 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:26.660523  1819 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.660897  1819 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.661306  1768 consensus_queue.cc:260] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.665575  1765 tablet_bootstrap.cc:654] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.667892  1768 ts_tablet_manager.cc:1434] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.001s
I20260709 13:01:26.669258  1768 tablet_bootstrap.cc:492] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:26.657200  1864 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.680328  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:26.680963  1720 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.681514  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:26.682611  1647 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.689911  1720 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:26.691207  1537 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:26.672856  1766 tablet_bootstrap.cc:654] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.692080  1769 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:26.692596  1769 raft_consensus.cc:697] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:26.694161  1769 consensus_queue.cc:237] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.696334  1864 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:26.701014  1766 tablet_bootstrap.cc:492] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:26.701841  1766 ts_tablet_manager.cc:1403] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.049s	user 0.015s	sys 0.001s
I20260709 13:01:26.672537  1471 catalog_manager.cc:5697] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:26.676483  1768 tablet_bootstrap.cc:654] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.708981  1765 tablet_bootstrap.cc:492] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:26.709784  1765 ts_tablet_manager.cc:1403] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.071s	user 0.026s	sys 0.023s
I20260709 13:01:26.711484  1766 raft_consensus.cc:359] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.713244  1765 raft_consensus.cc:359] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.676808  1819 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.713938  1765 raft_consensus.cc:385] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.714047  1766 raft_consensus.cc:385] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.714278  1765 raft_consensus.cc:740] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.714700  1766 raft_consensus.cc:740] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.715154  1765 consensus_queue.cc:260] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.716840  1766 consensus_queue.cc:260] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.719518  1765 ts_tablet_manager.cc:1434] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:01:26.720147  1766 ts_tablet_manager.cc:1434] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.000s
I20260709 13:01:26.720818  1765 tablet_bootstrap.cc:492] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:26.721997  1766 tablet_bootstrap.cc:492] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:26.719522  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:26.728814  1646 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.729615  1819 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:26.739149  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:26.740032  1721 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.742058  1863 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:26.745832  1647 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:26.753278  1469 catalog_manager.cc:5697] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:26.761153  1803 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.761812  1803 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.765386  1863 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.765972  1863 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.766691  1803 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:26.768266  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:26.768862  1863 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:26.769299  1573 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:26.772807  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:26.772964  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:26.775712  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:26.783082  1685 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:01:26.787092  1612 leader_election.cc:343] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:26.788650  1611 leader_election.cc:343] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:26.791884  1611 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:26.795771  1796 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:26.796047  1766 tablet_bootstrap.cc:654] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.790267  1647 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:26.799269  1803 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:26.799472  1721 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:26.799738  1803 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.801752  1803 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.805822  1537 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:26.808683  1819 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:26.810920  1646 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:26.811853  1765 tablet_bootstrap.cc:654] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.810438  1819 raft_consensus.cc:697] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:26.811800  1768 tablet_bootstrap.cc:492] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:26.817118  1803 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.819728  1768 ts_tablet_manager.cc:1403] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.151s	user 0.012s	sys 0.014s
I20260709 13:01:26.819995  1819 consensus_queue.cc:237] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.832692  1768 raft_consensus.cc:359] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.843713  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:26.844890  1863 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:26.845424  1573 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.853977  1768 raft_consensus.cc:385] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.854333  1766 tablet_bootstrap.cc:492] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:26.856498  1796 consensus_queue.cc:1048] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:26.858529  1766 ts_tablet_manager.cc:1403] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.137s	user 0.018s	sys 0.007s
I20260709 13:01:26.858723  1573 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:26.864907  1471 catalog_manager.cc:5697] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:26.867602  1685 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:26.868749  1766 raft_consensus.cc:359] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.869520  1766 raft_consensus.cc:385] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.869249  1791 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:26.870455  1768 raft_consensus.cc:740] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.871588  1768 consensus_queue.cc:260] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.869886  1766 raft_consensus.cc:740] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.876987  1768 ts_tablet_manager.cc:1434] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.057s	user 0.009s	sys 0.005s
I20260709 13:01:26.876348  1766 consensus_queue.cc:260] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.878367  1768 tablet_bootstrap.cc:492] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:26.882083  1803 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:26.883682  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:26.884403  1647 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.884696  1766 ts_tablet_manager.cc:1434] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.002s
I20260709 13:01:26.887349  1766 tablet_bootstrap.cc:492] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:26.891222  1765 tablet_bootstrap.cc:492] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:26.892025  1765 ts_tablet_manager.cc:1403] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.171s	user 0.011s	sys 0.007s
I20260709 13:01:26.892577  1796 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.894435  1796 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.898097  1765 raft_consensus.cc:359] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.901572  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:26.900645  1765 raft_consensus.cc:385] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.904826  1765 raft_consensus.cc:740] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.905761  1765 consensus_queue.cc:260] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.907146  1863 consensus_queue.cc:1048] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:26.907318  1573 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:26.893843  1791 raft_consensus.cc:697] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:01:26.913981  1611 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:26.916249  1796 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:26.917202  1796 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:26.918277  1796 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.918632  1796 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.921487  1766 tablet_bootstrap.cc:654] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.924961  1765 ts_tablet_manager.cc:1434] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.032s	user 0.012s	sys 0.003s
I20260709 13:01:26.927759  1765 tablet_bootstrap.cc:492] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:26.924868  1768 tablet_bootstrap.cc:654] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.916210  1647 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:26.931757  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:26.918300  1791 consensus_queue.cc:237] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:26.934026  1612 leader_election.cc:343] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:26.941596  1796 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.951617  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:26.952423  1571 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:26.954788  1766 tablet_bootstrap.cc:492] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:26.955391  1766 ts_tablet_manager.cc:1403] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.069s	user 0.018s	sys 0.006s
I20260709 13:01:26.959256  1765 tablet_bootstrap.cc:654] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.958729  1766 raft_consensus.cc:359] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.960412  1766 raft_consensus.cc:385] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.960772  1766 raft_consensus.cc:740] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.961584  1766 consensus_queue.cc:260] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.963835  1766 ts_tablet_manager.cc:1434] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:01:26.965072  1766 tablet_bootstrap.cc:492] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:26.969058  1768 tablet_bootstrap.cc:492] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:26.969993  1768 ts_tablet_manager.cc:1403] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.092s	user 0.009s	sys 0.012s
I20260709 13:01:26.973562  1766 tablet_bootstrap.cc:654] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:26.983439  1796 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:26.985541  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:26.987139  1819 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:01:26.987936  1612 leader_election.cc:343] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:26.988847  1571 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:26.990029  1796 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.982115  1468 catalog_manager.cc:5697] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:26.991078  1796 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.996841  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:26.979378  1768 raft_consensus.cc:359] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:26.998588  1768 raft_consensus.cc:385] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:26.998957  1768 raft_consensus.cc:740] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:26.999723  1768 consensus_queue.cc:260] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.000728  1611 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:27.002741  1768 ts_tablet_manager.cc:1434] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.000s
I20260709 13:01:27.003715  1913 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:27.004401  1863 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.005517  1768 tablet_bootstrap.cc:492] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:27.005053  1863 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.006589  1766 tablet_bootstrap.cc:492] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:27.007773  1766 ts_tablet_manager.cc:1403] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.043s	user 0.014s	sys 0.004s
I20260709 13:01:27.004463  1913 raft_consensus.cc:697] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:01:27.010756  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:27.011281  1863 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:27.016104  1766 raft_consensus.cc:359] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.017683  1766 raft_consensus.cc:385] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:27.032619  1766 raft_consensus.cc:740] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:27.016916  1720 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:27.033756  1766 consensus_queue.cc:260] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.017306  1875 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.035642  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:27.035674  1819 consensus_queue.cc:1048] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:01:27.036363  1913 consensus_queue.cc:237] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.038455  1863 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:27.035665  1875 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.038906  1863 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:27.002765  1611 leader_election.cc:343] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:27.039331  1863 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:27.040467  1875 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:27.042901  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:27.042923  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:27.043999  1796 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:27.048961  1766 ts_tablet_manager.cc:1434] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.040s	user 0.017s	sys 0.005s
I20260709 13:01:27.050166  1766 tablet_bootstrap.cc:492] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:27.044345  1573 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:27.051575  1863 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:27.056653  1611 leader_election.cc:343] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:27.058861  1863 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:27.063412  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:27.063709  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:27.060240  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:27.065258  1573 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:27.073239  1768 tablet_bootstrap.cc:654] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:27.073936  1765 tablet_bootstrap.cc:492] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:27.074440  1765 ts_tablet_manager.cc:1403] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.148s	user 0.026s	sys 0.024s
W20260709 13:01:27.078224  1612 leader_election.cc:343] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:27.078943  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:27.073239  1468 catalog_manager.cc:5697] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:27.080523  1863 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:27.092883  1765 raft_consensus.cc:359] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.093743  1765 raft_consensus.cc:385] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:27.094190  1765 raft_consensus.cc:740] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:27.095098  1765 consensus_queue.cc:260] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.099900  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:27.101504  1720 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:27.108806  1612 leader_election.cc:343] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:27.109818  1612 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:27.108553  1766 tablet_bootstrap.cc:654] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:27.117077  1863 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:27.116323  1765 ts_tablet_manager.cc:1434] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.042s	user 0.010s	sys 0.000s
I20260709 13:01:27.123898  1721 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:27.125080  1765 tablet_bootstrap.cc:492] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:27.142370  1798 consensus_queue.cc:1048] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:27.154431  1768 tablet_bootstrap.cc:492] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:27.154120  1573 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:27.161908  1798 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:01:27.167845  1611 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:27.162662  1768 ts_tablet_manager.cc:1403] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.158s	user 0.027s	sys 0.009s
I20260709 13:01:27.172242  1863 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:27.178387  1769 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:01:27.180305  1720 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:27.179067  1863 raft_consensus.cc:697] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:01:27.181996  1863 consensus_queue.cc:237] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.211254  1766 tablet_bootstrap.cc:492] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:27.211807  1766 ts_tablet_manager.cc:1403] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.162s	user 0.021s	sys 0.004s
I20260709 13:01:27.232707  1769 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:27.258881  1913 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.259464  1913 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.264443  1913 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:27.265316  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:27.267041  1766 raft_consensus.cc:359] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.267134  1768 raft_consensus.cc:359] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.267882  1768 raft_consensus.cc:385] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:27.268414  1768 raft_consensus.cc:740] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
W20260709 13:01:27.271118  1611 leader_election.cc:343] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:27.272595  1765 tablet_bootstrap.cc:654] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:27.275841  1768 consensus_queue.cc:260] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.309698  1913 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.326345  1766 raft_consensus.cc:385] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:27.326730  1766 raft_consensus.cc:740] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:27.329821  1768 ts_tablet_manager.cc:1434] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.160s	user 0.011s	sys 0.000s
I20260709 13:01:27.327332  1798 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:01:27.330760  1768 tablet_bootstrap.cc:492] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
W20260709 13:01:27.329365  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 47493.6us
I20260709 13:01:27.335428  1769 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:27.335626  1766 consensus_queue.cc:260] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.338798  1766 ts_tablet_manager.cc:1434] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.127s	user 0.008s	sys 0.003s
I20260709 13:01:27.325223  1913 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.298059  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:27.341754  1722 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:27.344915  1766 tablet_bootstrap.cc:492] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:27.348129  1612 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:01:27.349355  1796 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:27.349459  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:27.349860  1796 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.350283  1796 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:27.351802  1611 leader_election.cc:343] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:27.352984  1519 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:27.359216  1765 tablet_bootstrap.cc:492] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:27.360178  1765 ts_tablet_manager.cc:1403] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.238s	user 0.017s	sys 0.003s
I20260709 13:01:27.362443  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:27.369521  1765 raft_consensus.cc:359] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.359272  1913 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:27.370378  1765 raft_consensus.cc:385] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:27.373546  1765 raft_consensus.cc:740] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
W20260709 13:01:27.372844  1612 leader_election.cc:343] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:27.374285  1765 consensus_queue.cc:260] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.374639  1612 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:27.377943  1917 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:27.386444  1765 ts_tablet_manager.cc:1434] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:01:27.377396  1796 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.392059  1765 tablet_bootstrap.cc:492] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:27.396181  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:27.398218  1571 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:27.408829  1796 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:27.411316  1766 tablet_bootstrap.cc:654] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:27.413403  1468 catalog_manager.cc:5697] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:27.415776  1768 tablet_bootstrap.cc:654] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:27.417074  1571 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:27.418547  1611 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:27.419396  1796 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:27.420039  1796 raft_consensus.cc:697] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:01:27.420923  1796 consensus_queue.cc:237] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.425164  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:27.425890  1719 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:27.448378  1791 consensus_queue.cc:1048] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:27.448865  1719 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:27.460501  1765 tablet_bootstrap.cc:654] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:27.460222  1917 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.499470  1917 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.504541  1769 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.510197  1769 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.512642  1769 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:27.516695  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:27.521582  1768 tablet_bootstrap.cc:492] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:27.522190  1768 ts_tablet_manager.cc:1403] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.192s	user 0.010s	sys 0.007s
I20260709 13:01:27.527213  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:27.530387  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:27.531610  1646 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
W20260709 13:01:27.533047  1611 leader_election.cc:343] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:27.533859  1908 consensus_queue.cc:1048] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:27.535724  1917 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:01:27.537164  1537 leader_election.cc:343] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:27.540705  1766 tablet_bootstrap.cc:492] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:27.541263  1766 ts_tablet_manager.cc:1403] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.200s	user 0.042s	sys 0.051s
I20260709 13:01:27.542722  1768 raft_consensus.cc:359] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.544003  1766 raft_consensus.cc:359] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.545353  1766 raft_consensus.cc:385] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:27.545222  1768 raft_consensus.cc:385] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:27.546533  1766 raft_consensus.cc:740] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:27.546923  1768 raft_consensus.cc:740] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:27.548120  1768 consensus_queue.cc:260] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.549259  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:27.554219  1612 consensus_peers.cc:597] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b -> Peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Couldn't send request to peer 130bc5ff39b847b1b3204626f284f838. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:27.554910  1612 leader_election.cc:343] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:27.555783  1471 catalog_manager.cc:5697] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:27.560163  1766 consensus_queue.cc:260] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.555332  1612 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:27.567788  1917 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:27.567296  1766 ts_tablet_manager.cc:1434] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.004s
I20260709 13:01:27.570482  1538 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:27.570855  1766 tablet_bootstrap.cc:492] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:27.574546  1865 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:27.575006  1768 ts_tablet_manager.cc:1434] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.052s	user 0.009s	sys 0.005s
I20260709 13:01:27.575035  1865 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.575753  1865 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:27.576404  1768 tablet_bootstrap.cc:492] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:27.599131  1765 tablet_bootstrap.cc:492] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:27.599750  1765 ts_tablet_manager.cc:1403] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.210s	user 0.018s	sys 0.007s
I20260709 13:01:27.605746  1865 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.609004  1865 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:27.615773  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:27.615924  1917 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.620045  1765 raft_consensus.cc:359] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.620904  1913 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:27.621207  1766 tablet_bootstrap.cc:654] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:27.621629  1765 raft_consensus.cc:385] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:27.621101  1917 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.622077  1765 raft_consensus.cc:740] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:27.621162  1721 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:27.623697  1765 consensus_queue.cc:260] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.625958  1765 ts_tablet_manager.cc:1434] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 13:01:27.627326  1765 tablet_bootstrap.cc:492] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:27.631166  1917 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:27.633599  1917 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:27.637465  1768 tablet_bootstrap.cc:654] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:27.642138  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:27.644910  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:27.645846  1645 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:27.651779  1612 leader_election.cc:343] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:27.655105  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:01:27.657145  1611 leader_election.cc:343] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:27.657624  1611 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:01:27.658135  1937 log.cc:927] Time spent T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:01:27.660658  1917 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:27.673036  1765 tablet_bootstrap.cc:654] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:27.688125  1721 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:27.695220  1766 tablet_bootstrap.cc:492] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:27.695731  1766 ts_tablet_manager.cc:1403] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.125s	user 0.021s	sys 0.013s
I20260709 13:01:27.696229  1537 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:27.697777  1819 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:27.698324  1819 raft_consensus.cc:697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:27.698858  1766 raft_consensus.cc:359] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.699775  1766 raft_consensus.cc:385] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:27.699263  1819 consensus_queue.cc:237] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.701421  1913 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:27.703713  1765 tablet_bootstrap.cc:492] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:27.700135  1766 raft_consensus.cc:740] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:27.705971  1768 tablet_bootstrap.cc:492] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:27.706581  1768 ts_tablet_manager.cc:1403] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.130s	user 0.021s	sys 0.019s
I20260709 13:01:27.706504  1766 consensus_queue.cc:260] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.706908  1645 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:27.709363  1766 ts_tablet_manager.cc:1434] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:01:27.709492  1768 raft_consensus.cc:359] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.710206  1768 raft_consensus.cc:385] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:27.710470  1766 tablet_bootstrap.cc:492] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:27.710567  1768 raft_consensus.cc:740] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:27.710516  1765 ts_tablet_manager.cc:1403] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.083s	user 0.019s	sys 0.004s
I20260709 13:01:27.711539  1768 consensus_queue.cc:260] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.714488  1768 ts_tablet_manager.cc:1434] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:01:27.715461  1768 tablet_bootstrap.cc:492] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:27.717109  1765 raft_consensus.cc:359] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.719942  1765 raft_consensus.cc:385] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:27.720749  1765 raft_consensus.cc:740] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:27.724380  1768 tablet_bootstrap.cc:654] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:27.722374  1765 consensus_queue.cc:260] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.731129  1765 ts_tablet_manager.cc:1434] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:01:27.731592  1766 tablet_bootstrap.cc:654] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:27.739270  1765 tablet_bootstrap.cc:492] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:27.737571  1768 tablet_bootstrap.cc:492] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:27.740888  1471 catalog_manager.cc:5697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:27.742822  1768 ts_tablet_manager.cc:1403] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.009s
I20260709 13:01:27.749866  1768 raft_consensus.cc:359] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.750437  1768 raft_consensus.cc:385] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:27.750981  1768 raft_consensus.cc:740] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:27.751798  1765 tablet_bootstrap.cc:654] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:27.751672  1768 consensus_queue.cc:260] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.757853  1768 ts_tablet_manager.cc:1434] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:01:27.758872  1768 tablet_bootstrap.cc:492] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:27.759155  1766 tablet_bootstrap.cc:492] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:27.759608  1766 ts_tablet_manager.cc:1403] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.049s	user 0.014s	sys 0.000s
I20260709 13:01:27.765182  1768 tablet_bootstrap.cc:654] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:27.767650  1766 raft_consensus.cc:359] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.768460  1766 raft_consensus.cc:385] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:27.768800  1766 raft_consensus.cc:740] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:27.769848  1766 consensus_queue.cc:260] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.772434  1766 ts_tablet_manager.cc:1434] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:01:27.775583  1766 tablet_bootstrap.cc:492] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:27.779299  1765 tablet_bootstrap.cc:492] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:27.780019  1765 ts_tablet_manager.cc:1403] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.041s	user 0.024s	sys 0.009s
I20260709 13:01:27.783676  1766 tablet_bootstrap.cc:654] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:27.786903  1765 raft_consensus.cc:359] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.787774  1765 raft_consensus.cc:385] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:27.788054  1765 raft_consensus.cc:740] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:27.788686  1768 tablet_bootstrap.cc:492] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:27.789454  1768 ts_tablet_manager.cc:1403] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.031s	user 0.008s	sys 0.008s
I20260709 13:01:27.788996  1765 consensus_queue.cc:260] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.792680  1765 ts_tablet_manager.cc:1434] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:01:27.794364  1768 raft_consensus.cc:359] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.795869  1768 raft_consensus.cc:385] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:27.796348  1768 raft_consensus.cc:740] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:27.797267  1768 consensus_queue.cc:260] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.800132  1765 tablet_bootstrap.cc:492] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:27.808142  1765 tablet_bootstrap.cc:654] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:27.825169  1768 ts_tablet_manager.cc:1434] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 13:01:27.826100  1765 tablet_bootstrap.cc:492] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:27.826658  1765 ts_tablet_manager.cc:1403] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.009s
I20260709 13:01:27.828491  1768 tablet_bootstrap.cc:492] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:27.830381  1766 tablet_bootstrap.cc:492] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:27.831082  1766 ts_tablet_manager.cc:1403] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.056s	user 0.030s	sys 0.012s
I20260709 13:01:27.831813  1765 raft_consensus.cc:359] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.832588  1765 raft_consensus.cc:385] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:27.832890  1765 raft_consensus.cc:740] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:27.833571  1765 consensus_queue.cc:260] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.834127  1766 raft_consensus.cc:359] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.835041  1766 raft_consensus.cc:385] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:27.835352  1766 raft_consensus.cc:740] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:27.836091  1766 consensus_queue.cc:260] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.836735  1765 ts_tablet_manager.cc:1434] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:01:27.844791  1765 tablet_bootstrap.cc:492] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:27.847261  1917 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:27.848181  1766 ts_tablet_manager.cc:1434] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:01:27.854827  1769 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.854918  1765 tablet_bootstrap.cc:654] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:27.858435  1791 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.858091  1769 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.859130  1791 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.862353  1791 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:27.864571  1769 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:27.865480  1766 tablet_bootstrap.cc:492] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:27.867944  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:27.868656  1721 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:27.863168  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:27.870199  1570 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:27.868357  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:27.871553  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:27.871979  1645 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:27.872313  1646 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:27.872653  1685 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:27.872344  1537 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:27.874346  1769 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:27.874743  1769 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.875092  1769 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:27.884502  1791 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:27.892553  1791 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.893095  1791 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:27.894440  1768 tablet_bootstrap.cc:654] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:27.897156  1769 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.901909  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:27.902642  1646 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:27.903439  1917 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:27.909827  1646 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:27.908674  1791 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.915712  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:27.916810  1719 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:27.920279  1766 tablet_bootstrap.cc:654] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:27.920819  1791 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:27.921396  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:27.923087  1571 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:27.933744  1538 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:27.936172  1865 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:27.937918  1865 raft_consensus.cc:697] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:27.939225  1865 consensus_queue.cc:237] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.942327  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:27.942628  1917 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.943291  1645 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:27.943477  1917 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.917171  1769 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:27.956007  1917 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:27.957592  1719 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:27.958936  1571 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:27.960767  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:01:27.962261  1611 leader_election.cc:343] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:27.962774  1685 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:27.964000  1791 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:27.964476  1791 raft_consensus.cc:697] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:01:27.965250  1791 consensus_queue.cc:237] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:27.960777  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:27.968034  1612 leader_election.cc:343] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:27.968863  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:27.971632  1917 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:27.991966  1645 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:28.010960  1768 tablet_bootstrap.cc:492] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:28.011751  1768 ts_tablet_manager.cc:1403] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.183s	user 0.036s	sys 0.034s
I20260709 13:01:28.017124  1769 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.018054  1769 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.025148  1768 raft_consensus.cc:359] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.026132  1768 raft_consensus.cc:385] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:28.026599  1769 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:28.027951  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:28.028239  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:28.028631  1645 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
W20260709 13:01:28.029743  1537 leader_election.cc:343] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:28.031719  1765 tablet_bootstrap.cc:492] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:28.032442  1765 ts_tablet_manager.cc:1403] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.188s	user 0.015s	sys 0.051s
I20260709 13:01:28.033247  1469 catalog_manager.cc:5697] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:28.026467  1768 raft_consensus.cc:740] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:28.035341  1768 consensus_queue.cc:260] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.030637  1538 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:28.035329  1913 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.036734  1913 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.037034  1865 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:28.037606  1865 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.042430  1765 raft_consensus.cc:359] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.043179  1765 raft_consensus.cc:385] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:28.043196  1766 tablet_bootstrap.cc:492] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:28.043553  1765 raft_consensus.cc:740] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:28.043900  1766 ts_tablet_manager.cc:1403] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.195s	user 0.030s	sys 0.025s
I20260709 13:01:28.044291  1765 consensus_queue.cc:260] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.039569  1865 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:28.045326  1768 ts_tablet_manager.cc:1434] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.001s
I20260709 13:01:28.047746  1766 raft_consensus.cc:359] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.059008  1766 raft_consensus.cc:385] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:28.058519  1865 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.059154  1768 tablet_bootstrap.cc:492] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:28.059356  1766 raft_consensus.cc:740] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:28.060156  1766 consensus_queue.cc:260] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.050362  1765 ts_tablet_manager.cc:1434] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 13:01:28.061185  1865 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:28.062180  1765 tablet_bootstrap.cc:492] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:28.062824  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:28.063380  1645 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:28.050869  1913 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:28.064167  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:01:28.065904  1537 leader_election.cc:343] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:28.052425  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:28.071105  1766 ts_tablet_manager.cc:1434] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.001s
I20260709 13:01:28.072654  1766 tablet_bootstrap.cc:492] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:28.078449  1768 tablet_bootstrap.cc:654] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:28.080505  1765 tablet_bootstrap.cc:654] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:28.057885  1468 catalog_manager.cc:5697] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:28.073007  1645 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
W20260709 13:01:28.074054  1612 leader_election.cc:343] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:28.083690  1538 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:28.051292  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:28.085311  1865 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:28.086018  1865 raft_consensus.cc:697] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:28.086875  1766 tablet_bootstrap.cc:654] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:28.090755  1719 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:28.087555  1865 consensus_queue.cc:237] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:28.095497  1611 leader_election.cc:343] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:28.099403  1769 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.105317  1769 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.107931  1769 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:28.109252  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:28.110190  1721 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:28.111802  1537 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:28.112553  1819 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:28.112921  1819 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.113354  1819 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:28.099131  1611 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:28.116360  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:28.117156  1646 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:28.120004  1913 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:28.125314  1863 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.126052  1863 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.128747  1863 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:28.132488  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:28.134519  1612 leader_election.cc:343] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:28.137352  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:28.138221  1819 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.140687  1875 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.141237  1819 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:28.141362  1875 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.144357  1875 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:28.146222  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:28.149736  1612 leader_election.cc:343] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:28.140853  1611 leader_election.cc:343] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:28.150851  1611 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:28.155483  1875 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:28.155369  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:01:28.174482  1611 leader_election.cc:343] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:28.175024  1611 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:28.175850  1875 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:28.176749  1765 tablet_bootstrap.cc:492] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:28.176720  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:28.177744  1765 ts_tablet_manager.cc:1403] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.116s	user 0.015s	sys 0.000s
I20260709 13:01:28.177908  1721 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:28.181172  1765 raft_consensus.cc:359] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.182034  1765 raft_consensus.cc:385] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:28.182318  1765 raft_consensus.cc:740] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:28.182394  1819 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.182989  1765 consensus_queue.cc:260] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.183200  1819 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.184727  1765 ts_tablet_manager.cc:1434] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:01:28.186370  1819 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:28.189087  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:28.194016  1537 leader_election.cc:343] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:28.200451  1765 tablet_bootstrap.cc:492] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
W20260709 13:01:28.202471  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 77733us
I20260709 13:01:28.209216  1960 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:28.213587  1768 tablet_bootstrap.cc:492] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:28.214761  1768 ts_tablet_manager.cc:1403] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.156s	user 0.021s	sys 0.028s
I20260709 13:01:28.215906  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:28.221938  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:28.228645  1645 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:28.223238  1648 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:28.242648  1768 raft_consensus.cc:359] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.243463  1768 raft_consensus.cc:385] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:28.243728  1768 raft_consensus.cc:740] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:28.244675  1768 consensus_queue.cc:260] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.249610  1648 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:28.250905  1471 catalog_manager.cc:5697] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:28.253492  1721 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:28.254792  1537 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:28.255142  1768 ts_tablet_manager.cc:1434] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.001s
I20260709 13:01:28.255617  1769 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:28.256516  1769 raft_consensus.cc:697] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:28.256633  1768 tablet_bootstrap.cc:492] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:28.257405  1769 consensus_queue.cc:237] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.278226  1766 tablet_bootstrap.cc:492] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:28.278831  1766 ts_tablet_manager.cc:1403] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.207s	user 0.051s	sys 0.026s
I20260709 13:01:28.287561  1765 tablet_bootstrap.cc:654] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:28.276975  1538 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:28.291494  1962 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:28.292586  1962 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.292909  1962 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:28.307183  1766 raft_consensus.cc:359] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.315024  1766 raft_consensus.cc:385] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:28.315444  1766 raft_consensus.cc:740] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:28.316011  1819 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:28.304433  1962 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.316377  1766 consensus_queue.cc:260] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.319274  1766 ts_tablet_manager.cc:1434] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.001s
I20260709 13:01:28.321282  1766 tablet_bootstrap.cc:492] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:28.321532  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:28.327482  1645 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:28.342718  1962 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:28.346081  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:28.348115  1719 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:28.351692  1765 tablet_bootstrap.cc:492] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:28.352337  1765 ts_tablet_manager.cc:1403] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.157s	user 0.016s	sys 0.005s
I20260709 13:01:28.345779  1469 catalog_manager.cc:5697] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:28.353094  1645 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:28.356894  1719 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:28.356956  1538 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:28.358180  1969 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:28.358736  1969 raft_consensus.cc:697] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:28.359803  1969 consensus_queue.cc:237] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.373445  1768 tablet_bootstrap.cc:654] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:28.375881  1765 raft_consensus.cc:359] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.376956  1765 raft_consensus.cc:385] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:28.377341  1765 raft_consensus.cc:740] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:28.379129  1766 tablet_bootstrap.cc:654] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:28.379983  1765 consensus_queue.cc:260] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.383693  1765 ts_tablet_manager.cc:1434] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 13:01:28.400460  1962 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.400974  1962 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.409991  1791 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:01:28.411514  1962 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:28.413666  1907 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:28.413915  1765 tablet_bootstrap.cc:492] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:28.414425  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:28.418318  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:28.419039  1645 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:28.419580  1766 tablet_bootstrap.cc:492] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:28.420120  1766 ts_tablet_manager.cc:1403] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.099s	user 0.013s	sys 0.017s
I20260709 13:01:28.420692  1538 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:01:28.421743  1537 leader_election.cc:343] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:28.422068  1970 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:28.422441  1970 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.422761  1970 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:28.436731  1970 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:28.454066  1519 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:28.465238  1766 raft_consensus.cc:359] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.466171  1768 tablet_bootstrap.cc:492] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:28.466351  1766 raft_consensus.cc:385] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:28.466857  1768 ts_tablet_manager.cc:1403] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.210s	user 0.027s	sys 0.014s
I20260709 13:01:28.466789  1766 raft_consensus.cc:740] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:28.474916  1766 consensus_queue.cc:260] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.476560  1766 ts_tablet_manager.cc:1434] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.056s	user 0.012s	sys 0.012s
I20260709 13:01:28.481585  1768 raft_consensus.cc:359] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.482399  1768 raft_consensus.cc:385] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:28.482677  1768 raft_consensus.cc:740] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:28.483250  1768 consensus_queue.cc:260] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.483901  1765 tablet_bootstrap.cc:654] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:28.492796  1768 ts_tablet_manager.cc:1434] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.005s
I20260709 13:01:28.494073  1768 tablet_bootstrap.cc:492] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:28.499392  1469 catalog_manager.cc:5697] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:28.504894  1970 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:28.505769  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:28.506518  1646 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:28.506083  1766 tablet_bootstrap.cc:492] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:28.531664  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:28.537184  1962 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.538218  1962 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:28.536487  1537 leader_election.cc:343] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:28.547487  1969 consensus_queue.cc:1048] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:28.555687  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:28.556391  1648 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:28.559995  1538 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:28.559836  1969 consensus_queue.cc:1048] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:28.561674  1969 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:28.562139  1969 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.569186  1768 tablet_bootstrap.cc:654] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:28.569850  1969 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:28.564353  1765 tablet_bootstrap.cc:492] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:28.572412  1765 ts_tablet_manager.cc:1403] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.159s	user 0.029s	sys 0.022s
I20260709 13:01:28.572940  1766 tablet_bootstrap.cc:654] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:28.578621  1962 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:28.579651  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:28.582705  1721 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:28.550241  1646 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:28.596642  1863 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.597178  1863 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.595937  1765 raft_consensus.cc:359] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.596107  1538 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:28.597945  1765 raft_consensus.cc:385] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:28.598347  1765 raft_consensus.cc:740] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:28.599313  1863 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:28.599020  1765 consensus_queue.cc:260] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.602087  1970 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:28.602670  1970 raft_consensus.cc:697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:28.602882  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:28.603533  1970 consensus_queue.cc:237] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:28.607457  1612 leader_election.cc:343] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:28.607985  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:01:28.613976  1611 leader_election.cc:343] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:28.614645  1611 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:28.618487  1962 consensus_queue.cc:1048] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:28.619720  1917 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:28.621371  1765 ts_tablet_manager.cc:1434] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.001s
I20260709 13:01:28.627878  1768 tablet_bootstrap.cc:492] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:28.628372  1768 ts_tablet_manager.cc:1403] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.135s	user 0.010s	sys 0.006s
I20260709 13:01:28.629803  1765 tablet_bootstrap.cc:492] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:28.631023  1768 raft_consensus.cc:359] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.631752  1768 raft_consensus.cc:385] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:28.632030  1768 raft_consensus.cc:740] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:28.627144  1969 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.632699  1768 consensus_queue.cc:260] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.643823  1969 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:28.644234  1768 ts_tablet_manager.cc:1434] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 13:01:28.646517  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:28.647253  1645 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:28.648007  1765 tablet_bootstrap.cc:654] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:28.652688  1766 tablet_bootstrap.cc:492] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:28.653256  1766 ts_tablet_manager.cc:1403] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.148s	user 0.008s	sys 0.009s
I20260709 13:01:28.646412  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:28.661517  1768 tablet_bootstrap.cc:492] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:28.666517  1719 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:28.667496  1720 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:28.668717  1645 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:28.671454  1766 raft_consensus.cc:359] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.672573  1766 raft_consensus.cc:385] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:28.672899  1766 raft_consensus.cc:740] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:28.674170  1766 consensus_queue.cc:260] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.674132  1538 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:28.683605  1769 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:28.684305  1769 raft_consensus.cc:697] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:28.685231  1769 consensus_queue.cc:237] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.688272  1719 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:28.680776  1469 catalog_manager.cc:5697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:28.705365  1766 ts_tablet_manager.cc:1434] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.003s
I20260709 13:01:28.717185  1766 tablet_bootstrap.cc:492] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:28.720134  1969 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.034s
I20260709 13:01:28.768244  1765 tablet_bootstrap.cc:492] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:28.768777  1765 ts_tablet_manager.cc:1403] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.140s	user 0.017s	sys 0.003s
I20260709 13:01:28.782370  1765 raft_consensus.cc:359] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.783638  1765 raft_consensus.cc:385] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:28.784006  1765 raft_consensus.cc:740] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:28.790580  1962 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:28.803936  1765 consensus_queue.cc:260] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.810040  1766 tablet_bootstrap.cc:654] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:28.817291  1765 ts_tablet_manager.cc:1434] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.048s	user 0.003s	sys 0.006s
I20260709 13:01:28.816000  1791 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.818583  1791 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.819919  1768 tablet_bootstrap.cc:654] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:28.825119  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:28.837658  1765 tablet_bootstrap.cc:492] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:28.840088  1471 catalog_manager.cc:5697] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:28.855278  1769 consensus_queue.cc:1048] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:28.866878  1791 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:28.866724  1865 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:28.867535  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:28.869868  1687 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:28.880330  1978 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:28.882130  1819 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.882771  1819 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.883569  1766 tablet_bootstrap.cc:492] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:28.884274  1766 ts_tablet_manager.cc:1403] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.167s	user 0.006s	sys 0.031s
I20260709 13:01:28.885723  1917 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.888723  1917 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.890923  1917 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:28.892518  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:28.892367  1819 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:28.893414  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:28.894155  1573 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
W20260709 13:01:28.894903  1612 leader_election.cc:343] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:28.894053  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:28.895996  1611 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:01:28.897061  1537 leader_election.cc:343] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:28.898029  1917 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:28.904071  1766 raft_consensus.cc:359] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.905066  1766 raft_consensus.cc:385] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:28.905440  1766 raft_consensus.cc:740] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:28.900840  1917 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.906165  1917 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:28.906507  1766 consensus_queue.cc:260] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.898725  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:28.910283  1970 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:01:28.915462  1863 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.916182  1863 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.919911  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:28.920439  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:28.921115  1573 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
W20260709 13:01:28.921720  1612 leader_election.cc:343] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:28.921991  1863 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:28.921844  1917 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.922462  1611 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:28.918597  1768 tablet_bootstrap.cc:492] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:28.923559  1768 ts_tablet_manager.cc:1403] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.262s	user 0.025s	sys 0.019s
I20260709 13:01:28.923620  1875 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:28.924158  1875 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.925484  1875 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:28.930389  1645 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 1: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:01:28.931671  1766 ts_tablet_manager.cc:1434] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.047s	user 0.010s	sys 0.001s
I20260709 13:01:28.933393  1766 tablet_bootstrap.cc:492] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:28.934239  1538 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:28.935461  1819 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:28.947806  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:28.951946  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:28.952873  1917 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:28.955157  1819 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:01:28.961783  1612 leader_election.cc:343] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:28.967774  1765 tablet_bootstrap.cc:654] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:28.968637  1573 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:28.956094  1766 tablet_bootstrap.cc:654] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:28.964049  1875 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.965837  1768 raft_consensus.cc:359] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.976593  1611 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:28.977871  1917 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:28.978969  1917 raft_consensus.cc:697] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:01:28.979995  1917 consensus_queue.cc:237] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:28.982880  1875 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:28.984102  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:01:28.985493  1612 leader_election.cc:343] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:28.986671  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:28.990306  1573 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:28.994175  1766 tablet_bootstrap.cc:492] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:28.998934  1766 ts_tablet_manager.cc:1403] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.066s	user 0.010s	sys 0.007s
I20260709 13:01:28.976938  1768 raft_consensus.cc:385] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:29.002410  1768 raft_consensus.cc:740] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:29.002295  1766 raft_consensus.cc:359] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.003113  1766 raft_consensus.cc:385] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:29.003430  1766 raft_consensus.cc:740] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:29.003831  1573 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:29.004156  1766 consensus_queue.cc:260] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.003568  1768 consensus_queue.cc:260] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.006665  1765 tablet_bootstrap.cc:492] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:29.007488  1765 ts_tablet_manager.cc:1403] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.177s	user 0.007s	sys 0.011s
I20260709 13:01:29.008057  1766 ts_tablet_manager.cc:1434] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.005s
I20260709 13:01:29.009745  1766 tablet_bootstrap.cc:492] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:29.012985  1768 ts_tablet_manager.cc:1434] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.089s	user 0.004s	sys 0.004s
I20260709 13:01:29.014272  1768 tablet_bootstrap.cc:492] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:29.018612  1766 tablet_bootstrap.cc:654] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:29.020499  1611 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:29.020918  1768 tablet_bootstrap.cc:654] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:29.022222  1917 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:29.021811  1470 catalog_manager.cc:5697] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:29.023608  1917 raft_consensus.cc:697] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:01:29.029829  1917 consensus_queue.cc:237] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.034198  1768 tablet_bootstrap.cc:492] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:29.034654  1768 ts_tablet_manager.cc:1403] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.007s
I20260709 13:01:29.038833  1768 raft_consensus.cc:359] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.039861  1768 raft_consensus.cc:385] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:29.040352  1768 raft_consensus.cc:740] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:29.042011  1768 consensus_queue.cc:260] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.044816  1768 ts_tablet_manager.cc:1434] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 13:01:29.052767  1721 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:29.058753  1768 tablet_bootstrap.cc:492] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:29.017627  1765 raft_consensus.cc:359] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.060202  1765 raft_consensus.cc:385] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:29.061349  1865 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:29.067137  1768 tablet_bootstrap.cc:654] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:29.062242  1765 raft_consensus.cc:740] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
W20260709 13:01:29.070812  1748 tablet.cc:2406] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:01:29.084823  1768 tablet_bootstrap.cc:492] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:29.085310  1768 ts_tablet_manager.cc:1403] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.004s
I20260709 13:01:29.084514  1970 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:29.089043  1863 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.089587  1863 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.091948  1863 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:29.097157  1768 raft_consensus.cc:359] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.079747  1765 consensus_queue.cc:260] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.098008  1768 raft_consensus.cc:385] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:29.098721  1768 raft_consensus.cc:740] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:29.099515  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:01:29.101200  1611 leader_election.cc:343] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:29.101269  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:29.102367  1768 consensus_queue.cc:260] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.104684  1768 ts_tablet_manager.cc:1434] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.002s
I20260709 13:01:29.105746  1768 tablet_bootstrap.cc:492] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
W20260709 13:01:29.115425  1612 leader_election.cc:343] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:29.116576  1612 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:29.112093  1907 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.118031  1907 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.119489  1768 tablet_bootstrap.cc:654] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:29.122645  1875 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:29.132670  1907 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:29.134135  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:29.135099  1645 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:29.137077  1687 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:29.138615  1907 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:29.139297  1907 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.139673  1907 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:29.141748  1768 tablet_bootstrap.cc:492] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:29.142345  1768 ts_tablet_manager.cc:1403] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.005s
I20260709 13:01:29.144450  1865 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.145236  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:29.145344  1865 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.147454  1768 raft_consensus.cc:359] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.150768  1907 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.151185  1768 raft_consensus.cc:385] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:29.151666  1768 raft_consensus.cc:740] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:29.152434  1768 consensus_queue.cc:260] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.155189  1769 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:29.150041  1573 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:29.156172  1768 ts_tablet_manager.cc:1434] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:01:29.157477  1768 tablet_bootstrap.cc:492] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:29.161677  1907 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:29.165580  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:29.166461  1768 tablet_bootstrap.cc:654] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:29.167908  1865 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:29.165755  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:29.170475  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:29.170395  1573 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:29.171705  1721 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 1: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:01:29.166347  1647 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:29.183013  1765 ts_tablet_manager.cc:1434] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.175s	user 0.008s	sys 0.000s
I20260709 13:01:29.197769  1765 tablet_bootstrap.cc:492] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:29.200011  1470 catalog_manager.cc:5697] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:29.188282  1647 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:29.205330  1768 tablet_bootstrap.cc:492] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:29.217988  1768 ts_tablet_manager.cc:1403] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.061s	user 0.021s	sys 0.005s
I20260709 13:01:29.206894  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:29.220227  1646 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 1: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:01:29.226495  1917 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.226652  1538 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:29.228047  1917 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.229538  1766 tablet_bootstrap.cc:492] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:29.230268  1766 ts_tablet_manager.cc:1403] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.221s	user 0.069s	sys 0.116s
I20260709 13:01:29.230675  1917 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:29.230912  1573 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:29.231882  1768 raft_consensus.cc:359] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.232568  1768 raft_consensus.cc:385] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:29.232820  1768 raft_consensus.cc:740] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:29.234316  1765 tablet_bootstrap.cc:654] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:29.236651  1768 consensus_queue.cc:260] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.238804  1768 ts_tablet_manager.cc:1434] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 13:01:29.238618  1766 raft_consensus.cc:359] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.239569  1766 raft_consensus.cc:385] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:29.239828  1768 tablet_bootstrap.cc:492] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:29.239957  1766 raft_consensus.cc:740] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:29.240175  1970 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:29.242307  1685 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:29.243664  1907 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:29.244194  1907 raft_consensus.cc:697] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:01:29.245002  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:29.245105  1907 consensus_queue.cc:237] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.241807  1766 consensus_queue.cc:260] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.250515  1766 ts_tablet_manager.cc:1434] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:01:29.250999  1768 tablet_bootstrap.cc:654] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:29.251653  1766 tablet_bootstrap.cc:492] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:29.254594  1865 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:01:29.258261  1612 leader_election.cc:343] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:29.260140  1766 tablet_bootstrap.cc:654] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:29.273444  1978 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.274238  1978 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.278811  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:29.280519  1571 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 1: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:01:29.284981  1978 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:29.284715  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:29.288551  1687 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:29.292558  1978 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:29.289355  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:01:29.295254  1611 leader_election.cc:343] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:29.295806  1611 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:29.297215  1863 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:29.305938  1766 tablet_bootstrap.cc:492] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:29.305811  1907 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.306667  1766 ts_tablet_manager.cc:1403] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.055s	user 0.023s	sys 0.027s
I20260709 13:01:29.301772  1765 tablet_bootstrap.cc:492] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:29.308161  1765 ts_tablet_manager.cc:1403] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.111s	user 0.021s	sys 0.015s
I20260709 13:01:29.308620  1907 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.310748  1765 raft_consensus.cc:359] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.311453  1765 raft_consensus.cc:385] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:29.311700  1765 raft_consensus.cc:740] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:29.311295  1471 catalog_manager.cc:5697] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:29.312760  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:29.313784  1573 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 1: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:01:29.314260  1768 tablet_bootstrap.cc:492] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:29.314704  1768 ts_tablet_manager.cc:1403] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.075s	user 0.014s	sys 0.015s
I20260709 13:01:29.317497  1766 raft_consensus.cc:359] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.318164  1766 raft_consensus.cc:385] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:29.318428  1766 raft_consensus.cc:740] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:29.319018  1766 consensus_queue.cc:260] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.316129  1765 consensus_queue.cc:260] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.321126  1765 ts_tablet_manager.cc:1434] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:01:29.319419  1768 raft_consensus.cc:359] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.321126  1766 ts_tablet_manager.cc:1434] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.000s
I20260709 13:01:29.321329  1907 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:29.319238  1978 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:29.322795  1766 tablet_bootstrap.cc:492] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:29.323639  1765 tablet_bootstrap.cc:492] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:29.321892  1768 raft_consensus.cc:385] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:29.328967  1768 raft_consensus.cc:740] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:29.329934  1768 consensus_queue.cc:260] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.332922  1765 tablet_bootstrap.cc:654] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:29.332993  1768 ts_tablet_manager.cc:1434] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:01:29.333353  1766 tablet_bootstrap.cc:654] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:29.334136  1768 tablet_bootstrap.cc:492] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:29.343824  1768 tablet_bootstrap.cc:654] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:29.346868  1765 tablet_bootstrap.cc:492] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:29.347530  1765 ts_tablet_manager.cc:1403] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.005s
I20260709 13:01:29.352065  1765 raft_consensus.cc:359] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.353371  1765 raft_consensus.cc:385] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:29.353857  1765 raft_consensus.cc:740] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:29.358299  1766 tablet_bootstrap.cc:492] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:29.360082  1766 ts_tablet_manager.cc:1403] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.038s	user 0.016s	sys 0.003s
I20260709 13:01:29.364491  1765 consensus_queue.cc:260] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.379279  1765 ts_tablet_manager.cc:1434] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 13:01:29.380496  1765 tablet_bootstrap.cc:492] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:29.373752  1768 tablet_bootstrap.cc:492] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:29.366752  1766 raft_consensus.cc:359] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.382972  1766 raft_consensus.cc:385] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:29.383009  1768 ts_tablet_manager.cc:1403] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.049s	user 0.022s	sys 0.014s
I20260709 13:01:29.383484  1766 raft_consensus.cc:740] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:29.384287  1766 consensus_queue.cc:260] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.387976  1721 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:29.391134  1796 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:29.389560  1766 ts_tablet_manager.cc:1434] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.003s
I20260709 13:01:29.362174  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:29.389484  1768 raft_consensus.cc:359] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.398281  1768 raft_consensus.cc:385] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:29.399387  1687 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:29.405270  1768 raft_consensus.cc:740] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:29.402671  1765 tablet_bootstrap.cc:654] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:29.406191  1768 consensus_queue.cc:260] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.407171  1978 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:29.404781  1766 tablet_bootstrap.cc:492] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:29.414139  1768 ts_tablet_manager.cc:1434] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 13:01:29.414683  1863 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:29.415210  1768 tablet_bootstrap.cc:492] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:29.416522  1766 tablet_bootstrap.cc:654] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:29.425256  1970 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.426295  1970 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.432004  1768 tablet_bootstrap.cc:654] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:29.436250  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:29.436976  1647 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:29.439466  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:29.440737  1719 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:29.441514  1765 tablet_bootstrap.cc:492] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:29.439613  1538 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:29.442269  1765 ts_tablet_manager.cc:1403] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.062s	user 0.021s	sys 0.006s
I20260709 13:01:29.443156  1865 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:29.443599  1865 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.444000  1865 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:29.438915  1970 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:29.457724  1865 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.466078  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:29.466883  1648 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:29.463817  1765 raft_consensus.cc:359] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.475939  1765 raft_consensus.cc:385] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:29.476351  1765 raft_consensus.cc:740] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:29.477110  1765 consensus_queue.cc:260] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.481989  1648 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:29.489861  1538 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:29.506950  1960 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:29.507755  1960 raft_consensus.cc:697] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:29.522189  1796 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.522601  1765 ts_tablet_manager.cc:1434] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.080s	user 0.008s	sys 0.002s
I20260709 13:01:29.523069  1796 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.524122  1765 tablet_bootstrap.cc:492] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:29.532029  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:29.534097  1768 tablet_bootstrap.cc:492] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:29.535280  1768 ts_tablet_manager.cc:1403] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.120s	user 0.025s	sys 0.034s
I20260709 13:01:29.508936  1960 consensus_queue.cc:237] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.538204  1768 raft_consensus.cc:359] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.539728  1768 raft_consensus.cc:385] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:29.540580  1768 raft_consensus.cc:740] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
W20260709 13:01:29.540855  1611 leader_election.cc:343] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:29.552028  1969 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.552590  1969 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.552301  1768 consensus_queue.cc:260] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.563167  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:29.563560  1796 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:29.564725  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:01:29.566327  1612 leader_election.cc:343] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:29.565464  1648 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:29.567137  1612 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:29.568816  1538 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:29.570318  1969 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:29.572206  1796 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:29.576171  1768 ts_tablet_manager.cc:1434] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.001s
I20260709 13:01:29.576417  1907 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.577049  1907 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.577617  1978 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.578450  1766 tablet_bootstrap.cc:492] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:29.578166  1978 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.579133  1766 ts_tablet_manager.cc:1403] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.175s	user 0.030s	sys 0.052s
I20260709 13:01:29.582764  1766 raft_consensus.cc:359] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.583993  1766 raft_consensus.cc:385] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:29.584432  1766 raft_consensus.cc:740] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:29.585358  1766 consensus_queue.cc:260] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.596383  1907 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:29.574011  1969 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:29.600977  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:29.601791  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 67057.5us
I20260709 13:01:29.602546  1969 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.603647  1969 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:29.602619  1721 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:29.609625  1766 ts_tablet_manager.cc:1434] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.003s
I20260709 13:01:29.612982  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:01:29.615163  1685 leader_election.cc:343] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:29.588235  1978 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:29.617556  1766 tablet_bootstrap.cc:492] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:29.636636  2026 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.637223  2026 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.639739  2026 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:29.640476  1768 tablet_bootstrap.cc:492] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:29.648478  1765 tablet_bootstrap.cc:654] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:29.645675  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:29.647035  1470 catalog_manager.cc:5697] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:29.650818  1768 tablet_bootstrap.cc:654] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:29.653013  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:29.653950  1648 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:29.656612  1969 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.659832  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:29.663239  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:29.664103  1646 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:29.669301  1969 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:29.674347  1571 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 1: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:01:29.674642  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:29.675084  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:29.676915  1721 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:29.669375  1865 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:01:29.681767  1612 leader_election.cc:343] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:29.683578  2023 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:01:29.680984  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:29.685318  1647 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:29.685781  1719 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:29.698086  1721 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:29.697940  1978 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.700901  1537 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:29.701287  2020 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:01:29.702047  1978 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.720693  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:29.728364  1796 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.728922  1796 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.724939  1687 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:01:29.708503  1969 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:29.722585  1978 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:29.743786  1969 raft_consensus.cc:697] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:29.734534  2020 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:29.756541  2020 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.756996  2020 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:29.763517  1611 leader_election.cc:343] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:29.764231  1611 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:29.777802  1796 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:29.779343  1796 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:29.793476  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:29.802639  2024 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.803455  2024 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.822196  1969 consensus_queue.cc:237] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.747696  1687 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:01:29.848664  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 85213.4us
W20260709 13:01:29.851979  1611 leader_election.cc:343] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:29.854333  1768 tablet_bootstrap.cc:492] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:29.854786  1768 ts_tablet_manager.cc:1403] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.215s	user 0.018s	sys 0.025s
I20260709 13:01:29.756043  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:01:29.853677  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 114079us
I20260709 13:01:29.852521  1978 raft_consensus.cc:2764] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
W20260709 13:01:29.905238  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 90772.3us
I20260709 13:01:29.905248  1796 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:01:29.912919  2029 log.cc:927] Time spent T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:01:29.883428  1570 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:29.932536  1768 raft_consensus.cc:359] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.938820  1768 raft_consensus.cc:385] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:29.921304  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:29.935060  1719 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:29.934583  1765 tablet_bootstrap.cc:492] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:29.932296  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:29.938035  2024 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:29.940856  1766 tablet_bootstrap.cc:654] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:29.939347  1768 raft_consensus.cc:740] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:29.938678  1647 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:29.976102  1768 consensus_queue.cc:260] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:29.946475  1765 ts_tablet_manager.cc:1403] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.423s	user 0.018s	sys 0.029s
I20260709 13:01:29.984541  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:01:29.929013  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 65283.6us
I20260709 13:01:29.987747  1768 ts_tablet_manager.cc:1434] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.133s	user 0.012s	sys 0.004s
W20260709 13:01:29.987974  1612 leader_election.cc:343] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:29.989094  1612 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:01:29.991277  1519 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:29.995312  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:29.993459  1917 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:29.994567  1768 tablet_bootstrap.cc:492] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:29.979671  1645 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:30.003512  1978 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:01:30.005496  1612 leader_election.cc:343] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.006079  1687 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:30.007369  1978 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:30.007869  1978 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.008329  1978 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:30.010592  1766 tablet_bootstrap.cc:492] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:30.011317  1766 ts_tablet_manager.cc:1403] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.400s	user 0.023s	sys 0.052s
I20260709 13:01:30.014124  1768 tablet_bootstrap.cc:654] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:30.014717  1766 raft_consensus.cc:359] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.016077  1766 raft_consensus.cc:385] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.016425  1766 raft_consensus.cc:740] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.017359  1766 consensus_queue.cc:260] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.018285  1978 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:30.019182  1611 leader_election.cc:343] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.019680  1611 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:30.020406  2023 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:30.021314  1978 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:30.021847  1766 ts_tablet_manager.cc:1434] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:01:30.022938  1766 tablet_bootstrap.cc:492] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:30.023830  2019 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.024483  2019 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.027254  1765 raft_consensus.cc:359] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.027978  1765 raft_consensus.cc:385] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.028275  1765 raft_consensus.cc:740] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.029162  2019 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:30.025801  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:30.034446  1647 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:30.029241  1765 consensus_queue.cc:260] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.051366  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:30.052112  1572 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:30.066519  1765 ts_tablet_manager.cc:1434] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.088s	user 0.009s	sys 0.000s
I20260709 13:01:29.999843  2020 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.100274  1765 tablet_bootstrap.cc:492] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:30.105710  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:30.105355  1572 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:30.106412  1720 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:30.116633  1685 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:01:30.089952  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 68913.3us
I20260709 13:01:30.147171  1612 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:30.159967  1468 catalog_manager.cc:5697] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:30.179388  2023 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:30.195581  2023 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.196012  2023 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:30.228353  2021 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:30.229048  2021 raft_consensus.cc:697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:01:30.228343  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:30.233544  1962 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:30.242832  1766 tablet_bootstrap.cc:654] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:30.238632  1611 leader_election.cc:343] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.256521  2020 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:30.233562  2021 consensus_queue.cc:237] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.269073  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:01:30.270787  1685 leader_election.cc:343] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.279992  1647 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:30.280649  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:30.280061  1768 tablet_bootstrap.cc:492] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:30.282166  1768 ts_tablet_manager.cc:1403] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.288s	user 0.026s	sys 0.050s
I20260709 13:01:30.281422  1643 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:30.301882  1819 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:01:30.319293  1643 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:30.321871  1687 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:01:30.323979  1978 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:30.324626  1978 raft_consensus.cc:697] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:01:30.325415  1819 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.325788  1978 consensus_queue.cc:237] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.326200  1819 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.329938  1768 raft_consensus.cc:359] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.334160  1768 raft_consensus.cc:385] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.334578  1768 raft_consensus.cc:740] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.335822  1768 consensus_queue.cc:260] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.344990  1819 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:30.345484  1768 ts_tablet_manager.cc:1434] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.062s	user 0.009s	sys 0.000s
I20260709 13:01:30.346956  1768 tablet_bootstrap.cc:492] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:30.347618  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:30.348425  1719 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:30.355502  1537 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:30.361686  1865 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:30.362149  1865 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.362516  1865 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:30.377120  1765 tablet_bootstrap.cc:654] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:30.358767  2023 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.421510  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:30.419811  1865 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.424991  2023 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:30.427829  2038 consensus_queue.cc:1048] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:01:30.433051  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:30.428401  1643 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
W20260709 13:01:30.436269  1611 leader_election.cc:343] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.438138  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:30.439080  1719 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:30.450078  1766 tablet_bootstrap.cc:492] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:30.450747  1766 ts_tablet_manager.cc:1403] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.428s	user 0.043s	sys 0.051s
I20260709 13:01:30.458545  1768 tablet_bootstrap.cc:654] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:30.459784  1766 raft_consensus.cc:359] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.461472  1766 raft_consensus.cc:385] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.462096  1766 raft_consensus.cc:740] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.468417  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:30.472787  1720 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:30.486171  1766 consensus_queue.cc:260] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.483722  1865 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:30.491556  1766 ts_tablet_manager.cc:1434] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.000s
I20260709 13:01:30.493230  1766 tablet_bootstrap.cc:492] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:30.500586  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:30.497165  1720 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:30.503065  1537 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:30.505401  2038 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:30.507239  1766 tablet_bootstrap.cc:654] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:30.508904  1643 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:30.513484  2032 consensus_queue.cc:1048] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:30.515544  2038 raft_consensus.cc:697] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:30.516871  2038 consensus_queue.cc:237] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.523460  1768 tablet_bootstrap.cc:492] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:30.523959  1768 ts_tablet_manager.cc:1403] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.177s	user 0.016s	sys 0.002s
I20260709 13:01:30.529675  1768 raft_consensus.cc:359] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.530344  1768 raft_consensus.cc:385] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.530628  1768 raft_consensus.cc:740] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.531272  1768 consensus_queue.cc:260] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.534160  1768 ts_tablet_manager.cc:1434] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:01:30.535535  1768 tablet_bootstrap.cc:492] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:30.545682  1796 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.546355  1796 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.549929  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:01:30.551316  1611 leader_election.cc:343] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.552597  1796 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:30.554181  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:30.555060  1643 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:30.554323  1765 tablet_bootstrap.cc:492] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:30.556572  1765 ts_tablet_manager.cc:1403] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.469s	user 0.020s	sys 0.034s
I20260709 13:01:30.557621  1766 tablet_bootstrap.cc:492] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:30.558163  1766 ts_tablet_manager.cc:1403] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.065s	user 0.018s	sys 0.000s
I20260709 13:01:30.559912  1719 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:30.563565  1765 raft_consensus.cc:359] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.564420  1765 raft_consensus.cc:385] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.564736  1765 raft_consensus.cc:740] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.565402  1765 consensus_queue.cc:260] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.570832  1766 raft_consensus.cc:359] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.571642  1766 raft_consensus.cc:385] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.571942  1766 raft_consensus.cc:740] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.573446  1765 ts_tablet_manager.cc:1434] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:01:30.574630  1765 tablet_bootstrap.cc:492] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
W20260709 13:01:30.575420  1612 leader_election.cc:343] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.575736  1768 tablet_bootstrap.cc:654] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:30.581852  1765 tablet_bootstrap.cc:654] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:30.581984  1612 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:30.583428  1612 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:01:30.583858  1796 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:30.584343  2019 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:30.584932  2019 raft_consensus.cc:697] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:01:30.579234  1470 catalog_manager.cc:5697] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:30.585870  2019 consensus_queue.cc:237] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.572639  1766 consensus_queue.cc:260] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.590705  1796 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.591275  1796 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.594206  1766 ts_tablet_manager.cc:1434] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 13:01:30.595646  1766 tablet_bootstrap.cc:492] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:30.604300  1796 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:30.604879  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:30.606053  1468 catalog_manager.cc:5697] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:30.608533  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:30.612133  1468 catalog_manager.cc:5697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
W20260709 13:01:30.614681  1612 leader_election.cc:343] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:30.616504  1611 leader_election.cc:343] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.621487  1611 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:30.624296  2019 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:30.632023  1766 tablet_bootstrap.cc:654] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:30.634809  1765 tablet_bootstrap.cc:492] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:30.634606  1471 catalog_manager.cc:5697] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:30.635671  1765 ts_tablet_manager.cc:1403] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.061s	user 0.018s	sys 0.017s
I20260709 13:01:30.639600  1768 tablet_bootstrap.cc:492] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:30.640465  1768 ts_tablet_manager.cc:1403] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.105s	user 0.014s	sys 0.003s
I20260709 13:01:30.653571  1768 raft_consensus.cc:359] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.653744  1765 raft_consensus.cc:359] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.654620  1768 raft_consensus.cc:385] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.654676  1765 raft_consensus.cc:385] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.655151  1768 raft_consensus.cc:740] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.655187  1765 raft_consensus.cc:740] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.656131  1765 consensus_queue.cc:260] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.656507  1768 consensus_queue.cc:260] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.658366  1765 ts_tablet_manager.cc:1434] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 13:01:30.659546  1765 tablet_bootstrap.cc:492] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:30.661608  1768 ts_tablet_manager.cc:1434] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.004s
I20260709 13:01:30.662907  1768 tablet_bootstrap.cc:492] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:30.668071  1765 tablet_bootstrap.cc:654] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:30.670315  1766 tablet_bootstrap.cc:492] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:30.670969  1766 ts_tablet_manager.cc:1403] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.076s	user 0.014s	sys 0.000s
I20260709 13:01:30.677222  1768 tablet_bootstrap.cc:654] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:30.682099  1766 raft_consensus.cc:359] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.683063  1766 raft_consensus.cc:385] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.683435  1766 raft_consensus.cc:740] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.688299  1766 consensus_queue.cc:260] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.692991  1768 tablet_bootstrap.cc:492] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:30.693513  1768 ts_tablet_manager.cc:1403] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.031s	user 0.007s	sys 0.008s
I20260709 13:01:30.694492  1766 ts_tablet_manager.cc:1434] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.023s	user 0.013s	sys 0.000s
I20260709 13:01:30.697054  1766 tablet_bootstrap.cc:492] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:30.696544  1768 raft_consensus.cc:359] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.697768  1768 raft_consensus.cc:385] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.698122  1768 raft_consensus.cc:740] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.698827  1768 consensus_queue.cc:260] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.700946  1768 ts_tablet_manager.cc:1434] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:01:30.703009  1768 tablet_bootstrap.cc:492] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:30.706367  1766 tablet_bootstrap.cc:654] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:30.710354  1768 tablet_bootstrap.cc:654] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:30.718519  1865 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.719233  1865 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.721849  1765 tablet_bootstrap.cc:492] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:30.722443  1765 ts_tablet_manager.cc:1403] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.063s	user 0.016s	sys 0.011s
I20260709 13:01:30.724098  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:30.724509  1768 tablet_bootstrap.cc:492] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:30.725090  1768 ts_tablet_manager.cc:1403] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.001s
I20260709 13:01:30.725463  1765 raft_consensus.cc:359] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:30.726470  1537 leader_election.cc:343] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.726814  1765 raft_consensus.cc:385] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.727164  1765 raft_consensus.cc:740] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.727385  1865 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:30.732604  1766 tablet_bootstrap.cc:492] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:30.733165  1766 ts_tablet_manager.cc:1403] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.010s
I20260709 13:01:30.733228  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:30.728560  1768 raft_consensus.cc:359] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.733670  1765 consensus_queue.cc:260] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.734427  1768 raft_consensus.cc:385] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.734745  1768 raft_consensus.cc:740] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.736604  1765 ts_tablet_manager.cc:1434] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:01:30.737093  1768 consensus_queue.cc:260] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.739986  1768 ts_tablet_manager.cc:1434] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:01:30.741060  1768 tablet_bootstrap.cc:492] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:30.746505  1765 tablet_bootstrap.cc:492] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:30.748951  1768 tablet_bootstrap.cc:654] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:30.752893  1538 leader_election.cc:343] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.753322  1538 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:30.753429  1766 raft_consensus.cc:359] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.754518  1766 raft_consensus.cc:385] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.754889  1766 raft_consensus.cc:740] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.755115  2038 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:30.756063  1766 consensus_queue.cc:260] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.759348  1766 ts_tablet_manager.cc:1434] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.002s
I20260709 13:01:30.761214  1766 tablet_bootstrap.cc:492] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:30.778419  1907 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:30.779897  1765 tablet_bootstrap.cc:654] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:30.796213  1766 tablet_bootstrap.cc:654] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:30.796168  1865 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.797992  1865 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.813743  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:30.814648  1647 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:30.815846  1907 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.816571  1907 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.819746  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:30.820696  1720 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:30.822465  1907 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:30.827994  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:30.828990  1647 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:30.831063  1687 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:30.832942  1907 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:30.835393  1907 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.835960  1907 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:30.850837  1537 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:30.854962  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:30.855306  1962 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:30.856005  1962 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.856477  1962 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:30.868287  1978 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:30.867555  2021 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.870296  2021 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.878459  2021 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:30.877941  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:01:30.883296  1687 leader_election.cc:343] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.909874  1865 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:30.912828  2023 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.913767  2023 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.917810  2032 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.918874  2032 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.932168  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:30.935554  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:30.938597  1570 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:30.951540  2032 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:30.954293  2023 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:30.958388  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:30.959911  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:30.960368  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 61113us
W20260709 13:01:30.962281  1611 leader_election.cc:343] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.964689  1768 tablet_bootstrap.cc:492] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:30.958308  1907 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.965269  1768 ts_tablet_manager.cc:1403] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.224s	user 0.103s	sys 0.058s
W20260709 13:01:30.968979  1612 leader_election.cc:343] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.968983  1768 raft_consensus.cc:359] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.969830  1612 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:30.970175  1768 raft_consensus.cc:385] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.971079  1796 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:30.970635  1768 raft_consensus.cc:740] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.973198  1768 consensus_queue.cc:260] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.951447  1962 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:30.976014  1685 leader_election.cc:343] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.976658  1572 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 1: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:01:30.976603  1685 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:30.977427  1907 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:30.975915  1768 ts_tablet_manager.cc:1434] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:01:30.981317  1766 tablet_bootstrap.cc:492] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:30.992444  2075 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.993115  2075 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:30.993880  1766 ts_tablet_manager.cc:1403] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.233s	user 0.043s	sys 0.037s
I20260709 13:01:30.982477  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:30.995673  1648 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:30.978178  2076 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:30.990577  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:30.979131  1962 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:30.998850  1768 tablet_bootstrap.cc:492] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:31.003537  2075 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:31.011693  1768 tablet_bootstrap.cc:654] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.011905  1648 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:31.017535  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:31.018929  1538 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:31.019485  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:31.020186  1647 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:31.022060  2032 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:31.022854  2032 raft_consensus.cc:697] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:31.022540  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:31.024170  1648 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 1: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:01:31.023761  2032 consensus_queue.cc:237] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.025571  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:31.026304  1720 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 1: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:01:30.997581  1766 raft_consensus.cc:359] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.027303  1766 raft_consensus.cc:385] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.027812  1766 raft_consensus.cc:740] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.028647  1766 consensus_queue.cc:260] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.032313  1766 ts_tablet_manager.cc:1434] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 13:01:31.033411  1766 tablet_bootstrap.cc:492] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:31.040009  2075 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:31.016026  1765 tablet_bootstrap.cc:492] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:31.017222  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:01:31.036686  1519 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:31.059829  1687 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:31.059948  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:31.049715  1962 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:31.064548  1907 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:31.064958  1907 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.065287  1907 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.067021  1962 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:31.072546  2038 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:01:31.060693  1611 consensus_peers.cc:597] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:31.073710  1765 ts_tablet_manager.cc:1403] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.327s	user 0.059s	sys 0.062s
I20260709 13:01:31.063123  1766 tablet_bootstrap.cc:654] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:31.082712  1685 leader_election.cc:343] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.082049  1907 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.084945  1468 catalog_manager.cc:5697] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:31.084156  2076 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:01:31.084280  1685 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:31.102315  2076 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:01:31.103343  1907 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:31.094499  1765 raft_consensus.cc:359] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.107988  1765 raft_consensus.cc:385] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.110386  1765 raft_consensus.cc:740] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.115777  1765 consensus_queue.cc:260] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.113499  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:31.120430  1648 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.124545  1765 ts_tablet_manager.cc:1434] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.002s
I20260709 13:01:31.126011  1766 tablet_bootstrap.cc:492] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:31.126379  1765 tablet_bootstrap.cc:492] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:31.126622  1766 ts_tablet_manager.cc:1403] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.093s	user 0.016s	sys 0.000s
I20260709 13:01:31.129999  1766 raft_consensus.cc:359] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.132145  1766 raft_consensus.cc:385] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.132470  1766 raft_consensus.cc:740] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.133109  1766 consensus_queue.cc:260] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.133497  1648 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:31.135476  1766 ts_tablet_manager.cc:1434] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:01:31.137776  1766 tablet_bootstrap.cc:492] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:31.147231  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:31.148169  1687 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:31.151618  1907 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:31.152163  1907 raft_consensus.cc:697] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:01:31.153015  1907 consensus_queue.cc:237] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.148309  1571 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.169906  1796 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.172142  1796 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.181694  1796 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:31.198251  1768 tablet_bootstrap.cc:492] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:31.199132  1768 ts_tablet_manager.cc:1403] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.201s	user 0.015s	sys 0.012s
I20260709 13:01:31.208775  2076 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.209357  2076 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.211434  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:31.211925  2076 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:31.219830  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:31.220772  1648 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:31.218468  2077 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:01:31.228619  1687 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:31.230895  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:31.236533  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:31.237323  1573 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:31.242280  2074 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:31.242719  2074 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.243162  2074 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.248170  1611 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:31.252079  2019 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:31.252490  2019 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.252843  2019 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.254545  1768 raft_consensus.cc:359] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.255270  1768 raft_consensus.cc:385] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.255541  1768 raft_consensus.cc:740] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
W20260709 13:01:31.256867  1612 leader_election.cc:343] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.258168  1768 consensus_queue.cc:260] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.266763  1765 tablet_bootstrap.cc:654] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:31.269464  1685 leader_election.cc:343] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.269609  1768 ts_tablet_manager.cc:1434] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.070s	user 0.007s	sys 0.001s
I20260709 13:01:31.298092  1768 tablet_bootstrap.cc:492] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:31.299948  1571 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:31.307864  1766 tablet_bootstrap.cc:654] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.323315  2023 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.323784  2023 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.322765  1470 catalog_manager.cc:5697] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:31.303766  2019 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.335942  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:01:31.337460  1611 leader_election.cc:343] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.338436  2023 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:31.343448  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:31.344115  1571 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.348302  1765 tablet_bootstrap.cc:492] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:31.348927  1765 ts_tablet_manager.cc:1403] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.223s	user 0.015s	sys 0.008s
I20260709 13:01:31.350733  1768 tablet_bootstrap.cc:654] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.351672  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:31.352804  2019 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:31.354550  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:31.355125  1720 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.355407  1765 raft_consensus.cc:359] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.356248  1765 raft_consensus.cc:385] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.356537  1765 raft_consensus.cc:740] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.357726  1765 consensus_queue.cc:260] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:31.359916  1612 leader_election.cc:343] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.360520  1612 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:31.361303  2087 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:31.338702  2074 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.363862  2074 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:31.366451  1765 ts_tablet_manager.cc:1434] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.001s
I20260709 13:01:31.365149  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:31.369143  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:31.369848  1648 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.371910  1765 tablet_bootstrap.cc:492] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
W20260709 13:01:31.375528  1685 leader_election.cc:343] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.372524  1571 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:31.382787  1611 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:31.385537  1720 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:31.389957  2023 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:31.390712  2023 raft_consensus.cc:697] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:01:31.391716  2023 consensus_queue.cc:237] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.393748  1648 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:31.399482  1766 tablet_bootstrap.cc:492] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:31.400089  1766 ts_tablet_manager.cc:1403] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.263s	user 0.015s	sys 0.007s
I20260709 13:01:31.403522  1768 tablet_bootstrap.cc:492] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:31.404053  1768 ts_tablet_manager.cc:1403] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.115s	user 0.014s	sys 0.008s
I20260709 13:01:31.403730  1766 raft_consensus.cc:359] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.404430  1766 raft_consensus.cc:385] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.404742  1766 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.405735  1766 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.407819  1687 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:01:31.409231  2074 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:31.409941  2074 raft_consensus.cc:697] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:01:31.411013  2074 consensus_queue.cc:237] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.412375  1768 raft_consensus.cc:359] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.413322  1768 raft_consensus.cc:385] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.413620  1768 raft_consensus.cc:740] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.414333  1768 consensus_queue.cc:260] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.416869  1765 tablet_bootstrap.cc:654] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.418576  1768 ts_tablet_manager.cc:1434] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.004s
I20260709 13:01:31.417901  2087 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.422501  1768 tablet_bootstrap.cc:492] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:31.416828  1470 catalog_manager.cc:5697] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:31.417250  1766 ts_tablet_manager.cc:1434] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.001s
I20260709 13:01:31.422066  2087 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.424562  1766 tablet_bootstrap.cc:492] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:31.425997  2087 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:31.428048  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:31.428319  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:31.429915  1612 leader_election.cc:343] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:31.433183  1611 leader_election.cc:343] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.433838  1611 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:31.435119  2023 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:31.440428  1907 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.440990  1766 tablet_bootstrap.cc:654] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.441232  1907 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.444816  1907 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:31.446345  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:31.446805  1765 tablet_bootstrap.cc:492] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:31.452523  1765 ts_tablet_manager.cc:1403] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.081s	user 0.019s	sys 0.018s
I20260709 13:01:31.453087  1648 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:31.455842  1765 raft_consensus.cc:359] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.456606  1765 raft_consensus.cc:385] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.456856  1765 raft_consensus.cc:740] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.457566  1765 consensus_queue.cc:260] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.447815  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:31.481076  1468 catalog_manager.cc:5697] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:31.482311  1765 ts_tablet_manager.cc:1434] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.002s
I20260709 13:01:31.475414  1687 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:31.485323  2075 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:31.476563  1571 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:31.486227  2075 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.493384  1765 tablet_bootstrap.cc:492] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:31.491597  2075 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.513952  1766 tablet_bootstrap.cc:492] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:31.514487  1766 ts_tablet_manager.cc:1403] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.090s	user 0.021s	sys 0.003s
I20260709 13:01:31.524657  2075 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.528007  2075 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:31.530573  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:31.530647  1766 raft_consensus.cc:359] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.531334  1648 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.531536  1766 raft_consensus.cc:385] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.531926  1766 raft_consensus.cc:740] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.533113  1766 consensus_queue.cc:260] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.535549  1765 tablet_bootstrap.cc:654] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.541309  1766 ts_tablet_manager.cc:1434] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:01:31.543074  1766 tablet_bootstrap.cc:492] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:31.544309  2023 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.544902  2023 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.549340  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:31.552184  1572 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.552678  1766 tablet_bootstrap.cc:654] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.551429  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:31.553793  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:31.554751  1611 leader_election.cc:343] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:31.555405  1612 leader_election.cc:343] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.555948  1612 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:31.557260  2087 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:31.560120  1766 tablet_bootstrap.cc:492] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:31.560600  1766 ts_tablet_manager.cc:1403] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.004s
I20260709 13:01:31.560863  1571 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.561681  1648 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:31.564146  1765 tablet_bootstrap.cc:492] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:31.550490  2023 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:31.564649  1765 ts_tablet_manager.cc:1403] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.074s	user 0.013s	sys 0.011s
I20260709 13:01:31.568368  2038 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:31.569125  1768 tablet_bootstrap.cc:654] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.570801  1766 raft_consensus.cc:359] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.571529  1766 raft_consensus.cc:385] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.571782  1766 raft_consensus.cc:740] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.572410  1766 consensus_queue.cc:260] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.574563  1766 ts_tablet_manager.cc:1434] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:01:31.575790  1766 tablet_bootstrap.cc:492] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:31.576538  1687 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:31.577911  2075 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:31.578528  2075 raft_consensus.cc:697] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:01:31.579442  2075 consensus_queue.cc:237] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.583504  1572 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:31.588351  1766 tablet_bootstrap.cc:654] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.595238  1765 raft_consensus.cc:359] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.595954  1765 raft_consensus.cc:385] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.596218  1765 raft_consensus.cc:740] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.596870  1765 consensus_queue.cc:260] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.599371  1765 ts_tablet_manager.cc:1434] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.002s
I20260709 13:01:31.593752  1768 tablet_bootstrap.cc:492] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:31.601023  1768 ts_tablet_manager.cc:1403] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.179s	user 0.018s	sys 0.000s
I20260709 13:01:31.604115  1765 tablet_bootstrap.cc:492] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:31.617839  2023 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.619227  2023 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.622565  1768 raft_consensus.cc:359] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.623363  1768 raft_consensus.cc:385] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.623684  1768 raft_consensus.cc:740] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.623711  2023 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:31.624490  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:01:31.626639  1611 leader_election.cc:343] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.627161  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:31.628715  1768 consensus_queue.cc:260] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.630101  2074 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:31.630400  1722 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:31.631282  2087 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.632193  2087 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.632894  1612 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:01:31.634387  2019 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:31.634933  2019 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.635293  2019 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.640915  2087 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:31.641955  1768 ts_tablet_manager.cc:1434] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.041s	user 0.010s	sys 0.000s
I20260709 13:01:31.644454  1768 tablet_bootstrap.cc:492] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:31.660108  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:31.664323  1865 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:01:31.661039  1722 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:31.674373  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:31.686712  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:31.689527  1796 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:31.691557  1796 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.691931  1796 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.719174  1768 tablet_bootstrap.cc:654] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.727829  2019 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.732014  1766 tablet_bootstrap.cc:492] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:31.732535  1766 ts_tablet_manager.cc:1403] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.157s	user 0.016s	sys 0.004s
I20260709 13:01:31.734426  2074 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:31.739009  1468 catalog_manager.cc:5697] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
W20260709 13:01:31.753575  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 64237.2us
I20260709 13:01:31.730265  2019 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:31.758373  1765 tablet_bootstrap.cc:654] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.762851  1796 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:31.778822  1611 leader_election.cc:343] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.791991  1796 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:31.787320  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:31.801903  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:31.802474  1718 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.803728  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:31.804411  1721 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.806556  2075 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.807134  2075 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.811496  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:31.813261  1766 raft_consensus.cc:359] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.814126  1766 raft_consensus.cc:385] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.814527  1766 raft_consensus.cc:740] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.815218  1766 consensus_queue.cc:260] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.819089  1766 ts_tablet_manager.cc:1434] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.086s	user 0.006s	sys 0.001s
I20260709 13:01:31.819258  1718 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:31.822090  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:31.826131  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:31.826969  2023 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:31.827466  2023 raft_consensus.cc:697] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:01:31.826726  1611 leader_election.cc:343] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.814831  2075 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:31.820318  1766 tablet_bootstrap.cc:492] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
W20260709 13:01:31.831045  1611 leader_election.cc:343] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.831048  1721 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:31.832490  1612 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:01:31.834326  1796 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:31.835147  1796 raft_consensus.cc:697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:01:31.836201  1796 consensus_queue.cc:237] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:31.843858  1687 leader_election.cc:343] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.854882  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:31.828277  2023 consensus_queue.cc:237] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:31.864166  1685 leader_election.cc:343] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.864740  1685 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:31.867249  2075 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:31.888685  1768 tablet_bootstrap.cc:492] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:31.890097  1768 ts_tablet_manager.cc:1403] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.246s	user 0.029s	sys 0.023s
I20260709 13:01:31.923481  1765 tablet_bootstrap.cc:492] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:31.921620  1768 raft_consensus.cc:359] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.953146  1768 raft_consensus.cc:385] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.953593  1768 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.954329  1766 tablet_bootstrap.cc:654] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.955168  1768 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.959910  1768 ts_tablet_manager.cc:1434] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.069s	user 0.016s	sys 0.009s
I20260709 13:01:31.961319  1768 tablet_bootstrap.cc:492] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:31.968989  1768 tablet_bootstrap.cc:654] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.969934  1765 ts_tablet_manager.cc:1403] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.366s	user 0.018s	sys 0.007s
I20260709 13:01:31.969676  2108 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:31.974020  1765 raft_consensus.cc:359] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.974882  1765 raft_consensus.cc:385] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.970880  1469 catalog_manager.cc:5697] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:31.975175  1765 raft_consensus.cc:740] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.976552  1765 consensus_queue.cc:260] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:31.976912  1469 catalog_manager.cc:5697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:31.978947  1765 ts_tablet_manager.cc:1434] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:01:31.979419  2019 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:31.980121  1765 tablet_bootstrap.cc:492] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
W20260709 13:01:31.987169  1685 consensus_peers.cc:597] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:31.994092  1765 tablet_bootstrap.cc:654] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.012567  2023 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.013167  2023 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.018625  2023 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:32.019639  2075 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:01:32.022792  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:32.024288  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:32.024696  1768 tablet_bootstrap.cc:492] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
W20260709 13:01:32.024588  1611 leader_election.cc:343] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:32.025220  1768 ts_tablet_manager.cc:1403] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.003s
W20260709 13:01:32.027595  1612 leader_election.cc:343] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:32.028303  1612 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:32.030858  2023 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:32.037305  1768 raft_consensus.cc:359] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.038296  1768 raft_consensus.cc:385] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.038549  1768 raft_consensus.cc:740] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.039323  1768 consensus_queue.cc:260] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.040822  1766 tablet_bootstrap.cc:492] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:32.041334  1766 ts_tablet_manager.cc:1403] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.221s	user 0.016s	sys 0.025s
I20260709 13:01:32.046384  2106 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.046643  2076 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:32.046939  2106 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.049410  1768 ts_tablet_manager.cc:1434] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.024s	user 0.002s	sys 0.007s
I20260709 13:01:32.050685  2106 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:32.051746  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:32.053357  1537 leader_election.cc:343] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:32.056057  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:32.056756  1644 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:32.060695  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:32.060840  1768 tablet_bootstrap.cc:492] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:32.063259  1865 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:32.074772  1865 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.075213  1865 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.059810  2033 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.078282  2033 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.122282  1907 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.130522  1907 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.138731  1765 tablet_bootstrap.cc:492] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:32.136643  2075 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:32.139233  1765 ts_tablet_manager.cc:1403] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.159s	user 0.020s	sys 0.018s
I20260709 13:01:32.139413  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:32.140563  1907 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:32.141875  1765 raft_consensus.cc:359] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.142561  1765 raft_consensus.cc:385] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:01:32.149570  1685 leader_election.cc:343] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:32.142823  1765 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.151253  1765 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.154491  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:32.155117  1648 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:32.163466  1687 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:01:32.166647  2074 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:32.167137  2074 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.167533  2074 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.169260  1768 tablet_bootstrap.cc:654] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.149174  1865 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.154075  1765 ts_tablet_manager.cc:1434] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.002s
I20260709 13:01:32.189234  1765 tablet_bootstrap.cc:492] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:32.190896  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:32.190376  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:32.187450  1766 raft_consensus.cc:359] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.192498  1644 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.192520  1766 raft_consensus.cc:385] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.193833  1766 raft_consensus.cc:740] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
W20260709 13:01:32.195726  1537 leader_election.cc:343] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:32.197836  2074 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.198451  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:32.206597  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:32.201043  1766 consensus_queue.cc:260] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.209225  1721 raft_consensus.cc:2393] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 1: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:01:32.219494  1537 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:32.221050  2074 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:32.224229  1765 tablet_bootstrap.cc:654] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.224488  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:32.225162  1648 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.216813  1766 ts_tablet_manager.cc:1434] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.175s	user 0.012s	sys 0.017s
I20260709 13:01:32.226828  1766 tablet_bootstrap.cc:492] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:32.198325  2033 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:32.234409  1648 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:32.236419  1687 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:32.237432  1907 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:32.238183  1907 raft_consensus.cc:697] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:01:32.239110  1907 consensus_queue.cc:237] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.189296  1865 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:32.246582  1644 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:32.248003  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:32.251950  2033 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:32.256457  2033 raft_consensus.cc:697] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:32.257570  2033 consensus_queue.cc:237] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.263382  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:32.258867  2106 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.270789  1766 tablet_bootstrap.cc:654] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:32.276623  1685 leader_election.cc:343] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:32.281754  1768 tablet_bootstrap.cc:492] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:32.282325  1768 ts_tablet_manager.cc:1403] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.222s	user 0.021s	sys 0.003s
I20260709 13:01:32.283767  1765 tablet_bootstrap.cc:492] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:32.284438  1765 ts_tablet_manager.cc:1403] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.095s	user 0.032s	sys 0.023s
I20260709 13:01:32.287731  1765 raft_consensus.cc:359] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.288482  1765 raft_consensus.cc:385] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:01:32.289812  1519 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:32.290009  1796 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:32.288769  1765 raft_consensus.cc:740] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.292153  1765 consensus_queue.cc:260] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.294756  1765 ts_tablet_manager.cc:1434] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:01:32.295816  1765 tablet_bootstrap.cc:492] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:32.296828  2106 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:32.303736  1765 tablet_bootstrap.cc:654] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.316036  1765 tablet_bootstrap.cc:492] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:32.316546  1768 raft_consensus.cc:359] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.317392  1768 raft_consensus.cc:385] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.317683  1768 raft_consensus.cc:740] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.316598  1765 ts_tablet_manager.cc:1403] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.005s
I20260709 13:01:32.322196  1768 consensus_queue.cc:260] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.324806  1768 ts_tablet_manager.cc:1434] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.000s
I20260709 13:01:32.326315  1768 tablet_bootstrap.cc:492] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:32.324031  1468 catalog_manager.cc:5697] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:32.321290  1765 raft_consensus.cc:359] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.331848  1765 raft_consensus.cc:385] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.338107  1765 raft_consensus.cc:740] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.335580  1768 tablet_bootstrap.cc:654] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.338974  1765 consensus_queue.cc:260] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.342384  1765 ts_tablet_manager.cc:1434] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 13:01:32.346156  1765 tablet_bootstrap.cc:492] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:32.354401  1765 tablet_bootstrap.cc:654] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.355540  2023 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:32.366802  2106 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.367714  1960 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.367626  2106 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.368499  1960 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.371272  2106 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:32.371609  1960 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:32.377789  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:32.378939  1765 tablet_bootstrap.cc:492] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:32.378865  1643 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:32.379837  1765 ts_tablet_manager.cc:1403] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.006s
I20260709 13:01:32.382550  1765 raft_consensus.cc:359] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.382778  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:32.384485  1765 raft_consensus.cc:385] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.384948  1765 raft_consensus.cc:740] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.386009  1765 consensus_queue.cc:260] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.389158  1765 ts_tablet_manager.cc:1434] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:01:32.391553  1766 tablet_bootstrap.cc:492] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:32.392127  1766 ts_tablet_manager.cc:1403] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.166s	user 0.016s	sys 0.019s
I20260709 13:01:32.393283  1765 tablet_bootstrap.cc:492] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:32.402050  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:32.402758  1765 tablet_bootstrap.cc:654] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.402853  1720 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:32.408412  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:32.411088  1611 consensus_peers.cc:597] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:32.384768  1648 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:32.414987  1765 tablet_bootstrap.cc:492] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:32.415589  1765 ts_tablet_manager.cc:1403] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.003s
I20260709 13:01:32.419667  1766 raft_consensus.cc:359] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.420387  1766 raft_consensus.cc:385] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.420663  1766 raft_consensus.cc:740] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.421299  1766 consensus_queue.cc:260] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.422915  1537 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:32.424588  1765 raft_consensus.cc:359] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.424804  1907 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.425259  1765 raft_consensus.cc:385] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.425387  2106 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:32.425595  1765 raft_consensus.cc:740] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.425817  2106 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.426239  2106 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:32.427455  1537 leader_election.cc:343] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:32.434471  1573 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.437232  1538 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:32.438480  1960 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:32.438907  1960 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.439225  1960 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.444922  2077 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.445668  2077 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.426515  1765 consensus_queue.cc:260] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.448598  1960 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.450919  1765 ts_tablet_manager.cc:1434] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.000s
I20260709 13:01:32.450279  2106 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.452073  1765 tablet_bootstrap.cc:492] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:32.456493  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:32.457556  1717 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.459352  2106 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:32.466511  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:32.466794  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:01:32.470335  1537 leader_election.cc:343] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:32.473922  1611 leader_election.cc:343] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:32.425662  1907 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.463637  1960 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:32.478061  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:32.483116  1717 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:32.496903  1537 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:32.495133  1765 tablet_bootstrap.cc:654] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.501279  1960 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:32.504145  1960 raft_consensus.cc:697] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:32.444274  1766 ts_tablet_manager.cc:1434] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.052s	user 0.009s	sys 0.000s
I20260709 13:01:32.507488  1766 tablet_bootstrap.cc:492] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:32.464322  2077 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:32.508127  1470 catalog_manager.cc:5697] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:32.516187  1765 tablet_bootstrap.cc:492] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:32.505877  1960 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.516712  1765 ts_tablet_manager.cc:1403] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.065s	user 0.017s	sys 0.001s
I20260709 13:01:32.516546  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:32.517472  1648 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 1: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:01:32.501816  1907 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:32.521245  1765 raft_consensus.cc:359] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.522809  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:32.523478  1647 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.522805  1765 raft_consensus.cc:385] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.525463  1765 raft_consensus.cc:740] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.526274  1765 consensus_queue.cc:260] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.528805  1765 ts_tablet_manager.cc:1434] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:01:32.530100  1765 tablet_bootstrap.cc:492] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:32.538357  1765 tablet_bootstrap.cc:654] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.542162  1647 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:32.551599  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:32.552635  1647 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.555801  2038 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.557101  1765 tablet_bootstrap.cc:492] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:32.557121  2038 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.558071  1765 ts_tablet_manager.cc:1403] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.012s
I20260709 13:01:32.561101  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:32.562570  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:32.564447  1537 leader_election.cc:343] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:32.565089  2038 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:32.567024  1765 raft_consensus.cc:359] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.568593  1765 raft_consensus.cc:385] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.569113  1765 raft_consensus.cc:740] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.570134  1765 consensus_queue.cc:260] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.572500  1765 ts_tablet_manager.cc:1434] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:01:32.574944  1765 tablet_bootstrap.cc:492] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:32.581140  1819 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.581830  1819 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.583575  1766 tablet_bootstrap.cc:654] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.584080  1819 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:32.585367  1765 tablet_bootstrap.cc:654] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.591543  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:32.594347  1717 raft_consensus.cc:2393] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 1: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:01:32.596364  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 56836.7us
I20260709 13:01:32.597865  2077 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:01:32.600020  1612 leader_election.cc:343] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:32.600570  1612 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:32.601440  1538 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:32.602496  1819 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:32.602320  1687 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:32.602768  2077 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:32.603329  1819 raft_consensus.cc:697] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:32.604714  1907 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.631371  1768 tablet_bootstrap.cc:492] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:32.632445  2038 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.636255  1647 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:32.636961  1768 ts_tablet_manager.cc:1403] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.311s	user 0.081s	sys 0.084s
I20260709 13:01:32.604240  1819 consensus_queue.cc:237] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.633035  2038 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.696909  2019 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.697446  2019 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.701179  1768 raft_consensus.cc:359] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.703656  1768 raft_consensus.cc:385] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.704084  1768 raft_consensus.cc:740] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.705756  1768 consensus_queue.cc:260] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.701598  2019 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:32.722507  2038 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:32.735131  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:32.727573  1907 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:32.738759  1768 ts_tablet_manager.cc:1434] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.074s	user 0.010s	sys 0.004s
I20260709 13:01:32.735911  1571 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:32.753298  1768 tablet_bootstrap.cc:492] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:32.754338  1765 tablet_bootstrap.cc:492] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:32.754851  1765 ts_tablet_manager.cc:1403] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.180s	user 0.014s	sys 0.006s
I20260709 13:01:32.759605  1611 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:32.763265  1796 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:32.763705  1796 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.763978  1796 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.774748  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:32.774613  1765 raft_consensus.cc:359] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.775492  1765 raft_consensus.cc:385] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.776085  1765 raft_consensus.cc:740] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.776816  1765 consensus_queue.cc:260] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:32.780580  1537 leader_election.cc:343] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:32.782931  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:32.783591  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:32.783679  1648 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:32.796873  1765 ts_tablet_manager.cc:1434] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.002s
I20260709 13:01:32.809217  1766 tablet_bootstrap.cc:492] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:32.848766  1796 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.864825  1796 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:32.849737  1468 catalog_manager.cc:5697] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:32.869053  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 142868us
I20260709 13:01:32.911530  2145 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.912309  2145 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:32.917742  2145 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:32.983438  1766 ts_tablet_manager.cc:1403] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.476s	user 0.029s	sys 0.036s
I20260709 13:01:32.984773  1765 tablet_bootstrap.cc:492] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:32.983600  1768 tablet_bootstrap.cc:700] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838: Time spent opening tablet: real 0.197s	user 0.002s	sys 0.000s
W20260709 13:01:32.990064  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 116922us
I20260709 13:01:32.989951  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:32.991159  1571 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.992260  1768 tablet_bootstrap.cc:654] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.010313  1571 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:33.016968  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:33.017800  1648 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:33.022221  1766 raft_consensus.cc:359] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:33.023011  1766 raft_consensus.cc:385] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.025785  1766 raft_consensus.cc:740] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.013711  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:33.037876  1766 consensus_queue.cc:260] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:33.055583  1766 ts_tablet_manager.cc:1434] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.072s	user 0.003s	sys 0.005s
I20260709 13:01:33.115104  1765 tablet_bootstrap.cc:654] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.042444  1644 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:33.130124  2033 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.130852  2033 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:33.133106  2033 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:33.140823  1766 tablet_bootstrap.cc:492] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:33.147374  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:33.148020  1571 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:33.159467  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:33.160144  1648 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:33.163501  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:33.164444  1721 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:33.172173  1538 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:33.178467  2033 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.179292  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:33.180087  1720 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.182061  1538 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:33.183032  2038 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:33.183415  2038 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.183746  2038 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.186640  1687 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:33.227270  1611 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:33.212154  2148 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:33.222576  1720 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:33.201611  2106 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260709 13:01:33.236611  2109 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:33.243340  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:33.245946  1765 tablet_bootstrap.cc:492] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
W20260709 13:01:33.245465  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 45692.5us
I20260709 13:01:33.217847  2074 consensus_queue.cc:1048] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:01:33.251523  1765 ts_tablet_manager.cc:1403] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.267s	user 0.012s	sys 0.012s
I20260709 13:01:33.234436  2148 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.262512  2148 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:33.260383  1685 consensus_peers.cc:597] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:33.261885  2109 raft_consensus.cc:697] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:01:33.263619  1538 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:33.272141  1865 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:33.273120  1865 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.273495  1865 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.274610  1468 catalog_manager.cc:5697] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:33.280829  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 45222us
I20260709 13:01:33.286995  1819 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:33.289157  1766 tablet_bootstrap.cc:654] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.290536  2151 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:01:33.317016  2033 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:33.333436  1865 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:33.340181  1865 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:01:33.341629  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 67702.4us
I20260709 13:01:33.342669  1538 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:33.348182  1768 tablet_bootstrap.cc:492] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:33.348704  1768 ts_tablet_manager.cc:1403] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.598s	user 0.039s	sys 0.020s
I20260709 13:01:33.356096  1765 raft_consensus.cc:359] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:33.360046  1765 raft_consensus.cc:385] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.360443  1765 raft_consensus.cc:740] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.350121  2151 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:33.361018  1765 consensus_queue.cc:260] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:33.363348  2151 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.338833  2109 consensus_queue.cc:237] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:33.391438  1765 ts_tablet_manager.cc:1434] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.137s	user 0.008s	sys 0.000s
I20260709 13:01:33.386462  2151 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.403190  2146 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.403919  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:33.403820  2146 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:33.406626  1642 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.423069  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:33.431389  1765 tablet_bootstrap.cc:492] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
W20260709 13:01:33.447144  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 61927.5us
W20260709 13:01:33.476429  1537 leader_election.cc:343] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.487798  1865 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:33.481489  2038 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:33.492973  2038 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:33.481287  2162 consensus_queue.cc:1048] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
W20260709 13:01:33.521700  2161 log.cc:927] Time spent T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.136s	user 0.000s	sys 0.002s
I20260709 13:01:33.513897  2157 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.534510  1768 raft_consensus.cc:359] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:33.543881  1537 consensus_peers.cc:597] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 -> Peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Couldn't send request to peer 130bc5ff39b847b1b3204626f284f838. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:33.551829  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45697.1us
I20260709 13:01:33.563778  1768 raft_consensus.cc:385] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.564196  1768 raft_consensus.cc:740] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.564970  1768 consensus_queue.cc:260] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:33.551057  2159 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.211s	user 0.006s	sys 0.000s
I20260709 13:01:33.572127  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:33.574955  1768 ts_tablet_manager.cc:1434] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.226s	user 0.007s	sys 0.004s
W20260709 13:01:33.578496  1519 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:33.555619  1766 tablet_bootstrap.cc:492] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:33.582376  1766 ts_tablet_manager.cc:1403] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.447s	user 0.026s	sys 0.014s
W20260709 13:01:33.591344  1611 leader_election.cc:343] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.592259  1768 tablet_bootstrap.cc:492] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:33.596698  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:33.524986  2148 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:33.525048  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:33.620586  1766 raft_consensus.cc:359] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:33.621078  1766 raft_consensus.cc:385] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.621245  1766 raft_consensus.cc:740] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.622761  1766 consensus_queue.cc:260] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:33.626636  1718 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 1: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:01:33.629352  1642 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:33.618880  2148 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:33.647241  1766 ts_tablet_manager.cc:1434] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.065s	user 0.005s	sys 0.003s
I20260709 13:01:33.620290  1641 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.542135  2157 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:33.757673  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:33.654176  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:33.660431  1538 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:33.767408  2151 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:33.817276  2162 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:33.818171  2162 raft_consensus.cc:697] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:33.818713  2151 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:01:33.815727  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 155349us
W20260709 13:01:33.822857  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 165207us
W20260709 13:01:33.830264  1537 leader_election.cc:343] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.853047  1766 tablet_bootstrap.cc:492] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:33.863185  1766 tablet_bootstrap.cc:654] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.819180  2162 consensus_queue.cc:237] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:33.924304  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 97368us
W20260709 13:01:33.924398  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 88488.9us
W20260709 13:01:33.936162  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 104822us
W20260709 13:01:33.944679  2172 log.cc:927] Time spent T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.174s	user 0.002s	sys 0.000s
I20260709 13:01:33.957526  1765 tablet_bootstrap.cc:654] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.942557  2142 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.964221  1641 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:33.965301  2142 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:34.028277  1612 leader_election.cc:343] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.028800  1612 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:34.028885  1768 tablet_bootstrap.cc:654] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:34.123208  1766 tablet_bootstrap.cc:492] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:34.124320  1766 ts_tablet_manager.cc:1403] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.272s	user 0.010s	sys 0.017s
I20260709 13:01:34.026546  2157 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:34.116214  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:34.118801  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:34.146919  1568 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 1: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:01:34.159057  2146 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:34.159809  2023 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:34.177323  2142 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:34.214444  2149 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.165216  1713 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.215821  2149 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:34.367216  1766 raft_consensus.cc:359] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:34.370428  1766 raft_consensus.cc:385] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.370968  1766 raft_consensus.cc:740] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.372136  1766 consensus_queue.cc:260] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:34.375305  2149 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:01:34.377298  2184 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.305s	user 0.000s	sys 0.003s
I20260709 13:01:34.383927  1470 catalog_manager.cc:5697] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:34.406459  1766 ts_tablet_manager.cc:1434] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.282s	user 0.012s	sys 0.003s
W20260709 13:01:34.404300  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 376047us
I20260709 13:01:34.426870  1765 tablet_bootstrap.cc:492] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:34.428126  1765 ts_tablet_manager.cc:1403] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 1.015s	user 0.029s	sys 0.019s
W20260709 13:01:34.448627  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 322807us
I20260709 13:01:34.461803  1538 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:34.464946  1685 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:34.447575  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:34.489315  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:34.490691  1571 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 1: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:01:34.476830  2148 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:01:34.482321  2151 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:34.484418  1644 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:34.507638  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:34.495299  2019 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:34.517952  2019 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:34.520249  2019 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:34.511826  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:34.520990  1645 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.511060  2033 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:01:34.503257  2176 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:34.501842  2164 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:34.521989  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:34.536623  1768 tablet_bootstrap.cc:492] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:34.537734  1768 ts_tablet_manager.cc:1403] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.946s	user 0.039s	sys 0.041s
I20260709 13:01:34.538663  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:34.541445  1765 raft_consensus.cc:359] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:34.562978  1765 raft_consensus.cc:385] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.563526  1765 raft_consensus.cc:740] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.564405  1765 consensus_queue.cc:260] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:34.509174  2151 raft_consensus.cc:697] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:34.547013  2074 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.532250  2176 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:34.542940  2164 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:34.574122  2151 consensus_queue.cc:237] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:34.575112  2074 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:34.576809  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:34.592437  1612 leader_election.cc:343] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:34.599949  1538 leader_election.cc:343] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.532855  2033 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:34.717620  1768 raft_consensus.cc:359] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:34.590453  1713 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:34.607626  1765 ts_tablet_manager.cc:1434] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.179s	user 0.012s	sys 0.000s
I20260709 13:01:34.720705  1768 raft_consensus.cc:385] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.717861  1796 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:01:34.722313  1768 raft_consensus.cc:740] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.724107  1768 consensus_queue.cc:260] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:34.615097  2164 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:34.643055  2176 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:01:34.602794  1537 leader_election.cc:343] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.731947  1537 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:01:34.737289  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 134530us
I20260709 13:01:34.737946  1687 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:01:34.715780  2074 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:34.739755  2183 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:34.760196  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:34.761989  2033 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:34.764582  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:01:34.766893  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 46250.2us
I20260709 13:01:34.769486  1537 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:34.777415  1768 ts_tablet_manager.cc:1434] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.215s	user 0.010s	sys 0.001s
W20260709 13:01:34.779980  1537 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 -> Peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Couldn't send request to peer 130bc5ff39b847b1b3204626f284f838. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:34.840867  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 72693us
I20260709 13:01:34.844905  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:34.846401  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:34.848671  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:34.849856  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:34.853585  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:34.858088  2160 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.858906  2160 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:34.872571  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:34.878350  2176 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:34.781276  2074 raft_consensus.cc:2764] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:01:34.879506  2176 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:34.894402  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:34.896488  2192 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.897526  2192 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:34.917531  2176 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:34.879207  2164 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:01:34.931824  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 193962us
I20260709 13:01:34.781932  2033 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:34.817909  2151 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:01:34.951329  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 174571us
I20260709 13:01:34.954428  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:34.930455  1645 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:34.969040  1685 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:34.949774  2176 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:01:34.952837  2033 raft_consensus.cc:697] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:34.978489  2033 consensus_queue.cc:237] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:34.981087  2185 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:34.974817  2160 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:34.990211  2192 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:34.976392  2176 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:34.993759  2176 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:34.959208  2151 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:35.013955  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:35.021153  2151 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:35.023140  1611 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:35.023273  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:35.027298  2180 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:35.033500  1611 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:01:35.035053  1611 leader_election.cc:343] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.011184  2143 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.035497  1611 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:01:35.038117  1612 leader_election.cc:343] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.047039  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:35.040175  2197 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:35.049983  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:35.039239  2143 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:35.054306  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:35.055446  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:35.058426  2194 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:35.059074  2194 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:35.056150  2166 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:35.050913  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:35.063802  1713 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 0.
I20260709 13:01:35.068234  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:35.069977  1568 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:35.068490  2194 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:35.077795  1685 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:35.077797  1537 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:35.082311  2212 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:35.087271  2212 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:35.061187  1611 leader_election.cc:343] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.087992  2212 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.088451  1611 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:35.085915  2033 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:35.098176  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:35.104933  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 43784.3us
I20260709 13:01:35.076582  2143 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:35.110616  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:35.113252  1765 tablet_bootstrap.cc:492] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
W20260709 13:01:35.116429  2215 log.cc:927] Time spent T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.063s	user 0.001s	sys 0.001s
I20260709 13:01:35.104758  2176 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:01:35.120818  2216 log.cc:927] Time spent T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:01:35.104120  2033 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.134065  2033 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.142998  2033 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:35.143880  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:35.154191  1766 tablet_bootstrap.cc:492] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:35.149212  1470 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38662 (request call id 105) took 1440 ms. Trace:
I20260709 13:01:35.147958  2212 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:35.174511  1687 leader_election.cc:343] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.176072  1611 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:35.182065  2179 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:35.183476  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:35.184381  1768 tablet_bootstrap.cc:492] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:35.185708  1612 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:35.186672  2179 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:35.177472  1611 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:35.183547  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 305) took 1192 ms. Trace:
I20260709 13:01:35.193869  2019 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:35.197190  2033 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:35.155196  1470 rpcz_store.cc:276] 0709 13:01:33.724615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.725264 (+   649us) service_pool.cc:224] Handling call
0709 13:01:35.149011 (+1423747us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:34.488191 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:34.489791 (+  1600us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:34.489866 (+    75us) write_op.cc:435] Acquired schema lock
0709 13:01:34.489899 (+    33us) tablet.cc:662] Decoding operations
0709 13:01:34.490291 (+   392us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:34.490386 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:34.490429 (+    43us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:01:34.490961 (+   532us) tablet.cc:701] Row locks acquired
0709 13:01:34.490988 (+    27us) write_op.cc:260] PREPARE: finished
0709 13:01:34.491166 (+   178us) write_op.cc:270] Start()
0709 13:01:34.491279 (+   113us) write_op.cc:276] Timestamp: P: 1783602094491123 usec, L: 0
0709 13:01:34.491301 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:01:34.492413 (+  1112us) log.cc:844] Serialized 540 byte log entry
0709 13:01:34.728736 (+236323us) op_driver.cc:464] REPLICATION: finished
0709 13:01:35.010893 (+282157us) write_op.cc:301] APPLY: starting
0709 13:01:35.016905 (+  6012us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:35.024324 (+  7419us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:35.025183 (+   859us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:35.036316 (+ 11133us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:35.048922 (+ 12606us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:01:35.056342 (+  7420us) write_op.cc:312] APPLY: finished
0709 13:01:35.079322 (+ 22980us) log.cc:844] Serialized 22 byte log entry
0709 13:01:35.102845 (+ 23523us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:35.104613 (+  1768us) write_op.cc:454] Released schema lock
0709 13:01:35.106110 (+  1497us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:35.120324 (+ 14214us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:35.120360 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":258180,"apply.run_cpu_time_us":3684,"apply.run_wall_time_us":109825,"num_ops":1,"prepare.queue_time_us":96673,"prepare.run_cpu_time_us":8339,"prepare.run_wall_time_us":17551,"raft.queue_time_us":106287,"raft.run_cpu_time_us":20488,"raft.run_wall_time_us":235704,"replication_time_us":237216,"thread_start_us":392023,"threads_started":4,"wal-append.queue_time_us":105311}]]}
W20260709 13:01:35.198997  1519 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:35.202248  2212 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:35.202888  1611 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:35.203148  1538 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:35.204877  2166 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:35.191505  1717 rpcz_store.cc:276] 0709 13:01:33.991396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.995147 (+  3751us) service_pool.cc:224] Handling call
0709 13:01:34.546032 (+550885us) spinlock_profiling.cc:243] Waited 2.62 ms on lock 0x7b580009db98. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:01:34.551922 (+  5890us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:34.643489 (+ 91567us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:34.651012 (+  7523us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:34.729426 (+ 78414us) log.cc:844] Serialized 26 byte log entry
0709 13:01:34.956792 (+227366us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:01:34.967122 (+ 10330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.183176 (+216054us) spinlock_profiling.cc:243] Waited 45.7 ms on lock 0x7b54002c65d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:01:35.183229 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:35.183294 (+    65us) raft_consensus.cc:1685] finished
0709 13:01:35.183308 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:35.183449 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7530,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":19947,"spinlock_wait_cycles":111057664,"thread_start_us":186880,"threads_started":1,"wal-append.queue_time_us":236622}
I20260709 13:01:35.205583  2106 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:35.217921  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:35.219278  1766 tablet_bootstrap.cc:654] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:35.220654  1687 leader_election.cc:343] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.222597  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:35.223995  1566 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 1: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:01:35.231154  1685 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:35.239166  1765 tablet_bootstrap.cc:654] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.244869  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:35.244731  2196 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:01:35.246868  2142 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:01:35.248106  1768 tablet_bootstrap.cc:654] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.250552  1468 catalog_manager.cc:5697] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:35.256076  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:35.261521  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:35.262418  1720 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 1: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:01:35.280682  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:35.315074  1765 tablet_bootstrap.cc:492] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:35.315764  1765 ts_tablet_manager.cc:1403] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.203s	user 0.013s	sys 0.005s
I20260709 13:01:35.318804  1766 tablet_bootstrap.cc:492] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:35.319309  1766 ts_tablet_manager.cc:1403] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.165s	user 0.018s	sys 0.024s
I20260709 13:01:35.322017  1766 raft_consensus.cc:359] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:35.323010  1538 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:35.325194  2106 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:35.322717  1766 raft_consensus.cc:385] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.325876  1766 raft_consensus.cc:740] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.326680  1766 consensus_queue.cc:260] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:35.329046  1766 ts_tablet_manager.cc:1434] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:01:35.325484  1768 tablet_bootstrap.cc:492] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:35.330494  1768 ts_tablet_manager.cc:1403] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.146s	user 0.011s	sys 0.007s
I20260709 13:01:35.331296  1765 raft_consensus.cc:359] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:35.332024  1765 raft_consensus.cc:385] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.332302  1765 raft_consensus.cc:740] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.332907  1765 consensus_queue.cc:260] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:35.328895  1538 leader_election.cc:343] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.338105  1768 raft_consensus.cc:359] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:35.338888  1768 raft_consensus.cc:385] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.339187  1768 raft_consensus.cc:740] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
W20260709 13:01:35.339804  1538 leader_election.cc:343] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.340161  1538 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:35.343403  2188 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:01:35.342126  1765 ts_tablet_manager.cc:1434] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.003s
I20260709 13:01:35.344795  1765 tablet_bootstrap.cc:492] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:35.344977  1766 tablet_bootstrap.cc:492] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:35.344251  1768 consensus_queue.cc:260] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:35.351831  2106 consensus_queue.cc:1048] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:01:35.357777  2155 consensus_queue.cc:1048] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:35.367312  2185 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.368124  2185 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:35.376694  2185 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:35.384218  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:35.390763  1768 ts_tablet_manager.cc:1434] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.060s	user 0.008s	sys 0.000s
W20260709 13:01:35.399302  1685 leader_election.cc:343] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.405823  1768 tablet_bootstrap.cc:492] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:35.408094  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:35.435444  1470 catalog_manager.cc:5697] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:01:35.486294  2180 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.488950  2180 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:35.456254  1642 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:35.509878  1687 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:01:35.512987  2076 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:35.513520  2076 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.513919  2076 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.555115  2180 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:35.626358  2188 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.683211  1768 tablet_bootstrap.cc:654] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.627107  2188 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:35.671289  2076 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:35.706543  2180 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.733777  2180 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:35.737114  2180 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:35.739440  2076 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:35.739558  2188 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:35.747987  1766 tablet_bootstrap.cc:654] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.752306  1765 tablet_bootstrap.cc:654] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.866356  1468 catalog_manager.cc:5697] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:35.961529  1768 tablet_bootstrap.cc:492] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:36.015400  1766 tablet_bootstrap.cc:492] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:36.016549  1766 ts_tablet_manager.cc:1403] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.672s	user 0.061s	sys 0.057s
I20260709 13:01:36.038371  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:36.039660  1768 ts_tablet_manager.cc:1403] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.641s	user 0.019s	sys 0.007s
I20260709 13:01:36.042052  1765 tablet_bootstrap.cc:492] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:36.043124  1765 ts_tablet_manager.cc:1403] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.699s	user 0.030s	sys 0.016s
W20260709 13:01:36.023484  2225 log.cc:927] Time spent T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.635s	user 0.009s	sys 0.030s
I20260709 13:01:36.048393  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:36.050022  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:36.051647  1566 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:36.063989  1768 raft_consensus.cc:359] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.070086  1768 raft_consensus.cc:385] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.072001  1768 raft_consensus.cc:740] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.071975  1766 raft_consensus.cc:359] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.073704  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:36.073971  1766 raft_consensus.cc:385] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:01:36.028551  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 112134us
I20260709 13:01:36.079043  1768 consensus_queue.cc:260] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.081490  1768 ts_tablet_manager.cc:1434] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.002s
I20260709 13:01:36.075964  1766 raft_consensus.cc:740] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.079696  2155 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:01:36.083029  1766 consensus_queue.cc:260] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.075322  1644 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.088243  1766 ts_tablet_manager.cc:1434] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.071s	user 0.005s	sys 0.007s
I20260709 13:01:36.104067  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:36.103497  1766 tablet_bootstrap.cc:492] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: Bootstrap starting.
I20260709 13:01:36.121572  1644 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:36.122462  1567 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.142297  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:36.073251  1765 raft_consensus.cc:359] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.144456  1765 raft_consensus.cc:385] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.144907  1765 raft_consensus.cc:740] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.145788  1765 consensus_queue.cc:260] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.149585  1566 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.149886  1768 tablet_bootstrap.cc:492] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:36.155938  1687 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:36.158291  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:36.159113  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:36.162115  2212 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:36.151253  1765 ts_tablet_manager.cc:1434] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.096s	user 0.011s	sys 0.000s
I20260709 13:01:36.162886  2212 raft_consensus.cc:697] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:01:36.164546  1765 tablet_bootstrap.cc:492] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:36.164201  2212 consensus_queue.cc:237] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.176801  1566 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:36.225132  1537 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:36.243325  2157 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
W20260709 13:01:36.255056  1612 leader_election.cc:343] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:36.255803  1612 leader_election.cc:343] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.257000  1765 tablet_bootstrap.cc:654] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.261260  1720 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.280097  1611 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:36.285221  2241 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:36.285761  2241 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.286206  2241 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.304901  1768 tablet_bootstrap.cc:654] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:36.326808  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 46967.9us
I20260709 13:01:36.331830  1766 tablet_bootstrap.cc:654] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.345223  2235 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:36.358117  2241 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:36.375568  1611 leader_election.cc:343] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.376336  1611 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:36.398336  2245 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:36.454317  1765 tablet_bootstrap.cc:492] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:36.455240  1765 ts_tablet_manager.cc:1403] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.291s	user 0.028s	sys 0.018s
I20260709 13:01:36.546900  2164 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.547497  2164 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.547086  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:36.602528  2164 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:36.575801  1566 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.631160  1765 raft_consensus.cc:359] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.635174  1766 tablet_bootstrap.cc:492] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: No bootstrap required, opened a new log
I20260709 13:01:36.635748  1766 ts_tablet_manager.cc:1403] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: Time spent bootstrapping tablet: real 0.533s	user 0.015s	sys 0.015s
I20260709 13:01:36.635434  1765 raft_consensus.cc:385] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.636816  1765 raft_consensus.cc:740] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.640236  2241 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:36.643230  1768 tablet_bootstrap.cc:492] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:36.644107  1768 ts_tablet_manager.cc:1403] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.542s	user 0.022s	sys 0.035s
I20260709 13:01:36.660338  1765 consensus_queue.cc:260] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.678479  2243 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:36.679366  2243 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.669695  1766 raft_consensus.cc:359] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.682015  1766 raft_consensus.cc:385] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.682369  1766 raft_consensus.cc:740] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.687641  1768 raft_consensus.cc:359] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.689765  1768 raft_consensus.cc:385] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.690088  1768 raft_consensus.cc:740] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.697788  1766 consensus_queue.cc:260] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.702543  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:36.703567  1572 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 1: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:01:36.698542  2241 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.705389  2241 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.708081  2241 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:36.714411  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:36.715787  1568 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:36.716650  1766 ts_tablet_manager.cc:1434] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: Time spent starting tablet: real 0.080s	user 0.009s	sys 0.000s
I20260709 13:01:36.727430  1566 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:36.732213  1611 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:36.735038  2164 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:36.735641  2164 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.736055  2164 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.734176  1768 consensus_queue.cc:260] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.748896  2243 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:36.754395  1768 ts_tablet_manager.cc:1434] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.110s	user 0.012s	sys 0.004s
I20260709 13:01:36.756433  1768 tablet_bootstrap.cc:492] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:36.745698  1611 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:36.759490  2241 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:36.760475  2241 raft_consensus.cc:697] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:01:36.761582  2241 consensus_queue.cc:237] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.767601  1470 catalog_manager.cc:5697] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:36.770181  2243 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.771421  2243 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.778025  2243 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:36.779171  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:36.783279  1765 ts_tablet_manager.cc:1434] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.327s	user 0.023s	sys 0.024s
I20260709 13:01:36.784087  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:36.784937  1644 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:36.787111  1765 tablet_bootstrap.cc:492] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
W20260709 13:01:36.788203  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 125601us
W20260709 13:01:36.789810  1519 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:36.803951  2164 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.806172  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:36.807958  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:36.808658  1565 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.794117  1687 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:36.818073  1713 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 1: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:01:36.819093  2243 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:36.820302  1765 tablet_bootstrap.cc:654] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.820899  2243 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.821373  2243 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:36.808172  2164 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:36.794226  1768 tablet_bootstrap.cc:654] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.825883  2212 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:36.831599  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:36.842491  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:36.844408  1713 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.857544  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:36.861613  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:36.863368  1567 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:36.876127  1612 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:01:36.880448  1612 leader_election.cc:343] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.887862  1713 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:36.888959  2238 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.904143  2212 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:36.904781  1685 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:36.907136  2142 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:36.913837  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:36.914723  1716 raft_consensus.cc:2376] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1. Re-sending same reply.
I20260709 13:01:36.929240  1612 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:36.948192  2180 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:36.963893  2180 raft_consensus.cc:697] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:01:36.965005  2180 consensus_queue.cc:237] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.969276  1765 tablet_bootstrap.cc:492] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:36.984947  1765 ts_tablet_manager.cc:1403] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.198s	user 0.018s	sys 0.010s
I20260709 13:01:36.993516  1768 tablet_bootstrap.cc:492] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:36.994107  1768 ts_tablet_manager.cc:1403] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.238s	user 0.021s	sys 0.006s
I20260709 13:01:36.982966  1565 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:36.979458  2243 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.997562  1768 raft_consensus.cc:359] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:36.998371  1768 raft_consensus.cc:385] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.998633  1768 raft_consensus.cc:740] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.999356  1768 consensus_queue.cc:260] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:37.000010  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:37.000067  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:37.000963  1568 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:37.001974  1768 ts_tablet_manager.cc:1434] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:01:37.003145  1768 tablet_bootstrap.cc:492] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:37.014901  1768 tablet_bootstrap.cc:654] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:37.020038  2243 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:37.025544  1765 raft_consensus.cc:359] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:37.031785  1765 raft_consensus.cc:385] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:37.032347  1765 raft_consensus.cc:740] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:37.033978  1765 consensus_queue.cc:260] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:37.030792  2238 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:37.038203  1642 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:37.039772  1568 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:01:37.042272  2108 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.042920  2108 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:37.045344  2108 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:37.049265  1685 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:37.054983  2074 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:37.060042  2074 raft_consensus.cc:697] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:01:37.057377  1765 ts_tablet_manager.cc:1434] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.072s	user 0.004s	sys 0.009s
I20260709 13:01:37.069306  1765 tablet_bootstrap.cc:492] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504: Bootstrap starting.
I20260709 13:01:37.065115  2074 consensus_queue.cc:237] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:37.065011  1642 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:01:37.073191  1768 tablet_bootstrap.cc:492] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:37.074199  1768 ts_tablet_manager.cc:1403] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.071s	user 0.031s	sys 0.030s
I20260709 13:01:37.077140  1768 raft_consensus.cc:359] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:37.078377  1768 raft_consensus.cc:385] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:37.078657  1768 raft_consensus.cc:740] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:37.079783  1765 tablet_bootstrap.cc:654] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:37.079344  1768 consensus_queue.cc:260] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:37.081743  1768 ts_tablet_manager.cc:1434] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:01:37.082829  1768 tablet_bootstrap.cc:492] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:37.060977  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:37.088707  1565 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:37.091881  1611 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:37.093773  1768 tablet_bootstrap.cc:654] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:37.094175  2238 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:37.095283  2238 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.095767  2238 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.112010  2212 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.115163  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:37.115939  1716 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 0.
I20260709 13:01:37.113265  2212 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:37.185755  2212 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:37.186344  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:01:37.196703  1685 leader_election.cc:343] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:37.203070  1765 tablet_bootstrap.cc:492] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504: No bootstrap required, opened a new log
I20260709 13:01:37.203743  1765 ts_tablet_manager.cc:1403] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504: Time spent bootstrapping tablet: real 0.135s	user 0.044s	sys 0.046s
I20260709 13:01:37.224496  1765 raft_consensus.cc:359] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:37.225289  1765 raft_consensus.cc:385] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:37.226161  1765 raft_consensus.cc:740] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Initialized, Role: FOLLOWER
I20260709 13:01:37.229281  1765 consensus_queue.cc:260] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:37.186066  2238 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:37.244846  1612 consensus_peers.cc:597] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Couldn't send request to peer 130bc5ff39b847b1b3204626f284f838. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:37.206933  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:37.255556  1768 tablet_bootstrap.cc:492] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:37.256182  1768 ts_tablet_manager.cc:1403] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.174s	user 0.016s	sys 0.007s
I20260709 13:01:37.259711  1644 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 0.
I20260709 13:01:37.266826  1765 ts_tablet_manager.cc:1434] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504: Time spent starting tablet: real 0.063s	user 0.006s	sys 0.014s
I20260709 13:01:37.278156  1687 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:01:37.279179  2142 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:37.279652  2142 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.280026  2142 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.277980  1768 raft_consensus.cc:359] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:37.294276  1768 raft_consensus.cc:385] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:37.294952  1768 raft_consensus.cc:740] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:37.298959  2238 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:37.297365  1768 consensus_queue.cc:260] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:37.302809  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:37.303448  1716 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.304584  1768 ts_tablet_manager.cc:1434] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.048s	user 0.004s	sys 0.006s
I20260709 13:01:37.306722  1768 tablet_bootstrap.cc:492] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838: Bootstrap starting.
I20260709 13:01:37.318276  2142 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:37.328766  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:37.329298  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:37.329792  1568 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.330117  1646 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:37.316334  1537 consensus_peers.cc:597] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 -> Peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Couldn't send request to peer 130bc5ff39b847b1b3204626f284f838. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:01:37.338202  2108 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:37.338814  1646 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:37.340555  1716 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:37.341500  1687 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:37.342882  1768 tablet_bootstrap.cc:654] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:37.343230  2212 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:37.342024  2142 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:37.345135  2212 raft_consensus.cc:697] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:01:37.352910  2212 consensus_queue.cc:237] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:37.353683  1568 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:37.356221  1468 catalog_manager.cc:5697] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:37.358826  1768 tablet_bootstrap.cc:492] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838: No bootstrap required, opened a new log
I20260709 13:01:37.363662  1768 ts_tablet_manager.cc:1403] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838: Time spent bootstrapping tablet: real 0.058s	user 0.015s	sys 0.004s
I20260709 13:01:37.361173  1611 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:37.367290  2279 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:37.370450  2279 raft_consensus.cc:697] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:01:37.363900  1468 catalog_manager.cc:5697] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:37.371870  2108 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:01:37.371930  2279 consensus_queue.cc:237] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:37.386370  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:37.387436  1565 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.396036  1768 raft_consensus.cc:359] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:37.398087  1768 raft_consensus.cc:385] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:37.398465  1768 raft_consensus.cc:740] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Initialized, Role: FOLLOWER
I20260709 13:01:37.401710  1768 consensus_queue.cc:260] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:37.408535  1768 ts_tablet_manager.cc:1434] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838: Time spent starting tablet: real 0.044s	user 0.012s	sys 0.000s
I20260709 13:01:37.434235  1565 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:01:37.477855  2241 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:37.485754  1469 catalog_manager.cc:5697] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 0 to 2, leader changed from <none> to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:37.522961  1468 catalog_manager.cc:5697] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:37.540201  1470 catalog_manager.cc:5697] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:37.594910  2142 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.595497  2142 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:37.597430  2142 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:37.598189  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:37.598881  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:37.601105  1687 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:37.602211  2142 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:37.666366  2142 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.667141  2142 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:37.668993  2142 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:37.670593  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:37.671334  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:37.673211  1687 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:37.674144  2142 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:37.680598  1644 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:01:37.683012  2142 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:37.686609  1720 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.704216  1568 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:01:37.706271  2074 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:37.770524  2074 consensus_queue.cc:1048] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:37.781237  1720 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.803606  2142 consensus_queue.cc:1048] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:37.836134  2108 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:37.930989  2238 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:39.248024   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:39.281303  2302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:39.304646  2307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:39.307518  2303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:39.323171   431 server_base.cc:1061] running on GCE node
I20260709 13:01:39.324774   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:39.325067   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:39.325285   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602099325267 us; error 0 us; skew 500 ppm
I20260709 13:01:39.326045   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:39.330318   431 webserver.cc:533] Webserver started at http://127.0.107.196:35057/ using document root <none> and password file <none>
I20260709 13:01:39.330998   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:39.331257   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:39.331688   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:39.333527   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-3-root/instance:
uuid: "cf2f71592be8465fb18e36e15d0b2f89"
format_stamp: "Formatted at 2026-07-09 13:01:39 on dist-test-slave-mxd3"
I20260709 13:01:39.341974   431 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:01:39.350441  2311 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:39.352247   431 fs_manager.cc:730] Time spent opening block manager: real 0.007s	user 0.004s	sys 0.001s
I20260709 13:01:39.352681   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-3-root
uuid: "cf2f71592be8465fb18e36e15d0b2f89"
format_stamp: "Formatted at 2026-07-09 13:01:39 on dist-test-slave-mxd3"
I20260709 13:01:39.353261   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:39.373550   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:39.375500   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:39.378666   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:39.385673   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:39.386181   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.002s	sys 0.000s
I20260709 13:01:39.386682   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:39.386997   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:01:39.480026   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.196:39935
I20260709 13:01:39.480225  2373 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.196:39935 every 8 connection(s)
I20260709 13:01:39.488690   431 auto_leader_rebalancer-test.cc:384] add a tserver: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:39.517148  2374 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38281
I20260709 13:01:39.518006  2374 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:39.519963  2374 heartbeater.cc:507] Master 127.0.107.254:38281 requested a full tablet report, sending...
I20260709 13:01:39.523543  1469 ts_manager.cc:194] Registered new tserver with Master: cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:01:39.525501  1469 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47518
I20260709 13:01:40.197115  1642 consensus_queue.cc:237] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:40.220217  1565 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:40.220376  1720 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:40.222738  2108 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:40.223735  2378 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:40.253304  1612 consensus_peers.cc:597] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f59ca0c8e4c0491480502ebf944ba503. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:40.257592  2301 raft_consensus.cc:2955] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:40.259281  1716 raft_consensus.cc:2955] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:40.265753  1565 raft_consensus.cc:2955] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:40.292840  1469 catalog_manager.cc:5697] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:40.318215  1716 consensus_queue.cc:237] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:40.328025  1642 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:40.329315  1571 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:40.329397  2243 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:40.334069  2142 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:40.371649  2385 raft_consensus.cc:2955] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:40.384099  1642 raft_consensus.cc:2955] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
W20260709 13:01:40.393143  1686 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 27aa13d58e6d4987b2e34613996ef421. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:40.403203  1468 catalog_manager.cc:5697] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:40.411491  1568 raft_consensus.cc:2955] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:40.493870  1571 consensus_queue.cc:237] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:40.515017  1642 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:40.516891  2304 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:40.518931  1713 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:40.522408  2304 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:01:40.530403  1537 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ee0e81a514641709317e983fcf96fe4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:40.531266  2374 heartbeater.cc:499] Master 127.0.107.254:38281 was elected leader, sending a full tablet report...
I20260709 13:01:40.533362  2247 raft_consensus.cc:2955] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:40.535948  1713 raft_consensus.cc:2955] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:40.536998  1642 raft_consensus.cc:2955] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:40.581524  1644 consensus_queue.cc:237] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:40.570457  1470 catalog_manager.cc:5697] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:40.651969  1565 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:40.654788  1612 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bddd1ebc9af24fa282580ab028d643e0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:40.655408  2301 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:40.657265  1720 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:40.660113  2378 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:40.673990  2108 raft_consensus.cc:2955] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:40.678092  1571 raft_consensus.cc:2955] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:40.680778  2407 ts_tablet_manager.cc:933] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:40.692577  2407 tablet_copy_client.cc:323] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:44293
I20260709 13:01:40.742362  2413 ts_tablet_manager.cc:933] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:40.749292  2413 tablet_copy_client.cc:323] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:34553
I20260709 13:01:40.751519  1470 catalog_manager.cc:5697] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:40.778815  1720 raft_consensus.cc:2955] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:40.796495  1658 tablet_copy_service.cc:140] P f5f4b2707a9647e5bb99e065068dea2b: Received BeginTabletCopySession request for tablet f59ca0c8e4c0491480502ebf944ba503 from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:44424)
I20260709 13:01:40.797832  1658 tablet_copy_service.cc:161] P f5f4b2707a9647e5bb99e065068dea2b: Beginning new tablet copy session on tablet f59ca0c8e4c0491480502ebf944ba503 from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:44424: session id = cf2f71592be8465fb18e36e15d0b2f89-f59ca0c8e4c0491480502ebf944ba503
I20260709 13:01:40.819451  1658 tablet_copy_source_session.cc:215] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:40.833389  1732 tablet_copy_service.cc:140] P 130bc5ff39b847b1b3204626f284f838: Received BeginTabletCopySession request for tablet 27aa13d58e6d4987b2e34613996ef421 from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:34078)
I20260709 13:01:40.834252  1732 tablet_copy_service.cc:161] P 130bc5ff39b847b1b3204626f284f838: Beginning new tablet copy session on tablet 27aa13d58e6d4987b2e34613996ef421 from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:34078: session id = cf2f71592be8465fb18e36e15d0b2f89-27aa13d58e6d4987b2e34613996ef421
I20260709 13:01:40.836674  2407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f59ca0c8e4c0491480502ebf944ba503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:40.884065  1732 tablet_copy_source_session.cc:215] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:40.894511  2413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27aa13d58e6d4987b2e34613996ef421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:40.899091  2407 tablet_copy_client.cc:806] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:40.899896  2407 tablet_copy_client.cc:670] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:40.913339  2407 tablet_copy_client.cc:538] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:40.936857  2407 tablet_bootstrap.cc:492] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:01:40.943567  2413 tablet_copy_client.cc:806] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:40.944327  2413 tablet_copy_client.cc:670] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:40.950733  2413 tablet_copy_client.cc:538] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:40.962486  2413 tablet_bootstrap.cc:492] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:01:40.981755  2407 tablet_bootstrap.cc:492] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:40.983172  2407 tablet_bootstrap.cc:492] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:01:40.984724  2407 ts_tablet_manager.cc:1403] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 0.048s	user 0.016s	sys 0.020s
I20260709 13:01:40.992298  2407 raft_consensus.cc:359] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:40.993438  2407 raft_consensus.cc:740] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:01:40.994287  2407 consensus_queue.cc:260] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:41.000428  2413 tablet_bootstrap.cc:492] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:41.001776  2413 tablet_bootstrap.cc:492] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:01:41.002555  2413 ts_tablet_manager.cc:1403] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 0.040s	user 0.033s	sys 0.001s
I20260709 13:01:41.002945  2407 ts_tablet_manager.cc:1434] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.018s	user 0.011s	sys 0.003s
I20260709 13:01:41.006659  1658 tablet_copy_service.cc:342] P f5f4b2707a9647e5bb99e065068dea2b: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-f59ca0c8e4c0491480502ebf944ba503 received from {username='slave'} at 127.0.0.1:44424
I20260709 13:01:41.007467  1658 tablet_copy_service.cc:434] P f5f4b2707a9647e5bb99e065068dea2b: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-f59ca0c8e4c0491480502ebf944ba503 on tablet f59ca0c8e4c0491480502ebf944ba503 with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:41.005920  2413 raft_consensus.cc:359] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:41.008334  2413 raft_consensus.cc:740] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:01:41.009102  2413 consensus_queue.cc:260] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:41.013093  2413 ts_tablet_manager.cc:1434] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:01:41.015877  1732 tablet_copy_service.cc:342] P 130bc5ff39b847b1b3204626f284f838: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-27aa13d58e6d4987b2e34613996ef421 received from {username='slave'} at 127.0.0.1:34078
I20260709 13:01:41.016355  1732 tablet_copy_service.cc:434] P 130bc5ff39b847b1b3204626f284f838: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-27aa13d58e6d4987b2e34613996ef421 on tablet 27aa13d58e6d4987b2e34613996ef421 with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:41.107671  2349 raft_consensus.cc:1217] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:41.120838  2413 ts_tablet_manager.cc:933] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:01:41.149145  2413 tablet_copy_client.cc:323] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:39999
I20260709 13:01:41.156611  2407 ts_tablet_manager.cc:933] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:41.163832  2349 raft_consensus.cc:1217] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:41.188602  2407 tablet_copy_client.cc:323] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:44293
I20260709 13:01:41.191648  1658 tablet_copy_service.cc:140] P f5f4b2707a9647e5bb99e065068dea2b: Received BeginTabletCopySession request for tablet bddd1ebc9af24fa282580ab028d643e0 from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:44424)
I20260709 13:01:41.192731  1658 tablet_copy_service.cc:161] P f5f4b2707a9647e5bb99e065068dea2b: Beginning new tablet copy session on tablet bddd1ebc9af24fa282580ab028d643e0 from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:44424: session id = cf2f71592be8465fb18e36e15d0b2f89-bddd1ebc9af24fa282580ab028d643e0
I20260709 13:01:41.221745  1658 tablet_copy_source_session.cc:215] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:41.225761  2407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bddd1ebc9af24fa282580ab028d643e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:41.232082  1583 tablet_copy_service.cc:140] P 066ee0109aba4465ba17fa65e14ef504: Received BeginTabletCopySession request for tablet 3ee0e81a514641709317e983fcf96fe4 from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:46982)
I20260709 13:01:41.232813  1583 tablet_copy_service.cc:161] P 066ee0109aba4465ba17fa65e14ef504: Beginning new tablet copy session on tablet 3ee0e81a514641709317e983fcf96fe4 from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:46982: session id = cf2f71592be8465fb18e36e15d0b2f89-3ee0e81a514641709317e983fcf96fe4
I20260709 13:01:41.242496  1583 tablet_copy_source_session.cc:215] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:41.245924  2413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ee0e81a514641709317e983fcf96fe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:41.247864  2407 tablet_copy_client.cc:806] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:41.248543  2407 tablet_copy_client.cc:670] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:41.253129  2407 tablet_copy_client.cc:538] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:41.262446  2413 tablet_copy_client.cc:806] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:41.263154  2413 tablet_copy_client.cc:670] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:41.263352  2407 tablet_bootstrap.cc:492] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:01:41.275156  2413 tablet_copy_client.cc:538] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:41.305917  2413 tablet_bootstrap.cc:492] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:01:41.332930  2413 tablet_bootstrap.cc:492] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:41.333885  2407 tablet_bootstrap.cc:492] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:41.333983  2413 tablet_bootstrap.cc:492] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:01:41.335577  2407 tablet_bootstrap.cc:492] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:01:41.337596  2407 ts_tablet_manager.cc:1403] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 0.074s	user 0.028s	sys 0.017s
I20260709 13:01:41.341069  2413 ts_tablet_manager.cc:1403] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.008s
I20260709 13:01:41.344336  2407 raft_consensus.cc:359] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:41.350831  2407 raft_consensus.cc:740] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:01:41.349449  2413 raft_consensus.cc:359] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:41.351747  2407 consensus_queue.cc:260] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:41.353588  2413 raft_consensus.cc:740] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:01:41.356400  2413 consensus_queue.cc:260] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:41.358893  2413 ts_tablet_manager.cc:1434] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:01:41.361507  2407 ts_tablet_manager.cc:1434] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 13:01:41.364199  1658 tablet_copy_service.cc:342] P f5f4b2707a9647e5bb99e065068dea2b: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-bddd1ebc9af24fa282580ab028d643e0 received from {username='slave'} at 127.0.0.1:44424
I20260709 13:01:41.364708  1658 tablet_copy_service.cc:434] P f5f4b2707a9647e5bb99e065068dea2b: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-bddd1ebc9af24fa282580ab028d643e0 on tablet bddd1ebc9af24fa282580ab028d643e0 with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:41.369966  1583 tablet_copy_service.cc:342] P 066ee0109aba4465ba17fa65e14ef504: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-3ee0e81a514641709317e983fcf96fe4 received from {username='slave'} at 127.0.0.1:46982
I20260709 13:01:41.370739  1583 tablet_copy_service.cc:434] P 066ee0109aba4465ba17fa65e14ef504: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-3ee0e81a514641709317e983fcf96fe4 on tablet 3ee0e81a514641709317e983fcf96fe4 with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:41.535521  2349 raft_consensus.cc:1217] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:41.589318  2423 raft_consensus.cc:1064] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:01:41.593318  2423 consensus_queue.cc:237] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:41.602439  2349 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:41.604422  1646 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:41.604743  2422 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:41.607550  1565 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:41.606742  2423 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:01:41.611663  2422 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:41.628751  2422 raft_consensus.cc:2955] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:41.631655  1565 raft_consensus.cc:2955] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:41.646384  2349 raft_consensus.cc:1217] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:41.647110  2348 raft_consensus.cc:2955] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:41.659524  1640 raft_consensus.cc:2955] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:41.671425  1469 catalog_manager.cc:5697] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:41.718875  2424 raft_consensus.cc:1064] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:01:41.728665  2424 consensus_queue.cc:237] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:41.751253  1716 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:41.751760  1568 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:41.753023  2348 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:41.754101  2424 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:41.764348  2379 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:41.767658  2301 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:41.765007  1716 consensus_queue.cc:237] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:41.812570  2349 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:41.815754  2385 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:41.821174  1568 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:41.823694  2142 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:41.830574  2437 raft_consensus.cc:2955] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:41.836418  2423 raft_consensus.cc:2955] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:41.842772  1568 raft_consensus.cc:2955] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:41.874157  2348 raft_consensus.cc:2955] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:41.888993  2349 raft_consensus.cc:2955] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:41.907985  1469 catalog_manager.cc:5697] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: config changed from index 3 to 4, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:41.916109  1456 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 27aa13d58e6d4987b2e34613996ef421 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:41.922789  1629 tablet_service.cc:1558] Processing DeleteTablet for tablet 27aa13d58e6d4987b2e34613996ef421 with delete_type TABLET_DATA_TOMBSTONED (TS f5f4b2707a9647e5bb99e065068dea2b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39910
I20260709 13:01:41.926867  1468 catalog_manager.cc:5697] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:41.928782  1565 raft_consensus.cc:2955] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:41.957489  2442 tablet_replica.cc:333] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:01:41.959065  2442 raft_consensus.cc:2243] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:41.962307  2442 raft_consensus.cc:2272] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:41.960018  1722 raft_consensus.cc:2955] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:41.992009  2442 ts_tablet_manager.cc:1916] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:42.029536  1644 consensus_queue.cc:237] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:42.045482  2442 ts_tablet_manager.cc:1929] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:42.046517  2442 log.cc:1199] T 27aa13d58e6d4987b2e34613996ef421 P f5f4b2707a9647e5bb99e065068dea2b: Deleting WAL directory at /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-1-root/wals/27aa13d58e6d4987b2e34613996ef421
I20260709 13:01:42.049043  2436 raft_consensus.cc:1064] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:01:42.053226  2436 consensus_queue.cc:237] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:42.054654  2349 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:42.057029  2437 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:42.062530  1722 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:42.066032  2301 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:42.067260  1457 catalog_manager.cc:5028] TS f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): tablet 27aa13d58e6d4987b2e34613996ef421 (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
I20260709 13:01:42.070096  2405 raft_consensus.cc:1064] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:01:42.076066  2379 raft_consensus.cc:2955] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.073262  2405 consensus_queue.cc:237] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:42.082639  1722 raft_consensus.cc:2955] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.085155  2349 raft_consensus.cc:2955] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.105173  1457 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f59ca0c8e4c0491480502ebf944ba503 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:42.122181  2348 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:42.122589  1469 catalog_manager.cc:5697] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: config changed from index 3 to 4, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.124703  1716 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:42.126399  1565 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:42.128669  2436 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:42.135195  1558 tablet_service.cc:1558] Processing DeleteTablet for tablet f59ca0c8e4c0491480502ebf944ba503 with delete_type TABLET_DATA_TOMBSTONED (TS 066ee0109aba4465ba17fa65e14ef504 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40642
I20260709 13:01:42.138222  2437 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:42.142194  2301 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:42.211285  2447 tablet_replica.cc:333] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:01:42.222522  2348 raft_consensus.cc:2955] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.217886  2379 raft_consensus.cc:2955] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.235378  2447 raft_consensus.cc:2243] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:42.236384  2447 raft_consensus.cc:2272] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:42.237128  1565 raft_consensus.cc:2955] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.241621  2349 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:42.243688  1644 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:42.246006  2443 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:01:42.250994  2435 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:42.250048  1722 raft_consensus.cc:2955] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.296834  2447 ts_tablet_manager.cc:1916] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:42.273509  1468 catalog_manager.cc:5697] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:42.309986  1716 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:42.344686  1646 consensus_queue.cc:237] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:42.358343  2247 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:01:42.360294  2447 ts_tablet_manager.cc:1929] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:42.360788  2447 log.cc:1199] T f59ca0c8e4c0491480502ebf944ba503 P 066ee0109aba4465ba17fa65e14ef504: Deleting WAL directory at /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-0-root/wals/f59ca0c8e4c0491480502ebf944ba503
I20260709 13:01:42.362303  2349 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:42.363379  1565 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:42.365243  2379 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:42.368175  2424 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:42.376155  1455 catalog_manager.cc:5028] TS 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): tablet f59ca0c8e4c0491480502ebf944ba503 (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
I20260709 13:01:42.378957  2240 raft_consensus.cc:2955] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.382939  1565 raft_consensus.cc:2955] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.379762  2424 raft_consensus.cc:2955] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.396473  2349 raft_consensus.cc:2955] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.388324  2348 raft_consensus.cc:2955] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.409296  1716 raft_consensus.cc:2955] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.414307  1646 raft_consensus.cc:2955] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.437551  1468 catalog_manager.cc:5697] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.439179  1470 catalog_manager.cc:5697] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.442935  1457 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bddd1ebc9af24fa282580ab028d643e0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:42.474663  1708 tablet_service.cc:1558] Processing DeleteTablet for tablet bddd1ebc9af24fa282580ab028d643e0 with delete_type TABLET_DATA_TOMBSTONED (TS 130bc5ff39b847b1b3204626f284f838 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42266
I20260709 13:01:42.507364  1565 consensus_queue.cc:237] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:42.512905  2454 tablet_replica.cc:333] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
W20260709 13:01:42.511368  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 47204.8us
I20260709 13:01:42.514528  2454 raft_consensus.cc:2243] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:42.515452  2454 raft_consensus.cc:2272] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:42.520932  2454 ts_tablet_manager.cc:1916] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:42.528329  1644 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:42.532506  2435 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:42.536374  2348 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:42.546209  2405 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:42.548627  2443 raft_consensus.cc:2955] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.558602  2348 raft_consensus.cc:2955] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.560689  1640 raft_consensus.cc:2955] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.576237  2454 ts_tablet_manager.cc:1929] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:42.576889  2454 log.cc:1199] T bddd1ebc9af24fa282580ab028d643e0 P 130bc5ff39b847b1b3204626f284f838: Deleting WAL directory at /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-2-root/wals/bddd1ebc9af24fa282580ab028d643e0
I20260709 13:01:42.579205  1456 catalog_manager.cc:5028] TS 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): tablet bddd1ebc9af24fa282580ab028d643e0 (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
I20260709 13:01:42.579810  1471 catalog_manager.cc:5697] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:42.582564  1455 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3ee0e81a514641709317e983fcf96fe4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:42.596596  1708 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ee0e81a514641709317e983fcf96fe4 with delete_type TABLET_DATA_TOMBSTONED (TS 130bc5ff39b847b1b3204626f284f838 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42266
I20260709 13:01:42.597904  2454 tablet_replica.cc:333] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:01:42.600308  2454 raft_consensus.cc:2243] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:42.601291  2454 raft_consensus.cc:2272] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:42.612185  2454 ts_tablet_manager.cc:1916] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:42.632055  2454 ts_tablet_manager.cc:1929] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:42.632385  2454 log.cc:1199] T 3ee0e81a514641709317e983fcf96fe4 P 130bc5ff39b847b1b3204626f284f838: Deleting WAL directory at /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-2-root/wals/3ee0e81a514641709317e983fcf96fe4
I20260709 13:01:42.634171  1456 catalog_manager.cc:5028] TS 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): tablet 3ee0e81a514641709317e983fcf96fe4 (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
I20260709 13:01:43.698480  1644 consensus_queue.cc:237] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:43.710649  1565 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:43.712591  2436 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:43.719163  1716 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:43.722468  1612 consensus_peers.cc:597] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a5f5eedbd1406788cafbad6a95a4c7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:43.723007  2453 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:43.745450  2379 raft_consensus.cc:2955] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:43.751530  1717 raft_consensus.cc:2955] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:43.754794  1565 raft_consensus.cc:2955] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:43.787951  1716 consensus_queue.cc:237] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:43.792725  1470 catalog_manager.cc:5697] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:43.824913  1568 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:43.825428  1646 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:43.826845  1686 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9688a8a5839c4bb0b5a0196390670b14. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:43.827064  2456 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:43.829872  2456 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:43.874475  2457 raft_consensus.cc:2955] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:43.880000  1565 raft_consensus.cc:2955] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:43.889585  1646 raft_consensus.cc:2955] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:43.922657  1565 consensus_queue.cc:237] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:43.925473  1469 catalog_manager.cc:5697] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:43.942173  1537 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d2618e77160b44339d8499efcd1e8eca. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:43.944083  1713 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:43.946283  1646 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:43.948470  2240 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:43.950147  2467 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:43.967988  2455 raft_consensus.cc:2955] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:43.972801  1713 raft_consensus.cc:2955] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:43.982525  1644 raft_consensus.cc:2955] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:43.999850  1713 consensus_queue.cc:237] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:44.002238  1471 catalog_manager.cc:5697] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:44.045934  1644 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:44.051752  1565 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:44.052021  1686 consensus_peers.cc:597] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7b0d191c5dc46468a777fba0a666b32. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:44.053292  2457 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:44.055707  2456 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:44.068060  2456 raft_consensus.cc:2955] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:44.077001  1646 raft_consensus.cc:2955] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:44.101895  1565 raft_consensus.cc:2955] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:44.142370  1469 catalog_manager.cc:5697] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:44.152303  2478 ts_tablet_manager.cc:933] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:44.170526  2478 tablet_copy_client.cc:323] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:44293
I20260709 13:01:44.174715  1658 tablet_copy_service.cc:140] P f5f4b2707a9647e5bb99e065068dea2b: Received BeginTabletCopySession request for tablet e7a5f5eedbd1406788cafbad6a95a4c7 from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:44424)
I20260709 13:01:44.175514  1658 tablet_copy_service.cc:161] P f5f4b2707a9647e5bb99e065068dea2b: Beginning new tablet copy session on tablet e7a5f5eedbd1406788cafbad6a95a4c7 from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:44424: session id = cf2f71592be8465fb18e36e15d0b2f89-e7a5f5eedbd1406788cafbad6a95a4c7
I20260709 13:01:44.184794  1658 tablet_copy_source_session.cc:215] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:44.189594  2478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7a5f5eedbd1406788cafbad6a95a4c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:44.209239  2478 tablet_copy_client.cc:806] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:44.210052  2478 tablet_copy_client.cc:670] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:44.216894  2478 tablet_copy_client.cc:538] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:44.225530  2478 tablet_bootstrap.cc:492] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:01:44.258091  2478 tablet_bootstrap.cc:492] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:44.259042  2478 tablet_bootstrap.cc:492] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:01:44.259559  2478 ts_tablet_manager.cc:1403] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 0.034s	user 0.024s	sys 0.011s
I20260709 13:01:44.263120  2478 raft_consensus.cc:359] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:44.264171  2478 raft_consensus.cc:740] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:01:44.264892  2478 consensus_queue.cc:260] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:44.268787  2478 ts_tablet_manager.cc:1434] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.004s
I20260709 13:01:44.273185  1658 tablet_copy_service.cc:342] P f5f4b2707a9647e5bb99e065068dea2b: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-e7a5f5eedbd1406788cafbad6a95a4c7 received from {username='slave'} at 127.0.0.1:44424
I20260709 13:01:44.273772  1658 tablet_copy_service.cc:434] P f5f4b2707a9647e5bb99e065068dea2b: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-e7a5f5eedbd1406788cafbad6a95a4c7 on tablet e7a5f5eedbd1406788cafbad6a95a4c7 with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:44.380643  2478 ts_tablet_manager.cc:933] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:01:44.384513  2478 tablet_copy_client.cc:323] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:39999
I20260709 13:01:44.386730  1583 tablet_copy_service.cc:140] P 066ee0109aba4465ba17fa65e14ef504: Received BeginTabletCopySession request for tablet d2618e77160b44339d8499efcd1e8eca from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:46982)
I20260709 13:01:44.388042  1583 tablet_copy_service.cc:161] P 066ee0109aba4465ba17fa65e14ef504: Beginning new tablet copy session on tablet d2618e77160b44339d8499efcd1e8eca from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:46982: session id = cf2f71592be8465fb18e36e15d0b2f89-d2618e77160b44339d8499efcd1e8eca
I20260709 13:01:44.403584  1583 tablet_copy_source_session.cc:215] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:44.408219  2483 ts_tablet_manager.cc:933] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:44.410399  2478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2618e77160b44339d8499efcd1e8eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:44.420022  2483 tablet_copy_client.cc:323] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:34553
I20260709 13:01:44.422160  1732 tablet_copy_service.cc:140] P 130bc5ff39b847b1b3204626f284f838: Received BeginTabletCopySession request for tablet 9688a8a5839c4bb0b5a0196390670b14 from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:34078)
I20260709 13:01:44.422734  1732 tablet_copy_service.cc:161] P 130bc5ff39b847b1b3204626f284f838: Beginning new tablet copy session on tablet 9688a8a5839c4bb0b5a0196390670b14 from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:34078: session id = cf2f71592be8465fb18e36e15d0b2f89-9688a8a5839c4bb0b5a0196390670b14
I20260709 13:01:44.431277  2478 tablet_copy_client.cc:806] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:44.431996  2478 tablet_copy_client.cc:670] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:44.437018  2478 tablet_copy_client.cc:538] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:44.439101  1732 tablet_copy_source_session.cc:215] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:44.444877  2483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9688a8a5839c4bb0b5a0196390670b14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:44.457796  2478 tablet_bootstrap.cc:492] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:01:44.478593  2483 tablet_copy_client.cc:806] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:44.479468  2483 tablet_copy_client.cc:670] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:44.486898  2483 tablet_copy_client.cc:538] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:44.498028  2478 tablet_bootstrap.cc:492] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:44.499897  2478 tablet_bootstrap.cc:492] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:01:44.500730  2483 tablet_bootstrap.cc:492] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:01:44.500741  2478 ts_tablet_manager.cc:1403] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 0.043s	user 0.022s	sys 0.015s
I20260709 13:01:44.507285  2478 raft_consensus.cc:359] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:44.508419  2478 raft_consensus.cc:740] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:01:44.510504  2478 consensus_queue.cc:260] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:44.516901  2478 ts_tablet_manager.cc:1434] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.004s
I20260709 13:01:44.521797  1583 tablet_copy_service.cc:342] P 066ee0109aba4465ba17fa65e14ef504: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-d2618e77160b44339d8499efcd1e8eca received from {username='slave'} at 127.0.0.1:46982
I20260709 13:01:44.522296  1583 tablet_copy_service.cc:434] P 066ee0109aba4465ba17fa65e14ef504: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-d2618e77160b44339d8499efcd1e8eca on tablet d2618e77160b44339d8499efcd1e8eca with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:44.531160  2483 tablet_bootstrap.cc:492] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:44.531963  2483 tablet_bootstrap.cc:492] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:01:44.532545  2483 ts_tablet_manager.cc:1403] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.007s
I20260709 13:01:44.535386  2483 raft_consensus.cc:359] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:44.536278  2483 raft_consensus.cc:740] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:01:44.536962  2483 consensus_queue.cc:260] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:44.540799  2483 ts_tablet_manager.cc:1434] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:01:44.545136  1732 tablet_copy_service.cc:342] P 130bc5ff39b847b1b3204626f284f838: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-9688a8a5839c4bb0b5a0196390670b14 received from {username='slave'} at 127.0.0.1:34078
I20260709 13:01:44.546238  1732 tablet_copy_service.cc:434] P 130bc5ff39b847b1b3204626f284f838: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-9688a8a5839c4bb0b5a0196390670b14 on tablet 9688a8a5839c4bb0b5a0196390670b14 with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:44.655403  2483 ts_tablet_manager.cc:933] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:44.683247  2483 tablet_copy_client.cc:323] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:34553
I20260709 13:01:44.686935  1732 tablet_copy_service.cc:140] P 130bc5ff39b847b1b3204626f284f838: Received BeginTabletCopySession request for tablet b7b0d191c5dc46468a777fba0a666b32 from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:34078)
I20260709 13:01:44.687565  1732 tablet_copy_service.cc:161] P 130bc5ff39b847b1b3204626f284f838: Beginning new tablet copy session on tablet b7b0d191c5dc46468a777fba0a666b32 from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:34078: session id = cf2f71592be8465fb18e36e15d0b2f89-b7b0d191c5dc46468a777fba0a666b32
I20260709 13:01:44.696786  1732 tablet_copy_source_session.cc:215] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:44.706022  2483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7b0d191c5dc46468a777fba0a666b32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:44.729972  2483 tablet_copy_client.cc:806] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:44.732491  2483 tablet_copy_client.cc:670] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:44.737658  2483 tablet_copy_client.cc:538] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:44.739890  2348 raft_consensus.cc:1217] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:44.756819  2483 tablet_bootstrap.cc:492] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:01:44.785369  2483 tablet_bootstrap.cc:492] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:44.786197  2483 tablet_bootstrap.cc:492] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:01:44.786993  2483 ts_tablet_manager.cc:1403] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.012s
I20260709 13:01:44.790166  2483 raft_consensus.cc:359] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:44.791425  2483 raft_consensus.cc:740] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:01:44.792090  2483 consensus_queue.cc:260] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:44.795549  2483 ts_tablet_manager.cc:1434] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:01:44.797744  1732 tablet_copy_service.cc:342] P 130bc5ff39b847b1b3204626f284f838: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-b7b0d191c5dc46468a777fba0a666b32 received from {username='slave'} at 127.0.0.1:34078
I20260709 13:01:44.798240  1732 tablet_copy_service.cc:434] P 130bc5ff39b847b1b3204626f284f838: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-b7b0d191c5dc46468a777fba0a666b32 on tablet b7b0d191c5dc46468a777fba0a666b32 with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:44.821069  2348 raft_consensus.cc:1217] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:44.993321  2348 raft_consensus.cc:1217] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:45.209048  2348 raft_consensus.cc:1217] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:45.315294  2477 raft_consensus.cc:1064] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:01:45.318759  2477 consensus_queue.cc:237] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:45.322574  2461 raft_consensus.cc:1064] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:01:45.328258  2348 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:45.331204  2476 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:45.332324  1565 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:45.335023  2456 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:45.336441  1640 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:45.338989  2476 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:45.344655  2461 consensus_queue.cc:237] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:45.389765  2441 raft_consensus.cc:2955] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:45.395133  2348 raft_consensus.cc:2955] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:45.405095  2349 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:45.415912  2453 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:45.429091  1720 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:01:45.436512  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45989.7us
I20260709 13:01:45.437992  2489 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:45.442044  1565 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:45.446869  1640 raft_consensus.cc:2955] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:45.458474  1470 catalog_manager.cc:5697] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:45.464699  2461 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:01:45.546990  2499 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 13:01:46.084959  1571 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.006s	sys 0.001s
W20260709 13:01:46.093405  1571 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.011s	sys 0.001s
I20260709 13:01:46.155966  2520 raft_consensus.cc:2955] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:46.387830  2348 raft_consensus.cc:2955] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:01:46.394574  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.004s	sys 0.013s
W20260709 13:01:46.395707  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.004s	sys 0.013s
W20260709 13:01:46.402129  1537 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.922s	user 0.005s	sys 0.003s
W20260709 13:01:46.402868  1537 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.005s	sys 0.003s
W20260709 13:01:46.407342  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 927386us
W20260709 13:01:46.454566  1538 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.942s	user 0.019s	sys 0.044s
W20260709 13:01:46.455008  1538 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.944s	user 0.019s	sys 0.044s
I20260709 13:01:46.467142  2518 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:46.468559  2518 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:46.475698  2518 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:46.527186  2518 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:46.528414  2518 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:46.531788  2518 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:46.563862  2520 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:46.599818  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:46.601060  1720 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:46.601796  2504 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:46.602634  2504 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:46.603394  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:46.603777  2503 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:46.604369  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:46.604351  2503 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:46.606827  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:46.564765  2520 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:46.608328  2523 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:46.607613  2503 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:46.609056  2498 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:01:46.544220  1471 catalog_manager.cc:5697] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:46.630909  2498 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:46.646203  2518 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:46.647223  2518 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:46.630309  2523 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:46.656167  2518 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:46.671366  2520 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:46.684381  2504 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:46.697546  2506 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:01:46.699410  2506 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:46.702945  2498 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:46.704020  2506 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:46.707572  1568 raft_consensus.cc:2955] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:46.714701  2523 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:01:46.718089  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 100205us
I20260709 13:01:46.719617  2441 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:46.720166  2505 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:46.720474  2441 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:46.720858  2505 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:46.722329  2503 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:46.723325  2505 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:46.724076  2508 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:46.724597  2505 raft_consensus.cc:1064] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:01:46.724718  2508 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:46.725709  2441 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:46.723065  2503 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:46.731554  2503 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:46.727442  2511 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:46.733271  2511 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:46.735747  2511 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:46.728698  2508 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:46.746469  2498 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:46.748153  2498 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:46.751796  2498 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:46.758965  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:46.761956  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:46.763921  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:46.765064  1571 raft_consensus.cc:2955] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:46.766716  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:46.776737  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:46.779245  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:46.780221  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:46.778772  2505 consensus_queue.cc:237] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:46.781280  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:46.782125  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:46.781313  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:46.790627  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:46.783394  1722 consensus_queue.cc:237] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:46.807910  2461 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:01:46.808753  2461 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:46.809736  2498 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:46.811476  2461 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:46.811800  2498 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:46.817239  2240 raft_consensus.cc:1064] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:01:46.818794  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:46.817943  2498 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:46.820880  2240 consensus_queue.cc:237] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:46.822908  2498 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:01:46.823598  2498 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:46.825722  2498 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:46.834285  2511 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:46.834970  2511 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:46.837093  2511 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:01:46.840610  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:46.842185  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:46.914716  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:46.923205  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:46.923839  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:46.924863  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:46.925511  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:46.927764  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:46.929325  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:46.930421  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:46.932729  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:46.936069  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:46.937479  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:46.942489  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:46.942945  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:46.946259  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:46.947445  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:46.954310  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:46.962110  2348 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:46.964766  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:46.977195  2503 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:46.983734  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:47.014770  1612 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:01:47.023367  2348 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:47.023672  1612 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:47.025604  1612 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:47.026777  1612 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:47.026161  2518 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:47.028227  2518 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:47.028681  2518 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:47.035285  2520 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:47.036137  2519 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:47.039363  2527 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:47.039921  1612 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:47.047789  2520 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:47.051844  2518 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:47.060359  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 61882.9us
W20260709 13:01:47.060597  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49255.2us
I20260709 13:01:47.061717  1685 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:47.062933  2518 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:47.052747  1612 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:01:47.059576  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 67558.5us
I20260709 13:01:47.068606  2507 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:47.074417  2526 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:47.079203  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:47.081609  1642 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:01:47.087839  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.096951  2518 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:47.099002  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 46409.3us
I20260709 13:01:47.099936  1612 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:47.105214  2511 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:47.111426  2349 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:47.112076  2520 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:47.117048  1722 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:47.120334  1538 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:47.121840  2536 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:47.123317  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:47.125558  1720 raft_consensus.cc:2955] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:47.131731  1645 consensus_queue.cc:237] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:47.144636  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:47.145337  1611 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:47.147435  2527 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:47.152480  1639 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:47.150956  2537 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:01:47.169318  2522 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:47.181471  1611 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:47.183377  2518 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:47.192951  2346 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:47.196738  2527 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:47.204582  2506 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:47.223320  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:47.236922  2519 raft_consensus.cc:2955] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:47.240206  2345 raft_consensus.cc:2955] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:47.254102  1718 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:47.295644  1611 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:01:47.302345  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 238108us
I20260709 13:01:47.303768  1685 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:47.306386  1685 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:47.312878  1685 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:47.314954  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:47.292454  2240 raft_consensus.cc:2955] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:47.323025  1685 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:47.305367  2527 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:47.309127  2523 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:47.317232  2538 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:47.324448  2526 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:47.327633  1685 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:47.326546  2511 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:47.331521  2526 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:47.332064  1685 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:47.333051  2477 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:47.338740  2477 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:47.312523  2520 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:01:47.339675  1685 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:47.341712  2523 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:47.333997  2538 raft_consensus.cc:2955] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:47.340507  2347 raft_consensus.cc:2955] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:47.360430  2349 raft_consensus.cc:2955] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:47.405557  1571 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:47.391844  1570 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:47.404003  1713 raft_consensus.cc:2955] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:47.446158  1645 raft_consensus.cc:2955] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:01:47.480779  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 44720.8us
I20260709 13:01:47.487006  1457 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e7a5f5eedbd1406788cafbad6a95a4c7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:47.487280  2523 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:01:47.439349  2505 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:01:47.377285  2507 raft_consensus.cc:2955] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:01:47.504874  2370 debug-util.cc:398] Leaking SignalData structure 0x7b08002e0060 after lost signal to thread 434
W20260709 13:01:47.506493  2370 debug-util.cc:398] Leaking SignalData structure 0x7b08002e0740 after lost signal to thread 1505
W20260709 13:01:47.508471  2370 debug-util.cc:398] Leaking SignalData structure 0x7b08002c2fe0 after lost signal to thread 1597
I20260709 13:01:47.527477  1456 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9688a8a5839c4bb0b5a0196390670b14 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:01:47.527928  2370 debug-util.cc:398] Leaking SignalData structure 0x7b08002f0f40 after lost signal to thread 1672
W20260709 13:01:47.531476  2370 debug-util.cc:398] Leaking SignalData structure 0x7b080031e3a0 after lost signal to thread 1746
W20260709 13:01:47.536267  2370 debug-util.cc:398] Leaking SignalData structure 0x7b08002bd640 after lost signal to thread 2373
I20260709 13:01:47.548058  1471 catalog_manager.cc:5697] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: config changed from index 3 to 4, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.583053  1720 raft_consensus.cc:2955] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:47.576784  2345 raft_consensus.cc:2955] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:47.618283  1469 catalog_manager.cc:5697] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index 3 to 4, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.627978  1642 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:47.636368  1567 raft_consensus.cc:2955] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:47.654551  1642 raft_consensus.cc:2955] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:47.692914  1570 raft_consensus.cc:2955] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:47.697050  2522 consensus_queue.cc:1059] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:47.702456  2524 raft_consensus.cc:3055] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:47.702905  2524 raft_consensus.cc:740] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:47.703855  2524 consensus_queue.cc:260] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:47.707990  2524 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:47.740721  1468 catalog_manager.cc:5697] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.842000  1558 tablet_service.cc:1558] Processing DeleteTablet for tablet e7a5f5eedbd1406788cafbad6a95a4c7 with delete_type TABLET_DATA_TOMBSTONED (TS 066ee0109aba4465ba17fa65e14ef504 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40642
I20260709 13:01:47.863957  2557 tablet_replica.cc:333] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:01:47.865490  2557 raft_consensus.cc:2243] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:47.883059  2557 raft_consensus.cc:2272] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:47.886648  1629 tablet_service.cc:1558] Processing DeleteTablet for tablet 9688a8a5839c4bb0b5a0196390670b14 with delete_type TABLET_DATA_TOMBSTONED (TS f5f4b2707a9647e5bb99e065068dea2b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39910
I20260709 13:01:47.899924  2557 ts_tablet_manager.cc:1916] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:47.925542  2559 tablet_replica.cc:333] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:01:47.932331  1471 catalog_manager.cc:5697] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:47.968086  2557 ts_tablet_manager.cc:1929] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:47.968820  2557 log.cc:1199] T e7a5f5eedbd1406788cafbad6a95a4c7 P 066ee0109aba4465ba17fa65e14ef504: Deleting WAL directory at /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-0-root/wals/e7a5f5eedbd1406788cafbad6a95a4c7
I20260709 13:01:47.965524  1567 consensus_queue.cc:237] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:47.971685  1455 catalog_manager.cc:5028] TS 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): tablet e7a5f5eedbd1406788cafbad6a95a4c7 (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
I20260709 13:01:47.995082  1642 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:47.995273  2559 raft_consensus.cc:2243] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:47.998307  2537 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:01:48.017861  2559 raft_consensus.cc:2272] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:48.042313  2349 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:48.054993  2524 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:48.056442  1718 consensus_queue.cc:237] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:48.058004  2536 raft_consensus.cc:2955] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:48.063547  1647 raft_consensus.cc:2955] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:48.079372  2559 ts_tablet_manager.cc:1916] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:48.105974  2349 raft_consensus.cc:2955] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:48.110482  2345 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:48.114190  2477 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:48.128742  1642 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:48.133904  2549 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:48.138862  1470 catalog_manager.cc:5697] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:48.140132  2507 raft_consensus.cc:2955] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:48.152485  1642 raft_consensus.cc:2955] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:48.174611  2345 raft_consensus.cc:2955] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:48.183522  1708 tablet_service.cc:1558] Processing DeleteTablet for tablet d2618e77160b44339d8499efcd1e8eca with delete_type TABLET_DATA_TOMBSTONED (TS 130bc5ff39b847b1b3204626f284f838 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42266
I20260709 13:01:48.220188  1456 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b7b0d191c5dc46468a777fba0a666b32 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:48.227289  2560 tablet_replica.cc:333] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:01:48.236945  2560 raft_consensus.cc:2243] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:48.237929  2560 raft_consensus.cc:2272] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:48.250838  2560 ts_tablet_manager.cc:1916] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:48.253532  2559 ts_tablet_manager.cc:1929] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:48.258468  2559 log.cc:1199] T 9688a8a5839c4bb0b5a0196390670b14 P f5f4b2707a9647e5bb99e065068dea2b: Deleting WAL directory at /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-1-root/wals/9688a8a5839c4bb0b5a0196390670b14
I20260709 13:01:48.248878  1468 catalog_manager.cc:5697] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: config changed from index 3 to 4, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:48.261179  1455 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d2618e77160b44339d8499efcd1e8eca with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:48.266343  1457 catalog_manager.cc:5028] TS f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): tablet 9688a8a5839c4bb0b5a0196390670b14 (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
I20260709 13:01:48.297570  2560 ts_tablet_manager.cc:1929] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:48.298504  2560 log.cc:1199] T d2618e77160b44339d8499efcd1e8eca P 130bc5ff39b847b1b3204626f284f838: Deleting WAL directory at /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-2-root/wals/d2618e77160b44339d8499efcd1e8eca
I20260709 13:01:48.301738  1456 catalog_manager.cc:5028] TS 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): tablet d2618e77160b44339d8499efcd1e8eca (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
I20260709 13:01:48.350567  1558 tablet_service.cc:1558] Processing DeleteTablet for tablet b7b0d191c5dc46468a777fba0a666b32 with delete_type TABLET_DATA_TOMBSTONED (TS 066ee0109aba4465ba17fa65e14ef504 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40642
I20260709 13:01:48.351632  2557 tablet_replica.cc:333] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:01:48.352803  2557 raft_consensus.cc:2243] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:48.353574  2557 raft_consensus.cc:2272] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:48.363567  2557 ts_tablet_manager.cc:1916] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:48.390384  2557 ts_tablet_manager.cc:1929] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:48.390983  2557 log.cc:1199] T b7b0d191c5dc46468a777fba0a666b32 P 066ee0109aba4465ba17fa65e14ef504: Deleting WAL directory at /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-0-root/wals/b7b0d191c5dc46468a777fba0a666b32
I20260709 13:01:48.393117  1455 catalog_manager.cc:5028] TS 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): tablet b7b0d191c5dc46468a777fba0a666b32 (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
I20260709 13:01:48.628306  2477 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:48.628988  2477 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:48.631551  2477 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:48.634038  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:48.635103  1567 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:01:48.637403  1685 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:48.638806  2549 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:48.639236  2549 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:48.639637  2549 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:48.642064  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:48.642992  1642 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 2: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:01:48.661129  2549 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:48.664592  2549 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:48.664639  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:48.665041  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:48.666182  1642 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 2: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:01:48.680136  1567 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:01:48.682147  1685 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:48.683631  2477 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:48.684355  2477 raft_consensus.cc:697] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:01:48.686098  2477 consensus_queue.cc:237] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:48.720474  1470 catalog_manager.cc:5697] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 1 to 2, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:49.157567  2520 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.158169  2520 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:49.160100  2520 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:49.161598  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:49.162285  1567 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:01:49.162777  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:49.163532  1611 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:49.164583  2546 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:49.164975  2546 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.165333  2546 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:49.190330  2546 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:49.195843  2546 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:49.196174  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:49.197330  1567 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:49.198582  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:49.205691  1567 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:01:49.208349  1611 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:49.210223  2546 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:49.212246  2546 raft_consensus.cc:697] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:01:49.217458  2546 consensus_queue.cc:237] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:49.220372  1567 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:49.222510  2507 consensus_queue.cc:1059] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:49.223675  2477 raft_consensus.cc:3055] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:49.224380  2477 raft_consensus.cc:740] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:01:49.225346  2477 consensus_queue.cc:260] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:49.228276  2477 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:49.234630  1470 catalog_manager.cc:5697] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 2 to 3, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 3 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:49.476320  1567 consensus_queue.cc:237] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:49.497195  1718 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:49.497449  1537 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c6485a5991c348019b9779be81656a45. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:49.499819  1642 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:49.499752  2506 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:49.501468  2537 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:49.511031  2506 raft_consensus.cc:2955] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:49.516314  1720 raft_consensus.cc:2955] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:49.548553  1645 consensus_queue.cc:237] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:49.538300  1470 catalog_manager.cc:5697] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:49.553520  1642 raft_consensus.cc:2955] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:49.598419  1570 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:49.600314  1720 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:49.601677  1567 raft_consensus.cc:1275] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:49.601753  2546 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:49.603310  2519 consensus_queue.cc:1048] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:49.607270  2519 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:49.610163  1612 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0764c6a89eca47f39e28feb56e2123c7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:49.663854  2519 raft_consensus.cc:2955] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:49.705192  1570 raft_consensus.cc:2955] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:49.759538  1717 raft_consensus.cc:1275] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:01:49.760100  1717 pending_rounds.cc:85] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 1
I20260709 13:01:49.760524  1717 pending_rounds.cc:107] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:01:49.760923  1717 raft_consensus.cc:2889] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:49.765810  2577 consensus_queue.cc:1048] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:49.786938  1470 catalog_manager.cc:5697] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
W20260709 13:01:49.788435  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 46175.4us
I20260709 13:01:49.785832  1570 consensus_queue.cc:237] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:49.879267  1717 raft_consensus.cc:2955] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
W20260709 13:01:49.866106  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49571.1us
I20260709 13:01:49.951550  1720 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:49.953979  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 88987.7us
I20260709 13:01:49.958842  1645 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:49.969386  2522 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:49.962698  1537 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c6f11a07e734ec095c6185363fd6b52. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:49.975100  2536 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:50.060201  2587 ts_tablet_manager.cc:933] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:01:50.122148  1718 raft_consensus.cc:2955] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:50.084932  2537 raft_consensus.cc:2955] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:50.108569  1645 raft_consensus.cc:2955] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
W20260709 13:01:50.159981  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 60989.8us
I20260709 13:01:50.162168  2594 ts_tablet_manager.cc:933] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:50.162338  2587 tablet_copy_client.cc:323] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:39999
I20260709 13:01:50.165477  1583 tablet_copy_service.cc:140] P 066ee0109aba4465ba17fa65e14ef504: Received BeginTabletCopySession request for tablet c6485a5991c348019b9779be81656a45 from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:46982)
I20260709 13:01:50.170275  1583 tablet_copy_service.cc:161] P 066ee0109aba4465ba17fa65e14ef504: Beginning new tablet copy session on tablet c6485a5991c348019b9779be81656a45 from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:46982: session id = cf2f71592be8465fb18e36e15d0b2f89-c6485a5991c348019b9779be81656a45
I20260709 13:01:50.183457  1639 consensus_queue.cc:237] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:50.185482  1471 catalog_manager.cc:5697] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:50.238142  2594 tablet_copy_client.cc:323] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:44293
I20260709 13:01:50.234737  1583 tablet_copy_source_session.cc:215] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:50.248427  1658 tablet_copy_service.cc:140] P f5f4b2707a9647e5bb99e065068dea2b: Received BeginTabletCopySession request for tablet 0764c6a89eca47f39e28feb56e2123c7 from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:44424)
I20260709 13:01:50.249053  1658 tablet_copy_service.cc:161] P f5f4b2707a9647e5bb99e065068dea2b: Beginning new tablet copy session on tablet 0764c6a89eca47f39e28feb56e2123c7 from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:44424: session id = cf2f71592be8465fb18e36e15d0b2f89-0764c6a89eca47f39e28feb56e2123c7
W20260709 13:01:50.256265  1612 consensus_peers.cc:597] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 45c609109d4a4c40a3e74ff0917c7958. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:50.264492  1568 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:50.269726  2586 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:50.272462  1658 tablet_copy_source_session.cc:215] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:50.281440  1718 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:50.283830  2594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0764c6a89eca47f39e28feb56e2123c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:50.286509  2587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6485a5991c348019b9779be81656a45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:50.344532  2579 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:50.339270  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 53282.5us
I20260709 13:01:50.347594  2583 raft_consensus.cc:2955] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
W20260709 13:01:50.349941  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 69964.1us
I20260709 13:01:50.502421  2605 ts_tablet_manager.cc:933] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:01:50.508752  1566 raft_consensus.cc:2955] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:50.505721  2594 tablet_copy_client.cc:806] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:50.518018  1720 raft_consensus.cc:2955] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:50.527685  2594 tablet_copy_client.cc:670] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:50.568331  2605 tablet_copy_client.cc:323] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:39999
I20260709 13:01:50.585300  1470 catalog_manager.cc:5697] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:50.587651  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 45401.6us
I20260709 13:01:50.603302  2594 tablet_copy_client.cc:538] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:01:50.642213  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 104045us
I20260709 13:01:50.680224  2612 ts_tablet_manager.cc:933] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:50.691304  1583 tablet_copy_service.cc:140] P 066ee0109aba4465ba17fa65e14ef504: Received BeginTabletCopySession request for tablet 0c6f11a07e734ec095c6185363fd6b52 from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:46982)
I20260709 13:01:50.695598  2594 tablet_bootstrap.cc:492] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:01:50.692011  1583 tablet_copy_service.cc:161] P 066ee0109aba4465ba17fa65e14ef504: Beginning new tablet copy session on tablet 0c6f11a07e734ec095c6185363fd6b52 from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:46982: session id = cf2f71592be8465fb18e36e15d0b2f89-0c6f11a07e734ec095c6185363fd6b52
I20260709 13:01:50.732509  1583 tablet_copy_source_session.cc:215] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:50.733922  2612 tablet_copy_client.cc:323] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:44293
I20260709 13:01:50.753402  2587 tablet_copy_client.cc:806] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:50.773860  1658 tablet_copy_service.cc:140] P f5f4b2707a9647e5bb99e065068dea2b: Received BeginTabletCopySession request for tablet 45c609109d4a4c40a3e74ff0917c7958 from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:44424)
I20260709 13:01:50.775292  1658 tablet_copy_service.cc:161] P f5f4b2707a9647e5bb99e065068dea2b: Beginning new tablet copy session on tablet 45c609109d4a4c40a3e74ff0917c7958 from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:44424: session id = cf2f71592be8465fb18e36e15d0b2f89-45c609109d4a4c40a3e74ff0917c7958
I20260709 13:01:50.776069  2605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c6f11a07e734ec095c6185363fd6b52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:50.805884  2587 tablet_copy_client.cc:670] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:50.815091  2587 tablet_copy_client.cc:538] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:50.810263  1658 tablet_copy_source_session.cc:215] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:50.837458  2605 tablet_copy_client.cc:806] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:50.838299  2605 tablet_copy_client.cc:670] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:50.838363  2587 tablet_bootstrap.cc:492] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:01:50.843218  2605 tablet_copy_client.cc:538] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:50.844215  2612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45c609109d4a4c40a3e74ff0917c7958. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:50.848796  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 44786us
I20260709 13:01:50.867580  2605 tablet_bootstrap.cc:492] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:01:50.911860  2612 tablet_copy_client.cc:806] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:50.919673  2587 tablet_bootstrap.cc:492] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:50.920720  2587 tablet_bootstrap.cc:492] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:01:50.913522  2612 tablet_copy_client.cc:670] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:50.933599  2605 tablet_bootstrap.cc:492] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:50.936529  2605 tablet_bootstrap.cc:492] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:01:50.934077  2587 ts_tablet_manager.cc:1403] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 0.096s	user 0.019s	sys 0.011s
I20260709 13:01:50.937515  2605 ts_tablet_manager.cc:1403] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 0.070s	user 0.022s	sys 0.010s
I20260709 13:01:50.951834  2612 tablet_copy_client.cc:538] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:50.955220  2605 raft_consensus.cc:359] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:50.997290  2587 raft_consensus.cc:359] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:50.998548  2587 raft_consensus.cc:740] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:01:50.966677  2605 raft_consensus.cc:740] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:01:50.999452  2587 consensus_queue.cc:260] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:51.002734  2605 consensus_queue.cc:260] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:51.048144  2594 tablet_bootstrap.cc:492] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:51.049301  2594 tablet_bootstrap.cc:492] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:01:51.072960  2594 ts_tablet_manager.cc:1403] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 0.381s	user 0.033s	sys 0.010s
I20260709 13:01:51.136974  2612 tablet_bootstrap.cc:492] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:01:51.317989  2605 ts_tablet_manager.cc:1434] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.380s	user 0.006s	sys 0.009s
I20260709 13:01:51.320489  2594 raft_consensus.cc:359] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:51.329839  2594 raft_consensus.cc:740] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:01:51.979605  2594 consensus_queue.cc:260] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
W20260709 13:01:51.999990  2612 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.015s	sys 0.028s
I20260709 13:01:52.000289  2594 ts_tablet_manager.cc:1434] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.876s	user 0.012s	sys 0.008s
W20260709 13:01:52.000876  2612 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.015s	sys 0.028s
W20260709 13:01:52.000981  2594 ts_tablet_manager.cc:1467] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: Tablet startup took 876ms
W20260709 13:01:52.001456  2594 ts_tablet_manager.cc:1469] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: Trace:
0709 13:01:50.091102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.091803 (+   701us) service_pool.cc:224] Handling call
0709 13:01:50.162600 (+ 70797us) ts_tablet_manager.cc:934] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
0709 13:01:50.279082 (+116482us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:01:50.424314 (+145232us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:50.424444 (+   130us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:01:50.454989 (+ 30545us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:01:50.455357 (+   368us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:50.464937 (+  9580us) ts_tablet_manager.cc:989] Replica registered.
0709 13:01:50.470736 (+  5799us) inbound_call.cc:177] Queueing success response
0709 13:01:50.691823 (+221087us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:01:51.124674 (+432851us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:01:51.168730 (+ 44056us) tablet_replica.cc:261] Starting instrumentation
0709 13:01:51.169669 (+   939us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":5293,"raft.queue_time_us":17692,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":120,"tablet-copy.queue_time_us":69529,"tablet-download-pool.queue_time_us":1515,"tablet-download-pool.run_cpu_time_us":1766,"tablet-download-pool.run_wall_time_us":59358,"thread_start_us":130090,"threads_started":3}
I20260709 13:01:52.010771  2612 tablet_bootstrap.cc:492] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:52.012189  2612 tablet_bootstrap.cc:492] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:01:52.013058  2612 ts_tablet_manager.cc:1403] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 0.876s	user 0.044s	sys 0.034s
I20260709 13:01:52.016633  2612 raft_consensus.cc:359] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:52.017834  2612 raft_consensus.cc:740] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:01:52.018576  2612 consensus_queue.cc:260] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:52.021232  2612 ts_tablet_manager.cc:1434] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:01:52.065459  1583 tablet_copy_service.cc:342] P 066ee0109aba4465ba17fa65e14ef504: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-0c6f11a07e734ec095c6185363fd6b52 received from {username='slave'} at 127.0.0.1:46982
I20260709 13:01:52.066785  1583 tablet_copy_service.cc:434] P 066ee0109aba4465ba17fa65e14ef504: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-0c6f11a07e734ec095c6185363fd6b52 on tablet 0c6f11a07e734ec095c6185363fd6b52 with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:52.124794  2587 ts_tablet_manager.cc:1434] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 1.181s	user 0.021s	sys 0.038s
W20260709 13:01:52.125365  2587 ts_tablet_manager.cc:1467] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Tablet startup took 1182ms
I20260709 13:01:52.133471  2622 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:01:52.143961  2347 raft_consensus.cc:1217] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:52.134090  2622 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.152871  2586 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:52.153513  2586 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.156770  2622 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:52.156785  2586 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:52.164109  2586 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:52.168056  2586 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.176546  2586 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:52.179541  2596 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:52.181915  2596 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:01:52.180073  1538 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.876s	user 0.010s	sys 0.001s
W20260709 13:01:52.186378  1538 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.011s	sys 0.001s
I20260709 13:01:52.190411  2627 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:01:52.191951  2627 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.219010  2627 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:52.198012  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:52.179677  2597 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:01:52.125765  2587 ts_tablet_manager.cc:1469] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Trace:
0709 13:01:49.975679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.975846 (+   167us) service_pool.cc:224] Handling call
0709 13:01:50.060629 (+ 84783us) ts_tablet_manager.cc:934] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
0709 13:01:50.281598 (+220969us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:01:50.603707 (+322109us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:50.603856 (+   149us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:01:50.688150 (+ 84294us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:01:50.688626 (+   476us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:50.688970 (+   344us) ts_tablet_manager.cc:989] Replica registered.
0709 13:01:50.689212 (+   242us) inbound_call.cc:177] Queueing success response
0709 13:01:50.837739 (+148527us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:01:50.943898 (+106159us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:01:50.949855 (+  5957us) tablet_replica.cc:261] Starting instrumentation
0709 13:01:50.950429 (+   574us) tablet_replica.cc:280] Starting consensus
0709 13:01:52.116422 (+1165993us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b540010f970. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245de9bec 00007f824c7d8a13 00007f824c87d72f 00007f824de99e04 00007f824de9848d 00007f824dec9f4e 00007f824deaefdd 00007f824deaef89 00007f824deaef19 00007f824deaeee1
Metrics: {"dns_us":312,"raft.queue_time_us":102889,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":214,"spinlock_wait_cycles":3670272,"tablet-copy.queue_time_us":83597,"tablet-download-pool.queue_time_us":2748,"tablet-download-pool.run_cpu_time_us":1541,"tablet-download-pool.run_wall_time_us":5142,"thread_start_us":198964,"threads_started":3}
I20260709 13:01:52.226832  2597 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.229043  2597 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:52.211000  2628 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:01:52.241402  2628 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.243870  2628 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:52.213119  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:52.217015  1658 tablet_copy_service.cc:342] P f5f4b2707a9647e5bb99e065068dea2b: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-0764c6a89eca47f39e28feb56e2123c7 received from {username='slave'} at 127.0.0.1:44424
I20260709 13:01:52.245435  2627 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:01:52.231949  2583 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:52.246842  2583 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:52.234138  1583 tablet_copy_service.cc:342] P 066ee0109aba4465ba17fa65e14ef504: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-c6485a5991c348019b9779be81656a45 received from {username='slave'} at 127.0.0.1:46982
I20260709 13:01:52.249980  1583 tablet_copy_service.cc:434] P 066ee0109aba4465ba17fa65e14ef504: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-c6485a5991c348019b9779be81656a45 on tablet c6485a5991c348019b9779be81656a45 with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:52.234879  2603 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:52.251336  2603 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.207880  2596 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:52.236404  2601 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:01:52.254527  2601 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.257614  2601 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:52.249274  2583 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:52.249169  2628 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:52.260249  2628 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.262580  2628 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:52.258879  2603 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:52.264211  2627 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.193797  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:52.266464  2627 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:52.245376  1658 tablet_copy_service.cc:434] P f5f4b2707a9647e5bb99e065068dea2b: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-0764c6a89eca47f39e28feb56e2123c7 on tablet 0764c6a89eca47f39e28feb56e2123c7 with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:52.271260  2627 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:01:52.272405  1657 tablet_copy_service.cc:342] P f5f4b2707a9647e5bb99e065068dea2b: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-45c609109d4a4c40a3e74ff0917c7958 received from {username='slave'} at 127.0.0.1:44424
I20260709 13:01:52.274156  1657 tablet_copy_service.cc:434] P f5f4b2707a9647e5bb99e065068dea2b: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-45c609109d4a4c40a3e74ff0917c7958 on tablet 45c609109d4a4c40a3e74ff0917c7958 with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:52.272187  2627 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.279837  2627 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:52.280366  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:52.249847  2522 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:01:52.281888  2522 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.284752  2522 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:52.294369  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:01:52.294652  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 72832us
I20260709 13:01:52.295452  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:52.302788  2603 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:01:52.303428  2603 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.301877  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:52.318450  2611 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:01:52.319083  2611 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.322324  2611 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:52.322400  1471 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47518 (request call id 39) took 1206 ms. Trace:
I20260709 13:01:52.323020  1471 rpcz_store.cc:276] 0709 13:01:51.115767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.115932 (+   165us) service_pool.cc:224] Handling call
0709 13:01:52.322284 (+1206352us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1004098,"reported_tablets":2}
I20260709 13:01:52.326057  2603 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:52.327764  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:52.328511  1570 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:52.329722  1611 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:52.331336  2603 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:52.331853  2603 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:01:52.332242  2603 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:52.328630  2602 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:01:52.333410  2602 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.339999  2602 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:52.341379  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:52.342342  2596 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:52.342629  2609 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:01:52.343256  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:52.342970  2596 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.347826  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:52.350251  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:52.343122  2609 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.350988  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:52.355248  2609 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:52.356196  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:52.355268  2596 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:52.362414  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:52.362690  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:52.364405  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:52.364740  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:52.365195  1716 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:01:52.366091  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:52.368868  2348 raft_consensus.cc:1217] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:52.371249  2347 raft_consensus.cc:1217] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:52.374913  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:52.375886  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:52.377929  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:52.379914  2579 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:52.380652  2579 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:52.383142  2622 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:52.383773  2622 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.385980  2622 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:52.386567  2579 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:01:52.388641  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:52.392606  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:52.394035  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:01:52.396477  2609 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:01:52.396675  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:52.397117  2609 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:52.399233  2609 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:01:52.399349  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:52.400302  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:52.399362  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:52.403188  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:52.404253  1611 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:52.392546  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:52.405115  2579 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.409984  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:01:52.412909  1502 debug-util.cc:398] Leaking SignalData structure 0x7b080012c4a0 after lost signal to thread 434
I20260709 13:01:52.412789  2609 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:01:52.414373  1502 debug-util.cc:398] Leaking SignalData structure 0x7b08000386e0 after lost signal to thread 1505
W20260709 13:01:52.416203  1502 debug-util.cc:398] Leaking SignalData structure 0x7b080032f0a0 after lost signal to thread 1597
W20260709 13:01:52.417601  1502 debug-util.cc:398] Leaking SignalData structure 0x7b080032f840 after lost signal to thread 1672
W20260709 13:01:52.419651  1502 debug-util.cc:398] Leaking SignalData structure 0x7b08002f0e20 after lost signal to thread 1746
I20260709 13:01:52.413970  2609 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:01:52.423355  1502 debug-util.cc:398] Leaking SignalData structure 0x7b08000dce40 after lost signal to thread 2373
W20260709 13:01:52.424243  1502 debug-util.cc:398] Leaking SignalData structure 0x7b0800333180 after lost signal to thread 2614
I20260709 13:01:52.427205  2579 raft_consensus.cc:1064] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:01:52.429498  2579 consensus_queue.cc:237] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:52.432433  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:52.442399  1537 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:52.443421  2611 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.445075  1612 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:52.445278  2609 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:01:52.360790  2603 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.446043  2622 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.446054  1537 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:52.446316  1612 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:52.447937  1537 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:52.448786  2611 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.449235  2583 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.449549  2611 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.452957  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:52.453936  2611 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.454747  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:52.455914  1537 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:52.456192  1572 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:52.457126  2611 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.459682  1612 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:52.463894  2586 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.465101  1612 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:01:52.452267  2603 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:52.471540  2586 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:52.471990  2586 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:52.472404  2586 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:52.472818  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:52.474669  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:52.475960  2611 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:01:52.478792  1572 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:01:52.479611  1570 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:52.479346  2611 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.478268  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:52.482875  1612 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:52.484712  2611 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:52.476598  2609 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:01:52.486876  2609 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.487771  1611 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:52.491168  2609 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:52.493995  2583 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.495138  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:52.496769  2603 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:52.496954  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:52.497587  2603 raft_consensus.cc:697] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 2 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:01:52.498997  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:52.500002  1612 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:52.500545  1572 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:52.501245  2583 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.501348  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:52.504175  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:52.504761  2583 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:52.499194  2603 consensus_queue.cc:237] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.507700  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:52.521934  2347 raft_consensus.cc:1217] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:52.522409  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:52.516799  2586 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.528657  1718 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:52.532582  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:01:52.528671  2586 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:52.535172  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:52.536195  1646 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:52.536823  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:52.536374  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:52.534554  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:52.540896  2347 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:52.553120  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:52.553246  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:01:52.553473  2586 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:52.558971  2622 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:52.562278  1612 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:52.563638  2622 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.566700  2611 consensus_queue.cc:1059] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:52.567749  2611 raft_consensus.cc:3055] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:52.568071  2611 raft_consensus.cc:740] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:52.568877  2611 consensus_queue.cc:260] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.570748  2611 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:52.574064  1470 catalog_manager.cc:5697] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 1 to 2, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 2 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:52.597517  2597 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:01:52.598764  2597 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:52.601681  2597 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:52.602684  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:52.610884  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:01:52.614862  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:52.621068  1685 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:52.621981  2595 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.623097  2597 consensus_queue.cc:1059] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:01:52.625679  2507 raft_consensus.cc:3055] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:52.626098  2507 raft_consensus.cc:740] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:01:52.627005  2507 consensus_queue.cc:260] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:52.631759  2507 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:52.643893  2603 raft_consensus.cc:2955] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:52.646098  1685 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:52.649060  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:52.659273  1685 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:52.662478  1719 raft_consensus.cc:2955] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:52.664605  2627 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.647481  2638 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.664489  1572 raft_consensus.cc:2955] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:52.666893  1685 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:52.671129  2597 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.672554  1685 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:52.673358  2597 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.674899  1685 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:52.677558  1685 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:52.680833  1685 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:52.681959  2597 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.681959  2638 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.679639  2507 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.707520  2348 raft_consensus.cc:2955] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:52.713918  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:01:52.726608  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:52.732888  2611 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.736963  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:52.738235  2601 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.753165  1537 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:52.754722  2611 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.754802  1468 catalog_manager.cc:5697] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:52.756570  1537 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:52.758741  2601 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:52.762971  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:01:52.765201  2507 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:52.817031  1641 consensus_queue.cc:237] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:52.831385  2348 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:52.832018  1715 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:52.835966  2586 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:52.844291  2603 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:52.860527  2579 raft_consensus.cc:2955] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:52.884256  2347 raft_consensus.cc:2955] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:52.902017  1718 raft_consensus.cc:2955] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:52.903158  1457 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 45c609109d4a4c40a3e74ff0917c7958 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:52.931902  2586 raft_consensus.cc:1064] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:01:52.934646  2586 consensus_queue.cc:237] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:53.024783  1718 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:53.033260  1470 catalog_manager.cc:5697] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index 3 to 4, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:53.034204  2619 raft_consensus.cc:1064] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:01:53.036437  2579 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:53.054775  2619 consensus_queue.cc:237] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:53.074834  1572 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:53.078912  2348 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:53.082948  2603 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:53.087862  1722 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:53.097093  2579 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:53.114022  1572 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:53.140110  1558 tablet_service.cc:1558] Processing DeleteTablet for tablet 45c609109d4a4c40a3e74ff0917c7958 with delete_type TABLET_DATA_TOMBSTONED (TS 066ee0109aba4465ba17fa65e14ef504 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40642
I20260709 13:01:53.122339  2586 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:01:53.149931  2522 raft_consensus.cc:1064] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:01:53.154883  2522 consensus_queue.cc:237] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:53.160903  2622 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:53.210251  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 82468.8us
I20260709 13:01:53.325232  2649 tablet_replica.cc:333] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:01:53.344590  2649 raft_consensus.cc:2243] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:53.346287  2649 raft_consensus.cc:2272] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:53.333613  2647 raft_consensus.cc:2955] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:01:53.409682  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 91713.5us
I20260709 13:01:53.419114  2649 ts_tablet_manager.cc:1916] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:53.420789  2347 raft_consensus.cc:2955] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:53.439708  1642 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:53.444243  2348 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:53.446023  2349 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:53.450373  1722 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:53.490181  1566 raft_consensus.cc:2955] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:53.497706  2653 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:53.487565  1468 catalog_manager.cc:5697] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:53.522440  1722 raft_consensus.cc:2955] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:53.530123  2619 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:53.529239  2609 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:01:53.543936  2619 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:53.549508  2649 ts_tablet_manager.cc:1929] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:53.550424  2649 log.cc:1199] T 45c609109d4a4c40a3e74ff0917c7958 P 066ee0109aba4465ba17fa65e14ef504: Deleting WAL directory at /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-0-root/wals/45c609109d4a4c40a3e74ff0917c7958
I20260709 13:01:53.562711  1641 consensus_queue.cc:237] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:53.574710  1714 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:53.580861  1646 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:53.604148  1455 catalog_manager.cc:5028] TS 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): tablet 45c609109d4a4c40a3e74ff0917c7958 (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
W20260709 13:01:53.713253  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 55313.6us
I20260709 13:01:53.704876  2659 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:01:53.789999  1570 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:53.802570  2579 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
W20260709 13:01:53.864912  2670 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 13:01:53.881124  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 221435us
I20260709 13:01:53.911670  2349 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:53.930632  1714 raft_consensus.cc:2955] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:53.939498  2592 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:53.948864  2675 raft_consensus.cc:2955] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:53.956171  1566 raft_consensus.cc:2955] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:53.990595  1457 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0764c6a89eca47f39e28feb56e2123c7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:53.908269  2653 raft_consensus.cc:2955] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:53.999742  1646 raft_consensus.cc:2955] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:54.037493  2676 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:54.047652  1468 catalog_manager.cc:5697] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:54.129937  2592 raft_consensus.cc:2955] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:01:54.155045  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 94553.6us
I20260709 13:01:54.150681  1714 raft_consensus.cc:2955] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:01:54.228245  2679 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.071s	user 0.002s	sys 0.000s
I20260709 13:01:54.231590  2619 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.248085  2619 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:54.258862  2345 raft_consensus.cc:2955] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:54.258932  2619 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:54.265209  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:01:54.271441  1715 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:01:54.273753  1537 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:01:54.275296  2654 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:54.275837  2654 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.271598  2349 raft_consensus.cc:2955] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:54.240365  2347 raft_consensus.cc:2955] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:54.277678  2654 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:54.280790  1471 catalog_manager.cc:5697] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:54.324827  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:01:54.326007  1648 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:01:54.334208  1645 raft_consensus.cc:2955] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:54.359114  2654 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:54.388350  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:01:54.387346  2654 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:01:54.389289  1715 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:01:54.392752  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:01:54.393923  1642 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:54.418543  1642 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:01:54.423553  1538 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:01:54.432322  1470 catalog_manager.cc:5697] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:54.433629  2658 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:54.443886  1708 tablet_service.cc:1558] Processing DeleteTablet for tablet 0764c6a89eca47f39e28feb56e2123c7 with delete_type TABLET_DATA_TOMBSTONED (TS 130bc5ff39b847b1b3204626f284f838 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42266
I20260709 13:01:54.445096  2658 raft_consensus.cc:697] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:01:54.446646  2658 consensus_queue.cc:237] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:01:54.505023  2687 tablet_replica.cc:333] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:01:54.515620  1715 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:01:54.535847  1572 consensus_queue.cc:237] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:54.553714  2687 raft_consensus.cc:2243] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:54.554476  2687 raft_consensus.cc:2272] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:54.558566  2349 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:54.563167  2687 ts_tablet_manager.cc:1916] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:54.565428  1714 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:54.567606  2630 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:54.571491  2658 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:54.606319  2657 raft_consensus.cc:2955] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:54.613121  1715 raft_consensus.cc:2955] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:54.615067  1468 catalog_manager.cc:5697] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:54.620684  2349 raft_consensus.cc:2955] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:54.648941  1455 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0c6f11a07e734ec095c6185363fd6b52 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:54.655187  1572 consensus_queue.cc:237] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:54.665838  1720 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:54.665431  2345 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:54.681346  1469 catalog_manager.cc:5697] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index 3 to 4, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:54.682381  2592 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:54.691649  2687 ts_tablet_manager.cc:1929] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:54.692744  2687 log.cc:1199] T 0764c6a89eca47f39e28feb56e2123c7 P 130bc5ff39b847b1b3204626f284f838: Deleting WAL directory at /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-2-root/wals/0764c6a89eca47f39e28feb56e2123c7
I20260709 13:01:54.696902  1456 catalog_manager.cc:5028] TS 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): tablet 0764c6a89eca47f39e28feb56e2123c7 (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
I20260709 13:01:54.699497  2592 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:54.738006  2654 raft_consensus.cc:2955] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:54.745024  1629 tablet_service.cc:1558] Processing DeleteTablet for tablet 0c6f11a07e734ec095c6185363fd6b52 with delete_type TABLET_DATA_TOMBSTONED (TS f5f4b2707a9647e5bb99e065068dea2b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39910
I20260709 13:01:54.767483  2690 tablet_replica.cc:333] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:01:54.775615  1720 raft_consensus.cc:2955] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:54.769754  2345 raft_consensus.cc:2955] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:54.791388  2690 raft_consensus.cc:2243] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:54.794248  2690 raft_consensus.cc:2272] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:54.818147  2690 ts_tablet_manager.cc:1916] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:54.820145  1455 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c6485a5991c348019b9779be81656a45 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:54.851634  1469 catalog_manager.cc:5697] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: config changed from index 3 to 4, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:54.904772  1629 tablet_service.cc:1558] Processing DeleteTablet for tablet c6485a5991c348019b9779be81656a45 with delete_type TABLET_DATA_TOMBSTONED (TS f5f4b2707a9647e5bb99e065068dea2b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39910
I20260709 13:01:54.935117  2690 ts_tablet_manager.cc:1929] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:54.935827  2690 log.cc:1199] T 0c6f11a07e734ec095c6185363fd6b52 P f5f4b2707a9647e5bb99e065068dea2b: Deleting WAL directory at /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-1-root/wals/0c6f11a07e734ec095c6185363fd6b52
I20260709 13:01:54.939731  2690 tablet_replica.cc:333] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:01:54.941098  1457 catalog_manager.cc:5028] TS f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): tablet 0c6f11a07e734ec095c6185363fd6b52 (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
I20260709 13:01:54.943866  2690 raft_consensus.cc:2243] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:54.944880  2690 raft_consensus.cc:2272] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:54.962158  1714 raft_consensus.cc:1275] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:54.965080  2592 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:54.979923  2690 ts_tablet_manager.cc:1916] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:54.993465  1641 raft_consensus.cc:1275] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:55.003823  2592 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:55.042146  2690 ts_tablet_manager.cc:1929] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:55.043998  2690 log.cc:1199] T c6485a5991c348019b9779be81656a45 P f5f4b2707a9647e5bb99e065068dea2b: Deleting WAL directory at /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-1-root/wals/c6485a5991c348019b9779be81656a45
I20260709 13:01:55.048957  1457 catalog_manager.cc:5028] TS f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): tablet c6485a5991c348019b9779be81656a45 (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
W20260709 13:01:55.207849  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 73131.5us
W20260709 13:01:56.168795  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 81823.9us
W20260709 13:01:56.179211  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 111687us
W20260709 13:01:56.345875  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 144565us
W20260709 13:01:56.428046  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 63113.5us
I20260709 13:01:56.480876  1572 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
W20260709 13:01:56.561357  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 54365.4us
I20260709 13:01:56.620594  1715 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:56.625919  2611 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:56.628413  1537 consensus_peers.cc:597] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 08cf0535992b4db395d567393667593b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:56.704157  2611 raft_consensus.cc:2955] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:56.708087  1647 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:56.715968  1717 raft_consensus.cc:2955] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:56.724699  2701 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:56.737912  1715 consensus_queue.cc:237] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:56.745273  1470 catalog_manager.cc:5697] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
W20260709 13:01:56.802862  1686 consensus_peers.cc:597] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7ae86dda8d6f4caf8f08a1607327d9f9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:56.808375  1641 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:56.811242  2702 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:56.823915  1572 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:56.831812  2702 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:01:56.819680  1647 raft_consensus.cc:2955] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:56.904510  2714 raft_consensus.cc:2955] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:56.908749  1647 raft_consensus.cc:2955] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:56.924358  1572 raft_consensus.cc:2955] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:57.022240  1715 consensus_queue.cc:237] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:57.035184  1471 catalog_manager.cc:5697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:57.055966  1686 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 380ceca851ad4a84b89cc2e55313d10b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:57.057297  1647 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:57.060777  1572 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:57.063119  2714 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:57.074489  2725 ts_tablet_manager.cc:933] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:01:57.076578  2714 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:57.103892  2714 raft_consensus.cc:2955] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:57.114744  1647 raft_consensus.cc:2955] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:57.143754  2725 tablet_copy_client.cc:323] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:39999
I20260709 13:01:57.193509  1583 tablet_copy_service.cc:140] P 066ee0109aba4465ba17fa65e14ef504: Received BeginTabletCopySession request for tablet 08cf0535992b4db395d567393667593b from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:46982)
I20260709 13:01:57.194628  1583 tablet_copy_service.cc:161] P 066ee0109aba4465ba17fa65e14ef504: Beginning new tablet copy session on tablet 08cf0535992b4db395d567393667593b from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:46982: session id = cf2f71592be8465fb18e36e15d0b2f89-08cf0535992b4db395d567393667593b
I20260709 13:01:57.214375  1583 tablet_copy_source_session.cc:215] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:57.221692  1469 catalog_manager.cc:5697] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:57.224566  1642 consensus_queue.cc:237] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:57.237303  2725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08cf0535992b4db395d567393667593b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:57.243588  1572 raft_consensus.cc:2955] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:57.259220  2725 tablet_copy_client.cc:806] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:57.260419  2725 tablet_copy_client.cc:670] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:57.267939  2725 tablet_copy_client.cc:538] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:01:57.273092  1612 consensus_peers.cc:597] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f0ee72a7f576482aabe40ef837964708. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:57.286912  2725 tablet_bootstrap.cc:492] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:01:57.312809  1572 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:57.327081  1720 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:57.327827  2725 tablet_bootstrap.cc:492] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:57.330132  2731 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:57.344363  2725 tablet_bootstrap.cc:492] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:01:57.369491  2725 ts_tablet_manager.cc:1403] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 0.083s	user 0.028s	sys 0.007s
I20260709 13:01:57.410504  2732 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:01:57.414134  2734 ts_tablet_manager.cc:933] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:01:57.417955  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 86477.9us
I20260709 13:01:57.423652  2734 tablet_copy_client.cc:323] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:34553
I20260709 13:01:57.426133  1732 tablet_copy_service.cc:140] P 130bc5ff39b847b1b3204626f284f838: Received BeginTabletCopySession request for tablet 7ae86dda8d6f4caf8f08a1607327d9f9 from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:34078)
I20260709 13:01:57.426800  1732 tablet_copy_service.cc:161] P 130bc5ff39b847b1b3204626f284f838: Beginning new tablet copy session on tablet 7ae86dda8d6f4caf8f08a1607327d9f9 from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:34078: session id = cf2f71592be8465fb18e36e15d0b2f89-7ae86dda8d6f4caf8f08a1607327d9f9
I20260709 13:01:57.412158  2725 raft_consensus.cc:359] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:57.444550  2725 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:01:57.445555  2725 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:57.472532  1732 tablet_copy_source_session.cc:215] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:57.471642  1720 raft_consensus.cc:2955] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:57.502977  2734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ae86dda8d6f4caf8f08a1607327d9f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:57.510152  2725 ts_tablet_manager.cc:1434] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.140s	user 0.011s	sys 0.006s
I20260709 13:01:57.499174  1570 raft_consensus.cc:2955] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:57.509438  2586 raft_consensus.cc:2955] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:57.530949  1583 tablet_copy_service.cc:342] P 066ee0109aba4465ba17fa65e14ef504: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-08cf0535992b4db395d567393667593b received from {username='slave'} at 127.0.0.1:46982
I20260709 13:01:57.536276  1583 tablet_copy_service.cc:434] P 066ee0109aba4465ba17fa65e14ef504: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-08cf0535992b4db395d567393667593b on tablet 08cf0535992b4db395d567393667593b with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:57.549827  1469 catalog_manager.cc:5697] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:01:57.551656  2725 ts_tablet_manager.cc:933] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:01:57.558068  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 60587.2us
I20260709 13:01:57.592581  2349 raft_consensus.cc:1217] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:57.632629  2725 tablet_copy_client.cc:323] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:34553
I20260709 13:01:57.634788  1732 tablet_copy_service.cc:140] P 130bc5ff39b847b1b3204626f284f838: Received BeginTabletCopySession request for tablet 380ceca851ad4a84b89cc2e55313d10b from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:34078)
I20260709 13:01:57.635461  1732 tablet_copy_service.cc:161] P 130bc5ff39b847b1b3204626f284f838: Beginning new tablet copy session on tablet 380ceca851ad4a84b89cc2e55313d10b from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:34078: session id = cf2f71592be8465fb18e36e15d0b2f89-380ceca851ad4a84b89cc2e55313d10b
I20260709 13:01:57.637629  2734 tablet_copy_client.cc:806] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:57.638486  2734 tablet_copy_client.cc:670] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:57.643873  2734 tablet_copy_client.cc:538] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:57.651139  1732 tablet_copy_source_session.cc:215] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:57.655475  2725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 380ceca851ad4a84b89cc2e55313d10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:57.656642  2734 tablet_bootstrap.cc:492] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:01:57.712992  2725 tablet_copy_client.cc:806] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:57.720011  2734 tablet_bootstrap.cc:492] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:57.725797  2734 tablet_bootstrap.cc:492] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:01:57.728698  2734 ts_tablet_manager.cc:1403] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 0.072s	user 0.026s	sys 0.014s
I20260709 13:01:57.746546  2743 ts_tablet_manager.cc:933] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:01:57.749387  2734 raft_consensus.cc:359] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:57.758242  2734 raft_consensus.cc:740] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:01:57.759610  2734 consensus_queue.cc:260] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:57.761950  2725 tablet_copy_client.cc:670] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:57.772822  2734 ts_tablet_manager.cc:1434] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.001s
I20260709 13:01:57.775831  2743 tablet_copy_client.cc:323] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:44293
I20260709 13:01:57.778667  1731 tablet_copy_service.cc:342] P 130bc5ff39b847b1b3204626f284f838: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-7ae86dda8d6f4caf8f08a1607327d9f9 received from {username='slave'} at 127.0.0.1:34078
I20260709 13:01:57.779295  1731 tablet_copy_service.cc:434] P 130bc5ff39b847b1b3204626f284f838: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-7ae86dda8d6f4caf8f08a1607327d9f9 on tablet 7ae86dda8d6f4caf8f08a1607327d9f9 with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:57.780900  1657 tablet_copy_service.cc:140] P f5f4b2707a9647e5bb99e065068dea2b: Received BeginTabletCopySession request for tablet f0ee72a7f576482aabe40ef837964708 from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:44424)
I20260709 13:01:57.781594  1657 tablet_copy_service.cc:161] P f5f4b2707a9647e5bb99e065068dea2b: Beginning new tablet copy session on tablet f0ee72a7f576482aabe40ef837964708 from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:44424: session id = cf2f71592be8465fb18e36e15d0b2f89-f0ee72a7f576482aabe40ef837964708
I20260709 13:01:57.782641  2725 tablet_copy_client.cc:538] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:57.807425  1657 tablet_copy_source_session.cc:215] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:57.810081  2725 tablet_bootstrap.cc:492] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:01:57.816550  2743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0ee72a7f576482aabe40ef837964708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:57.922258  2725 tablet_bootstrap.cc:492] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:57.923621  2725 tablet_bootstrap.cc:492] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:01:57.931237  2725 ts_tablet_manager.cc:1403] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 0.121s	user 0.034s	sys 0.001s
I20260709 13:01:57.940106  2725 raft_consensus.cc:359] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:57.941756  2725 raft_consensus.cc:740] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:01:57.942814  2725 consensus_queue.cc:260] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:57.948112  2725 ts_tablet_manager.cc:1434] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 13:01:57.957651  1732 tablet_copy_service.cc:342] P 130bc5ff39b847b1b3204626f284f838: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-380ceca851ad4a84b89cc2e55313d10b received from {username='slave'} at 127.0.0.1:34078
I20260709 13:01:57.958129  1732 tablet_copy_service.cc:434] P 130bc5ff39b847b1b3204626f284f838: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-380ceca851ad4a84b89cc2e55313d10b on tablet 380ceca851ad4a84b89cc2e55313d10b with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:58.017279  2345 raft_consensus.cc:1217] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:58.027222  2743 tablet_copy_client.cc:806] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:58.027931  2743 tablet_copy_client.cc:670] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:58.141837  2743 tablet_copy_client.cc:538] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:58.152341  2743 tablet_bootstrap.cc:492] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:01:58.178013  2345 raft_consensus.cc:1217] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:58.247313  2592 raft_consensus.cc:1064] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:01:58.258459  2592 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:58.304807  1642 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:58.309079  2710 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:58.317749  2743 tablet_bootstrap.cc:492] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:58.320598  2743 tablet_bootstrap.cc:492] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:01:58.321585  2743 ts_tablet_manager.cc:1403] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 0.169s	user 0.025s	sys 0.009s
I20260709 13:01:58.325129  2743 raft_consensus.cc:359] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:58.326404  2743 raft_consensus.cc:740] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:01:58.327721  2743 consensus_queue.cc:260] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:01:58.332808  2345 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:58.335476  2743 ts_tablet_manager.cc:1434] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.003s
I20260709 13:01:58.332803  1720 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:58.338418  1657 tablet_copy_service.cc:342] P f5f4b2707a9647e5bb99e065068dea2b: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-f0ee72a7f576482aabe40ef837964708 received from {username='slave'} at 127.0.0.1:44424
I20260709 13:01:58.339030  1657 tablet_copy_service.cc:434] P f5f4b2707a9647e5bb99e065068dea2b: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-f0ee72a7f576482aabe40ef837964708 on tablet f0ee72a7f576482aabe40ef837964708 with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:58.382959  2592 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:58.415809  2710 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:58.443970  2592 raft_consensus.cc:2955] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:58.485419  2345 raft_consensus.cc:2955] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:58.515197  1642 raft_consensus.cc:2955] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:58.517313  1469 catalog_manager.cc:5697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:58.560401  1720 raft_consensus.cc:2955] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:58.643683  2746 raft_consensus.cc:1064] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:01:58.647571  2746 consensus_queue.cc:237] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:58.656332  1566 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:58.736984  2345 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:58.742769  2592 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:58.756626  1641 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:58.756800  2349 raft_consensus.cc:1217] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:58.759590  2592 raft_consensus.cc:2955] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:58.763195  2349 raft_consensus.cc:2955] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:58.785348  2747 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:01:58.781594  2699 raft_consensus.cc:1064] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:01:58.808779  1641 raft_consensus.cc:2955] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:58.826988  1468 catalog_manager.cc:5697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:58.827520  2349 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:58.841553  1708 tablet_service.cc:1558] Processing DeleteTablet for tablet 08cf0535992b4db395d567393667593b with delete_type TABLET_DATA_TOMBSTONED (TS 130bc5ff39b847b1b3204626f284f838 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42266
I20260709 13:01:58.876817  1639 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:58.855373  2702 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
W20260709 13:01:58.922674  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 67912.4us
I20260709 13:01:58.924285  2699 consensus_queue.cc:237] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:58.939769  2765 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:01:58.944191  2760 tablet_replica.cc:333] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:01:58.946499  2760 raft_consensus.cc:2243] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.984241  2760 raft_consensus.cc:2272] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.956791  1455 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 08cf0535992b4db395d567393667593b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:59.004670  1572 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:59.043736  2760 ts_tablet_manager.cc:1916] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:59.063172  2762 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:59.064235  2347 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:59.082158  2759 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:59.094408  1572 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:59.097716  1639 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:59.115549  2760 ts_tablet_manager.cc:1929] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:59.116242  2760 log.cc:1199] T 08cf0535992b4db395d567393667593b P 130bc5ff39b847b1b3204626f284f838: Deleting WAL directory at /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-2-root/wals/08cf0535992b4db395d567393667593b
I20260709 13:01:59.124100  2746 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:59.125448  2699 raft_consensus.cc:2955] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.131253  2764 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:59.130285  2347 raft_consensus.cc:2955] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.134562  1456 catalog_manager.cc:5028] TS 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): tablet 08cf0535992b4db395d567393667593b (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
I20260709 13:01:59.186354  2764 raft_consensus.cc:2955] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.202116  1566 raft_consensus.cc:2955] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.232471  1471 catalog_manager.cc:5697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.312319  1641 raft_consensus.cc:2955] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.303812  2347 raft_consensus.cc:2955] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.353376  1572 raft_consensus.cc:2955] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.363546  1471 catalog_manager.cc:5697] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.379402  1641 raft_consensus.cc:2955] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.445007  1720 consensus_queue.cc:237] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:59.449581  1714 consensus_queue.cc:237] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:59.498273  1645 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:59.499111  2348 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:59.503031  2758 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:59.508726  2746 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:59.514397  2765 raft_consensus.cc:2955] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.518023  1645 raft_consensus.cc:2955] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.539463  1456 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7ae86dda8d6f4caf8f08a1607327d9f9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:59.540071  2348 raft_consensus.cc:2955] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.548424  1566 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:59.569008  2349 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:59.576480  2740 raft_consensus.cc:1064] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:01:59.579977  2740 consensus_queue.cc:237] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:01:59.590776  2746 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:59.593904   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:59.614140  2765 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:59.663509  2759 raft_consensus.cc:2955] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.669551  1572 raft_consensus.cc:2955] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.674661  1471 catalog_manager.cc:5697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index 3 to 4, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:59.686326  2349 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:59.691376  2740 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:59.702597  1715 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:59.705614  1469 catalog_manager.cc:5697] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: config changed from index 3 to 4, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.715848  1558 tablet_service.cc:1558] Processing DeleteTablet for tablet 7ae86dda8d6f4caf8f08a1607327d9f9 with delete_type TABLET_DATA_TOMBSTONED (TS 066ee0109aba4465ba17fa65e14ef504 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40642
I20260709 13:01:59.733578  2697 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:59.743327  1566 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:59.756611  1629 tablet_service.cc:1558] Processing DeleteTablet for tablet 380ceca851ad4a84b89cc2e55313d10b with delete_type TABLET_DATA_TOMBSTONED (TS f5f4b2707a9647e5bb99e065068dea2b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39910
I20260709 13:01:59.757735  2348 raft_consensus.cc:2955] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.760249  2777 tablet_replica.cc:333] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:01:59.783468  2777 raft_consensus.cc:2243] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.784279  2777 raft_consensus.cc:2272] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.762688  2779 tablet_replica.cc:333] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:01:59.765019  1456 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 380ceca851ad4a84b89cc2e55313d10b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:59.792564  2779 raft_consensus.cc:2243] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.792593  2776 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:59.798476  2777 ts_tablet_manager.cc:1916] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:59.800966  2779 raft_consensus.cc:2272] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.826334  2779 ts_tablet_manager.cc:1916] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:59.861269  2776 raft_consensus.cc:2955] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.873997  1715 raft_consensus.cc:2955] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.889731  1572 raft_consensus.cc:2955] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.918231  1566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08cf0535992b4db395d567393667593b"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:01:59.919099  1566 raft_consensus.cc:606] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Received request to transfer leadership to TS cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:59.940474  1566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:01:59.941375  1566 raft_consensus.cc:606] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Received request to transfer leadership to TS cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:59.950773  2348 raft_consensus.cc:2955] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:01:59.963670  1469 catalog_manager.cc:5697] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:59.973306  1573 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:01:59.974431  1573 raft_consensus.cc:606] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Received request to transfer leadership to TS 130bc5ff39b847b1b3204626f284f838
I20260709 13:01:59.974830  2779 ts_tablet_manager.cc:1929] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:59.975265  2779 log.cc:1199] T 380ceca851ad4a84b89cc2e55313d10b P f5f4b2707a9647e5bb99e065068dea2b: Deleting WAL directory at /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-1-root/wals/380ceca851ad4a84b89cc2e55313d10b
I20260709 13:01:59.977821  1457 catalog_manager.cc:5028] TS f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): tablet 380ceca851ad4a84b89cc2e55313d10b (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
I20260709 13:01:59.981436  1573 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6485a5991c348019b9779be81656a45"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:01:59.982664  1573 raft_consensus.cc:606] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Received request to transfer leadership to TS cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:01:59.984294  2777 ts_tablet_manager.cc:1929] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:59.993033  2777 log.cc:1199] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 066ee0109aba4465ba17fa65e14ef504: Deleting WAL directory at /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-0-root/wals/7ae86dda8d6f4caf8f08a1607327d9f9
I20260709 13:02:00.005478  1455 catalog_manager.cc:5028] TS 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): tablet 7ae86dda8d6f4caf8f08a1607327d9f9 (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
I20260709 13:02:00.007236  2774 raft_consensus.cc:993] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: : Instructing follower cf2f71592be8465fb18e36e15d0b2f89 to start an election
I20260709 13:02:00.007921  2663 raft_consensus.cc:1081] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Signalling peer cf2f71592be8465fb18e36e15d0b2f89 to start an election
I20260709 13:02:00.028383  2345 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6485a5991c348019b9779be81656a45"
dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
 from {username='slave'} at 127.0.0.1:55890
I20260709 13:02:00.029026  2345 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:00.029363  2345 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.036417  1645 consensus_queue.cc:237] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:00.040014  2345 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:00.046238  1573 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:00.047739  2776 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:00.050065  2348 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:00.058319  2769 raft_consensus.cc:2955] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:02:00.061101  1645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
mode: GRACEFUL
new_leader_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:49580
I20260709 13:02:00.062219  1645 raft_consensus.cc:606] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Received request to transfer leadership to TS 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:00.067655  2784 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:00.076213  1573 raft_consensus.cc:2955] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:02:00.082890  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:00.085080  1720 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.084447  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:00.093962  1566 raft_consensus.cc:3055] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:00.094321  1566 raft_consensus.cc:740] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:00.095286  1566 consensus_queue.cc:260] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:00.096504  1566 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.099031  1572 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:02:00.099891  1572 raft_consensus.cc:606] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Received request to transfer leadership to TS 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:00.101863   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:02:00.087216  2348 raft_consensus.cc:2955] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:02:00.103711   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:00.114535  2345 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:00.118471  1720 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:00.120368  2313 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:00.121412  2738 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:00.135027  2738 raft_consensus.cc:697] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:00.141387  1469 catalog_manager.cc:5697] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: config changed from index 3 to 4, VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:02:00.137272  2738 consensus_queue.cc:237] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:00.147588  1457 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f0ee72a7f576482aabe40ef837964708 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:00.109414  1566 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:00.175885  1708 tablet_service.cc:1558] Processing DeleteTablet for tablet f0ee72a7f576482aabe40ef837964708 with delete_type TABLET_DATA_TOMBSTONED (TS 130bc5ff39b847b1b3204626f284f838 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42266
I20260709 13:02:00.232422  2774 raft_consensus.cc:993] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: : Instructing follower cf2f71592be8465fb18e36e15d0b2f89 to start an election
I20260709 13:02:00.239301  2788 tablet_replica.cc:333] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:02:00.241124  2747 raft_consensus.cc:993] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504: : Instructing follower 130bc5ff39b847b1b3204626f284f838 to start an election
I20260709 13:02:00.241823  2747 raft_consensus.cc:1081] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Signalling peer 130bc5ff39b847b1b3204626f284f838 to start an election
I20260709 13:02:00.244016  2592 raft_consensus.cc:993] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: : Instructing follower cf2f71592be8465fb18e36e15d0b2f89 to start an election
I20260709 13:02:00.244518  2592 raft_consensus.cc:1081] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Signalling peer cf2f71592be8465fb18e36e15d0b2f89 to start an election
I20260709 13:02:00.246196  2788 raft_consensus.cc:2243] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:00.247072  2788 raft_consensus.cc:2272] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:00.246222  2345 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52"
dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
 from {username='slave'} at 127.0.0.1:55890
I20260709 13:02:00.247809  2345 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:00.248165  2774 raft_consensus.cc:1081] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Signalling peer cf2f71592be8465fb18e36e15d0b2f89 to start an election
I20260709 13:02:00.248231  2345 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.250653  2348 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08cf0535992b4db395d567393667593b"
dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
 from {username='slave'} at 127.0.0.1:55890
I20260709 13:02:00.251128  1714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:42276
I20260709 13:02:00.251511  2348 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:00.252050  2788 ts_tablet_manager.cc:1916] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:00.252200  2348 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.252839  1471 catalog_manager.cc:5697] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 1 to 2, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:00.251748  1714 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:00.264211  1714 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.265410  2345 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:00.276424  2345 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:00.277325  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:00.281365  1566 raft_consensus.cc:3055] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:00.281739  1566 raft_consensus.cc:740] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:00.282553  1566 consensus_queue.cc:260] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:00.283830  1566 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.287999  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:00.288655  1720 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.277215  2348 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:00.314546  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:00.314646  2348 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:00.315207  1572 raft_consensus.cc:3055] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:00.315614  1572 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:00.297307  1566 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:00.316502  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:00.316398  1572 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:00.317221  1639 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.317442  2312 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:00.317721  1572 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.320937  2738 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:00.309835  1720 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:00.321573  2738 raft_consensus.cc:697] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:00.322618  2738 consensus_queue.cc:237] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:00.303294  1714 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:00.328359  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:00.329037  1639 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:00.329128  1645 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.344872  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:00.347983  1572 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:00.353104  2312 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:00.354387  2786 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:00.348590  1566 raft_consensus.cc:3055] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:00.355862  1566 raft_consensus.cc:740] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:00.361871  1714 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:00.362823  1566 consensus_queue.cc:260] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:00.365602  1566 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.360877  1645 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:00.371070  1687 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:00.383456  2788 ts_tablet_manager.cc:1929] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:00.383949  2788 log.cc:1199] T f0ee72a7f576482aabe40ef837964708 P 130bc5ff39b847b1b3204626f284f838: Deleting WAL directory at /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-2-root/wals/f0ee72a7f576482aabe40ef837964708
I20260709 13:02:00.398952  2759 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:00.400763  2759 raft_consensus.cc:697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:00.401871  2759 consensus_queue.cc:237] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:00.405169  1456 catalog_manager.cc:5028] TS 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): tablet f0ee72a7f576482aabe40ef837964708 (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
I20260709 13:02:00.396467  1471 catalog_manager.cc:5697] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 1 to 2, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:00.460925  2747 raft_consensus.cc:993] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: : Instructing follower 130bc5ff39b847b1b3204626f284f838 to start an election
I20260709 13:02:00.461994  2747 raft_consensus.cc:1081] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Signalling peer 130bc5ff39b847b1b3204626f284f838 to start an election
I20260709 13:02:00.464941  2786 raft_consensus.cc:697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:00.466372  2786 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:00.475124  1714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:42276
I20260709 13:02:00.478600  1714 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:00.478991  1566 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:00.480796  1714 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.505807  2784 raft_consensus.cc:993] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b: : Instructing follower 130bc5ff39b847b1b3204626f284f838 to start an election
I20260709 13:02:00.506667  2753 raft_consensus.cc:1081] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Signalling peer 130bc5ff39b847b1b3204626f284f838 to start an election
I20260709 13:02:00.519289  1720 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:42278
I20260709 13:02:00.520028  1720 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:00.520449  1720 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.579236  1471 catalog_manager.cc:5697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 1 to 2, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:00.581744  1469 catalog_manager.cc:5697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 1 to 2, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:00.623458  1566 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:00.632369  2786 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:00.642581  1720 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:00.649531  1720 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:00.658321  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:00.659129  1645 raft_consensus.cc:3055] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:00.659423  1645 raft_consensus.cc:740] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:00.663439  1714 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:00.677557  1714 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:00.683336  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:00.684211  1645 consensus_queue.cc:260] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:00.696491  1641 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.704288  1645 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.706786  1641 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
W20260709 13:02:00.739198  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 47160.8us
I20260709 13:02:00.739964  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:00.740774  1572 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:00.742033  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 56658.4us
I20260709 13:02:00.743425  1687 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:00.744472  2746 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:00.745025  2746 raft_consensus.cc:697] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:00.746052  2746 consensus_queue.cc:237] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:00.753191  1714 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:00.762183  2786 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:00.764230  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:00.764943  1570 raft_consensus.cc:3055] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:00.765264  1570 raft_consensus.cc:740] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:00.766299  1570 consensus_queue.cc:260] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:00.767148  1572 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:00.767664  1645 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:00.767990  1570 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:00.789055  1685 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:00.790228  2765 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:00.791000  2765 raft_consensus.cc:697] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:00.791957  2765 consensus_queue.cc:237] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:00.840799  1570 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:00.872438  1471 catalog_manager.cc:5697] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 1 to 2, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:00.879565  1566 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:00.881989  1566 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:00.882274  2765 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:00.883931  2786 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:00.897336  1565 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:00.897550  1720 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:00.899362  2786 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:00.900960  2786 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:00.923343  1645 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:00.927227  2764 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:00.943915  1639 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:00.946744  2790 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:01.089751  1471 catalog_manager.cc:5697] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 1 to 2, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:01.107245   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:01.179031  1566 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:01.182669  2759 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:01.222892  1639 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:01.224790  2765 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:01.230379   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:01.301190  1639 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:01.303841  2758 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:01.394272  1565 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:01.402709  1570 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:02:01.403343  1570 raft_consensus.cc:606] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Received request to transfer leadership to TS cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:01.417958  2764 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:02:01.477177  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 57749us
I20260709 13:02:01.531023  2774 raft_consensus.cc:993] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: : Instructing follower cf2f71592be8465fb18e36e15d0b2f89 to start an election
I20260709 13:02:01.531816  2797 raft_consensus.cc:1081] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Signalling peer cf2f71592be8465fb18e36e15d0b2f89 to start an election
I20260709 13:02:01.532940  1566 consensus_queue.cc:237] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:01.534811  2348 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4"
dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
 from {username='slave'} at 127.0.0.1:55890
I20260709 13:02:01.535431  2348 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:01.535801  2348 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:01.545403  1717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:02:01.546391  1717 raft_consensus.cc:606] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Received request to transfer leadership to TS cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:01.551452  1717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:02:01.552230  1717 raft_consensus.cc:606] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:01.555259  1720 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:01.557847  1571 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:02:01.558559  1571 raft_consensus.cc:606] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Received request to transfer leadership to TS 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:01.566553  2348 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:01.568503  1537 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eca30b43c26f420c965ee25f475581a0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:01.569046  2797 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:01.570499  2348 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:01.570889  1646 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:01.577271  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:01.578024  1645 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:01.580289  1639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
mode: GRACEFUL
new_leader_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:49580
I20260709 13:02:01.582819  1639 raft_consensus.cc:606] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Received request to transfer leadership to TS 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:01.575487  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:01.583590  1565 raft_consensus.cc:3055] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:01.583335  2774 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:01.583971  1565 raft_consensus.cc:740] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:01.585173  1565 consensus_queue.cc:260] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:01.587809  1565 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:01.596340  1645 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:01.601747  2314 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:01.602070  2758 raft_consensus.cc:993] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: : Instructing follower 066ee0109aba4465ba17fa65e14ef504 to start an election
I20260709 13:02:01.607503  2790 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:01.608222  2790 raft_consensus.cc:697] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:01.611732  2790 consensus_queue.cc:237] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:01.613421  2765 raft_consensus.cc:993] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: : Instructing follower cf2f71592be8465fb18e36e15d0b2f89 to start an election
I20260709 13:02:01.614024  2765 raft_consensus.cc:1081] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Signalling peer cf2f71592be8465fb18e36e15d0b2f89 to start an election
I20260709 13:02:01.616178  2348 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9"
dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
 from {username='slave'} at 127.0.0.1:55888
I20260709 13:02:01.617161  2348 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:01.617738  1571 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:02:01.618016  2348 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:01.618531  1571 raft_consensus.cc:606] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Received request to transfer leadership to TS cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:01.623440   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:02:01.609540  2746 raft_consensus.cc:1081] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Signalling peer 066ee0109aba4465ba17fa65e14ef504 to start an election
I20260709 13:02:01.626307   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:01.629192  1571 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:40670
I20260709 13:02:01.631848  1571 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:01.630999  2774 raft_consensus.cc:2955] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:02:01.635201  1720 raft_consensus.cc:2955] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:02:01.637473  1565 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:01.662457  1571 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:01.702617  1469 catalog_manager.cc:5697] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:02:01.716933  1470 catalog_manager.cc:5697] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 1 to 2, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:01.724468  1639 raft_consensus.cc:2955] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:02:01.737265  2348 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:01.737926  1571 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:01.750126  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:01.750871  1720 raft_consensus.cc:3055] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:01.751217  1720 raft_consensus.cc:740] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:01.752105  1720 consensus_queue.cc:260] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:01.752588  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:01.724076  1646 consensus_queue.cc:237] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:01.754057  1645 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:01.756172  1720 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:01.761371  1571 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:01.763588  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:01.764314  1714 raft_consensus.cc:3055] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:01.764642  1714 raft_consensus.cc:740] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:01.765434  1714 consensus_queue.cc:260] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:01.767190  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:01.768385  1648 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:01.767323  1714 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:01.783982  1648 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:01.784216  2348 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:01.800851  1565 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:01.787145  1720 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:01.803139  2313 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:01.827011  2790 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:01.827632  2790 raft_consensus.cc:697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:01.806509  1538 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:01.828961  2797 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:01.828778  2790 consensus_queue.cc:237] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:01.829515  2797 raft_consensus.cc:697] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:01.831038  2797 consensus_queue.cc:237] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:01.815225  1720 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:02:01.840579  1612 consensus_peers.cc:597] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 58b222526a1a4b52bdcda825e15c46f9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:01.840888  2784 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:01.844376  2800 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:01.801549  1645 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:01.898855  1714 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:01.908763  2753 raft_consensus.cc:993] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: : Instructing follower 130bc5ff39b847b1b3204626f284f838 to start an election
I20260709 13:02:01.909523  2753 raft_consensus.cc:1081] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Signalling peer 130bc5ff39b847b1b3204626f284f838 to start an election
I20260709 13:02:01.926937  2769 raft_consensus.cc:2955] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:02:01.930269  1720 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:42278
I20260709 13:02:01.930984  1720 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:01.931346  1720 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:01.944067  1565 raft_consensus.cc:2955] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:02:01.952179  1720 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:01.974052  2828 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.076s	user 0.000s	sys 0.004s
I20260709 13:02:01.978331  1720 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:01.980077  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:01.981209  1566 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:01.994681  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:01.995818  1646 raft_consensus.cc:3055] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:01.996177  1646 raft_consensus.cc:740] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:02.006318  1714 raft_consensus.cc:2955] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:02:02.022557  2592 raft_consensus.cc:993] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: : Instructing follower cf2f71592be8465fb18e36e15d0b2f89 to start an election
I20260709 13:02:02.023182  2592 raft_consensus.cc:1081] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Signalling peer cf2f71592be8465fb18e36e15d0b2f89 to start an election
I20260709 13:02:02.025094  2349 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca"
dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
 from {username='slave'} at 127.0.0.1:55890
I20260709 13:02:02.026027  2349 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:02.026417  2349 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:02.028430  1471 catalog_manager.cc:5697] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 1 to 2, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:01.997494  1646 consensus_queue.cc:260] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:02.062705  1566 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:02.074150  1646 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:02.062953  1470 catalog_manager.cc:5697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 1 to 2, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:02.074211  2833 ts_tablet_manager.cc:933] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
W20260709 13:02:02.074666  2800 consensus_queue.cc:1175] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "066ee0109aba4465ba17fa65e14ef504" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 13:02:02.103777  2349 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:02.109273  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:02.110028  1565 raft_consensus.cc:3055] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:02.110376  1565 raft_consensus.cc:740] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:02.120877  1565 consensus_queue.cc:260] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:02.122295  1565 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:02.130892  2349 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:02.133896  1565 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:02.143553  1685 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:02.145838  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 114174us
W20260709 13:02:02.157670  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 74923us
I20260709 13:02:02.158890  1566 consensus_queue.cc:237] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
W20260709 13:02:02.164976  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 80255.2us
I20260709 13:02:02.211614  2794 raft_consensus.cc:993] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: : Instructing follower 130bc5ff39b847b1b3204626f284f838 to start an election
I20260709 13:02:02.212245  2794 raft_consensus.cc:1081] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Signalling peer 130bc5ff39b847b1b3204626f284f838 to start an election
I20260709 13:02:02.152058  2312 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:02:02.217512  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 71327.2us
I20260709 13:02:02.221729  1646 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:02.173427  2836 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:02.225263  1714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:42276
I20260709 13:02:02.218480  2790 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:02.227452  2790 raft_consensus.cc:697] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:02.228370  2790 consensus_queue.cc:237] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:02.226502  2836 raft_consensus.cc:697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:02.228614  2833 tablet_copy_client.cc:323] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:39999
I20260709 13:02:02.233512  2836 consensus_queue.cc:237] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:02.237944  1583 tablet_copy_service.cc:140] P 066ee0109aba4465ba17fa65e14ef504: Received BeginTabletCopySession request for tablet eca30b43c26f420c965ee25f475581a0 from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:46982)
I20260709 13:02:02.238687  1583 tablet_copy_service.cc:161] P 066ee0109aba4465ba17fa65e14ef504: Beginning new tablet copy session on tablet eca30b43c26f420c965ee25f475581a0 from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:46982: session id = cf2f71592be8465fb18e36e15d0b2f89-eca30b43c26f420c965ee25f475581a0
I20260709 13:02:02.242103  1468 catalog_manager.cc:5697] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:02:02.247823  1714 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:02.248466  1714 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:02.253295  1565 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:02.274374  1583 tablet_copy_source_session.cc:215] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:02.286015  1714 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:02.287660  2790 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:02:02.304419  1714 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:02.306253  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:02.306874  1648 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:02.293076  1470 catalog_manager.cc:5697] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 1 to 2, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:02.321880  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:02.322916  1645 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:02.323338  1639 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:02.335975  1639 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:02:02.346378  2844 log.cc:927] Time spent T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.058s	user 0.000s	sys 0.002s
I20260709 13:02:02.349092  1645 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
W20260709 13:02:02.358336  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 101281us
I20260709 13:02:02.363579  1648 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
W20260709 13:02:02.368978  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 111473us
I20260709 13:02:02.378952  1717 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:02:02.406384  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 50297.2us
W20260709 13:02:02.408447  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 90976.1us
I20260709 13:02:02.413811  1687 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:02.419052  2833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eca30b43c26f420c965ee25f475581a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:02.424693  2765 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:02.440874  2765 raft_consensus.cc:697] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:02.442106  2765 consensus_queue.cc:237] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:02.467532  2832 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:02:02.506695  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:02.507884  1566 raft_consensus.cc:3055] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:02.508306  1566 raft_consensus.cc:740] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:02.509270  1566 consensus_queue.cc:260] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:02.518322  1566 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:02.526257  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 154559us
I20260709 13:02:02.538239  2832 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:02:02.541936  1537 consensus_peers.cc:597] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 59d236b930214b1f91fb55ece6fecd68. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:02.573105  1566 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
W20260709 13:02:02.644486  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 63882.4us
I20260709 13:02:02.645390   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:02.656487  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 113705us
I20260709 13:02:02.777925  2846 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.185s
W20260709 13:02:02.866990  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 469403us
I20260709 13:02:02.892207  1717 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:02:02.899613  2855 log.cc:927] Time spent T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.407s	user 0.003s	sys 0.004s
I20260709 13:02:02.920361  1641 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:02.982627  2790 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
W20260709 13:02:03.043111  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 477710us
I20260709 13:02:03.102394  1648 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:03.102491  1646 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:02:03.104576  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.119678  1646 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:03.041463  2832 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
W20260709 13:02:03.128314  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 465771us
I20260709 13:02:03.147450  2758 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:03.176097  2867 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:03.199553  2746 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:03.230914  1643 raft_consensus.cc:1275] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:02:03.232656  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 72890.8us
I20260709 13:02:03.242002  2867 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:03.244958  2867 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:03.222285  2758 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:03.261749  1639 raft_consensus.cc:1275] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:02:03.285605  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 67182.8us
I20260709 13:02:03.284379  2875 ts_tablet_manager.cc:933] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:03.235394  2746 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:03.303632  1571 raft_consensus.cc:1275] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:03.298918  2758 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:03.306120  2833 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.579s	user 0.016s	sys 0.001s
I20260709 13:02:03.307636  2833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:55890 (request call id 139) took 1367 ms. Trace:
I20260709 13:02:03.368451  2746 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:03.308259  2833 rpcz_store.cc:276] 0709 13:02:01.940682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.940829 (+   147us) service_pool.cc:224] Handling call
0709 13:02:02.113573 (+172744us) ts_tablet_manager.cc:934] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
0709 13:02:02.311412 (+197839us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:02:02.649165 (+337753us) tablet_metadata.cc:665] Metadata flushed
0709 13:02:02.656056 (+  6891us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:02:03.306782 (+650726us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:02:03.307133 (+   351us) tablet_replica.cc:206] Creating consensus instance
0709 13:02:03.307483 (+   350us) ts_tablet_manager.cc:989] Replica registered.
0709 13:02:03.307544 (+    61us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":5092,"tablet-copy.queue_time_us":42423,"thread_start_us":186860,"threads_started":2}
I20260709 13:02:03.369519  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:03.370067  2833 tablet_copy_client.cc:806] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:03.370680  2833 tablet_copy_client.cc:670] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:03.371418  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:03.376304  1721 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:02:03.384893  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 125289us
I20260709 13:02:03.400307  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:03.451640  2753 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:03.475063  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:03.491551  2849 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:03.497242  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:03.505789  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 129307us
I20260709 13:02:03.507642  2888 ts_tablet_manager.cc:933] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:02:03.469538  2753 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:03.502761  2875 tablet_copy_client.cc:323] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:44293
I20260709 13:02:03.468680  2794 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 13:02:03.506165  2846 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:03.589386  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.523713  2883 consensus_queue.cc:1048] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.113s
I20260709 13:02:03.494783  2849 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:03.644657  2796 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.648150  1468 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38650 (request call id 191) took 1104 ms. Trace:
I20260709 13:02:03.648762  1468 rpcz_store.cc:276] 0709 13:02:02.543319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.543787 (+   468us) service_pool.cc:224] Handling call
0709 13:02:03.647969 (+1104182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":321447,"reported_tablets":2}
I20260709 13:02:03.667474  2857 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:03.690636  2592 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:03.691948  2592 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:03.697359  2592 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:03.702591  1470 catalog_manager.cc:5697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 1 to 2, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:03.648953  2796 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:03.720247  2753 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:03.722440  2796 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:03.710922  2849 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:03.669106  2857 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:03.726406  2786 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:03.747982  2786 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:03.726598  2873 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:03.728083  2864 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:03.727895  1470 catalog_manager.cc:5697] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 1 to 2, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:03.764405  1570 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:03.738860  2871 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:03.778165  2864 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:03.802024  2871 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:03.762919  2873 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:03.811753  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:03.819449  2873 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:03.824841  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 405896us
W20260709 13:02:03.826098  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 318332us
W20260709 13:02:03.822713  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 319670us
I20260709 13:02:03.835103  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:03.822988  2786 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:03.876288  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:03.887750  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 51044.2us
I20260709 13:02:03.898020  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:03.897398  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:03.894708  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:03.853222  2872 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:03.846071  2883 consensus_queue.cc:1048] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:02:03.922986  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:03.938959  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:03.911860  2835 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:03.920112  2888 tablet_copy_client.cc:323] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:39999
I20260709 13:02:03.922330  2758 consensus_queue.cc:1048] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:02:03.893957  2867 raft_consensus.cc:2955] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
W20260709 13:02:03.976629  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 71595.5us
I20260709 13:02:03.947469  2857 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:03.935482  2872 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:03.992588  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:03.995994  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:03.997809  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 88622.8us
I20260709 13:02:03.965327  2865 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:04.020355  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:04.020541  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:03.958456  2851 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:02:04.042420  1687 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:04.032299  1572 raft_consensus.cc:1275] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:04.038149   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:03.997850  2883 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:03.999400  2841 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:04.077158  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:04.011325  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 65865.4us
I20260709 13:02:04.099074  2837 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:04.083353  2841 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:04.102046  2841 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:04.043592  2871 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:04.119174  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:04.042661  2865 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:03.970541  2835 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:04.173381  2797 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:04.182150  2797 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:04.120662  2856 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.162s
I20260709 13:02:04.185595  2838 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.154685  2800 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:04.188162  2864 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:04.185622  2872 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:04.202795  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:04.096333  2883 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:04.196300  2800 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:04.238747  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:04.187777  2838 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:04.257762  2835 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:04.233592  2883 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:04.238794  2797 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:04.278955  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:04.280066  1718 raft_consensus.cc:2955] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:02:04.281293  1720 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:02:04.285671  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:04.278956  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:04.309281  1687 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:04.320614  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 267123us
I20260709 13:02:04.324714  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:04.328754  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:04.330647  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 295910us
W20260709 13:02:04.332587  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 265061us
I20260709 13:02:04.294739  2838 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:04.348300  2909 log.cc:927] Time spent T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Append to log took a long time: real 0.081s	user 0.000s	sys 0.002s
I20260709 13:02:04.330515  2797 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.343088  2800 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:04.373972  2868 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:04.418824  2765 consensus_queue.cc:1048] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:02:04.438908  2908 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.382030  2797 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:04.442062  2908 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:04.538921  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:04.550920  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:04.558092  2746 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:04.558295  1583 tablet_copy_service.cc:140] P 066ee0109aba4465ba17fa65e14ef504: Received BeginTabletCopySession request for tablet 59d236b930214b1f91fb55ece6fecd68 from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:46982)
I20260709 13:02:04.558979  1583 tablet_copy_service.cc:161] P 066ee0109aba4465ba17fa65e14ef504: Beginning new tablet copy session on tablet 59d236b930214b1f91fb55ece6fecd68 from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:46982: session id = cf2f71592be8465fb18e36e15d0b2f89-59d236b930214b1f91fb55ece6fecd68
I20260709 13:02:04.543339  2833 tablet_copy_client.cc:538] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:04.560333  2852 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:04.558128  2902 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:02:04.561008  2852 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:04.545677  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:04.569126  1568 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:02:04.569948  1568 raft_consensus.cc:606] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Received request to transfer leadership to TS 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:04.563611  2800 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:04.574297  2797 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:04.574196  2800 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:04.575678  1686 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:04.582429  2758 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:04.585995  2758 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:04.591017  2800 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:04.588641  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 257137us
I20260709 13:02:04.582904  2746 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:04.611949  2908 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:04.617313  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:04.619249  2746 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:04.620091  2746 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:04.622991  2852 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:04.624320  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:04.597321  2758 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:04.633762  2852 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:04.638590  2890 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:04.642025  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:04.643204  2910 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:04.643992  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:04.643906  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:04.644486  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:04.647086  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:04.651064  2881 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:04.674978  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:04.610201  2800 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:04.728065  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:04.691787  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:04.682624  2891 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:04.738137  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:04.739233  1646 raft_consensus.cc:2410] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:04.741255  1657 tablet_copy_service.cc:140] P f5f4b2707a9647e5bb99e065068dea2b: Received BeginTabletCopySession request for tablet 58b222526a1a4b52bdcda825e15c46f9 from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:44424)
I20260709 13:02:04.741968  1657 tablet_copy_service.cc:161] P f5f4b2707a9647e5bb99e065068dea2b: Beginning new tablet copy session on tablet 58b222526a1a4b52bdcda825e15c46f9 from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:44424: session id = cf2f71592be8465fb18e36e15d0b2f89-58b222526a1a4b52bdcda825e15c46f9
I20260709 13:02:04.702697  2800 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:04.744378  2800 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:04.702648  2765 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:04.733567  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:04.748311  2312 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:04.752599  2863 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:04.763856  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:04.764695  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:04.770843  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:04.687022  2852 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:04.762612  2873 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:04.753254  2863 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:04.745421  2765 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:04.776078  2837 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:04.795405  2914 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:04.804397  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:04.804709  1537 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:04.726730  2890 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:04.781234  2851 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:04.731143  2881 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:04.804857  2837 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:04.843514  1612 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:04.727530  2910 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:04.815289  2794 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:04.767797  2891 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:04.825040  2865 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:04.853327  1612 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:04.851054  2794 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:04.860812  2794 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:04.861027  2921 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:04.864495  2894 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:04.838779  2851 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:04.909965  1639 raft_consensus.cc:2955] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:02:04.919571  2851 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:04.913545  1685 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.059s	sys 0.119s
W20260709 13:02:04.926455  1685 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.059s	sys 0.120s
W20260709 13:02:04.933445  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 527106us
I20260709 13:02:04.865262  2894 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:04.914270  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 61074.7us
I20260709 13:02:04.956563  2881 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:04.959720  1612 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:04.834766  2914 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:04.914193  2852 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:04.921120  2746 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:04.982667  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:04.982932  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:04.925765  2797 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:02:04.982201  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 44898.4us
I20260709 13:02:05.022351  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:05.022505  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:04.999967  1538 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:05.027311  2840 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:05.028986  2840 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:05.019166  1685 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:05.035959  1538 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:04.967979  2914 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:04.973558  2915 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:05.037220  2592 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:04.980856  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:04.980173  2890 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:05.001145  2867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:46968 (request call id 6) took 2913 ms. Trace:
I20260709 13:02:05.105872  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:05.106547  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 87309.1us
I20260709 13:02:05.001536  2852 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:05.110261  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:05.055343  2774 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:05.035372  2865 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:05.136471  1685 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:05.142131  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 452270us
I20260709 13:02:05.148602  1611 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:05.101832  2835 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:05.099174  2840 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:05.142532  2800 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:05.101115  2797 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:05.108851  2314 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:05.221415  2922 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.036s	sys 0.096s
W20260709 13:02:05.221880  2922 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.039s	sys 0.096s
I20260709 13:02:05.221726  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:05.222230  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 123642us
I20260709 13:02:05.220340  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:05.191598  2863 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:05.223107  2314 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:05.223124  1565 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:05.230085  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:05.203243  2923 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:05.142331  2852 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:05.223874  2894 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:05.278385  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:05.274978  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:05.196295  2865 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:05.283776  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 228806us
W20260709 13:02:05.286275  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 91138.3us
I20260709 13:02:05.289398  1538 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:05.241077  2796 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:05.224277  2910 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:05.319438  1538 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:05.244614  2837 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:05.249207  2891 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:05.326251  1538 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:05.282284  2873 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:05.336328  1611 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:05.351300  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:05.354547  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:05.355700  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:05.357962  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:05.361836  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:05.328825  2879 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:05.323602  2796 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:05.382376  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:05.384557  2889 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 1.030s	user 0.012s	sys 0.000s
W20260709 13:02:05.384375  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 161385us
I20260709 13:02:05.355777  2849 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:05.402063  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:05.393039  1657 tablet_copy_source_session.cc:215] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:05.403575  2901 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:05.425278  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:05.529589  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:05.528486  2906 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:05.535367  2906 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:05.130929  2867 rpcz_store.cc:276] 0709 13:02:02.087730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.087883 (+   153us) service_pool.cc:224] Handling call
0709 13:02:02.222501 (+134618us) log.cc:844] Serialized 506 byte log entry
0709 13:02:02.317992 (+ 95491us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b18000331c0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d98557 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:02:02.345131 (+ 27139us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:02:05.001002 (+2655871us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6541,"raft.queue_time_us":2405,"raft.run_cpu_time_us":2080,"raft.run_wall_time_us":5249,"spinlock_wait_cycles":29419008,"thread_start_us":45749,"threads_started":2,"wal-append.queue_time_us":45284,"wal-append.run_cpu_time_us":4941,"wal-append.run_wall_time_us":1086760}
W20260709 13:02:05.543038  2927 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.387s	user 0.002s	sys 0.000s
I20260709 13:02:05.528291  2937 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:05.528025  2901 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:02:05.482232  2929 log.cc:927] Time spent T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.253s	user 0.001s	sys 0.001s
I20260709 13:02:05.398094  2868 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:05.564952  2938 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:05.551707  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:05.561692  2906 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:05.452971  2926 log.cc:927] Time spent T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.289s	user 0.000s	sys 0.002s
I20260709 13:02:05.550386  2852 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:05.584013  2936 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:05.584648  2936 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:05.576297  2944 log.cc:927] Time spent T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
W20260709 13:02:05.593377  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 267255us
I20260709 13:02:05.468178  2797 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:05.570672  2938 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:05.422286  2908 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:05.601799  2794 consensus_meta.cc:287] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.736s	user 0.015s	sys 0.004s
I20260709 13:02:05.429265  2851 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:05.575122  2868 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:05.640303  2945 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:05.577910  2937 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:05.641134  2945 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:05.656649  2854 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:05.626936  2873 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:05.571740  2901 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:05.679774  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1522) took 1031 ms. Trace:
I20260709 13:02:05.636195  2794 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:05.680446  1639 rpcz_store.cc:276] 0709 13:02:04.649108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.650037 (+   929us) service_pool.cc:224] Handling call
0709 13:02:04.652399 (+  2362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:04.652956 (+   557us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.652997 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:04.654996 (+  1999us) log.cc:844] Serialized 506 byte log entry
0709 13:02:04.908967 (+253971us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:05.125747 (+216780us) log.cc:844] Serialized 14 byte log entry
0709 13:02:05.135793 (+ 10046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.136167 (+   374us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:05.679428 (+543261us) raft_consensus.cc:1685] finished
0709 13:02:05.679460 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.679666 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9617,"raft.queue_time_us":16453,"raft.run_cpu_time_us":955,"raft.run_wall_time_us":39225,"thread_start_us":171683,"threads_started":1,"wal-append.queue_time_us":442689}
I20260709 13:02:05.657657  2797 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:05.676002  2873 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:05.666811  2865 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:05.863421  2940 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:02:05.865514  1611 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.014s	sys 0.026s
W20260709 13:02:05.866119  1611 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.014s	sys 0.026s
I20260709 13:02:05.864723  2940 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:05.868112  2931 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:05.878334  2931 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:05.814507  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:05.912962  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 81616.5us
I20260709 13:02:05.955924  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:06.392236  1722 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
W20260709 13:02:05.856978  2929 log.cc:927] Time spent T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:02:05.919840  2934 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:05.920840  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:05.729460  2937 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:05.726227  2938 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:05.747725  2833 tablet_bootstrap.cc:492] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:02:05.921962  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:05.924681  2865 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:06.383721  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:06.460383  1565 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:06.336553  2901 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:06.047775  2315 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.024s	sys 0.074s
I20260709 13:02:06.114684  2945 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:06.385819  1566 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:05.671262  2854 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:06.845098  2854 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:06.061728  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:06.045789  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:02:06.052543  1568 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.029s	sys 0.070s
I20260709 13:02:06.433656  1717 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:02:05.845079  2794 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:06.163002  1573 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.001s	sys 0.012s
I20260709 13:02:06.064554  2875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58b222526a1a4b52bdcda825e15c46f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:06.455217  1713 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:02:06.452863  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 963) took 1407 ms. Trace:
I20260709 13:02:06.846069  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:06.848783  1568 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.494s	user 0.030s	sys 0.090s
W20260709 13:02:06.306742  2925 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.755s	user 0.007s	sys 0.003s
W20260709 13:02:06.457199  1646 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.823s	user 0.022s	sys 0.040s
I20260709 13:02:06.478346  2934 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:06.560350  2901 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:02:06.477162  2315 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.021s	user 0.024s	sys 0.095s
W20260709 13:02:06.595796  2928 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.002s	sys 0.007s
W20260709 13:02:06.867219  2928 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.286s	user 0.008s	sys 0.013s
I20260709 13:02:06.867406  2966 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:06.009159  2947 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:06.870229  2901 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:06.870862  2959 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:06.870945  2947 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:06.871549  2959 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:06.753832  2951 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:05.936841  2936 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:06.874217  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 935) took 2846 ms. Trace:
I20260709 13:02:06.874107  2951 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:06.874698  1645 rpcz_store.cc:276] 0709 13:02:04.027947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.028080 (+   133us) service_pool.cc:224] Handling call
0709 13:02:04.066204 (+ 38124us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:04.066687 (+   483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.066722 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:04.498092 (+431370us) spinlock_profiling.cc:243] Waited 5.11 ms on lock 0x7b2c000120e8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824777ea11 00007f824778e372 00007f8248ec0a01 00007f8248e8543a 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2
0709 13:02:04.502340 (+  4248us) log.cc:844] Serialized 26 byte log entry
0709 13:02:04.746744 (+244404us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:04.746779 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.746845 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:05.725622 (+978777us) spinlock_profiling.cc:243] Waited 9.93 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:02:06.873749 (+1148127us) raft_consensus.cc:1685] finished
0709 13:02:06.873773 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.874098 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":74873,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":17191,"spinlock_wait_cycles":34588800,"thread_start_us":236277,"threads_started":1,"wal-append.queue_time_us":419063}
I20260709 13:02:06.877393  2902 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:06.830343  2952 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:06.830622  2961 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:05.728766  2797 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:06.868985  2966 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:06.878146  2902 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:06.878834  2961 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:06.879966  2765 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:06.882005  2966 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:06.878834  2952 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:06.180987  2950 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:02:06.643105  2882 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.735s	user 0.014s	sys 0.026s
I20260709 13:02:06.874076  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:06.499677  2865 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:06.556625  2906 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:06.873387  2901 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:06.452793  2904 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:06.425269  2941 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:02:06.892486  2882 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.994s	user 0.015s	sys 0.031s
I20260709 13:02:06.384647  2940 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:06.832322  2965 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:06.865787  2874 raft_consensus.cc:993] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: : Instructing follower 130bc5ff39b847b1b3204626f284f838 to start an election
I20260709 13:02:06.884038  2950 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:06.877449  2959 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:06.885020  1713 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:06.890226  2906 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:06.897486  2965 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:06.897861  2874 raft_consensus.cc:1081] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Signalling peer 130bc5ff39b847b1b3204626f284f838 to start an election
I20260709 13:02:06.897125  2941 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:06.918322  2961 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:06.893121  2951 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:06.896775  2904 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:06.935890  2865 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:06.936108  2952 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:06.921428  2934 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:06.930708  2950 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:06.944427  2953 log.cc:927] Time spent T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Append to log took a long time: real 0.089s	user 0.001s	sys 0.001s
I20260709 13:02:06.937184  2965 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:06.947229  2947 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:06.947362  2941 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:06.850666  1573 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.268s	user 0.002s	sys 0.017s
I20260709 13:02:06.852267  1570 rpcz_store.cc:276] 0709 13:02:05.006704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.006944 (+   240us) service_pool.cc:224] Handling call
0709 13:02:05.007358 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.242485 (+235127us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:05.242535 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:05.242595 (+    60us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:05.242639 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.412548 (+1169909us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54004056d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:06.412611 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.412783 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":539234688}
I20260709 13:02:06.524947  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:06.696640  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:06.598528  1614 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.227s	user 0.000s	sys 0.037s
I20260709 13:02:06.994825  1565 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
W20260709 13:02:06.996088  1614 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.632s	user 0.009s	sys 0.058s
I20260709 13:02:06.513433  1583 tablet_copy_source_session.cc:215] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:02:06.857613  1646 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.224s	user 0.023s	sys 0.040s
W20260709 13:02:06.791559  1685 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.000s	sys 0.008s
W20260709 13:02:07.016633  1685 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.000s	sys 0.010s
W20260709 13:02:06.654232  1686 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.001s	sys 0.009s
W20260709 13:02:06.685849  1538 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.046s	user 0.091s	sys 0.138s
I20260709 13:02:07.002563  2800 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:06.936627  2873 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:07.002775  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:07.082743  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:07.086545  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 38) took 2311 ms. Trace:
W20260709 13:02:07.195319  1715 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1260) took 2195 ms (client timeout 2122 ms). Trace:
I20260709 13:02:07.198496  2890 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:07.199872  2865 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:07.089385  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:07.077062  2969 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:07.070156  2920 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:07.078512  2908 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:07.077737  2947 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:07.204355  1537 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:07.186395  1646 rpcz_store.cc:276] 0709 13:02:04.775347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.276390 (+501043us) service_pool.cc:224] Handling call
0709 13:02:05.296821 (+ 20431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.297127 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:05.633538 (+336411us) log.cc:844] Serialized 14 byte log entry
0709 13:02:07.085092 (+1451554us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:07.085243 (+   151us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:07.085436 (+   193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.085660 (+   224us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:07.086278 (+   618us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68825,"prepare.queue_time_us":59440,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":63,"raft.queue_time_us":70936,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":297,"spinlock_wait_cycles":203443584,"thread_start_us":1689576,"threads_started":2,"wal-append.queue_time_us":1296526}
I20260709 13:02:07.073124  2904 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.188997  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:07.078145  2874 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:07.141072  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 981) took 1575 ms. Trace:
I20260709 13:02:07.141865  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:07.093088  2882 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:07.203040  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:07.085064  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:07.215936  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:07.210898  1715 rpcz_store.cc:269] 0709 13:02:04.999671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.194202 (+2194531us) service_pool.cc:211] Skipping call since client already timed out
0709 13:02:07.195060 (+   858us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:02:07.218391  1713 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:07.216276  2890 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:07.214520  2934 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:07.289876  2753 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:07.079627  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:07.070910  2941 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:07.069204  1583 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:46982 (request call id 51) took 2513 ms. Trace:
I20260709 13:02:07.299770  2947 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:07.301836  2800 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.203119  2796 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.303714  2874 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.087514  2956 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:07.304585  2931 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.071586  2915 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:07.313787  2753 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.078795  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:07.066594  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:07.078258  2838 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:07.079384  2870 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:07.033138  2906 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:07.314363  1583 rpcz_store.cc:276] 0709 13:02:04.556064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.558081 (+  2017us) service_pool.cc:224] Handling call
0709 13:02:06.425096 (+1867015us) spinlock_profiling.cc:243] Waited 953 ms on lock 0x7b5400013e54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e8f3f7 00007f824de25036 00007f824de2ea3e 00007f82456f227b 00007f824de2e9ba 00007f824de29acc 00007f824de24c7b 00007f824de198d1 00007f824cecd4b4 00007f824cecd442
0709 13:02:07.068838 (+643742us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2192169728}
I20260709 13:02:07.216980  2865 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.033825  2863 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:07.349426  2972 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:07.139930  2964 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:07.328058  2941 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.330083  2947 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:07.329931  2911 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:07.331674  1538 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.673s	user 0.118s	sys 0.196s
I20260709 13:02:07.079319  2971 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:02:07.337903  1686 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.277s	user 0.003s	sys 0.025s
I20260709 13:02:07.101526  1612 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:07.220399  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 144944us
I20260709 13:02:07.295483  1573 rpcz_store.cc:276] 0709 13:02:05.565272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.565447 (+   175us) service_pool.cc:224] Handling call
0709 13:02:05.571260 (+  5813us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.572731 (+  1471us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:06.986580 (+1413849us) log.cc:844] Serialized 14 byte log entry
0709 13:02:07.113984 (+127404us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:07.114097 (+   113us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:07.114311 (+   214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.114504 (+   193us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:07.140792 (+ 26288us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":488533,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":1448,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":1449273,"threads_started":2,"wal-append.queue_time_us":199457}
I20260709 13:02:07.358273  1639 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:02:07.083936  2902 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:07.218575  2908 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.350252  2863 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:07.365399  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 372536us
I20260709 13:02:07.365823  2873 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:07.366859  2908 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.367201  2863 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.367645  2838 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:07.367235  2873 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:07.349864  2915 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.375494  2874 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.376510  2874 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:07.376381  2904 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:07.356338  2956 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.352674  2964 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.217940  2920 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.363276  2931 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:07.386610  2947 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:02:07.382861  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 1.34237e+06us
I20260709 13:02:07.217319  2969 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:07.377543  2971 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:07.369534  2753 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.377785  2870 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.406843  2947 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.412048  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:07.419533  2941 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.420454  2947 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.387744  2800 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.399048  2934 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.422369  2931 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.350769  2972 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:07.359542  2882 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.427814  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1537) took 2560 ms. Trace:
I20260709 13:02:07.425028  2934 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:07.435446  1685 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:07.450985  1612 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:07.416826  2865 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:07.408085  2874 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.434010  1719 rpcz_store.cc:276] 0709 13:02:04.869017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.874123 (+  5106us) service_pool.cc:224] Handling call
0709 13:02:06.893435 (+2019312us) spinlock_profiling.cc:243] Waited 44.8 ms on lock 0x7b580009db98. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60f8c 00007f8245d60ef6 0000000000375939 00007f824ddd41e0 00007f824de9d0c4 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:02:06.893613 (+   178us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:06.893913 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:07.033962 (+140049us) log.cc:844] Serialized 14 byte log entry
0709 13:02:07.355940 (+321978us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:07.362584 (+  6644us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:07.386006 (+ 23422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.399063 (+ 13057us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:07.427409 (+ 28346us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":329,"prepare.queue_time_us":54601,"prepare.run_cpu_time_us":166,"prepare.run_wall_time_us":248,"raft.queue_time_us":9102,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":251,"spinlock_wait_cycles":103042048,"thread_start_us":443106,"threads_started":2,"wal-append.queue_time_us":397498}
I20260709 13:02:07.471586  2800 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:07.466488  2971 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:07.483513  2890 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:07.486858  1639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:49580
I20260709 13:02:07.476938  2882 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.495178  1722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:42276
I20260709 13:02:07.492595  1639 raft_consensus.cc:606] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
W20260709 13:02:07.496785  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 94485.2us
I20260709 13:02:07.483922  2865 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:07.500047  1722 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:07.496817  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1525) took 2832 ms. Trace:
I20260709 13:02:07.500550  1722 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:07.500697  1642 rpcz_store.cc:276] 0709 13:02:04.664823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.722346 (+ 57523us) service_pool.cc:224] Handling call
0709 13:02:04.722757 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.870501 (+147744us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.870596 (+    95us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:04.870670 (+    74us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:04.870721 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.496135 (+2625414us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54001fefd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:07.496246 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:07.496579 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338923776}
I20260709 13:02:07.498992  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:07.503947  1715 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:07.511219  2797 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.512210  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:07.514355  1717 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:02:07.515247  2964 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.517187  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:07.519328  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:07.520102  2347 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:07.522277  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:07.529415  2873 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.529227  2871 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:07.530519  2906 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:07.530853  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:07.530982  2865 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:07.531268  2906 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.532618  2865 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:07.498329  2972 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:07.498435  2838 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:07.516546  1612 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:07.542976  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 117961us
I20260709 13:02:07.547681  2956 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.547768  2873 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:07.549276  2902 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:07.550923  2902 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:07.551894  1722 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.554299  1722 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:07.554528  2874 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.554991  2867 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:07.556036  2964 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:07.563261  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:07.549685  2969 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:07.563931  2867 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.563875  2964 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:07.570397  2964 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:07.571724  2867 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.572589  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:07.574581  1565 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 1.
I20260709 13:02:07.575336  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:07.576285  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:07.569190  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:07.577687  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:07.576421  1570 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 1.
I20260709 13:02:07.579299  1566 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 1.
I20260709 13:02:07.579396  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:07.580179  1569 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 1.
W20260709 13:02:07.559141  2975 log.cc:927] Time spent T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.609s	user 0.000s	sys 0.004s
I20260709 13:02:07.582271  2915 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.536604  2906 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:02:07.585558  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:07.586827  1572 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:02:07.587829  2972 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:07.595575  2972 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.596501  2920 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:07.597389  2957 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.649s	user 0.000s	sys 0.014s
I20260709 13:02:07.537477  2971 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:07.590662  2867 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:07.549288  2873 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:07.591632  1685 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:07.609841  1572 raft_consensus.cc:606] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Received request to transfer leadership to TS 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:07.624737  2971 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:07.627061  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:07.611852  2915 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:07.635810  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 119521us
I20260709 13:02:07.601140  2865 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:07.640761  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49568.4us
I20260709 13:02:07.649712  2870 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.644294  2931 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.641618  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:07.656407  1716 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:02:07.641119  2906 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:07.658975  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:07.659291  2906 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:07.659791  2906 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:07.661863  2867 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.664610  2867 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.639261  2902 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.694175  2875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:55874 (request call id 202) took 4713 ms. Trace:
I20260709 13:02:07.695078  2875 rpcz_store.cc:276] 0709 13:02:02.980682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.980808 (+   126us) service_pool.cc:224] Handling call
0709 13:02:03.304565 (+323757us) ts_tablet_manager.cc:934] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
0709 13:02:06.062982 (+2758417us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:02:07.476295 (+1413313us) tablet_metadata.cc:665] Metadata flushed
0709 13:02:07.499894 (+ 23599us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:02:07.691223 (+191329us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:02:07.692746 (+  1523us) tablet_replica.cc:206] Creating consensus instance
0709 13:02:07.693723 (+   977us) ts_tablet_manager.cc:989] Replica registered.
0709 13:02:07.693890 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":64873,"tablet-copy.queue_time_us":251504,"thread_start_us":283391,"threads_started":2}
I20260709 13:02:07.696630  1469 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47518 (request call id 85) took 3192 ms. Trace:
I20260709 13:02:07.698428  2875 tablet_copy_client.cc:806] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:07.698348  1469 rpcz_store.cc:276] 0709 13:02:04.504023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.504200 (+   177us) service_pool.cc:224] Handling call
0709 13:02:07.696252 (+3192052us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 13:02:07.699528  2875 tablet_copy_client.cc:670] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:07.705432  2873 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.713967  1537 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:07.719187  2867 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:07.720005  2867 raft_consensus.cc:697] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:07.721557  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:07.722203  2867 consensus_queue.cc:237] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.723609  2833 tablet_bootstrap.cc:492] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:07.725034  2833 tablet_bootstrap.cc:492] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:02:07.725545  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:07.728598  2833 ts_tablet_manager.cc:1403] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 2.026s	user 0.076s	sys 0.040s
W20260709 13:02:07.730947  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 48509.2us
I20260709 13:02:07.751818  2888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59d236b930214b1f91fb55ece6fecd68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:07.755596  2868 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:07.758348  2954 log.cc:927] Time spent T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.818s	user 0.006s	sys 0.009s
W20260709 13:02:07.759552  2949 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.829s	user 0.008s	sys 0.007s
I20260709 13:02:07.768555  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 965) took 2747 ms. Trace:
I20260709 13:02:07.769169  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:07.770960  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:07.769268  1568 rpcz_store.cc:276] 0709 13:02:05.020397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.027128 (+  6731us) service_pool.cc:224] Handling call
0709 13:02:05.028100 (+   972us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:05.275409 (+247309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:05.275446 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:05.287928 (+ 12482us) log.cc:844] Serialized 26 byte log entry
0709 13:02:07.528232 (+2240304us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b18000c6160. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a250d1
0709 13:02:07.528652 (+   420us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:07.528703 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.537054 (+  8351us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:07.758479 (+221425us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:02:07.767279 (+  8800us) raft_consensus.cc:1685] finished
0709 13:02:07.767303 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:07.767628 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26336,"raft.queue_time_us":80993,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":122,"spinlock_wait_cycles":315647616,"thread_start_us":1682021,"threads_started":2,"wal-append.queue_time_us":1520980}
I20260709 13:02:07.771860  1713 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 1.
I20260709 13:02:07.772300  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:07.772050  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:07.769279  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:07.775902  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:07.779141  1470 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38666 (request call id 209) took 4727 ms. Trace:
I20260709 13:02:07.743048  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:07.785269  2971 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:07.779811  1470 rpcz_store.cc:276] 0709 13:02:03.051807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.052111 (+   304us) service_pool.cc:224] Handling call
0709 13:02:07.778887 (+4726776us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:04.050298 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:04.124183 (+ 73885us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:04.124235 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:02:04.124261 (+    26us) tablet.cc:662] Decoding operations
0709 13:02:04.124804 (+   543us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:04.124907 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:04.124949 (+    42us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:02:04.125370 (+   421us) tablet.cc:701] Row locks acquired
0709 13:02:04.125395 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:02:04.125586 (+   191us) write_op.cc:270] Start()
0709 13:02:04.125772 (+   186us) write_op.cc:276] Timestamp: P: 1783602124125532 usec, L: 0
0709 13:02:04.125801 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:02:04.126832 (+  1031us) log.cc:844] Serialized 928 byte log entry
0709 13:02:04.593288 (+466456us) op_driver.cc:464] REPLICATION: finished
0709 13:02:05.322175 (+728887us) write_op.cc:301] APPLY: starting
0709 13:02:05.332116 (+  9941us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:05.354043 (+ 21927us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:05.399118 (+ 45075us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:05.456439 (+ 57321us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:05.524420 (+ 67981us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:02:05.529676 (+  5256us) write_op.cc:312] APPLY: finished
0709 13:02:05.550314 (+ 20638us) log.cc:844] Serialized 28 byte log entry
0709 13:02:06.867976 (+1317662us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:06.868083 (+   107us) write_op.cc:454] Released schema lock
0709 13:02:06.868130 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:06.892530 (+ 24400us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:06.899721 (+  7191us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":701577,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":685248,"apply.run_cpu_time_us":29021,"apply.run_wall_time_us":1577581,"num_ops":2,"prepare.queue_time_us":210013,"prepare.run_cpu_time_us":5362,"prepare.run_wall_time_us":119946,"raft.queue_time_us":242731,"raft.run_cpu_time_us":135046,"raft.run_wall_time_us":629511,"replication_time_us":467274,"thread_start_us":2272359,"threads_started":5,"wal-append.queue_time_us":1001372,"wal-append.run_cpu_time_us":38959,"wal-append.run_wall_time_us":1374146}]]}
I20260709 13:02:07.789949  2937 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:07.791842  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:07.793994  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:07.787985  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 53828.1us
I20260709 13:02:07.794875  2937 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:07.795157  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:07.801971  2937 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:07.806250  1647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
mode: GRACEFUL
new_leader_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:49580
I20260709 13:02:07.807866  1647 raft_consensus.cc:606] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Received request to transfer leadership to TS 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:07.810559  1468 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38650 (request call id 193) took 2911 ms. Trace:
I20260709 13:02:07.813371  1468 rpcz_store.cc:276] 0709 13:02:04.907804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.921195 (+ 13391us) service_pool.cc:224] Handling call
0709 13:02:07.069766 (+2148571us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b54002b0054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248def090 00007f8248e8df3a 00007f824ec441cc 00007f824ed4237e 00007f824d2032a4 00007f824d203232 00007f824d203161 00007f824d2030dc 00007f824d202392 00007f82471c048c 00007f82471bf8c6
0709 13:02:07.795156 (+725390us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":550854,"reported_tablets":1,"spinlock_wait_cycles":698499968}
I20260709 13:02:07.815168  1646 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:49580
I20260709 13:02:07.793534  2972 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:07.817898  1646 raft_consensus.cc:606] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
W20260709 13:02:07.812074  1538 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.977s (SENT) [suppressed 3 similar messages]
I20260709 13:02:07.815804  1537 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:07.822453  1612 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:07.823658  1612 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:07.823745  2960 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:07.824853  1612 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:07.825168  2952 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:07.825794  2952 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:07.826220  2952 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:07.830120  2965 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:07.831008  2965 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:07.831494   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:07.831651  2965 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:07.833449  2312 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:07.825182  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:07.834873  1568 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 1.
I20260709 13:02:07.825820  2313 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:07.824855  2960 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:07.839550  2960 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:07.828284  2833 raft_consensus.cc:359] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:07.840651  2833 raft_consensus.cc:740] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:02:07.841725  2833 consensus_queue.cc:260] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:07.837425   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:07.851049  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:07.860661  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 45321.9us
W20260709 13:02:07.863387  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 115498us
I20260709 13:02:07.869021  2963 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:07.870123  2963 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:07.870810  2963 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:07.890482  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:07.902307  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 75487.5us
I20260709 13:02:07.787765  2906 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
W20260709 13:02:07.910858  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 98833.7us
W20260709 13:02:07.906280  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 80861.3us
I20260709 13:02:07.875489  2902 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:07.904881  2871 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:07.933976  2871 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:07.934948  2871 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:07.865607  2937 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:07.940205  1537 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:07.914604  2868 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:02:07.911302  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 78121.3us
I20260709 13:02:07.937357  2902 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:07.959940  2902 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:07.944521  2874 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:07.945245  2964 raft_consensus.cc:993] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: : Instructing follower 130bc5ff39b847b1b3204626f284f838 to start an election
I20260709 13:02:07.954073  2951 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:07.958035  2797 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:07.963174  2947 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:07.971246  2947 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.975986  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:07.978981  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:07.977560  2937 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:08.019758  2868 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.932783  2972 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:08.025488  1714 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:07.957223  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:07.965373  2797 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:07.965885  2951 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:08.033994  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.035578  2947 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:08.037046  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.038414  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.039974  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.042241  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:07.987668  2312 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:08.017894  2874 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:08.046025  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.046694  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:08.034063  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:08.048141  2797 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.041946  2972 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:07.986395  2964 raft_consensus.cc:1081] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Signalling peer 130bc5ff39b847b1b3204626f284f838 to start an election
I20260709 13:02:08.049891  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.050907  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.045198  2874 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:08.061100  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:08.062306  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 92155us
I20260709 13:02:08.056034  1646 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:08.065948  2984 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:08.066488  2984 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:08.067066  2984 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.050349  2951 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:08.083060  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 125952us
I20260709 13:02:08.092465  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:08.093284  2345 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:02:08.063988  2312 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:08.080684  2874 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:08.147472  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 243115us
I20260709 13:02:08.152418  2960 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:08.138005  2963 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:08.184968  2965 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:08.187613  2902 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:08.218591  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.174728  2971 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:08.229619  1646 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:08.188963  2965 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:08.170555  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 106827us
I20260709 13:02:08.176100  2972 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:08.234124  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.219573  2960 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:08.215938  2906 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:08.230809  2971 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:08.237277  2963 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:08.255604  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.266053  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.253152  2971 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.264554  1646 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:08.289471  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:08.276796  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.255767  2970 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:08.277468  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.301959  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.276134  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.311066  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.260085  2915 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:08.310782  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.254693  2312 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:08.306162  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.321581  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.274520  2965 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:08.270532  2960 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:08.318759  1645 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:08.328135  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.323800  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.324549  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.275116  2902 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:08.304337  2874 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.327498  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.323701  1718 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:02:08.280901  1647 consensus_queue.cc:237] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:08.333916  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.324421  2986 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:08.341688  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:08.341231  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:08.342201  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.343122  1648 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:08.340327  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.347728  1645 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
W20260709 13:02:08.328747  2952 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.502s	user 0.013s	sys 0.008s
I20260709 13:02:08.350384  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.352156  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.353818  2797 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:08.324468  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 79450.5us
I20260709 13:02:08.357216  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.373178  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.362133  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.374509  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.375272  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:08.375401  2312 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:08.376434  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:08.376500  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.364562  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.378547  2873 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:08.379066  2873 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:08.365278  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.379920  2833 ts_tablet_manager.cc:1434] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.651s	user 0.011s	sys 0.065s
I20260709 13:02:08.380155  1717 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
W20260709 13:02:08.380297  2833 ts_tablet_manager.cc:1467] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Tablet startup took 651ms
I20260709 13:02:08.346959  2986 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:08.365582  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.366677  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.359313  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:08.359663  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.357154  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:08.379527  2873 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.354050  1639 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:08.390763  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:08.393410  1640 raft_consensus.cc:2393] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 2: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:08.393324  2986 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.326839  2960 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:08.402195  1715 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:02:08.402369  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.368793  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.412997  1643 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:08.404934  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.406962  1714 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:08.407814  2871 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:08.408234  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.382123  2952 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:08.416507  2871 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:08.409310  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.335301  2965 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:08.401717  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.418020  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:08.362085  2797 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:08.321239  2970 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:08.367756  2915 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:08.462932  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 81164.8us
I20260709 13:02:08.474834  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:08.481068  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 113828us
I20260709 13:02:08.483711  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:08.356892  2951 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:08.487759  1641 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:08.464334  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.498404  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.502127  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:08.502956  1567 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.504022  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.504340  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.505235  2952 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:08.510704  2873 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:08.511636  2951 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:08.512010  2348 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:08.515970  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.515668  2874 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:08.497789  2797 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:08.536238  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.525381  2915 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:08.533252  2874 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:08.549629  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.525175  2937 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:02:08.562067  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 62378.3us
I20260709 13:02:08.521871  2986 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:08.567924  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:08.570808  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 97365.4us
I20260709 13:02:08.578258  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.579280  1564 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:08.582131  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.583060  1571 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:02:08.583184  1567 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:08.529799  2984 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:08.550484  2873 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:08.584834  1567 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:08.548796  2868 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:08.533710  2797 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:08.588079  1719 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:42276
I20260709 13:02:08.601373  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.599604  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.581947  2874 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:08.604521  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.605906  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:08.584896  2937 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:08.606218  2349 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 2: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:08.607484  1570 raft_consensus.cc:3055] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:08.608227  1570 raft_consensus.cc:740] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:08.609283  2937 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:08.609930  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.610072  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.611339  2960 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:08.611707  1565 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:02:08.602442  1573 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:02:08.613410  2970 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:08.613823  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.615048  1568 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:02:08.614485  2965 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:08.601787  1719 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:08.621076  2345 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:08.609272  1570 consensus_queue.cc:260] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:08.622408  1719 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.628916  1570 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.629772  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.601205  2963 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:08.634690  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:08.634833  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.636334  1717 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.638182  2346 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
W20260709 13:02:08.380710  2833 ts_tablet_manager.cc:1469] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Trace:
0709 13:02:01.940682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.940829 (+   147us) service_pool.cc:224] Handling call
0709 13:02:02.113573 (+172744us) ts_tablet_manager.cc:934] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
0709 13:02:02.311412 (+197839us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:02:02.649165 (+337753us) tablet_metadata.cc:665] Metadata flushed
0709 13:02:02.656056 (+  6891us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:02:03.306782 (+650726us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:02:03.307133 (+   351us) tablet_replica.cc:206] Creating consensus instance
0709 13:02:03.307483 (+   350us) ts_tablet_manager.cc:989] Replica registered.
0709 13:02:03.307544 (+    61us) inbound_call.cc:177] Queueing success response
0709 13:02:05.686489 (+2378945us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:02:07.729126 (+2042637us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:02:07.745105 (+ 15979us) tablet_replica.cc:261] Starting instrumentation
0709 13:02:07.746452 (+  1347us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":5092,"mutex_wait_us":4351,"raft.queue_time_us":16254,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":16460,"tablet-copy.queue_time_us":42423,"tablet-download-pool.queue_time_us":208,"tablet-download-pool.run_cpu_time_us":3560,"tablet-download-pool.run_wall_time_us":1157781,"thread_start_us":414760,"threads_started":3}
I20260709 13:02:08.640753  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:08.642159  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.642488  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:08.643894  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:08.648427  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.650187  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.647771  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.658421  2988 consensus_queue.cc:1059] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:08.660425  2906 raft_consensus.cc:3055] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:02:08.658391  1611 connection.cc:256] RPC call timeout handler was delayed by -1.1653583960000446s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21309999999999998s for a response
I20260709 13:02:08.661471  2906 raft_consensus.cc:740] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:08.662705  2906 consensus_queue.cc:260] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:08.664440  2906 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.667946  2797 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:08.668761  2963 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:08.671352  2963 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:08.674358  1570 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:08.678488  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:08.680481  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.678575  1583 tablet_copy_service.cc:342] P 066ee0109aba4465ba17fa65e14ef504: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-eca30b43c26f420c965ee25f475581a0 received from {username='slave'} at 127.0.0.1:46982
I20260709 13:02:08.682747  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:08.684643  1569 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:08.687071  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.688550  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.678506  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.623078  2868 consensus_queue.cc:1059] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 13:02:08.691102  2868 raft_consensus.cc:3055] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:08.691506  2868 raft_consensus.cc:740] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:08.682134  1583 tablet_copy_service.cc:434] P 066ee0109aba4465ba17fa65e14ef504: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-eca30b43c26f420c965ee25f475581a0 on tablet eca30b43c26f420c965ee25f475581a0 with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:08.692808  2868 consensus_queue.cc:260] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:08.694156  2868 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.602684  2971 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:08.696949  1687 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:08.698042  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:08.698833  2988 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.700049  1564 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:08.671871  2963 consensus_queue.cc:1059] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:02:08.701385  2986 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:08.701653  2312 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:08.702327  2963 raft_consensus.cc:3055] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:08.708431  2963 raft_consensus.cc:740] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:08.681388  1573 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:08.687011  2888 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:55890 (request call id 143) took 5365 ms. Trace:
I20260709 13:02:08.703230  1566 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:08.710022  1687 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:08.676999  1719 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:08.723536  1716 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
W20260709 13:02:08.725206  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 97707.1us
I20260709 13:02:08.725410  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:08.726718  1568 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:08.732713  1717 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:08.735594  1719 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:08.709599  2888 rpcz_store.cc:276] 0709 13:02:03.322065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.322246 (+   181us) service_pool.cc:224] Handling call
0709 13:02:03.508342 (+186096us) ts_tablet_manager.cc:934] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
0709 13:02:07.749675 (+4241333us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:02:08.260063 (+510388us) tablet_metadata.cc:665] Metadata flushed
0709 13:02:08.268713 (+  8650us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:02:08.685975 (+417262us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:02:08.686352 (+   377us) tablet_replica.cc:206] Creating consensus instance
0709 13:02:08.686699 (+   347us) ts_tablet_manager.cc:989] Replica registered.
0709 13:02:08.686939 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":64938,"tablet-copy.queue_time_us":184229,"thread_start_us":506556,"threads_started":2}
I20260709 13:02:08.742129  2989 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:08.743259  2888 tablet_copy_client.cc:806] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:08.744129  2888 tablet_copy_client.cc:670] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:08.743527  2989 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:08.746830  2963 consensus_queue.cc:260] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:08.748571  2994 raft_consensus.cc:993] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b: : Instructing follower 066ee0109aba4465ba17fa65e14ef504 to start an election
W20260709 13:02:08.746821  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 48275.4us
I20260709 13:02:08.753875  1687 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:08.715816  2984 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:08.722822  2971 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:08.758427  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:08.760689  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:08.728689  2797 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:08.731035  2988 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.732441  2873 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:08.781788  2797 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:08.765604  2746 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.779201  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:08.782086  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:08.757880  2952 raft_consensus.cc:1081] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Signalling peer 066ee0109aba4465ba17fa65e14ef504 to start an election
I20260709 13:02:08.764881  2947 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:02:08.786228  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 123923us
I20260709 13:02:08.795644  1717 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:08.797868  1685 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:08.799098  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 45295.8us
I20260709 13:02:08.796535  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:08.806742  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:08.808077  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.808622  1564 raft_consensus.cc:3055] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:08.811182  2947 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:08.813898  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.794852  2873 raft_consensus.cc:697] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:08.777667  2994 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:08.817952  2994 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:08.785750  1468 catalog_manager.cc:5697] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:02:08.818284  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.818397  2994 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.808957  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:08.814867  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.777957  2963 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.824110  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:08.824870  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:08.827862  1687 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:08.816495  2873 consensus_queue.cc:237] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:08.826318  1642 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:08.828207  2746 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:08.809541  1564 raft_consensus.cc:740] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:08.845340  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.844966  2937 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:08.836205  2972 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:08.837450  1569 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:08.854157  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:08.848229  2345 raft_consensus.cc:1217] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:08.858111  2797 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:08.858950  2937 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:08.872258  2998 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.876034  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.883901  2937 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:08.885982  1641 raft_consensus.cc:1240] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:08.873818  2991 consensus_queue.cc:1059] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:02:08.877410  1642 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
W20260709 13:02:08.907898  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.StartTabletCopy -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 66504.6us
I20260709 13:02:08.870077  2989 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:08.909807  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.913157  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:08.892947  2991 raft_consensus.cc:3055] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:02:08.931452  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 105079us
I20260709 13:02:08.934939  1685 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:08.932693  1611 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:08.939987  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.941787  1687 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:08.943351  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.946503  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.929677  2991 raft_consensus.cc:740] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:08.931454  2937 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:08.940668  2952 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:08.955384  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:08.956743  1687 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:08.957515  1685 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:08.957798  2746 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:08.958565   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:08.945997  2906 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:08.960695  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.911190  1564 consensus_queue.cc:260] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:08.964481  1564 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.966044  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.968765  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.975996  2937 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.979038  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.980288  1611 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:08.981331  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.956554  2952 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:08.979244  2947 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:08.968756  2888 tablet_copy_client.cc:538] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:08.992938  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:08.989897  1685 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:08.980701  2989 consensus_queue.cc:1059] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:08.986843  2746 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.986915  2952 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:08.982390  2906 raft_consensus.cc:697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:08.992614  2972 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:09.000602  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:09.032676  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:09.040123  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 83466.5us
W20260709 13:02:09.046765  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.040858  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:09.057987  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.027994  2988 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:09.029510  2972 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:02:09.079348  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 53106.9us
I20260709 13:02:09.027124  2991 consensus_queue.cc:260] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:09.041126  2874 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:09.106194  2997 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:09.107424  2997 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:09.108383  2997 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:09.110406  2906 consensus_queue.cc:237] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:09.082681  2972 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.076762  2988 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:09.083932  2871 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:02:09.134537  2937 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:09.123785  2988 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.134589  2994 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:09.156661  1611 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:09.148607  1687 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:09.160193  2991 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:09.174841  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:09.175410  1611 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:09.178653  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:09.183133  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:09.202172  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.159601  2994 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:09.178265  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.174118  2965 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:09.230469  1611 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:09.173035  3000 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:09.241423  1611 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:09.209951  3010 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:09.235046  1687 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:09.249861  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 178414us
I20260709 13:02:09.232296  2965 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:09.255717  2746 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:09.260238  2746 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:09.260900  2746 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:09.258049  1687 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:09.261397  2989 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:09.240047  3000 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:09.280561  1687 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:09.262848  2965 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:09.308280  1564 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:09.310680  1685 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:09.246031  3010 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:02:09.314038  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 72710.4us
I20260709 13:02:09.320060  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:09.320453  1687 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:09.317205  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.330559  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:09.331617  1716 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:09.240624  3009 consensus_queue.cc:1059] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:02:09.262203  2989 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:09.262171  2952 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:09.339468  1611 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:09.310778  3000 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:09.328001  2868 raft_consensus.cc:2764] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:02:09.350382  2868 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:09.334828  2906 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:09.351132  2906 raft_consensus.cc:697] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:09.318917  2836 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:09.321403  3010 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.345661  2989 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.352001  2906 consensus_queue.cc:237] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:09.361130  2836 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:09.366400  3016 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:09.417742  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:09.427770  2952 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:09.428836  1646 raft_consensus.cc:3055] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:02:09.429188  3012 log.cc:927] Time spent T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:02:09.429464  1646 raft_consensus.cc:740] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:09.385546  2960 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:09.407053  2836 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:09.468947  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 147379us
I20260709 13:02:09.447264  1646 consensus_queue.cc:260] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:09.487865  1646 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.494364  1687 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:09.497722  1687 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:09.496124  3003 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:09.415045  3016 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:09.514034  1646 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:02:09.516558  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 194055us
I20260709 13:02:09.517083  1687 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:02:09.518214  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 101654us
I20260709 13:02:09.526547  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:09.526803  3022 consensus_queue.cc:1059] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:09.530260  1643 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:09.529598  3022 raft_consensus.cc:3055] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:09.531073  3022 raft_consensus.cc:740] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:09.531780  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:09.533178  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:09.534348  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:09.535331  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:09.535867  3022 consensus_queue.cc:260] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:09.536309  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:09.538121  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:09.538004  3022 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.538604  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:09.452312  2960 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:09.542855  1685 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:09.544021  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:09.493260  2988 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:09.558804  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:09.562294  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:09.509506  3003 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:09.562593  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:09.620105  1568 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:09.617398  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:09.584223  1573 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:02:09.590723  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.602576  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:09.632128  1639 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
W20260709 13:02:09.610008  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.633625  1687 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:09.514600  3016 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.549613  2960 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.638178  1566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:40656
I20260709 13:02:09.649787  1566 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:09.544211  3021 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:09.646889  1687 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:09.650295  1566 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:09.552098  2868 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:09.606099  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:09.653105  2947 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:09.608302  2888 tablet_bootstrap.cc:492] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:02:09.628839  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:09.521998  2972 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:09.617187  2963 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:09.629462  3003 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.691654  3024 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:09.679324  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:09.637930  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 151383us
I20260709 13:02:09.621246  2997 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:09.604388  2989 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:09.679282  3004 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:02:09.680961  3015 log.cc:927] Time spent T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Append to log took a long time: real 0.229s	user 0.000s	sys 0.004s
I20260709 13:02:09.685756  2998 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:09.659507  2868 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:09.681011  3010 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:09.685807  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.661881  2947 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:09.691200  2988 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:09.736949  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.660810  1537 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:09.656083  3021 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:09.686446  2963 raft_consensus.cc:697] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:09.691712  2915 raft_consensus.cc:993] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: : Instructing follower 066ee0109aba4465ba17fa65e14ef504 to start an election
I20260709 13:02:09.729539  2972 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:09.730376  2997 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:09.751348  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 205824us
W20260709 13:02:09.738559  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 77861.1us
I20260709 13:02:09.692309  3024 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:09.756776  2836 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:09.734426  2868 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.742376  3021 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.769284  2989 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:09.770828  3010 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:09.752315  3005 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:02:09.787534  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.760880  3024 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.795672  1719 raft_consensus.cc:1240] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:09.742765  2746 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:09.785183  2952 raft_consensus.cc:1081] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Signalling peer 066ee0109aba4465ba17fa65e14ef504 to start an election
W20260709 13:02:09.809954  2970 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:02:09.811219  2994 raft_consensus.cc:3055] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:09.790696  3005 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:09.822865  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:02:09.834261  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.834980  1568 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.820206  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:02:09.831446  2965 consensus_meta.cc:287] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.523s	user 0.014s	sys 0.003s
I20260709 13:02:09.849486  1567 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:09.765377  2998 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:09.836669  3005 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:09.830914  2994 raft_consensus.cc:740] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:09.861402  3027 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.804912  2915 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:02:09.895819  1568 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:09.854454  2963 consensus_queue.cc:237] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:09.931766  2746 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:09.935874  2947 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:09.945850  1568 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:09.940953  1470 catalog_manager.cc:5697] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 1 to 2, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:09.958295  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:09.960203  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:09.860119  2965 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:09.973472  3000 consensus_meta.cc:287] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.616s	user 0.014s	sys 0.002s
I20260709 13:02:09.997965  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:09.998004  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 153728us
W20260709 13:02:10.002388  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 355577us
I20260709 13:02:10.004005  1570 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:40656
I20260709 13:02:10.004621  1570 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:10.004984  1570 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:10.005167  1687 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:10.006939  1647 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:10.002610  1566 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:10.019021  1568 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 3. Ops: []
I20260709 13:02:10.029157  1645 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:09.980528  2906 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.041191  1685 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:10.048010  1566 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:10.027155  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:10.001591  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:10.056703  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.046818  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:10.062116  2952 consensus_queue.cc:1059] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:09.994372  2873 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:02:10.073282  1570 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:09.908120  3027 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:10.081439  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:10.080340  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.084594  1570 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:09.987215  2960 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:10.097276  1640 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:10.102319  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.016019  3000 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:10.086673  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 48883.9us
I20260709 13:02:10.034853  2906 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:10.080961  2314 leader_election.cc:336] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.657s (SENT)
I20260709 13:02:10.009320  2994 consensus_queue.cc:260] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:10.078397  2836 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:10.137042  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:10.134402  1612 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:10.137625  1722 raft_consensus.cc:3055] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:10.138046  1722 raft_consensus.cc:740] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:10.139768  1714 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:10.142769  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:10.143355  1715 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:10.145105  3022 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:10.119565  2964 consensus_queue.cc:1059] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:02:10.136466  3007 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:10.162074  3007 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:10.162523  3007 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:10.167481  1722 consensus_queue.cc:260] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:10.171119  1722 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:10.174614  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.179029  1612 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:10.148737  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 73051.5us
W20260709 13:02:10.191675  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 186570us
I20260709 13:02:10.206476  3025 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:10.207502  3025 raft_consensus.cc:697] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:02:10.211522  2314 leader_election.cc:336] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.624s (SENT)
W20260709 13:02:10.215675  2314 leader_election.cc:336] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.549s (SENT)
I20260709 13:02:10.229923  2868 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:10.231451  1719 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:10.218590  1647 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
W20260709 13:02:10.212086  3037 log.cc:927] Time spent T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:02:10.230458  3030 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.223531  1642 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:10.125039  3033 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.096s
W20260709 13:02:10.239768  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 198745us
I20260709 13:02:10.237372  1715 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
W20260709 13:02:10.237010  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.240286  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.244086  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:10.244761  2313 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:10.217422  2998 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:10.199365  3019 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:10.255117  1612 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:10.257455  1612 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:10.263499  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:10.266848  1612 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:10.268702  1612 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:10.242961  2868 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:10.253791  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:10.263274  2991 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.285444  2989 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:10.299798  2313 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:10.302663  1722 raft_consensus.cc:2410] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 3 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:02:10.271744  2970 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.282907  3009 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:10.298730  3016 consensus_meta.cc:287] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.641s	user 0.013s	sys 0.002s
I20260709 13:02:10.278475  3010 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:10.209260  3025 consensus_queue.cc:237] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:10.317587  1538 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.977s (SENT)
I20260709 13:02:10.303732  2971 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.308274  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:10.307981  2989 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:02:10.336964  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 48806.4us
I20260709 13:02:10.337970  2989 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:10.338933  1612 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:10.286901  3019 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:10.335124  2991 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:10.346874  3009 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.352291  3010 raft_consensus.cc:993] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b: : Instructing follower 130bc5ff39b847b1b3204626f284f838 to start an election
W20260709 13:02:10.348013  3040 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:02:10.324553  1538 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:10.349303  3027 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:10.319654  3016 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:10.375255  2991 raft_consensus.cc:697] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:10.359997  2965 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:10.361061  3010 raft_consensus.cc:1081] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Signalling peer 130bc5ff39b847b1b3204626f284f838 to start an election
I20260709 13:02:10.363308  3019 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:10.379787  3026 raft_consensus.cc:3055] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:10.380352  3026 raft_consensus.cc:740] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:02:10.363742  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.366961  2989 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:10.337116  2971 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:10.381858  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.400342  3019 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.400938  3019 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:10.401274  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:10.402206  2346 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:10.395320  3027 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.402590  2971 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:10.403009  3019 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:10.376688  2991 consensus_queue.cc:237] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:10.402546  3016 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:10.344746  3003 consensus_meta.cc:287] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.652s	user 0.019s	sys 0.004s
I20260709 13:02:10.341233  1612 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:10.405745  1686 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:02:10.395972  3021 consensus_meta.cc:287] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.600s	user 0.011s	sys 0.004s
W20260709 13:02:10.398810  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.420163  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 106578us
W20260709 13:02:10.397722  3024 consensus_meta.cc:287] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.582s	user 0.009s	sys 0.009s
I20260709 13:02:10.425009  3024 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:10.427821  3024 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:10.432056  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:10.431553  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:10.403545  2947 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:10.441311  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 100222us
I20260709 13:02:10.445088  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:10.445868  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:10.428973  2836 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:10.448385  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:10.483418  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:10.431031  2797 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:10.429432  3021 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:10.403256  3027 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:10.443101  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 65826.1us
I20260709 13:02:10.398859  3026 consensus_queue.cc:260] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:10.448820  3030 consensus_queue.cc:1059] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:02:10.441504  3039 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:10.482575  2836 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.548669  2836 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:10.549715  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.521625  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.522399  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.522806  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.557403  3039 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:10.555649  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:10.561285  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.562507  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.488925  3031 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:10.564842  3039 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:10.544482  3019 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.562574  1571 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:10.361515  3005 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:10.526822  1572 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:10.557159  3030 raft_consensus.cc:3055] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:10.568656  3030 raft_consensus.cc:740] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:10.569655  1567 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:10.526944  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 83512.1us
I20260709 13:02:10.571291  3005 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:10.570838  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:10.578778  1647 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:10.580003  1715 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:10.569041  3000 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:10.533661  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.533977  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.524917  3007 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:10.536414  2797 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:10.540212  1564 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:10.543866  1716 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:10.566563  3019 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:10.585194  1572 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:10.423483  3003 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:10.590337  1538 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.589s (SENT)
I20260709 13:02:10.601184  1538 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:10.589848  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:10.591552  1612 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:10.604969  1640 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:10.597474  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.570328  3031 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:10.580857  3018 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:10.607837  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:10.613421  1714 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:10.584415  3021 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:10.616205  1721 raft_consensus.cc:1240] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:10.616683  1720 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 3. Ops: []
I20260709 13:02:10.615760  3005 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:10.619436  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:10.613562  3031 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:10.612704  3010 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:10.624998  2875 tablet_copy_client.cc:538] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:10.612704  3018 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:10.630976  3018 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:10.632513  2960 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:10.634135  1713 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
W20260709 13:02:10.634829  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.612627  2997 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:10.617434  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:10.360548   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:10.687624  1715 raft_consensus.cc:1240] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
W20260709 13:02:10.642665  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.679186  1468 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38666 (request call id 210) took 2129 ms. Trace:
I20260709 13:02:10.699218  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:10.571750  3030 consensus_queue.cc:260] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:10.638636  2797 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:10.743239  1644 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:10.638087  2312 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:10.655617  3019 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:10.714955  3039 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:10.744892  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 107032us
I20260709 13:02:10.745055  3039 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:10.701112  3026 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:10.701110  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:10.747069  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.705312  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:10.749531  1713 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:10.625239  3010 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:10.708741  2313 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:10.732451  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:10.723230  1469 catalog_manager.cc:5697] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 1 to 2, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:10.736585  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:10.737715  1538 leader_election.cc:336] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.624s (SENT)
I20260709 13:02:10.738440  2964 consensus_queue.cc:1059] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:10.756526  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:10.757022  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 48470.5us
I20260709 13:02:10.757416  1641 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:10.739917  1537 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:10.671195  2873 consensus_queue.cc:1059] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:02:10.742868  3030 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:02:10.659189  3013 raft_consensus.cc:3055] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:10.746577  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:10.700567  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:10.750804  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:10.751542  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:10.800173  1646 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 2: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:10.703692  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:10.629340  2994 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:10.756526  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:10.757596  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.739285  1718 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:10.755656  1538 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:10.759603  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:10.760576  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:10.637136  2906 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:10.740475  1640 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:10.760646  3010 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:10.807833  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:10.808233  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:10.808661  1640 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:10.809202  1644 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:10.714946  3003 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:10.760774  2989 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:10.747283  3039 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:10.641152  3027 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:10.777696  2971 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:10.765357  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:10.767211  2873 raft_consensus.cc:3055] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:10.781072  3049 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:10.761220  1642 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
W20260709 13:02:10.774963  2314 leader_election.cc:336] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.921s (SENT)
I20260709 13:02:10.810248  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:10.810592  1646 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:10.837761  1647 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:10.838835  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:10.839315  1646 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:10.808398  2947 consensus_queue.cc:1059] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:10.814172  1640 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:10.804338  3013 raft_consensus.cc:740] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:10.837260  1687 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:10.757481  2346 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
W20260709 13:02:10.815409  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 77761.3us
I20260709 13:02:10.786201  1641 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:10.789592  2797 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:10.790612  2964 consensus_queue.cc:1059] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:02:10.826690  1639 raft_consensus.cc:1240] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:10.819414  3005 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:10.844808  3049 raft_consensus.cc:697] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:10.785223  2836 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:10.856647  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.868654  1611 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:10.856606  1538 leader_election.cc:336] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.652s (SENT)
I20260709 13:02:10.843379  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:10.716516  1468 rpcz_store.cc:276] 0709 13:02:08.550064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.558010 (+  7946us) service_pool.cc:224] Handling call
0709 13:02:10.679081 (+2121071us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:09.149267 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:09.150029 (+   762us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:09.150267 (+   238us) write_op.cc:435] Acquired schema lock
0709 13:02:09.150294 (+    27us) tablet.cc:662] Decoding operations
0709 13:02:09.150954 (+   660us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:09.151056 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:09.151114 (+    58us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:02:09.151438 (+   324us) tablet.cc:701] Row locks acquired
0709 13:02:09.151493 (+    55us) write_op.cc:260] PREPARE: finished
0709 13:02:09.151748 (+   255us) write_op.cc:270] Start()
0709 13:02:09.151906 (+   158us) write_op.cc:276] Timestamp: P: 1783602129151666 usec, L: 0
0709 13:02:09.151948 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 13:02:09.153479 (+  1531us) log.cc:844] Serialized 598 byte log entry
0709 13:02:10.141994 (+988515us) op_driver.cc:464] REPLICATION: finished
0709 13:02:10.321996 (+180002us) write_op.cc:301] APPLY: starting
0709 13:02:10.322174 (+   178us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:10.322401 (+   227us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:10.322541 (+   140us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:10.322934 (+   393us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:10.323206 (+   272us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:02:10.323246 (+    40us) write_op.cc:312] APPLY: finished
0709 13:02:10.350893 (+ 27647us) log.cc:844] Serialized 22 byte log entry
0709 13:02:10.351293 (+   400us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:10.351414 (+   121us) write_op.cc:454] Released schema lock
0709 13:02:10.352671 (+  1257us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:10.353529 (+   858us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:10.353574 (+    45us) write_op.cc:454] Released schema lock
0709 13:02:10.375922 (+ 22348us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:10.375966 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":43767,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":174847,"apply.run_cpu_time_us":3501,"apply.run_wall_time_us":31254,"num_ops":1,"prepare.queue_time_us":309891,"prepare.run_cpu_time_us":80783,"prepare.run_wall_time_us":220875,"raft.queue_time_us":319303,"raft.run_cpu_time_us":37888,"raft.run_wall_time_us":221057,"replication_time_us":968476,"thread_start_us":689152,"threads_started":4,"wal-append.queue_time_us":202569}]]}
I20260709 13:02:10.871413  1538 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:10.865545  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:10.873351  1642 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:10.835794  2971 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.875479  1647 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:10.878964  2344 raft_consensus.cc:1240] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:10.880728  3031 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:10.882083  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:10.882547  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 45448.4us
I20260709 13:02:10.882759  3003 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:10.907855  3005 raft_consensus.cc:3055] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:10.908435  3005 raft_consensus.cc:740] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:10.886739  1640 raft_consensus.cc:1275] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:10.834496  2873 raft_consensus.cc:740] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:10.910274  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:10.910517  2873 consensus_queue.cc:260] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:10.910835  3049 consensus_queue.cc:237] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:10.887425  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:10.893862  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.895304  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.898072  1646 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:10.824245  3042 consensus_queue.cc:1059] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 13:02:10.914326  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.914764  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.915809  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.914326  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:10.899976  1722 raft_consensus.cc:2368] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 2. Current term is 3.
I20260709 13:02:10.917753  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:10.865209  2997 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:10.915891  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.917384  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:10.918975  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:10.912173  2873 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:10.889878  1644 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:10.883958  2960 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:10.938807  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.919626  1687 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:10.888319  2971 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:10.936661  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:10.941584  1646 raft_consensus.cc:2393] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 2: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:10.942762  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:10.852313  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:10.944159  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.944914  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.947305  1648 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 2: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:10.896355  3007 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:02:10.955546  3058 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:10.949172  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 132326us
I20260709 13:02:10.967087  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.968529  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:10.969975  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 50483.3us
I20260709 13:02:10.975124  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:10.897082  3022 consensus_queue.cc:1059] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:02:10.976665  1566 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:10.980746  3060 raft_consensus.cc:3055] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:11.000039  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:11.004941  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:10.956823  3003 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.987545  1642 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:10.989528  1687 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:11.009929  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 110991us
I20260709 13:02:10.990108  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:11.020129  1644 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 2: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:10.993000  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:11.021695  1567 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:11.010998  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:11.013182  1565 raft_consensus.cc:1275] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:02:10.984500  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 127948us
I20260709 13:02:11.016016  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:11.017591  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:10.918629  3048 consensus_queue.cc:1059] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 13:02:10.939528  2997 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:11.030516  1687 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:10.997349  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.022805  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:11.025928  1568 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:11.025408  1571 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:11.037344  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:11.038213  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:11.043306  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:11.043792  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:11.001035  2797 raft_consensus.cc:697] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:10.998358  3060 raft_consensus.cc:740] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:11.046183  1687 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:11.023422  3027 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.999166  3013 consensus_queue.cc:260] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:11.090703  2345 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:11.059557  2906 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.103065  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:11.126735  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:11.128175  1642 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:11.090950  3062 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:02:11.073071  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:11.124132  2960 raft_consensus.cc:697] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 2 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:02:11.156395  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 97229.8us
I20260709 13:02:11.086659  3047 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.056140  2972 consensus_queue.cc:1059] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
W20260709 13:02:11.165470  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 62512.8us
I20260709 13:02:11.099788  2997 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:11.085451  3031 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:11.161057  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 114908us
W20260709 13:02:11.076853  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 301953us
I20260709 13:02:11.024402  3005 consensus_queue.cc:260] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:11.086635  2836 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:11.172272  1685 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:11.173728  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.175009  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:11.180428  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 144954us
I20260709 13:02:11.092948  3007 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.072s
I20260709 13:02:11.191838  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:11.162039  3033 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:11.196892  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.092442  2989 consensus_queue.cc:1059] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 13:02:11.211440  1612 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:11.085490  3025 consensus_queue.cc:1059] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.087s
I20260709 13:02:11.101147  3018 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:11.084411  3059 consensus_queue.cc:1059] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.116s
I20260709 13:02:11.246258  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:11.092309  2797 consensus_queue.cc:237] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:11.235759  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:11.238230  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.199129  3017 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:11.228286  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:11.209208  2963 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:11.215936  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:11.277843  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 83434.7us
W20260709 13:02:11.281343  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 70030.9us
I20260709 13:02:11.283551  1612 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:11.265022  3017 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.177600  3060 consensus_queue.cc:260] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:11.241088  3003 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:02:11.322001  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 142048us
I20260709 13:02:11.312955  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:11.323380  1642 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 2: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:11.285972  1565 raft_consensus.cc:2955] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
W20260709 13:02:11.262319  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 50210.8us
W20260709 13:02:11.327725  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 44316.7us
I20260709 13:02:11.332759  1612 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:11.270639  2963 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.240036  2960 consensus_queue.cc:237] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:11.341411  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:11.342566  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:11.348297  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:11.356349  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:11.362505  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:11.365577  1719 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:11.315714  3017 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:11.370296  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:11.372234  1714 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 2: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:11.375717  1612 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:11.379500  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:11.320254  3018 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:11.396086  1716 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:42278
I20260709 13:02:11.396941  1716 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:11.397365  1716 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:11.387742  1612 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:11.375684  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:11.343072  2963 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:11.333940  2989 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:02:11.406388  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 107168us
I20260709 13:02:11.391062  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:11.378042  1718 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
W20260709 13:02:11.394170  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 46332.7us
I20260709 13:02:11.331102  3064 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.342015  2346 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:11.401942  1612 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:11.413430  1612 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:11.405712  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:11.375226  3068 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.338873  3003 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:11.426661  1642 raft_consensus.cc:2376] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2. Re-sending same reply.
I20260709 13:02:11.437747  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:11.417484  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.404206  1714 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:11.388119  1719 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:11.338585  3065 raft_consensus.cc:3055] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:11.441983  3065 raft_consensus.cc:740] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:11.443181  3065 consensus_queue.cc:260] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:11.422451  1687 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:11.445495  3065 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:11.457576  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:11.458968  1572 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:02:11.468235  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.403849  2997 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:11.405045  1564 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:11.470643  1721 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
W20260709 13:02:11.472913  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.403132  3007 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:11.405140  3031 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.420964  3018 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.411761  2972 consensus_queue.cc:1059] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.079s
W20260709 13:02:11.537981  3057 log.cc:927] Time spent T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:02:11.538957  3071 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:02:11.515705  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 89369.2us
W20260709 13:02:11.517458  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 89612.3us
I20260709 13:02:11.521967  3074 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:11.407657  3048 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.100s
I20260709 13:02:11.508186  3007 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:11.534094  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:02:11.556455  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 134094us
I20260709 13:02:11.581615  1687 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:11.428390  3059 consensus_queue.cc:1059] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.089s
I20260709 13:02:11.545497  1716 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:11.579238  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 165980us
W20260709 13:02:11.582554  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 71099.6us
I20260709 13:02:11.596992  1687 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:11.541472  2971 consensus_meta.cc:287] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.573s	user 0.011s	sys 0.002s
I20260709 13:02:11.411505  2989 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:11.508438  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:11.603735  1612 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:11.600562  1572 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:11.541199  3069 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:02:11.538998  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 65440us
I20260709 13:02:11.615976  1568 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:11.615849  1716 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:11.629024  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:11.630628  1713 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:02:11.572774  3068 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.551735  3027 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:11.627394  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:11.634178  3027 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.635380  1715 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 2: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:02:11.627514  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.637936  1642 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:11.639212  3007 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:11.605351  3026 consensus_meta.cc:287] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.858s	user 0.017s	sys 0.000s
I20260709 13:02:11.645257  1719 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:11.616022  2875 tablet_bootstrap.cc:492] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
W20260709 13:02:11.650640  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 454422us
I20260709 13:02:11.643652  1645 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:11.604724  2836 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:02:11.666026  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 69118.9us
W20260709 13:02:11.662401  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:11.650633  3030 consensus_meta.cc:287] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.833s	user 0.011s	sys 0.004s
W20260709 13:02:11.691852  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 93087us
I20260709 13:02:11.679494  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:02:11.625677  1612 consensus_peers.cc:597] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82d3b49001741eda872554a7ecb2db3. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:11.644083  2994 consensus_meta.cc:287] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.829s	user 0.012s	sys 0.004s
I20260709 13:02:11.684861  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:11.687178  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:11.645550  3019 raft_consensus.cc:3055] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:11.692798  1538 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:11.580585  3074 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:11.658749  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:11.703719  2349 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:11.698248  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 91539.5us
I20260709 13:02:11.610463  2971 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:11.704571  1687 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:11.704973  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:11.736040  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:11.742933  3073 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 13:02:11.712476  2314 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:11.745455  2989 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:11.748631  3087 consensus_queue.cc:1059] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:11.751938  3080 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:11.760596  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:11.763561  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:11.794983  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 91701.5us
I20260709 13:02:11.795148  2349 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:11.799216  1686 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:11.804761  1646 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:11.716979  1639 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:11.711691  3019 raft_consensus.cc:740] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:02:11.808416  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 91146.9us
I20260709 13:02:11.812065  1612 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:11.816380  3094 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.818274  2963 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:11.818053  3019 consensus_queue.cc:260] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:11.821552  3074 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:11.810680  2965 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.825977  2965 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:11.716113  3058 consensus_queue.cc:1059] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.095s
W20260709 13:02:11.816433  2873 consensus_meta.cc:287] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.876s	user 0.016s	sys 0.004s
I20260709 13:02:11.835748  3005 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:11.766655  2836 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:11.840713  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 155010us
I20260709 13:02:11.803448  3007 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.854513  1716 raft_consensus.cc:1240] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:11.741652  2994 consensus_queue.cc:1059] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 13:02:11.856190  1612 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:11.862524  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 149218us
I20260709 13:02:11.866930  1612 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:11.837937  2989 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:11.817054  3017 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:11.864872  3064 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:11.863389  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 163234us
I20260709 13:02:11.878189  2971 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:11.857466  3093 consensus_queue.cc:1059] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:02:11.873554  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:11.884635  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:11.886415  1565 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:11.887290  1612 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:11.889218  1612 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:11.862929  3003 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:11.894677  2314 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:11.884677  3030 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.896569  2314 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:11.881947  3088 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:11.898759  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1030) took 1065 ms. Trace:
I20260709 13:02:11.889846  3086 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:11.899796  1719 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:11.861761  3091 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:11.900456  1612 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:11.872577  2963 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:11.881904  3094 raft_consensus.cc:3055] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:02:11.900780  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 137322us
I20260709 13:02:11.912952  1538 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:11.900121  1647 rpcz_store.cc:276] 0709 13:02:10.833453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.896981 (+1063528us) service_pool.cc:224] Handling call
0709 13:02:11.897317 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.897502 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.897531 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:11.897576 (+    45us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:11.897607 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.898568 (+   961us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:11.898699 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:11.897886  2873 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.890765  2947 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:11.857906  2836 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
W20260709 13:02:11.904541  2315 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.013s	sys 0.017s
I20260709 13:02:11.897444  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:11.890213  2989 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:11.898913  3065 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:11.921722  3067 raft_consensus.cc:1064] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:02:11.902033  2971 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.935189  1647 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:11.891408  2988 consensus_queue.cc:1059] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:02:11.918085  2994 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.909554  3094 raft_consensus.cc:740] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:11.943856  1716 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:11.909883  3030 consensus_queue.cc:1059] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:02:11.948065  3074 raft_consensus.cc:3055] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:11.936959  2346 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:11.919893  2947 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:11.952427  3039 raft_consensus.cc:3055] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:02:11.966764  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 93627us
I20260709 13:02:11.968502  1687 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:11.929421  2315 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.014s	sys 0.018s
I20260709 13:02:11.971690  3030 raft_consensus.cc:3055] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:11.972190  3030 raft_consensus.cc:740] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:11.972949  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:11.937088  3086 raft_consensus.cc:697] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:11.965380  3058 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.974620  3086 consensus_queue.cc:237] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:11.976076  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:11.932144  3097 consensus_queue.cc:1059] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 13:02:11.961833  3017 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:11.962749  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 57) took 1103 ms. Trace:
I20260709 13:02:11.938839  1612 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:11.982935  1647 rpcz_store.cc:276] 0709 13:02:10.859805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.934667 (+1074862us) service_pool.cc:224] Handling call
0709 13:02:11.934963 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.962377 (+ 27414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.962671 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:11.983572  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 44892.1us
I20260709 13:02:11.964900  1565 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:11.954432  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:11.967509  3039 raft_consensus.cc:740] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:11.979642  2345 raft_consensus.cc:2368] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3.
I20260709 13:02:11.963533  3074 raft_consensus.cc:740] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:11.978161  3094 consensus_queue.cc:260] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:11.985064  1612 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:11.988828  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 76001.7us
I20260709 13:02:11.976810  1469 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38650 (request call id 194) took 3166 ms. Trace:
I20260709 13:02:11.991454  3080 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.990175  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:11.992404  2836 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:11.969108  2947 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:11.996186  1687 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:11.996388  1612 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:02:12.004575  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 71469.5us
I20260709 13:02:12.002612  1538 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:11.986063  3017 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:12.009869  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:12.011188  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:11.987251  2989 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:12.010649  1568 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:11.996840  3080 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:12.018608  1687 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:12.018659  1612 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:11.973075  3030 consensus_queue.cc:260] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:12.020264  1538 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:12.025274  2888 tablet_bootstrap.cc:492] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:12.011495  3094 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:12.004153  2836 consensus_queue.cc:1059] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:02:12.025790  3099 consensus_queue.cc:1059] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.105s
I20260709 13:02:12.026695  3069 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:12.017525  3068 consensus_queue.cc:1059] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:02:12.006785  3065 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:12.009053  3033 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:12.057466  1721 raft_consensus.cc:1240] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:12.058383  3033 raft_consensus.cc:697] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:12.020073  3017 raft_consensus.cc:697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:12.059449  3033 consensus_queue.cc:237] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.030611  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:12.031860  1640 raft_consensus.cc:3055] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:12.063650  1640 raft_consensus.cc:740] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:12.051579  1538 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:11.990409  1469 rpcz_store.cc:276] 0709 13:02:08.810625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.823960 (+ 13335us) service_pool.cc:224] Handling call
0709 13:02:11.976738 (+3152778us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:10.772182 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:10.772858 (+   676us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:10.772909 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:02:10.772931 (+    22us) tablet.cc:662] Decoding operations
0709 13:02:10.773336 (+   405us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:10.773410 (+    74us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:10.773451 (+    41us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:02:10.773704 (+   253us) tablet.cc:701] Row locks acquired
0709 13:02:10.773728 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:02:10.773878 (+   150us) write_op.cc:270] Start()
0709 13:02:10.774006 (+   128us) write_op.cc:276] Timestamp: P: 1783602130773827 usec, L: 0
0709 13:02:10.774027 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:02:10.774812 (+   785us) log.cc:844] Serialized 541 byte log entry
0709 13:02:10.777571 (+  2759us) op_driver.cc:464] REPLICATION: finished
0709 13:02:10.778399 (+   828us) write_op.cc:301] APPLY: starting
0709 13:02:10.778611 (+   212us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:10.778712 (+   101us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:10.778824 (+   112us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:10.779644 (+   820us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:10.779815 (+   171us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:02:10.779856 (+    41us) write_op.cc:312] APPLY: finished
0709 13:02:10.780053 (+   197us) log.cc:844] Serialized 23 byte log entry
0709 13:02:10.780230 (+   177us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:10.780311 (+    81us) write_op.cc:454] Released schema lock
0709 13:02:10.780356 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:10.780842 (+   486us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:10.780879 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":37556,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":268,"apply.run_cpu_time_us":2284,"apply.run_wall_time_us":2283,"num_ops":1,"prepare.queue_time_us":3190,"prepare.run_cpu_time_us":2973,"prepare.run_wall_time_us":2972,"replication_time_us":2600}]]}
I20260709 13:02:12.036038  1572 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:12.040120  3083 consensus_queue.cc:1059] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:02:12.065997  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:12.066593  1572 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:12.048327  3100 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:02:12.042160  2963 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:12.049342  3065 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:12.066562  1538 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:12.048943  2836 raft_consensus.cc:3055] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:12.043712  3069 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:12.074949  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:12.077538  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1599) took 1175 ms. Trace:
I20260709 13:02:12.071650  1685 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:12.084313  3042 raft_consensus.cc:3055] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:12.090385  3042 raft_consensus.cc:740] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:12.091547  2888 tablet_bootstrap.cc:492] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:02:12.087626  1646 rpcz_store.cc:276] 0709 13:02:10.902725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.942444 (+ 39719us) service_pool.cc:224] Handling call
0709 13:02:12.077263 (+1134819us) spinlock_profiling.cc:243] Waited 753 ms on lock 0x7b5400098654. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:12.077424 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1732627328}
I20260709 13:02:12.087265  3065 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:12.092753  2888 ts_tablet_manager.cc:1403] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 2.548s	user 0.045s	sys 0.042s
I20260709 13:02:12.090215  3098 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:12.087072  2836 raft_consensus.cc:740] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:12.091548  3042 consensus_queue.cc:260] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.095095  3042 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:12.090253  3083 consensus_queue.cc:1243] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:02:12.054904  3068 raft_consensus.cc:3055] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:12.029054  3101 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:12.068735  3097 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:12.071621  2971 consensus_queue.cc:1059] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
W20260709 13:02:12.111295  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 92753.1us
I20260709 13:02:12.075310  3030 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:12.044464  3039 consensus_queue.cc:260] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.088928  3091 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:12.112942  1687 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:12.101845  2963 raft_consensus.cc:697] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:12.091960  3069 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:12.105151  2875 tablet_bootstrap.cc:700] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Time spent opening tablet: real 0.140s	user 0.002s	sys 0.000s
I20260709 13:02:12.105314  3099 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:12.135283  1687 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:12.103356  3068 raft_consensus.cc:740] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:12.137866  1685 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:12.135802  3066 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.709s	user 0.004s	sys 0.001s
I20260709 13:02:12.059468  3017 consensus_queue.cc:237] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:12.144567  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.124526  3098 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:12.149070  3085 raft_consensus.cc:3055] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:12.049751  3074 consensus_queue.cc:260] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:12.158035  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.167163  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.168291  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.152675  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1042) took 1184 ms. Trace:
I20260709 13:02:12.136222  3087 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:12.109007  3084 consensus_queue.cc:1059] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.133s
I20260709 13:02:12.174392  3074 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:12.173254  1570 rpcz_store.cc:276] 0709 13:02:10.967848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.968002 (+   154us) service_pool.cc:224] Handling call
0709 13:02:10.977481 (+  9479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:10.977855 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:10.977884 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.002427 (+ 24543us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.163084 (+160657us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:11.163147 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.163495 (+   348us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.104471 (+940976us) spinlock_profiling.cc:243] Waited 2.51 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:02:12.151429 (+ 46958us) raft_consensus.cc:1685] finished
0709 13:02:12.151473 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.152459 (+   986us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3871,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":27606,"spinlock_wait_cycles":5778816,"thread_start_us":119340,"threads_started":1,"wal-append.queue_time_us":295922}
I20260709 13:02:12.095275  2836 consensus_queue.cc:260] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.185105  2836 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:12.139895  3099 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:02:12.192616  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 126142us
I20260709 13:02:12.203289  1538 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:12.205113  1537 leader_election.cc:400] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3.
I20260709 13:02:12.242206  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:12.243116  1537 leader_election.cc:403] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:12.243494  1570 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:12.243144  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 83615.9us
W20260709 13:02:12.244292  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 63943.4us
I20260709 13:02:12.139382  2972 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:02:12.157226  3088 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:12.176259  3039 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:12.171579  3100 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:12.177815  3105 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:12.205587  3099 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:12.202613  3048 raft_consensus.cc:3055] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:12.190472  3085 raft_consensus.cc:740] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:12.250720  1538 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:12.239938  3097 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:12.278532  1538 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:12.256762  3098 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3.
I20260709 13:02:12.166980  2963 consensus_queue.cc:237] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.256840  2994 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:12.286656  3068 consensus_queue.cc:260] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.257791  3033 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:12.256834  3084 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:12.266973  3048 raft_consensus.cc:740] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:12.289222  3068 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:12.259148  3105 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.282395  3005 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:12.261420  3087 raft_consensus.cc:697] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:12.315467  1570 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:12.298187  2952 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.301702  3088 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:12.319418  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:12.291303  2994 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.311378  3105 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:12.336159  1570 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:12.305943  3033 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:12.318549  3085 consensus_queue.cc:260] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.309968  3084 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.322247  2952 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.334617  2346 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:12.363171  2972 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:12.364006  2972 raft_consensus.cc:697] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:12.364464  3033 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:12.362528  3087 consensus_queue.cc:237] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.364970  2972 consensus_queue.cc:237] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:12.368002  2952 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:12.378031  2994 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:12.378917  3111 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.088s	user 0.000s	sys 0.003s
I20260709 13:02:12.337740  2947 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:12.371206  3085 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:12.382824  1572 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:12.398404  3075 raft_consensus.cc:3055] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:12.399132  3075 raft_consensus.cc:740] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:12.369179  1569 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:12.407146  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:12.401405  3013 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:12.415421  3016 raft_consensus.cc:2955] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:02:12.421710  3064 raft_consensus.cc:3055] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:12.433305  1572 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:12.409890  3100 raft_consensus.cc:697] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:12.369267  2888 raft_consensus.cc:359] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
W20260709 13:02:12.436986  1722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 57) took 2002 ms (client timeout 2137 ms). Trace:
I20260709 13:02:12.414146  3005 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:12.443643  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 165226us
I20260709 13:02:12.445010  1538 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:12.446209  1538 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:12.451841  1538 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:12.439193  1722 rpcz_store.cc:269] 0709 13:02:10.430652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.431016 (+   364us) service_pool.cc:224] Handling call
0709 13:02:12.433147 (+2002131us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b54000230d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:12.433349 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1072628480}
I20260709 13:02:12.463357  1538 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:12.421820  3084 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:12.433889  3064 raft_consensus.cc:740] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:12.468274  1538 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:12.478837  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:12.480361  1722 raft_consensus.cc:2368] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3.
I20260709 13:02:12.480623  1538 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:12.481988  1538 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:12.370889  3048 consensus_queue.cc:260] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:12.462685  3098 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:12.465487  3101 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:12.400338  3075 consensus_queue.cc:260] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.462320  3097 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:12.439105  2888 raft_consensus.cc:740] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:02:12.487449  1538 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:12.471872  3005 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:12.498832  1537 leader_election.cc:400] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3.
I20260709 13:02:12.499403  1537 leader_election.cc:403] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 13:02:12.503222  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 185367us
I20260709 13:02:12.488734  2964 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:12.513376  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1019) took 1767 ms (client timeout 1901 ms). Trace:
I20260709 13:02:12.497192  3048 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:12.514936  1648 rpcz_store.cc:269] 0709 13:02:10.746102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.764613 (+ 18511us) service_pool.cc:224] Handling call
0709 13:02:12.512895 (+1748282us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54002ab2d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:12.513290 (+   395us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":276166912}
I20260709 13:02:12.495092  2972 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:12.480294  3107 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:12.502286  3098 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:12.504180  3075 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:12.506676  3101 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:12.558686  1722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:02:12.559708  1722 raft_consensus.cc:606] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:12.560169  1719 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:12.560883  1565 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:12.553956  3116 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:12.563819  1721 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:12.563511  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:12.565945  1719 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:02:12.565868  1687 leader_election.cc:336] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.731s (SENT)
I20260709 13:02:12.550124  2972 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.575091  3109 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:12.573196  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:12.572288  1687 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:12.547509  3065 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:12.580623  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.585530  1468 catalog_manager.cc:5697] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 1 to 2, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:12.586686  3069 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.588841  3069 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:12.535646  2888 consensus_queue.cc:260] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:12.591660  2888 ts_tablet_manager.cc:1434] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.498s	user 0.017s	sys 0.016s
I20260709 13:02:12.595498  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:12.598065  2994 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.598445  1641 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:12.598654  2994 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:12.600018  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 96913.1us
W20260709 13:02:12.600965  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.597358  1644 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:12.604553  2313 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:12.608516  1719 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:12.610458  1583 tablet_copy_service.cc:342] P 066ee0109aba4465ba17fa65e14ef504: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-59d236b930214b1f91fb55ece6fecd68 received from {username='slave'} at 127.0.0.1:46982
I20260709 13:02:12.611291  1583 tablet_copy_service.cc:434] P 066ee0109aba4465ba17fa65e14ef504: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-59d236b930214b1f91fb55ece6fecd68 on tablet 59d236b930214b1f91fb55ece6fecd68 with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:12.599819  3025 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:12.615033  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:12.615139  2971 consensus_queue.cc:1059] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:12.599386  3080 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:12.603837  2952 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:02:12.620313  1645 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1000) took 2098 ms (client timeout 1731 ms). Trace:
I20260709 13:02:12.616235  3091 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
W20260709 13:02:12.634112  1645 rpcz_store.cc:269] 0709 13:02:10.522313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.522446 (+   133us) service_pool.cc:224] Handling call
0709 13:02:12.620130 (+2097684us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54004114d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:12.620235 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2487908736}
I20260709 13:02:12.622633  3099 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.623508  3117 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3.
I20260709 13:02:12.623618  2906 raft_consensus.cc:3055] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:12.624773  3065 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:12.625397  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.627945  1648 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
mode: GRACEFUL
new_leader_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:49580
I20260709 13:02:12.618863  2345 raft_consensus.cc:1240] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:12.571957  3120 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:12.603960  3084 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:12.655676  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.615566  2994 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:12.609385  3042 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:12.533708  3064 consensus_queue.cc:260] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.656720  2952 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.687556  1643 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:12.578353  3116 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.576125  3109 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.667420  2906 raft_consensus.cc:740] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:02:12.682749  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1615) took 1654 ms (client timeout 2156 ms). Trace:
I20260709 13:02:12.679759  3042 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.632544  3083 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:12.657371  3080 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.672151  2971 consensus_queue.cc:1243] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:12.681452  1640 consensus_queue.cc:260] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.701584  1640 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:12.674618  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 55265.5us
W20260709 13:02:12.696220  1642 rpcz_store.cc:269] 0709 13:02:11.028303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.437247 (+408944us) service_pool.cc:224] Handling call
0709 13:02:12.682518 (+1245271us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54001fefd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:12.682677 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":457773824}
W20260709 13:02:12.705555  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.707921  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1327) took 1281 ms. Trace:
I20260709 13:02:12.713974  1718 rpcz_store.cc:276] 0709 13:02:11.422603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.422754 (+   151us) service_pool.cc:224] Handling call
0709 13:02:11.423366 (+   612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.624180 (+200814us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.624229 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:11.624292 (+    63us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:11.624337 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.703897 (+1079560us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b5400249354. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:12.703948 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.704069 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":461164416}
I20260709 13:02:12.696249  2972 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:12.698123  3065 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:12.683350  3084 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.708415  3064 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:12.706707  2971 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:12.601593  3100 consensus_queue.cc:237] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.731065  3104 raft_consensus.cc:3055] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:12.733040  2873 raft_consensus.cc:3055] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:12.720566  3010 raft_consensus.cc:3055] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:02:12.741271  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.697345  3083 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:12.744385  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.745889  3058 raft_consensus.cc:3055] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:12.756093  3058 raft_consensus.cc:740] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:12.743145  3010 raft_consensus.cc:740] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:12.748752  1717 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:12.758023  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1325) took 1396 ms. Trace:
W20260709 13:02:12.749902  2348 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55888 (request call id 186) took 2046 ms (client timeout 1735 ms). Trace:
I20260709 13:02:12.739332  2873 raft_consensus.cc:740] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:12.750628  2906 consensus_queue.cc:260] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.678596  3120 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:12.759133  2348 rpcz_store.cc:269] 0709 13:02:10.703095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.703349 (+   254us) service_pool.cc:224] Handling call
0709 13:02:12.749677 (+2046328us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b5400600254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:12.749818 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1052220160}
I20260709 13:02:12.760479  3010 consensus_queue.cc:260] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.754356  3003 raft_consensus.cc:3055] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:12.768553  1640 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:12.758394  1717 rpcz_store.cc:276] 0709 13:02:11.361786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.411770 (+ 49984us) service_pool.cc:224] Handling call
0709 13:02:12.748343 (+1336573us) spinlock_profiling.cc:243] Waited 3.81 ms on lock 0x7b580009db98. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:02:12.748555 (+   212us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.757745 (+  9190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.757963 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8769792}
I20260709 13:02:12.718178  3121 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.773660  1648 raft_consensus.cc:606] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Received request to transfer leadership to TS 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:12.779855  1648 raft_consensus.cc:614] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:12.774268  1640 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1600) took 1871 ms (client timeout 1687 ms). Trace:
I20260709 13:02:12.775465  2873 consensus_queue.cc:260] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:12.781457  2873 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:12.782017  2906 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:12.783461  2952 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:12.770268  3116 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:12.787459  3042 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:12.738059  3104 raft_consensus.cc:740] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:12.778265  3003 raft_consensus.cc:740] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:12.727262  2312 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:12.737304  3065 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.734694  3016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:44408 (request call id 7) took 4450 ms. Trace:
W20260709 13:02:12.788862  1640 rpcz_store.cc:269] 0709 13:02:10.903493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.909975 (+  6482us) service_pool.cc:224] Handling call
0709 13:02:12.031649 (+1121674us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54002772a0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248dfcdc0 00007f8248e88e18 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6
0709 13:02:12.681031 (+649382us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54002772a0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e0306f 00007f8248e7bf60 00007f8248e7d7de 00007f8248e8927c 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2
0709 13:02:12.774166 (+ 93135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":510865664}
I20260709 13:02:12.772725  3005 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:12.791723  3020 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:02:12.788473  2965 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:12.760491  3058 consensus_queue.cc:260] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.804374  3058 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:12.814738  1571 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:12.815234  1570 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:12.816453  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:12.817490  1571 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:12.824366  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:12.821444  3099 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:12.836431  1646 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:12.821060  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:12.838440  1564 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:12.840402  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:12.843526  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:12.846043  3080 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:12.791143  3104 consensus_queue.cc:260] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:12.821789  2965 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.814451  2971 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:02:12.847743  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.810526  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 89195.8us
I20260709 13:02:12.797537  3003 consensus_queue.cc:260] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.838445  3120 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:12.834419  3048 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.864672  3048 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:12.850769  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 246355us
I20260709 13:02:12.864750  2971 raft_consensus.cc:697] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:12.853741  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:12.828352  3083 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:12.883049  3030 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.885133  1571 raft_consensus.cc:2410] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:12.892462  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:12.895262  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:12.896508  3120 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:12.798727  3121 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:12.865180  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:12.877923  3104 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:12.906508  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:12.890710  3003 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:12.909670  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:02:12.898545  3088 consensus_meta.cc:287] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.575s	user 0.011s	sys 0.003s
I20260709 13:02:12.860203  2965 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:12.890383  3069 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:12.903512  3068 consensus_meta.cc:287] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.592s	user 0.009s	sys 0.008s
I20260709 13:02:12.797761  3016 rpcz_store.cc:276] 0709 13:02:08.274884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.277796 (+  2912us) service_pool.cc:224] Handling call
0709 13:02:08.487951 (+210155us) log.cc:844] Serialized 506 byte log entry
0709 13:02:08.616896 (+128945us) spinlock_profiling.cc:243] Waited 4.24 ms on lock 0x7b1800004a80. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d12c0a 00007f824f452219 00007f8245d9aa40 00007f8245d954e4 00007f8245d95455 00007f8248ddeb0b 00007f8248e63b63 00007f8248e8b752 00007f824de4bd29
0709 13:02:12.722949 (+4106053us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":178478,"raft.queue_time_us":195800,"raft.run_cpu_time_us":3662,"raft.run_wall_time_us":209091,"spinlock_wait_cycles":9873536,"thread_start_us":24510,"threads_started":1,"wal-append.queue_time_us":70876,"wal-append.run_cpu_time_us":2751,"wal-append.run_wall_time_us":1057993}
I20260709 13:02:12.904368  3027 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:02:12.892452  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 261606us
I20260709 13:02:12.919401  3084 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:12.939314  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.897253  3030 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.928653  3088 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:12.909050  3120 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:12.943537  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.957705  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 452222us
I20260709 13:02:12.958509  1567 raft_consensus.cc:1275] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:02:12.960562  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.940587  2873 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:12.964613  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.965462  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.969343  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.929960  3069 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:12.982182  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.943040  3027 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:12.967142  3098 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:12.939153  2971 consensus_queue.cc:237] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:12.999929  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.008674  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:13.009560  1571 raft_consensus.cc:2393] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 2: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:12.980286  3095 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:13.021344  2348 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:13.009553  1686 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:12.984128  3025 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:02:13.030154  1538 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.028s	sys 0.067s
W20260709 13:02:13.030833  1538 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.029s	sys 0.067s
I20260709 13:02:12.974107  2873 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:13.029722  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:13.032866  3075 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:13.032924  1714 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 2: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:13.033797  3027 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:13.035569  3121 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:13.035892  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 548574us
I20260709 13:02:13.036906  1538 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:12.999008  2965 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:13.038702  1538 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:13.041203  3078 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:13.042146  3078 raft_consensus.cc:697] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:13.045969  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:13.045917  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:13.046819  1538 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:13.049580  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:13.053822  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:13.055069  1715 raft_consensus.cc:2410] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:13.054423  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:13.059319  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:13.061728  1538 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:13.063043  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:13.063941  1720 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:02:13.058745  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:13.047744  3074 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.016337  3084 consensus_queue.cc:1059] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:02:13.068437  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:13.070521  1719 raft_consensus.cc:3055] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:13.030067  3098 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:13.074282  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:13.077368  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:13.077623  1538 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:13.078181  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:13.044013  3042 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:13.084452  1612 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:13.080626  1719 raft_consensus.cc:740] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:13.083467  2348 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:13.093501  1644 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:13.095166  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:13.044395  3048 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:13.075809  3094 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:13.097481  1719 consensus_queue.cc:260] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:13.096964  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:13.097285  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1076) took 1055 ms. Trace:
I20260709 13:02:13.099074  1644 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:13.101399  1640 raft_consensus.cc:2393] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 2: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:13.030146  3095 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:13.104501  3122 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:13.106000  1641 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:13.109295  3101 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:13.098508  3117 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:13.100222  3080 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:13.112460  3120 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:13.114269  3094 raft_consensus.cc:697] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:13.075889  3081 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:13.039808  3025 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:13.118408  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:13.138646  1640 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:13.101477  1565 rpcz_store.cc:276] 0709 13:02:12.030099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.030229 (+   130us) service_pool.cc:224] Handling call
0709 13:02:13.084246 (+1054017us) spinlock_profiling.cc:243] Waited 498 ms on lock 0x7b54002b9654. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:13.084336 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1144545024}
I20260709 13:02:13.122066  1644 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:13.139617  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:13.139986  1640 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:13.110792  1642 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:13.016291  3128 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:02:13.132589  1719 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:13.142663  1641 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:13.135753  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:13.145394  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.151769  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:13.154299  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:13.148867  2965 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.134145  3101 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:13.161506  2965 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:13.128321  3080 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.164464  3080 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:13.163981  2965 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:13.172104  3025 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:13.131870  3075 raft_consensus.cc:697] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:13.174647  3030 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:13.175315  3075 consensus_queue.cc:237] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:13.175948  3069 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:13.082778  3074 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:13.098521  3078 consensus_queue.cc:237] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:13.184811  3094 consensus_queue.cc:237] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:13.205533  1719 raft_consensus.cc:1275] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:13.197799  1565 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
W20260709 13:02:13.218899  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:13.383610  1685 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.755s (SENT) [suppressed 43 similar messages]
W20260709 13:02:14.140615  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 757039us
W20260709 13:02:13.242885  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:14.144719  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:14.145305  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.242183  1718 raft_consensus.cc:2955] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:02:13.226805  1722 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:13.428120  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:13.261871  2875 tablet_bootstrap.cc:492] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:13.218926  3025 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:14.183921  3025 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:13.317166  3060 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:13.282271  3064 consensus_meta.cc:287] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.544s	user 0.001s	sys 0.015s
I20260709 13:02:13.291476  3074 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:13.304661  3080 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.197268  2875 tablet_bootstrap.cc:492] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:02:14.198176  3080 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:13.287910  3085 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.906s	user 0.013s	sys 0.006s
I20260709 13:02:13.370018  3084 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.195699  3025 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:14.200937  3084 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.203815  3084 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:13.345111  3093 raft_consensus.cc:3055] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:14.205541  3093 raft_consensus.cc:740] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:14.206813  3093 consensus_queue.cc:260] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.208357  3093 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:13.265865  3095 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:13.096197  3098 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:14.212762  3159 ts_tablet_manager.cc:933] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:13.164019  3101 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:13.266516  3105 consensus_meta.cc:287] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.925s	user 0.012s	sys 0.008s
I20260709 13:02:13.157224  3113 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.137653  3117 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:13.136936  3122 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:13.593035  1564 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:13.646677  3131 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:13.235488  1611 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:13.674614  3096 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:13.681335  2947 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:13.696920  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:13.707000  1468 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47518 (request call id 90) took 2511 ms. Trace:
I20260709 13:02:13.723707  3067 consensus_queue.cc:237] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:13.725368  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1029) took 2902 ms. Trace:
I20260709 13:02:13.742172  3073 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:13.748642  3016 raft_consensus.cc:3055] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:13.754662  3051 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:13.763913  2873 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:13.769246  1569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1055) took 2755 ms (client timeout 1678 ms). Trace:
W20260709 13:02:13.773593  3090 log.cc:927] Time spent T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.889s	user 0.014s	sys 0.005s
I20260709 13:02:14.213927  3013 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.837534  3068 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.228729  3013 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.229517  3068 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.217155  3105 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:14.232208  3105 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:13.837778  2994 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:13.862407  1572 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1077) took 1827 ms (client timeout 2045 ms). Trace:
I20260709 13:02:13.870510  3061 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:13.917193  3069 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:14.219851  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:14.234613  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.234508  3069 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:13.925704  3099 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:13.928386  3018 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:14.235522  3013 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.213539  3098 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:13.934403  3152 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.942644  3020 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.221140  3117 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.238049  3020 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:14.222222  3122 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.240893  3020 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:13.944557  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:14.235780  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1332) took 1205 ms. Trace:
W20260709 13:02:14.223292  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.952284  3030 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.958559  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:13.975335  3039 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:13.977742  2314 leader_election.cc:336] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.078s (SENT)
W20260709 13:02:14.225528  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 990211us
I20260709 13:02:14.023470  3109 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:14.027225  1537 consensus_peers.cc:597] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:14.243848  2314 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:14.244544  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 217797us
I20260709 13:02:14.245054  3152 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:14.245419  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 267779us
I20260709 13:02:14.247701  1611 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:14.048806  3091 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.247961  3152 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.040534  3153 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:14.222820  3131 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:14.029238  3115 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.225925  3096 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:14.247843  2314 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:14.251907  3131 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:14.252908  3096 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:14.055989  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.255129  1721 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.254369  1714 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:14.233599  1572 rpcz_store.cc:269] 0709 13:02:12.034618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.074414 (+ 39796us) service_pool.cc:224] Handling call
0709 13:02:13.862192 (+1787778us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b54002251d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:13.862338 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":679397120}
I20260709 13:02:14.054366  1568 raft_consensus.cc:1240] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:14.242091  2348 raft_consensus.cc:2368] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3.
I20260709 13:02:14.234045  3061 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.258371  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1075) took 2238 ms. Trace:
I20260709 13:02:14.227542  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1341) took 1174 ms. Trace:
I20260709 13:02:14.258817  1568 rpcz_store.cc:276] 0709 13:02:12.019602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.019894 (+   292us) service_pool.cc:224] Handling call
0709 13:02:12.590706 (+570812us) spinlock_profiling.cc:243] Waited 7.78 ms on lock 0x7b5800011198. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:02:12.590920 (+   214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.257484 (+1666564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.258097 (+   613us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54000f73d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:14.258308 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3382637824}
I20260709 13:02:14.259850  3061 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.065642  2344 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:02:14.243599  3039 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.223066  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1090) took 1241 ms. Trace:
I20260709 13:02:14.233273  2994 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.073529  2347 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:14.263855  1573 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.078913  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1084) took 1476 ms. Trace:
I20260709 13:02:14.243088  3069 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:14.100122  2746 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
W20260709 13:02:14.097335  1639 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 54) took 3245 ms (client timeout 1624 ms). Trace:
I20260709 13:02:14.241497  3020 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:02:14.266299  1639 rpcz_store.cc:269] 0709 13:02:10.852257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.861472 (+  9215us) service_pool.cc:224] Handling call
0709 13:02:14.097048 (+3235576us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b54002db754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:14.097233 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1873067904}
I20260709 13:02:14.120193  3104 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:14.121354  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:14.127557  3103 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:14.227409  3016 raft_consensus.cc:740] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:02:14.138725  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.218560  3113 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:14.271476  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.226738  1468 rpcz_store.cc:276] 0709 13:02:11.196210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.196422 (+   212us) service_pool.cc:224] Handling call
0709 13:02:13.706874 (+2510452us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:12.704496 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:12.833938 (+129442us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:12.848305 (+ 14367us) write_op.cc:435] Acquired schema lock
0709 13:02:12.850292 (+  1987us) tablet.cc:662] Decoding operations
0709 13:02:12.909938 (+ 59646us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:12.919291 (+  9353us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:12.926438 (+  7147us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:02:12.938708 (+ 12270us) tablet.cc:701] Row locks acquired
0709 13:02:12.941840 (+  3132us) write_op.cc:260] PREPARE: finished
0709 13:02:12.956632 (+ 14792us) write_op.cc:270] Start()
0709 13:02:12.966150 (+  9518us) write_op.cc:276] Timestamp: P: 1783602132950257 usec, L: 0
0709 13:02:12.968858 (+  2708us) op_driver.cc:348] REPLICATION: starting
0709 13:02:13.064011 (+ 95153us) log.cc:844] Serialized 536 byte log entry
0709 13:02:13.289240 (+225229us) op_driver.cc:464] REPLICATION: finished
0709 13:02:13.324042 (+ 34802us) write_op.cc:301] APPLY: starting
0709 13:02:13.324383 (+   341us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:13.324549 (+   166us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:13.333972 (+  9423us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:13.334372 (+   400us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:13.334571 (+   199us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:02:13.334636 (+    65us) write_op.cc:312] APPLY: finished
0709 13:02:13.334975 (+   339us) log.cc:844] Serialized 23 byte log entry
0709 13:02:13.335199 (+   224us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:13.335298 (+    99us) write_op.cc:454] Released schema lock
0709 13:02:13.335345 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:13.335768 (+   423us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:13.335799 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":29273,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":34401,"apply.run_cpu_time_us":2419,"apply.run_wall_time_us":11738,"num_ops":1,"prepare.queue_time_us":45029,"prepare.run_cpu_time_us":54355,"prepare.run_wall_time_us":550419,"raft.queue_time_us":49582,"raft.run_cpu_time_us":6102,"raft.run_wall_time_us":18802,"replication_time_us":320185,"thread_start_us":207087,"threads_started":4,"wal-append.queue_time_us":165232}]]}
I20260709 13:02:14.271314  3099 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:14.152782  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.232422  3068 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.154124  2988 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.310093  3084 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.310148  3101 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.318219  2988 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.318998  3101 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:14.319710  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.322108  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:14.323778  3093 raft_consensus.cc:993] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838: : Instructing follower 066ee0109aba4465ba17fa65e14ef504 to start an election
I20260709 13:02:14.324280  3093 raft_consensus.cc:1081] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Signalling peer 066ee0109aba4465ba17fa65e14ef504 to start an election
I20260709 13:02:14.154955  3075 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:14.263888  3039 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.325615  3075 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.157393  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:14.317337  2746 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.326429  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:14.326813  3122 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.327082  2746 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.160466  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.164731   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:02:14.199244  2875 ts_tablet_manager.cc:1403] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 2.622s	user 0.034s	sys 0.046s
I20260709 13:02:14.328377  1713 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:14.329722  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.330104  2746 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:14.330507   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:14.331003  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.162909  1719 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:14.236433  3018 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.163833  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.331859  2875 raft_consensus.cc:359] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:14.179802  3010 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.333868  3018 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:13.929188  2906 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.334713  2906 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.227289  3073 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.340124  3073 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.036111  3019 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.343364  1685 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:14.321305  3101 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:14.321223  2988 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:14.248489  3091 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:14.348807  3075 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:14.049420  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:14.251579  3115 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.256913  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:14.243224  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:14.222419  1571 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:14.056746  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:14.058889  2346 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:14.227608  3051 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:14.098534  1647 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:14.268144  3104 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:14.261976  2344 pending_rounds.cc:85] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Aborting all ops after (but not including) 4
I20260709 13:02:14.149591  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:14.140623  3065 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.265300  2994 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.351255  3095 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:14.272605  3113 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:14.243199  3030 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:14.227963  1716 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:14.258834  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:14.265843  1570 rpcz_store.cc:276] 0709 13:02:12.602523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.602756 (+   233us) service_pool.cc:224] Handling call
0709 13:02:12.604523 (+  1767us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.816010 (+211487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.078565 (+1262555us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b5400012554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:14.078819 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":484076800}
I20260709 13:02:14.226487  1566 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:14.266028  2873 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:14.259967  1722 rpcz_store.cc:276] 0709 13:02:13.053560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.053788 (+   228us) service_pool.cc:224] Handling call
0709 13:02:13.077564 (+ 23776us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.179067 (+1101503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.227270 (+ 48203us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54002c60d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:14.227484 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":342681856}
I20260709 13:02:14.280241  3096 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.280570  3098 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:14.404495  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:14.281016  3013 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.261094  3129 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.281819  3117 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.410296  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:14.410230  3117 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:14.412837  1642 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:14.282375  3061 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:14.413758  3098 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.414448  3061 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.243569  1648 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:14.269755  3103 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:14.227944  1569 rpcz_store.cc:269] 0709 13:02:11.013588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.016349 (+  2761us) service_pool.cc:224] Handling call
0709 13:02:13.768854 (+2752505us) spinlock_profiling.cc:243] Waited 976 ms on lock 0x7b5400306954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:13.769119 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2245884672}
I20260709 13:02:14.283478  3131 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:14.281073  1538 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:14.242424  1718 rpcz_store.cc:276] 0709 13:02:13.030662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.030926 (+   264us) service_pool.cc:224] Handling call
0709 13:02:13.031589 (+   663us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:13.165778 (+134189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.165817 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:13.206131 (+ 40314us) log.cc:844] Serialized 506 byte log entry
0709 13:02:13.241964 (+ 35833us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:14.229120 (+987156us) log.cc:844] Serialized 14 byte log entry
0709 13:02:14.234593 (+  5473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.235367 (+   774us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b5400053f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:14.235419 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:14.235466 (+    47us) raft_consensus.cc:1685] finished
0709 13:02:14.235554 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.235718 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7337,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":112,"spinlock_wait_cycles":307541376,"thread_start_us":4013,"threads_started":1,"wal-append.queue_time_us":12390}
I20260709 13:02:14.227062  1643 rpcz_store.cc:276] 0709 13:02:10.823530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.702652 (+879122us) service_pool.cc:224] Handling call
0709 13:02:12.683363 (+980711us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.688910 (+  5547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.724989 (+1036079us) spinlock_profiling.cc:243] Waited 937 ms on lock 0x7b54001fefd0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:13.725275 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2155552128}
I20260709 13:02:14.291810  1721 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:14.294409  1470 catalog_manager.cc:5697] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 1 to 2, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:14.298718  3016 consensus_queue.cc:260] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.303169  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:14.303800  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:02:14.427513  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.440905  2349 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
W20260709 13:02:14.441797  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:14.442876  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 161917us
I20260709 13:02:14.442801  1639 raft_consensus.cc:2393] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 2: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:14.260828  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:14.310698  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 63182us
I20260709 13:02:14.418128  2348 raft_consensus.cc:2393] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 2: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:14.317463  3099 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:14.326161  1471 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38662 (request call id 205) took 3681 ms. Trace:
I20260709 13:02:14.450723  1471 rpcz_store.cc:276] 0709 13:02:10.622837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.624203 (+  1366us) service_pool.cc:224] Handling call
0709 13:02:14.304223 (+3680020us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54875,"reported_tablets":9}
I20260709 13:02:14.327333  1641 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:14.333019  2875 raft_consensus.cc:740] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:02:13.920007  3151 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:13.947613  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.021524  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1088) took 1043 ms. Trace:
I20260709 13:02:14.344547  3122 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.350373  1567 raft_consensus.cc:2376] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2. Re-sending same reply.
I20260709 13:02:14.443358  3067 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:14.359094  1571 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1092) took 1358 ms (client timeout 1657 ms). Trace:
W20260709 13:02:14.359553  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 303255us
I20260709 13:02:14.468439  2746 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.359932  3105 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:14.376802  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:14.464622  1571 rpcz_store.cc:269] 0709 13:02:13.000834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.047298 (+ 46464us) service_pool.cc:224] Handling call
0709 13:02:14.358894 (+1311596us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400305a54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:14.359023 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2621063552}
W20260709 13:02:14.378511  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 46627.7us
I20260709 13:02:14.379562  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:14.382474  3104 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.366361  2344 pending_rounds.cc:107] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:02:14.382189  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:14.374843  3051 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:14.349610  3091 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.385433  3095 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:14.400368  3115 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:14.402782  1566 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:14.397887  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 60) took 3522 ms. Trace:
I20260709 13:02:14.463424  3080 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.505111  1647 rpcz_store.cc:276] 0709 13:02:10.875547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.983837 (+1108290us) service_pool.cc:224] Handling call
0709 13:02:14.098104 (+2114267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.362728 (+264624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.397576 (+ 34848us) spinlock_profiling.cc:243] Waited 2.11 s on lock 0x7b54002db750. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:14.397746 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4861921152}
I20260709 13:02:14.402124  2342 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:14.280840  2947 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:14.411260  2994 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.283941  1564 rpcz_store.cc:276] 0709 13:02:12.981253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.591323 (+610070us) service_pool.cc:224] Handling call
0709 13:02:13.592624 (+  1301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.222521 (+629897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.222937 (+   416us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:14.442692  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.481673  3075 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:14.448925  1470 catalog_manager.cc:5697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 1 to 2, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:14.313545  1714 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:14.452121  1611 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:14.522521  1538 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:14.522800  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 72386.5us
I20260709 13:02:14.523736  1611 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:14.523357  1470 catalog_manager.cc:5697] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 1 to 2, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:14.454037  1565 rpcz_store.cc:276] 0709 13:02:12.978538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.155447 (+176909us) service_pool.cc:224] Handling call
0709 13:02:13.155915 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.732775 (+576860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.021205 (+288430us) spinlock_profiling.cc:243] Waited 38.1 ms on lock 0x7b5400307354. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:14.021450 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87694080}
I20260709 13:02:14.459844  1573 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:14.417357  3061 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.462522  1567 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:14.356499  2345 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:14.442112  3042 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.469142  2746 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.417373  3098 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.444139  3131 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.472594  3067 consensus_queue.cc:1059] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:14.419790  2906 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:14.475613  1568 raft_consensus.cc:2410] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 2 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:14.443579  3039 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:14.406347  3013 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.423105  3159 tablet_copy_client.cc:323] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:44293
W20260709 13:02:14.495808  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.483330  3059 consensus_queue.cc:1059] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:02:14.545962  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.546388  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:14.494277  1594 debug-util.cc:398] Leaking SignalData structure 0x7b08000d5720 after lost signal to thread 434
I20260709 13:02:14.397073  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 97) took 1374 ms. Trace:
I20260709 13:02:14.507937  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 98) took 1467 ms. Trace:
I20260709 13:02:14.460765  3153 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:14.477043  3113 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.463413  3099 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:14.504568  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1095) took 1477 ms. Trace:
I20260709 13:02:14.483202  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:14.478094  3105 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.557938  1566 rpcz_store.cc:276] 0709 13:02:13.027699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.392711 (+1365012us) service_pool.cc:224] Handling call
0709 13:02:14.402366 (+  9655us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.504168 (+101802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.504494 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.517299  3122 consensus_queue.cc:1059] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:02:14.406842  3096 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.516834  1537 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:14.444387  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.483829  2344 raft_consensus.cc:2889] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:14.526569  1538 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:14.487293  3109 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.527257  1611 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:14.528362  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1096) took 1490 ms. Trace:
I20260709 13:02:14.530961  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1097) took 1484 ms. Trace:
I20260709 13:02:14.522190  2947 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.600246  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1100) took 1529 ms. Trace:
W20260709 13:02:14.532689  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.465114  3151 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.507866  3095 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.603216  1720 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:14.461671  3101 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:14.551061  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.553928  1571 rpcz_store.cc:276] 0709 13:02:13.038408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.478819 (+1440411us) service_pool.cc:224] Handling call
0709 13:02:14.504942 (+ 26123us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.505165 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.505208 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:14.505272 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:14.505322 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.505403 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.505580 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.521211  3060 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:14.555938  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.506942  2875 consensus_queue.cc:260] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:14.566700  3105 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.554577  1570 rpcz_store.cc:276] 0709 13:02:13.022794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.391126 (+1368332us) service_pool.cc:224] Handling call
0709 13:02:14.396039 (+  4913us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.396269 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.396437 (+   168us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:14.396548 (+   111us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:14.396749 (+   201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.396831 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.396996 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.558490  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.535480  3102 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:14.529805  3068 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.536000  3084 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:14.635071  3084 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:14.636109  3084 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.636600  3161 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.520951  2994 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.637163  3161 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:14.516336  3051 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:02:14.565404  3099 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:14.527891  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 275683us
I20260709 13:02:14.475975  2988 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.641745  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1101) took 1007 ms. Trace:
I20260709 13:02:14.641458  2988 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.565456  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.642241  1570 rpcz_store.cc:276] 0709 13:02:13.634992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.624658 (+989666us) service_pool.cc:224] Handling call
0709 13:02:14.641159 (+ 16501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.641368 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.641395 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:14.641450 (+    55us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:14.641476 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.641536 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.641682 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.565380  3113 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:14.571171  1569 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 1.
I20260709 13:02:14.617161  3059 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:14.644934  3059 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:14.645361  3059 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.572566  3117 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:14.646457  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1103) took 1010 ms. Trace:
I20260709 13:02:14.645596  3104 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:02:14.647059  1569 rpcz_store.cc:276] 0709 13:02:13.635876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.645125 (+1009249us) service_pool.cc:224] Handling call
0709 13:02:14.645791 (+   666us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.646042 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.646094 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:14.646145 (+    51us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:14.646180 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.646244 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.646370 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.625882  3103 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.594249  3153 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.650347  2875 ts_tablet_manager.cc:1434] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.321s	user 0.005s	sys 0.004s
I20260709 13:02:14.575870  2746 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:14.545115  3067 raft_consensus.cc:3055] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:14.651495  3067 raft_consensus.cc:740] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:14.517148  3080 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.655125  3098 raft_consensus.cc:3055] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:02:14.592854  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.656008  3098 raft_consensus.cc:740] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:02:14.656872  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.545010  3131 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:14.532346  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 160262us
I20260709 13:02:14.620119  3151 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.577060  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.596951  1567 rpcz_store.cc:276] 0709 13:02:13.038302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.461678 (+1423376us) service_pool.cc:224] Handling call
0709 13:02:14.462082 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.525604 (+ 63522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.528304 (+  2700us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.600695  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:14.600916  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 73734.4us
I20260709 13:02:14.597224  1565 rpcz_store.cc:276] 0709 13:02:13.046426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.530399 (+1483973us) service_pool.cc:224] Handling call
0709 13:02:14.530649 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.530761 (+   112us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.530776 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:14.530801 (+    25us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:14.530819 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.530850 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.530921 (+    71us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.660336  1611 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:14.605034  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.602581  1566 rpcz_store.cc:276] 0709 13:02:13.070587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.563529 (+1492942us) service_pool.cc:224] Handling call
0709 13:02:14.599702 (+ 36173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.599918 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.599944 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:14.599986 (+    42us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:14.600018 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.600072 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.600184 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.498355  3085 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:02:14.660748  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:14.661825  3010 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:14.541563  3042 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.662573  3010 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:14.662936  3010 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.663303  3010 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.551721  3039 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.624073  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.596056  2873 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.660820  1565 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:14.664978  2873 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:14.666320   436 debug-util.cc:398] Leaking SignalData structure 0x7b080000fe40 after lost signal to thread 1611
I20260709 13:02:14.665035  1719 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:14.649427  1717 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:14.669112  3068 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:14.669628  3018 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.605077  3109 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:14.670542  3068 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:14.618734  3061 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:14.673291  3061 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:14.673902  3061 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.676443  3030 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.678205  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.614348  3101 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.615672  3152 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.614460  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 99) took 1529 ms. Trace:
I20260709 13:02:14.679759  1571 rpcz_store.cc:276] 0709 13:02:13.085607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.613361 (+1527754us) service_pool.cc:224] Handling call
0709 13:02:14.613767 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.613955 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.613985 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:14.614033 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:14.614059 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.614259 (+   200us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.614403 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.617689  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.681238  1714 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
W20260709 13:02:14.618449  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 91970.4us
I20260709 13:02:14.683213  1538 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:14.614243  1594 debug-util.cc:398] Leaking SignalData structure 0x7b08000afca0 after lost signal to thread 1505
I20260709 13:02:14.600143  3065 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.631803  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.625859  3102 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.617810  3020 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:14.620676  3060 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.644132  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1102) took 1008 ms. Trace:
I20260709 13:02:14.624599  1564 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:14.531427  3075 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:14.678905  1719 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:14.684461  1611 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:14.729595  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45236.1us
I20260709 13:02:14.688601  1716 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:14.730692  1611 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:14.732216  1611 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:14.697324  1646 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:14.698776  1612 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:14.616959  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 100720us
I20260709 13:02:14.734661  1537 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:14.735286  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.613816  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:14.736428  1719 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:14.702354  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:14.704021  1647 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:14.712464  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.718981  1570 rpcz_store.cc:276] 0709 13:02:13.635613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.643260 (+1007647us) service_pool.cc:224] Handling call
0709 13:02:14.643606 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.643791 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.643815 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:14.643863 (+    48us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:14.643906 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.643959 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.644075 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.737079  3103 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.727909  3013 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.715992  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.709926  3065 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:14.724476  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.726565  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.703469  1645 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:14.708873  2746 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.706490  3005 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.708881  3102 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.705442  3085 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:14.670833  3018 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.755287  3069 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:14.767995  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.751130  3068 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.728986  3042 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.751529  1715 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:02:14.753755  1611 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:14.743897  2947 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:14.756397  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 116163us
I20260709 13:02:14.742524  1641 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:14.742520  1720 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:14.812522  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 67775.8us
W20260709 13:02:14.815383  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 76702.6us
I20260709 13:02:14.758185  3091 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.743156  3152 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:14.687333  3098 consensus_queue.cc:260] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:14.817553  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 63936.5us
I20260709 13:02:14.753142  2746 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
W20260709 13:02:14.817947  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 119467us
I20260709 13:02:14.764849  3005 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:14.836534  3005 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:14.750125  2994 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.746234  3075 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:14.810876  1594 debug-util.cc:398] Leaking SignalData structure 0x7b08003a8880 after lost signal to thread 1597
W20260709 13:02:14.840438  1594 debug-util.cc:398] Leaking SignalData structure 0x7b08003a7fe0 after lost signal to thread 1672
I20260709 13:02:14.840593  3097 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.743306  3115 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:02:14.841259  3097 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.842976  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:14.747725  3103 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.083s
I20260709 13:02:14.844202  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:14.844633  3097 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.746507  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1098) took 1688 ms. Trace:
I20260709 13:02:14.754561  3109 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:14.828651  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:14.737171  3030 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:14.829376  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:14.744524  3060 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.752178  3065 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:14.785630  3085 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.744186  2906 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:02:14.721354  3020 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.743125  2312 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:14.834435  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:14.748687  3013 consensus_queue.cc:1059] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 13:02:14.745185  3122 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.076s
I20260709 13:02:14.881651  3084 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:14.882265  1566 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:02:14.883273  1573 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1069) took 2875 ms (client timeout 1755 ms). Trace:
W20260709 13:02:14.883764  1573 rpcz_store.cc:269] 0709 13:02:12.008128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.008286 (+   158us) service_pool.cc:224] Handling call
0709 13:02:13.427931 (+1419645us) spinlock_profiling.cc:243] Waited 6.13 ms on lock 0x7b5800011198. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:02:14.883064 (+1455133us) spinlock_profiling.cc:243] Waited 85 ms on lock 0x7b54001b4754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:14.883191 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":209711744}
I20260709 13:02:14.884529  3018 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.844735  2988 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:14.885215  1573 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:40670
I20260709 13:02:14.847113  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:14.886256  1565 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:14.843022  2746 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:14.846091  1572 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:14.852911  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.856060  2345 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:14.859295  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:14.889858  1567 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:14.890255  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1106) took 1253 ms. Trace:
I20260709 13:02:14.862287  1611 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:14.890748  1568 rpcz_store.cc:276] 0709 13:02:13.636599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.260509 (+623910us) service_pool.cc:224] Handling call
0709 13:02:14.890034 (+629525us) spinlock_profiling.cc:243] Waited 29.9 ms on lock 0x7b5400225bd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:14.890183 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68860672}
I20260709 13:02:14.854079  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:14.828959  3080 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.892504  3080 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.844236  3161 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.894579  1647 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:14.895824  1648 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:14.870379  3153 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.875589  2873 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:14.896133  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1091) took 1908 ms. Trace:
I20260709 13:02:14.823688  3168 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.870743  3152 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:02:14.841073  3010 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:14.896979  1572 rpcz_store.cc:276] 0709 13:02:12.987771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.355473 (+1367702us) service_pool.cc:224] Handling call
0709 13:02:14.355820 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.887579 (+531759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.895765 (+  8186us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b54004074d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:14.896048 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1126845312}
I20260709 13:02:14.901523  3010 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:14.879264  3073 raft_consensus.cc:3055] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:14.883946  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:14.810307  3068 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:14.902311  3073 raft_consensus.cc:740] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:14.902757  3068 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.903294  3068 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:14.845629  1571 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:14.909062  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:14.729460  3067 consensus_queue.cc:260] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.910362  1648 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 2: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:14.910945  3067 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.854668  1564 rpcz_store.cc:276] 0709 13:02:13.058200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.539370 (+1481170us) service_pool.cc:224] Handling call
0709 13:02:14.578669 (+ 39299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.728239 (+149570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.746432 (+ 18193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.890450  1720 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:14.894373  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:14.895782  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:14.914652  2873 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.828470  3099 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:14.917232  3013 consensus_queue.cc:1243] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 2.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:02:14.918008  1573 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:14.918694  1573 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:14.919021  3117 consensus_queue.cc:1175] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "130bc5ff39b847b1b3204626f284f838" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 13:02:14.919425  2797 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.920118  2344 raft_consensus.cc:1240] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:02:14.920027  2797 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.923240  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:14.924288  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:14.876597  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.926900  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.927597  1720 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:14.928588  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.928501  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.929092  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.930408  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:14.931303  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:14.932185  1646 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:14.933720  2313 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:02:14.935495  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.935519  1639 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:14.936741  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:14.904196  3073 consensus_queue.cc:260] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.929436  1717 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:14.924962  1647 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
W20260709 13:02:14.863287  1594 debug-util.cc:398] Leaking SignalData structure 0x7b0800389380 after lost signal to thread 1746
I20260709 13:02:14.960470  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:14.940094  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:14.957794  3151 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.962339  1572 raft_consensus.cc:2376] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3. Re-sending same reply.
I20260709 13:02:14.941104  1648 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 2 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:14.940039  1573 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.962854  3151 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.935091  3020 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.944538  3105 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:14.877106  3095 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.947896  3101 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.951346  3059 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.940840  3084 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.974283  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:14.951355  3068 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:14.952682  3073 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.953040  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:14.953621  3161 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:14.954263  1642 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:14.994948  1572 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:14.954439  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:14.956501  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.995705  1718 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:14.847093  3096 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.931063  1715 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:14.961392  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:14.966953  1647 raft_consensus.cc:2368] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3.
I20260709 13:02:14.864143  3085 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.967708  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:02:14.997918  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 43553.1us
I20260709 13:02:14.998034  1567 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:14.998569  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:14.969476  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:14.948534  3061 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.832647  3091 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:14.918510  3013 raft_consensus.cc:3055] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:14.838578  3093 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.120s
I20260709 13:02:14.976658  3099 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:14.978068  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:14.978317  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:15.017437  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:15.019307  1641 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 2: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:14.965512  2906 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:02:14.973336  3060 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:14.983875  2797 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:15.020673  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:14.986279  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:14.985751  1570 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:14.985371  1648 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:14.985517  1645 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 2: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:14.988840  1685 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.678s (SENT)
I20260709 13:02:15.022277  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:14.992023  1538 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:14.999295  1612 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:15.035326  1612 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:15.036520  1612 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:15.036880  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:15.020012  3013 raft_consensus.cc:740] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:15.039105  3099 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:15.042042  3168 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:15.042585  3168 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.042938  3168 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:15.046522  3152 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:15.047488  3152 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.048282  3152 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:15.049384  3091 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:15.050282  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:15.051189  1641 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:15.051592  3060 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:15.052973  3010 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:15.054198  3060 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:15.055315  1612 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:15.056557  3105 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:15.041465  3098 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:15.060595  3068 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:15.062899  3099 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:15.062989  3153 consensus_queue.cc:1059] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:15.065263  3153 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:15.067006  3030 raft_consensus.cc:3055] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:15.072264  1639 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 2: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:15.068748  3030 raft_consensus.cc:740] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:15.009908  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:15.076975  1565 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:15.033414  1537 leader_election.cc:400] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3.
I20260709 13:02:14.982443  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:16.326799  1564 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
W20260709 13:02:16.327450  1564 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1123) took 1410 ms (client timeout 1527 ms). Trace:
W20260709 13:02:16.328164  1564 rpcz_store.cc:269] 0709 13:02:14.917317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.939714 (+ 22397us) service_pool.cc:224] Handling call
0709 13:02:16.327382 (+1387668us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:15.014356  1566 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
W20260709 13:02:16.332922  1566 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1119) took 1461 ms (client timeout 1811 ms). Trace:
W20260709 13:02:16.333209  1566 rpcz_store.cc:269] 0709 13:02:14.871670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.883563 (+ 11893us) service_pool.cc:224] Handling call
0709 13:02:16.332738 (+1449175us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54004056d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:16.332868 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256886528}
I20260709 13:02:15.022063  1571 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
W20260709 13:02:14.960314  1594 debug-util.cc:398] Leaking SignalData structure 0x7b0800389360 after lost signal to thread 1747
I20260709 13:02:15.023056  1647 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:16.414155  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1062) took 1393 ms. Trace:
I20260709 13:02:16.414529  1647 rpcz_store.cc:276] 0709 13:02:15.020691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.022043 (+  1352us) service_pool.cc:224] Handling call
0709 13:02:16.414092 (+1392049us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.415674  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:14.991082  1643 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:16.417176  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1045) took 1481 ms. Trace:
I20260709 13:02:16.417521  1643 rpcz_store.cc:276] 0709 13:02:14.935372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.935608 (+   236us) service_pool.cc:224] Handling call
0709 13:02:16.417114 (+1481506us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:16.418298  1594 debug-util.cc:398] Leaking SignalData structure 0x7b0800388a60 after lost signal to thread 2373
I20260709 13:02:16.418577  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:14.999891  1640 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
W20260709 13:02:16.420677  1640 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1050) took 1451 ms (client timeout 1609 ms). Trace:
W20260709 13:02:16.421061  1640 rpcz_store.cc:269] 0709 13:02:14.968827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.969107 (+   280us) service_pool.cc:224] Handling call
0709 13:02:16.420613 (+1451506us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.422042  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:16.422628  1640 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:15.096130  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:16.428180  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:16.428817  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.030267  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:15.090519  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.037361  1569 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:16.432240  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.432876  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:16.045321  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:16.434212  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:16.080708  1570 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1115) took 1250 ms (client timeout 1580 ms). Trace:
W20260709 13:02:15.027464  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 284465us
I20260709 13:02:16.141507  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.441895  3189 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.442111  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.442884  1646 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:16.443751  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:16.023311  2343 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.984s	user 0.008s	sys 0.014s
I20260709 13:02:15.024940  3095 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:16.144004  1641 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:16.451012  3095 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:15.093518  3061 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:15.007174  3067 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
W20260709 13:02:15.995126  2348 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.002s	sys 0.019s
W20260709 13:02:16.484503  2348 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.351s	user 0.003s	sys 0.019s
I20260709 13:02:15.635661  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:16.506425  2349 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:02:16.010876  2340 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.902s	user 0.001s	sys 0.007s
W20260709 13:02:16.515223  2340 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.407s	user 0.002s	sys 0.009s
I20260709 13:02:16.516126  2340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 203) took 1411 ms. Trace:
I20260709 13:02:15.016445  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:02:16.523211  2342 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55888 (request call id 202) took 1537 ms (client timeout 1606 ms). Trace:
I20260709 13:02:15.012900  3059 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:16.028127  3073 consensus_meta.cc:287] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.038s	user 0.010s	sys 0.000s
W20260709 13:02:15.923696  3085 consensus_meta.cc:287] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.904s	user 0.010s	sys 0.000s
I20260709 13:02:16.540932  3085 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:16.543788  3085 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:15.053979  3010 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.545467  3010 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:15.101856  3030 consensus_queue.cc:260] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:16.551980  3030 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:15.007248  3039 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:15.011121  3084 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:16.596077  3084 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.596711  3084 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:14.990617  3016 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:15.932732  3091 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.031607  3096 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.603329  3096 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:16.015471  3098 consensus_meta.cc:287] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.956s	user 0.011s	sys 0.001s
W20260709 13:02:15.936627  3102 consensus_meta.cc:287] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.128s	user 0.013s	sys 0.000s
I20260709 13:02:16.605428  3190 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.602633  3091 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:16.605692  3102 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:16.606045  3190 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:16.607977  3102 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.972782  3103 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:02:15.027596  3104 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:02:16.021898  3152 consensus_meta.cc:287] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.973s	user 0.011s	sys 0.006s
I20260709 13:02:16.615991  3152 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:16.609269  3091 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:16.613443  3010 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:16.618718  3152 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.990777  3151 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:15.080896  3153 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.624019  3153 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:16.627525  3190 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:15.013592  3097 consensus_queue.cc:1059] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:02:14.973120  3113 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:14.994306  3161 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.632836  3161 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:16.620246  3152 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.634259  3152 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:15.098572  3168 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:16.640970  3168 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:14.975558  3170 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:16.011219  3172 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.908s	user 0.011s	sys 0.041s
I20260709 13:02:16.150952  1639 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:02:16.157908  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 1.10311e+06us
I20260709 13:02:16.648634  1612 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:16.188892  3180 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:02:16.191035  1568 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.114s	user 0.002s	sys 0.007s
W20260709 13:02:16.194054  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 1.20215e+06us
I20260709 13:02:16.219640  3181 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.227296  3182 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:02:16.247316  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.650990  3168 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:16.650799  3181 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:16.651573  3168 raft_consensus.cc:697] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:16.252661  3183 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.652534  3183 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:16.650799  3182 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:16.324383  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 1.29158e+06us
I20260709 13:02:16.657557  3103 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.658233  1537 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:16.658784  3103 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:16.343008  3184 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:16.659888  3184 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:15.068138  1567 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:16.370139  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.662896  3184 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:15.078289  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:16.663439  1567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1135) took 1631 ms (client timeout 1577 ms). Trace:
I20260709 13:02:16.663584  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1059) took 1674 ms. Trace:
I20260709 13:02:16.329437  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:16.379736  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:16.663980  1565 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
W20260709 13:02:16.664757  1565 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1131) took 1634 ms (client timeout 1666 ms). Trace:
I20260709 13:02:16.382789  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:16.666049  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:16.666551  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.668022  1645 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:16.669245  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.431464  1642 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
W20260709 13:02:16.684778  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1061) took 1688 ms (client timeout 1577 ms). Trace:
W20260709 13:02:16.685245  1642 rpcz_store.cc:269] 0709 13:02:14.996691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.009604 (+ 12913us) service_pool.cc:224] Handling call
0709 13:02:16.684677 (+1675073us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.686708  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.687427  1642 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:16.688469  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.689078  1642 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:16.433792  1647 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 2: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:16.690532  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:16.435057  1573 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:16.691910  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:40670 (request call id 1118) took 1827 ms. Trace:
I20260709 13:02:16.692320  1573 rpcz_store.cc:276] 0709 13:02:14.863886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.885073 (+ 21187us) service_pool.cc:224] Handling call
0709 13:02:16.434940 (+1549867us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b4c0006f200. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8247151a20 00007f8247162a29 00007f824716313f 00007f82471272ab 00007f8247154f3c 00007f82471553fe 00007f82473e288b 00007f8248de1495 00007f8248e3230f 00007f8248e7ca55 00007f824de4ccd0
0709 13:02:16.691375 (+256435us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7688064}
W20260709 13:02:16.435068  1570 rpcz_store.cc:269] 0709 13:02:14.830457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.843751 (+ 13294us) service_pool.cc:224] Handling call
0709 13:02:16.080396 (+1236645us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b5400306454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:16.080629 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323707264}
I20260709 13:02:16.694396  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:16.444308  1646 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:16.696802  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.698268  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.698843  1646 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:16.699648  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.700165  1646 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:16.700868  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:16.448354  2343 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.409s	user 0.009s	sys 0.016s
I20260709 13:02:16.451911  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.706607  1641 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 2: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:16.708055  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.708730  1641 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:16.343345   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:16.463760  1640 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:16.710841  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.712134  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:16.712916  1640 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:16.713692  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.468725  1470 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38666 (request call id 211) took 5039 ms. Trace:
I20260709 13:02:16.464656  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1104) took 2825 ms. Trace:
I20260709 13:02:16.716069  1569 rpcz_store.cc:276] 0709 13:02:13.636131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.648463 (+1012332us) service_pool.cc:224] Handling call
0709 13:02:14.648834 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:16.432275 (+1783441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.437664 (+  5389us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54002242d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:16.460913 (+ 23249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3192774400}
I20260709 13:02:16.718329  3098 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.718616  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:16.718986  3098 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:16.714457  1640 raft_consensus.cc:2393] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:16.491428  1685 leader_election.cc:336] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.657s (SENT)
I20260709 13:02:16.721553  1640 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:16.485378  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 204) took 1380 ms. Trace:
I20260709 13:02:16.722344  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:16.723268  1573 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:16.722258  2348 rpcz_store.cc:276] 0709 13:02:15.104922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.105048 (+   126us) service_pool.cc:224] Handling call
0709 13:02:15.105351 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.131541 (+ 26190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.131585 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.132956 (+  1371us) log.cc:844] Serialized 26 byte log entry
0709 13:02:16.484984 (+1352028us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:16.485027 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.485104 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:16.485147 (+    43us) raft_consensus.cc:1685] finished
0709 13:02:16.485160 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.485314 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25728,"raft.queue_time_us":29501,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"thread_start_us":1351609,"threads_started":1,"wal-append.queue_time_us":896824}
W20260709 13:02:16.723830  1573 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1134) took 1692 ms (client timeout 2154 ms). Trace:
W20260709 13:02:16.724247  1573 rpcz_store.cc:269] 0709 13:02:15.031416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.693308 (+1661892us) service_pool.cc:224] Handling call
0709 13:02:16.723764 (+ 30456us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.724840  2348 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:16.498643  3193 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:16.499483  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 65326us
W20260709 13:02:16.726385  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.726111  3193 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:16.726830  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:16.727206  1687 connection.cc:256] RPC call timeout handler was delayed by -1.0317528530000004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15300000000000002s for a response
I20260709 13:02:16.714948  1470 rpcz_store.cc:276] 0709 13:02:11.429224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.429831 (+   607us) service_pool.cc:224] Handling call
0709 13:02:16.468651 (+5038820us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:14.825423 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:14.835313 (+  9890us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:14.835355 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:02:14.835374 (+    19us) tablet.cc:662] Decoding operations
0709 13:02:14.836055 (+   681us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:14.836147 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:14.836176 (+    29us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:02:14.836563 (+   387us) tablet.cc:701] Row locks acquired
0709 13:02:14.836582 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:02:14.836756 (+   174us) write_op.cc:270] Start()
0709 13:02:14.836896 (+   140us) write_op.cc:276] Timestamp: P: 1783602134836703 usec, L: 0
0709 13:02:14.837034 (+   138us) op_driver.cc:348] REPLICATION: starting
0709 13:02:14.949376 (+112342us) log.cc:844] Serialized 1317 byte log entry
0709 13:02:16.040958 (+1091582us) op_driver.cc:464] REPLICATION: finished
0709 13:02:16.053005 (+ 12047us) write_op.cc:301] APPLY: starting
0709 13:02:16.053135 (+   130us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:16.054500 (+  1365us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:16.055281 (+   781us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:16.056316 (+  1035us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:16.056692 (+   376us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:02:16.056764 (+    72us) write_op.cc:312] APPLY: finished
0709 13:02:16.057119 (+   355us) log.cc:844] Serialized 35 byte log entry
0709 13:02:16.057368 (+   249us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:16.057510 (+   142us) write_op.cc:454] Released schema lock
0709 13:02:16.057566 (+    56us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:16.058120 (+   554us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:16.058481 (+   361us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":436818,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":11065,"apply.run_cpu_time_us":4986,"apply.run_wall_time_us":5516,"num_ops":3,"prepare.queue_time_us":252698,"prepare.run_cpu_time_us":11033,"prepare.run_wall_time_us":185518,"raft.queue_time_us":937035,"raft.run_cpu_time_us":9133,"raft.run_wall_time_us":10969,"replication_time_us":1203699,"thread_start_us":171133,"threads_started":4,"wal-append.queue_time_us":68402}]]}
I20260709 13:02:16.728753  3193 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:16.501161  3196 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:16.731148  3196 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:16.722002  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 230651us
I20260709 13:02:16.734081  3186 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:02:16.736057  1685 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.755s (SENT)
I20260709 13:02:16.473457  3187 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:16.738368  3187 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:16.740294  1685 leader_election.cc:336] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.045s (SENT)
I20260709 13:02:16.468556  3189 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:16.511624  3185 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:16.513554  2349 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
W20260709 13:02:16.506171  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 56097.1us
I20260709 13:02:16.745110  3098 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:16.747197  2343 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 201) took 1814 ms. Trace:
I20260709 13:02:16.473843  3192 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:16.747725  2343 rpcz_store.cc:276] 0709 13:02:14.932553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.938644 (+  6091us) service_pool.cc:224] Handling call
0709 13:02:14.942061 (+  3417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.038071 (+ 96010us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.038108 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.038758 (+   650us) log.cc:844] Serialized 26 byte log entry
0709 13:02:16.702095 (+1663337us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:16.702172 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.705185 (+  3013us) spinlock_profiling.cc:243] Waited 77.9 ms on lock 0x7b540065b3d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:16.705254 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:16.705319 (+    65us) raft_consensus.cc:1685] finished
0709 13:02:16.705341 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.747118 (+ 41777us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55,"raft.queue_time_us":2229,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":303,"spinlock_wait_cycles":179140480,"thread_start_us":1662832,"threads_started":1,"wal-append.queue_time_us":992750}
I20260709 13:02:16.516525  2340 rpcz_store.cc:276] 0709 13:02:15.104577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.104814 (+   237us) service_pool.cc:224] Handling call
0709 13:02:15.105264 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.106544 (+  1280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.106579 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.107412 (+   833us) log.cc:844] Serialized 26 byte log entry
0709 13:02:16.515690 (+1408278us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:16.515736 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.515844 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:16.515893 (+    49us) raft_consensus.cc:1685] finished
0709 13:02:16.515909 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.516062 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28634,"raft.run_cpu_time_us":629,"raft.run_wall_time_us":630,"thread_start_us":1407782,"threads_started":1,"wal-append.queue_time_us":926762}
I20260709 13:02:16.608294  3096 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:16.612900  3104 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:16.614353  3013 consensus_queue.cc:260] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:16.620476  3010 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:16.630967  3084 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:16.630827  3097 raft_consensus.cc:3055] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:16.765554  3097 raft_consensus.cc:740] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:16.767385  3097 consensus_queue.cc:260] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:16.768985  3097 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:16.769130  1685 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.577s (SENT)
I20260709 13:02:16.641661  3152 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:16.641960  1643 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:16.643240  3170 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:16.770242  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:16.649283  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.650369  1612 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:16.654119  3181 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:16.772521  3170 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:16.772966  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 123339us
I20260709 13:02:16.659274  3039 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:16.775388  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:16.779726  1713 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:16.779558  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:16.381733  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 200) took 1532 ms. Trace:
I20260709 13:02:16.780603  3013 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:16.399695  1571 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1122) took 1490 ms (client timeout 1925 ms). Trace:
W20260709 13:02:16.781888  1571 rpcz_store.cc:269] 0709 13:02:14.911191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.914303 (+  3112us) service_pool.cc:224] Handling call
0709 13:02:14.985547 (+ 71244us) spinlock_profiling.cc:243] Waited 3.26 ms on lock 0x7b5800011198. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:02:16.399594 (+1414047us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7497600}
I20260709 13:02:16.783252  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:16.663934  1567 rpcz_store.cc:269] 0709 13:02:15.031681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.036511 (+  4830us) service_pool.cc:224] Handling call
0709 13:02:16.663265 (+1626754us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.785061  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:16.670136  3103 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:16.670718  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:16.402500  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:16.657522  3190 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:16.670325  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:16.809091  1718 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 2: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:16.652666  3168 consensus_queue.cc:237] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:16.649871  3180 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:16.664078  1648 rpcz_store.cc:276] 0709 13:02:14.988929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.023083 (+ 34154us) service_pool.cc:224] Handling call
0709 13:02:16.663520 (+1640437us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.663627  3184 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.674721  3161 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:16.675594  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:16.676863  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:16.676472  3153 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:16.677208  1572 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:16.676542  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:16.678998  1644 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:16.852375  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:16.811147  3190 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:16.853319  1721 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:16.856909  3039 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:16.857451  2340 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Advancing to term 3
I20260709 13:02:16.679466  1564 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 2: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:16.862169  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:16.683228  1719 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:16.864073  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:16.719501  1569 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:16.867484  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:16.868291  1569 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:02:16.452003  3095 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:16.741604  3187 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:16.740801  1647 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 2: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:16.871796  3095 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:16.743665  3189 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:16.743674  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55888 (request call id 205) took 1638 ms. Trace:
I20260709 13:02:16.818173  3103 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:16.875452  3103 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:16.875082  2349 rpcz_store.cc:276] 0709 13:02:15.105292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.634125 (+528833us) service_pool.cc:224] Handling call
0709 13:02:16.743587 (+1109462us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.744527  3113 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:02:16.877136  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:16.753985  3198 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:16.845309  3184 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:16.877998  1611 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:16.878425  3198 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:16.879416  3187 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:16.879884  3187 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:16.880257  3184 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:16.881148  3198 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:16.524140  2342 rpcz_store.cc:269] 0709 13:02:14.986287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.990355 (+  4068us) service_pool.cc:224] Handling call
0709 13:02:16.523132 (+1532777us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:16.754653  1568 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.677s	user 0.004s	sys 0.008s
W20260709 13:02:16.755141  3172 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.652s	user 0.011s	sys 0.045s
I20260709 13:02:16.755220  3192 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:16.756904  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:16.865497  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:16.760032  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:16.884971  2348 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:16.887162  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:16.760282  3196 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:16.762140  3183 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:16.895731  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1635) took 2491 ms (client timeout 2764 ms). Trace:
W20260709 13:02:16.899768  1644 rpcz_store.cc:269] 0709 13:02:14.404061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.404211 (+   150us) service_pool.cc:224] Handling call
0709 13:02:16.895397 (+2491186us) spinlock_profiling.cc:243] Waited 2.27 s on lock 0x7b54001fe5d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:16.895555 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5229148544}
I20260709 13:02:16.725381  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:16.904850  1573 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:16.715864  1645 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:16.701431  1646 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:16.764662  3104 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:16.665140  1565 rpcz_store.cc:269] 0709 13:02:15.030566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.077943 (+ 47377us) service_pool.cc:224] Handling call
0709 13:02:16.664689 (+1586746us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.774026  1570 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:16.773331  3181 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:16.780932  2344 rpcz_store.cc:276] 0709 13:02:14.849341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.855001 (+  5660us) service_pool.cc:224] Handling call
0709 13:02:14.902036 (+ 47035us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:14.920702 (+ 18666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.381294 (+1460592us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b5400164754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:16.381596 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40863488}
I20260709 13:02:16.896919  1470 catalog_manager.cc:5697] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:02:16.783972  1571 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:16.786515  1722 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
W20260709 13:02:16.912276  1570 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1133) took 1881 ms (client timeout 2273 ms). Trace:
I20260709 13:02:16.799986  1566 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:16.800925  1612 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:16.912921  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:16.912878  1570 rpcz_store.cc:269] 0709 13:02:15.031092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.694080 (+1662988us) service_pool.cc:224] Handling call
0709 13:02:16.912044 (+217964us) spinlock_profiling.cc:243] Waited 78.5 ms on lock 0x7b5400013e54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:16.912177 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180486400}
I20260709 13:02:16.818959  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:16.821893  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 92521.6us
W20260709 13:02:16.915060  1566 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1132) took 1884 ms (client timeout 2068 ms). Trace:
W20260709 13:02:16.915613  1566 rpcz_store.cc:269] 0709 13:02:15.030842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.401872 (+1371030us) service_pool.cc:224] Handling call
0709 13:02:16.914992 (+513120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.824133  3170 raft_consensus.cc:476] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Not starting pre-election -- already a leader
I20260709 13:02:16.824050  3188 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:16.824931  3096 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:16.826469  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 96861.7us
I20260709 13:02:16.932394  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:16.827301  3180 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:16.827585  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:16.908787  3181 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.838899  2347 raft_consensus.cc:2410] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:16.825526  3098 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:16.846432  1572 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1136) took 1814 ms (client timeout 2007 ms). Trace:
W20260709 13:02:16.946717  1572 rpcz_store.cc:269] 0709 13:02:15.031995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.379251 (+1347256us) service_pool.cc:224] Handling call
0709 13:02:16.846325 (+467074us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.811249  3152 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.830088  3059 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:16.862315  1564 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1129) took 1841 ms (client timeout 1804 ms). Trace:
I20260709 13:02:16.862561  3190 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:16.658300  3182 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:16.753688  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.949988  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:16.950278  1641 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:16.950627  1573 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:16.880301  3187 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:16.883458  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:16.954434  1714 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:16.955149  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:16.895669  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:16.762079  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:16.869450  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:16.897382  2345 raft_consensus.cc:2393] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:16.903031  1639 raft_consensus.cc:2368] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3.
I20260709 13:02:16.908842  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:16.909168  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:16.962392  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:16.963260  1721 raft_consensus.cc:3055] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:02:16.912216  1537 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:40656 dropped due to backpressure: service queue is full with 50 items
I20260709 13:02:16.963433  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:16.913906  1565 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:16.964126  1717 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:16.913331  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 112513us
I20260709 13:02:16.967602  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:16.823200  3013 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.916055  1720 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:02:16.978097  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 148926us
I20260709 13:02:16.912241  1470 catalog_manager.cc:5697] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:02:16.921839  2343 raft_consensus.cc:2410] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:16.894157  3103 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:16.922410  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:16.922551  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 120482us
W20260709 13:02:16.829456  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.938161  1611 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:16.938836  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:16.941869  1716 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:16.883641  3203 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.084s
I20260709 13:02:16.921409  3183 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:16.956106  1715 raft_consensus.cc:2393] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 2: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:16.960876  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:16.798961  1567 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:16.963618  1721 raft_consensus.cc:740] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:16.813126  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:16.916106  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:16.962791  1719 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:16.982107  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.982182  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.980544  1470 catalog_manager.cc:5697] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:16.983203  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.952966  3059 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:16.984962  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.963763  1537 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:40656 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1391)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1387)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1388)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1390)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1371)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1401)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1385)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1373)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 108)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1137)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1138)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1382)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1139)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1140)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1395)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1402)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1141)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1377)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1404)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1405)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1398)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1142)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 103)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1375)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1403)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1399)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 111)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 106)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1393)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1400)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 104)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1396)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1128)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1130)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1379)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1389)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1392)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 105)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 107)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 109)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 110)
W20260709 13:02:17.113529  2345 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 244) took 2120 ms (client timeout 2363 ms). Trace:
I20260709 13:02:17.017813  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:17.126647  1572 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:02:17.108078  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:16.989074  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.948951  3181 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:17.125730  2345 rpcz_store.cc:269] 0709 13:02:14.933015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.933179 (+   164us) service_pool.cc:224] Handling call
0709 13:02:17.052772 (+2119593us) spinlock_profiling.cc:243] Waited 1.9 s on lock 0x7b5400651b54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:17.053027 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4367216000}
I20260709 13:02:16.989429  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.132462  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1121) took 2223 ms. Trace:
I20260709 13:02:17.132895  1640 raft_consensus.cc:2368] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4.
I20260709 13:02:17.129011  1470 catalog_manager.cc:5697] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:02:17.129418  1642 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:16.992233  1722 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:02:17.126878  1570 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:17.128043  1644 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:02:17.137986  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.139303  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.140324  1644 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:17.141263  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.018920  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.143586  1644 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:17.137001  1648 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:02:17.150826  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:17.151157  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:16.984843  3182 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:17.130949  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:16.931025  3188 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:17.145143  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:17.145963  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:17.146195  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.173893  1573 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:17.146270  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:16.958916  3152 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:17.179332  3153 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:17.146607  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.180168  3152 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:16.949591  1564 rpcz_store.cc:269] 0709 13:02:15.029868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.329088 (+1299220us) service_pool.cc:224] Handling call
0709 13:02:16.862227 (+533139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.149457  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.149961  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.140019  1645 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:17.181779  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1390) took 1009 ms. Trace:
I20260709 13:02:17.182657  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:17.183517  1645 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:16.937896  3096 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:17.181779  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1391) took 1001 ms. Trace:
W20260709 13:02:17.152496  1743 debug-util.cc:398] Leaking SignalData structure 0x7b08001cb0c0 after lost signal to thread 434
I20260709 13:02:17.193619  1572 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:17.142769  1717 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:17.132890  1568 rpcz_store.cc:276] 0709 13:02:14.908619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.908741 (+   122us) service_pool.cc:224] Handling call
0709 13:02:15.064927 (+156186us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b5800011198. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:02:15.065167 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:15.065439 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:16.883038 (+1817599us) log.cc:844] Serialized 14 byte log entry
0709 13:02:17.131130 (+248092us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:17.131482 (+   352us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:17.132142 (+   660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.132280 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.132409 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":960041,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"spinlock_wait_cycles":31395072,"thread_start_us":2052915,"threads_started":2}
I20260709 13:02:17.193815  1565 rpcz_store.cc:276] 0709 13:02:16.180306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.149721 (+969415us) service_pool.cc:224] Handling call
0709 13:02:17.181699 (+ 31978us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.195083  1566 raft_consensus.cc:2410] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 13:02:17.125417  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:17.197841  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.138695  1646 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:17.143724  1571 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:17.199378  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1370) took 1038 ms. Trace:
I20260709 13:02:16.988302  1569 raft_consensus.cc:2393] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:17.154057  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.125594  1721 consensus_queue.cc:260] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:16.989302  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:17.202538  1641 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:17.165869  1647 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:17.180583  1640 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:17.151508  1644 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:17.185045  1714 raft_consensus.cc:2410] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:02:17.147439  3178 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:02:17.182252  1570 rpcz_store.cc:276] 0709 13:02:16.171972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.149114 (+977142us) service_pool.cc:224] Handling call
0709 13:02:17.181699 (+ 32585us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.959789  3098 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:17.185215  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 247220us
I20260709 13:02:17.194340  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1388) took 1024 ms. Trace:
I20260709 13:02:17.238967  1572 rpcz_store.cc:276] 0709 13:02:16.170459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.146017 (+975558us) service_pool.cc:224] Handling call
0709 13:02:17.194279 (+ 48262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.240401  1611 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:17.197957  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.244441  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:17.126753  1713 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:02:17.241741  1568 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:17.151655  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:17.246204  1648 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:17.206193  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.246749  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.131475  3183 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.218920  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.204579  1571 rpcz_store.cc:276] 0709 13:02:16.160988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.961487 (+800499us) service_pool.cc:224] Handling call
0709 13:02:17.199309 (+237822us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.201208  1639 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:17.174561  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1367) took 1014 ms. Trace:
I20260709 13:02:17.248317  1573 rpcz_store.cc:276] 0709 13:02:16.159938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.145766 (+985828us) service_pool.cc:224] Handling call
0709 13:02:17.174477 (+ 28711us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:17.223385  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.225214  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:17.250272  1715 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:17.225289  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:17.251636  1719 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:17.252419  1537 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:17.226635  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:17.254009  1716 raft_consensus.cc:2393] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 2: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:17.227095  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:17.206912  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 227536us
I20260709 13:02:17.165521  3182 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:17.237381  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.259600  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.157752  3186 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:17.245499  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1381) took 1080 ms. Trace:
I20260709 13:02:17.249085  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.220476  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:17.226104  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:17.272578  1717 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:17.260458  1572 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:17.275197  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:17.180166  3188 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:17.174569  3010 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.274169  3152 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:17.295523  3152 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.296061  3152 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:17.297035  3096 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:17.303562  3098 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:17.305425  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.306602  1571 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
W20260709 13:02:17.194497  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 62256.2us
I20260709 13:02:17.264264  3182 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:17.272408  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.181831  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.276691  1567 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:17.262257  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.264016  1611 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:17.263648  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1385) took 1096 ms. Trace:
I20260709 13:02:17.186020  3103 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:17.268709  1566 rpcz_store.cc:276] 0709 13:02:16.164674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.916440 (+751766us) service_pool.cc:224] Handling call
0709 13:02:17.245269 (+328829us) spinlock_profiling.cc:243] Waited 69.7 ms on lock 0x7b5400406ad4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:17.245440 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160395008}
I20260709 13:02:16.990053  3013 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:17.406387  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1387) took 1238 ms. Trace:
W20260709 13:02:17.243455  1743 debug-util.cc:398] Leaking SignalData structure 0x7b0800084600 after lost signal to thread 1505
I20260709 13:02:17.242249  3193 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:17.290844  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.417537  1639 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:17.419001  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.277263  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1369) took 1104 ms. Trace:
I20260709 13:02:17.316414  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:17.388484  1640 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
W20260709 13:02:17.389461  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 125536us
I20260709 13:02:17.426435  1572 raft_consensus.cc:2410] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:02:17.427012  1611 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:17.427316  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.428169  1572 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:17.404254  3209 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:17.406870  1567 rpcz_store.cc:276] 0709 13:02:16.167956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.150790 (+982834us) service_pool.cc:224] Handling call
0709 13:02:17.406178 (+255388us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b54004065d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:17.406313 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":286346112}
I20260709 13:02:17.389882  1565 rpcz_store.cc:276] 0709 13:02:16.167041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.205824 (+1038783us) service_pool.cc:224] Handling call
0709 13:02:17.263550 (+ 57726us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.432543  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.433382  1565 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:17.434475  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.435369  1565 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:17.436108  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1377) took 1272 ms. Trace:
I20260709 13:02:17.260550  3183 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:17.419919  1639 raft_consensus.cc:2368] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4.
I20260709 13:02:17.308332  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.431633  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:17.312041  3210 log.cc:927] Time spent T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:02:17.437950  1571 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:17.464478  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.428932  1611 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:17.464812  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1382) took 1299 ms. Trace:
I20260709 13:02:17.465734  1571 rpcz_store.cc:276] 0709 13:02:16.164981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.307708 (+1142727us) service_pool.cc:224] Handling call
0709 13:02:17.464731 (+157023us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.439167  1567 raft_consensus.cc:2410] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:02:17.440066  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 187803us
I20260709 13:02:17.436543  1565 rpcz_store.cc:276] 0709 13:02:16.163455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.434180 (+1270725us) service_pool.cc:224] Handling call
0709 13:02:17.436047 (+  1867us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.387172  1647 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:17.460673  1644 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:17.468437  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:17.471093  1572 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:02:17.468670  1537 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:17.468937  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.475962  1565 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 2: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:02:17.478608  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49749.6us
I20260709 13:02:17.480474  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.482183  1611 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:17.481312  1639 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:17.486476  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.487429  1639 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:17.491230  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.474253  1537 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:17.310401  3161 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:17.501271  1641 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:17.496429  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.416123  3084 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:17.505162  1565 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:17.505352  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:17.470085  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.285282  3039 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.181s
I20260709 13:02:17.512362  1644 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:17.503221  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.517802  3186 raft_consensus.cc:697] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:17.523499  1572 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:17.513759  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.524699  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.525152  1644 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:17.525242  1572 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:17.526197  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.528937  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.507002  1567 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:17.529875  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1375) took 1367 ms. Trace:
I20260709 13:02:17.514746  1639 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:17.531960  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.559984  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.560437  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.561576  1641 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:17.530210  1567 rpcz_store.cc:276] 0709 13:02:16.162720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.496066 (+1333346us) service_pool.cc:224] Handling call
0709 13:02:17.529798 (+ 33732us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.563789  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.564164  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.564965  1573 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:17.565501  1567 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:17.567130  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.567777  1567 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:17.568648  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:17.570313  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.571111  1647 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:02:17.316049  3192 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:17.479461  3196 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:17.485210  3098 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:17.388155  3010 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:17.576350  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 314369us
I20260709 13:02:17.332824  3152 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:17.478232  3096 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:17.587354  1639 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:17.575687  3190 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:17.582755  3196 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.510532  3188 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:17.502938  3202 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:17.317250  2312 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:02:17.559134  3168 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.061s	sys 0.094s
I20260709 13:02:17.599222  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.419742  3103 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:17.606281  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.510903  3084 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:17.616403  1685 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:17.600117  1743 debug-util.cc:398] Leaking SignalData structure 0x7b08003671e0 after lost signal to thread 1597
W20260709 13:02:17.620944  1687 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.008s	sys 0.008s
W20260709 13:02:17.629336  1687 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.008s	sys 0.008s
I20260709 13:02:17.629720  1685 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:17.629837  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 691128us
W20260709 13:02:17.624279  3168 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.064s	sys 0.095s
I20260709 13:02:17.621991  3196 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:17.621495  3188 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:17.633100  3113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.002s	sys 0.013s
I20260709 13:02:17.630584  3084 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:17.624428  3192 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:02:17.643247  3188 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:17.641738  3161 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:17.647048  3113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.003s	sys 0.013s
I20260709 13:02:17.665562  3013 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:17.652308  3186 consensus_queue.cc:237] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:17.652956  3168 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:17.678014  3168 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:17.678540  3168 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:17.683656  3193 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:17.684320  3193 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:17.678020  3212 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.093s	user 0.001s	sys 0.001s
W20260709 13:02:17.690502  3097 consensus_meta.cc:287] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.921s	user 0.017s	sys 0.000s
I20260709 13:02:17.654898  3059 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:17.657167  3161 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:17.689361  3181 consensus_meta.cc:287] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.523s	user 0.006s	sys 0.001s
I20260709 13:02:17.662184  3202 raft_consensus.cc:697] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 2 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:17.191550   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:17.561321  1566 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:17.810842  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1398) took 1170 ms. Trace:
I20260709 13:02:17.811647  1566 rpcz_store.cc:276] 0709 13:02:16.640006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.478922 (+838916us) service_pool.cc:224] Handling call
0709 13:02:17.810316 (+331394us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.570448  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:17.619050  1644 raft_consensus.cc:2393] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:17.617013  1646 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:17.679860  3103 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:17.760843  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.510403  1564 raft_consensus.cc:2393] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:17.764073  1570 raft_consensus.cc:2410] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:02:17.764935  1743 debug-util.cc:398] Leaking SignalData structure 0x7b080034d780 after lost signal to thread 1672
I20260709 13:02:17.697088  3161 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:17.768704  3232 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:02:17.703999  3202 consensus_queue.cc:237] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
W20260709 13:02:17.777961  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 148172us
W20260709 13:02:17.777597  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 295323us
I20260709 13:02:17.616639  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:17.779081  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 147473us
I20260709 13:02:17.779852  1568 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:17.626698  1647 raft_consensus.cc:2376] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2. Re-sending same reply.
W20260709 13:02:17.636451  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 319309us
I20260709 13:02:17.786703  3010 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:17.788837  3152 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:17.795037  3095 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:17.798527  3113 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:17.802518  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:17.804752  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:17.806617  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:17.804754  3182 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:17.823300  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:17.837322  1686 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.904s	user 0.002s	sys 0.019s
I20260709 13:02:17.821427  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 103) took 1072 ms. Trace:
I20260709 13:02:17.425803  1569 rpcz_store.cc:276] 0709 13:02:16.160684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.869136 (+708452us) service_pool.cc:224] Handling call
0709 13:02:17.264669 (+395533us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.923125  3095 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.925892  3113 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:17.933089  1686 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.000s	user 0.014s	sys 0.039s
I20260709 13:02:17.828366  1721 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:17.935774  1643 raft_consensus.cc:2368] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3.
W20260709 13:02:17.937925  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.934314  2340 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:17.829202  3059 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:17.954149  3059 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:17.956053  3182 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:17.943703  3193 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:17.960354  3193 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:17.945013  1564 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1371) took 1783 ms (client timeout 2098 ms). Trace:
I20260709 13:02:17.939508  3095 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:17.964370  3180 consensus_meta.cc:287] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.013s	user 0.017s	sys 0.008s
W20260709 13:02:17.965767  1564 rpcz_store.cc:269] 0709 13:02:16.161278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.181488 (+1020210us) service_pool.cc:224] Handling call
0709 13:02:17.944743 (+763255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:17.967887  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:17.969094  1564 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:17.951407  2312 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:02:17.983232  1570 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1373) took 1821 ms (client timeout 2165 ms). Trace:
W20260709 13:02:17.983990  1570 rpcz_store.cc:269] 0709 13:02:16.162112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.244008 (+1081896us) service_pool.cc:224] Handling call
0709 13:02:17.982957 (+738949us) spinlock_profiling.cc:243] Waited 517 ms on lock 0x7b54002baa54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:17.983164 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1188146560}
I20260709 13:02:17.822705  3181 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:17.939774  3113 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.000016  1570 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:17.934168  1639 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:17.928673  1565 rpcz_store.cc:276] 0709 13:02:16.748693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.490950 (+742257us) service_pool.cc:224] Handling call
0709 13:02:17.820852 (+329902us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b54001b5654. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:17.821167 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28508544}
I20260709 13:02:17.979362  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:17.950172  1611 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:17.981340  1568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1137) took 1155 ms (client timeout 1520 ms). Trace:
I20260709 13:02:17.952307  1573 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:17.984586  3168 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:17.983250  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.986027  1572 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:02:17.983259  3059 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:17.836280  3183 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:17.992209  3187 consensus_meta.cc:287] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.039s	user 0.011s	sys 0.007s
I20260709 13:02:17.952463  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:18.004776  1568 rpcz_store.cc:269] 0709 13:02:16.826200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.246268 (+420068us) service_pool.cc:224] Handling call
0709 13:02:17.981074 (+734806us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b5400225bd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:17.981248 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1223199488}
I20260709 13:02:17.974257  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:18.003255  1641 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:17.965081  3180 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:18.023850  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1128) took 2989 ms. Trace:
I20260709 13:02:18.024600  1570 rpcz_store.cc:276] 0709 13:02:15.018103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.999301 (+2981198us) service_pool.cc:224] Handling call
0709 13:02:17.999756 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:18.006096 (+  6340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.007427 (+  1331us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:18.022861  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1397) took 1391 ms. Trace:
I20260709 13:02:18.027849  1573 rpcz_store.cc:276] 0709 13:02:16.631318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.566102 (+934784us) service_pool.cc:224] Handling call
0709 13:02:18.018308 (+452206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:18.033170  3084 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:18.034041  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:18.034730  3202 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:18.035130  1647 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:18.036226  3084 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:18.035169  3202 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.037019  3202 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.039486  3095 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:18.041867  3186 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:18.043267  3183 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:18.043807  3095 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:18.044624  1642 raft_consensus.cc:2368] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3.
I20260709 13:02:18.046090  1643 raft_consensus.cc:2376] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3. Re-sending same reply.
I20260709 13:02:18.048187  1643 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:02:18.048995  3104 consensus_meta.cc:287] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 1.129s	user 0.013s	sys 0.010s
I20260709 13:02:18.051227  1569 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:18.051493  3168 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:18.051920  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 104) took 1302 ms. Trace:
I20260709 13:02:18.052415  1569 rpcz_store.cc:276] 0709 13:02:16.749083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.951762 (+1202679us) service_pool.cc:224] Handling call
0709 13:02:18.051840 (+100078us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:18.053391  1566 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:18.053109  3202 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:18.053960  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1400) took 1381 ms. Trace:
I20260709 13:02:18.054327  1566 rpcz_store.cc:276] 0709 13:02:16.672980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.924595 (+1251615us) service_pool.cc:224] Handling call
0709 13:02:18.053877 (+129282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:18.056146  3180 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:18.057142  3202 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:18.057582  1571 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:02:18.029024  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 1.09683e+06us
W20260709 13:02:18.058104  1743 debug-util.cc:398] Leaking SignalData structure 0x7b08003b19c0 after lost signal to thread 1746
W20260709 13:02:18.059085  1743 debug-util.cc:398] Leaking SignalData structure 0x7b08003b19a0 after lost signal to thread 1747
I20260709 13:02:18.062158  3192 consensus_queue.cc:1059] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:18.064064  3192 raft_consensus.cc:3055] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:18.064517  3192 raft_consensus.cc:740] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:18.067241  3181 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:18.067797  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1655) took 1080 ms. Trace:
I20260709 13:02:18.079006  1687 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:18.082327  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:18.068852  1639 rpcz_store.cc:276] 0709 13:02:16.987317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.030387 (+1043070us) service_pool.cc:224] Handling call
0709 13:02:18.066768 (+ 36381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:18.067222 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:18.067253 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:18.067313 (+    60us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:18.067358 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.067461 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.067649 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:18.084432  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 134559us
I20260709 13:02:18.079110  1565 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:18.086061  1572 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:18.083557  1644 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
W20260709 13:02:18.092743  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 63352.8us
I20260709 13:02:18.100289  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1130) took 3064 ms. Trace:
I20260709 13:02:18.101786  1565 rpcz_store.cc:276] 0709 13:02:15.030295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.049860 (+3019565us) service_pool.cc:224] Handling call
0709 13:02:18.078765 (+ 28905us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:18.087232 (+  8467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.094230 (+  6998us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:18.103428  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1404) took 1275 ms. Trace:
I20260709 13:02:18.104671  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1392) took 1922 ms. Trace:
I20260709 13:02:18.027402  3187 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:18.107491  3187 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:18.104004  1571 rpcz_store.cc:276] 0709 13:02:16.828494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.466617 (+638123us) service_pool.cc:224] Handling call
0709 13:02:18.103039 (+636422us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b5400223dd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:18.103337 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1213447936}
I20260709 13:02:18.109378  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1379) took 1945 ms. Trace:
I20260709 13:02:18.105268  1565 rpcz_store.cc:276] 0709 13:02:16.183071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.103016 (+1919945us) service_pool.cc:224] Handling call
0709 13:02:18.103663 (+   647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:18.104135 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:18.104160 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:18.104228 (+    68us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:18.104265 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.104341 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.104559 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:18.114473  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1399) took 1441 ms. Trace:
I20260709 13:02:18.114946  1572 rpcz_store.cc:276] 0709 13:02:16.672676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.569953 (+897277us) service_pool.cc:224] Handling call
0709 13:02:18.110882 (+540929us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:18.110507  1568 rpcz_store.cc:276] 0709 13:02:16.164019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.108156 (+1944137us) service_pool.cc:224] Handling call
0709 13:02:18.108679 (+   523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:18.109042 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:18.109071 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:18.109121 (+    50us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:18.109146 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.109194 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.109316 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:18.122514  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 596324us
I20260709 13:02:18.122984  1611 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:18.125214  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:18.127208  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 105) took 1377 ms. Trace:
I20260709 13:02:18.125612  1611 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:18.130863  1570 rpcz_store.cc:276] 0709 13:02:16.749417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.109506 (+1360089us) service_pool.cc:224] Handling call
0709 13:02:18.126270 (+ 16764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:18.126790 (+   520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:18.126818 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:18.126886 (+    68us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:18.126915 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.126974 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.127115 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:18.127064  3181 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:18.128988  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1659) took 1037 ms. Trace:
I20260709 13:02:18.134390  1642 rpcz_store.cc:276] 0709 13:02:17.091952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.134362 (+ 42410us) service_pool.cc:224] Handling call
0709 13:02:18.117106 (+982744us) spinlock_profiling.cc:243] Waited 871 ms on lock 0x7b54001f2a54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:18.128799 (+ 11693us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2003514624}
I20260709 13:02:18.133409  3181 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:18.136956  3181 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.121001  3095 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:18.143127  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 68801.5us
I20260709 13:02:18.144433  1687 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:18.129756  1648 consensus_meta.cc:287] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.866s	user 0.010s	sys 0.009s
I20260709 13:02:18.146047  1648 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
W20260709 13:02:18.147156  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 162932us
W20260709 13:02:18.120735  1743 debug-util.cc:398] Leaking SignalData structure 0x7b08002a0ba0 after lost signal to thread 2373
W20260709 13:02:18.148942  1743 debug-util.cc:398] Leaking SignalData structure 0x7b08003b1040 after lost signal to thread 2875
I20260709 13:02:18.149381  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:18.150729  2340 raft_consensus.cc:2393] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
W20260709 13:02:18.150960  1743 debug-util.cc:398] Leaking SignalData structure 0x7b08003952e0 after lost signal to thread 3159
I20260709 13:02:18.152666  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:18.154875  1538 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:18.156490  3180 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:18.157187  3180 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.158560  3180 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.159993  1646 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
W20260709 13:02:18.160954  1645 consensus_meta.cc:287] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.977s	user 0.015s	sys 0.002s
I20260709 13:02:18.161489  1645 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:18.074602  3104 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:18.165557  3181 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:18.168448  3181 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:18.125898  3226 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
W20260709 13:02:18.111572  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 160644us
W20260709 13:02:18.171887  3188 consensus_meta.cc:287] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.507s	user 0.000s	sys 0.015s
I20260709 13:02:18.171375  2312 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:02:18.176000  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 43587.3us
I20260709 13:02:18.172655  3188 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:18.074647  3192 consensus_queue.cc:260] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:18.188410  1721 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:18.210438  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:18.215459  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 110) took 1386 ms. Trace:
I20260709 13:02:18.216320  1568 rpcz_store.cc:276] 0709 13:02:16.829242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.121056 (+1291814us) service_pool.cc:224] Handling call
0709 13:02:18.205185 (+ 84129us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:18.214591 (+  9406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:18.214619 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:18.214696 (+    77us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:18.214736 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.215193 (+   457us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.215362 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:18.198987  3059 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:18.179471  1564 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:18.218924  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1396) took 1595 ms. Trace:
I20260709 13:02:18.218217  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 107) took 1468 ms. Trace:
I20260709 13:02:18.219300  1567 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:18.219480  1571 rpcz_store.cc:276] 0709 13:02:16.750130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.110489 (+1360359us) service_pool.cc:224] Handling call
0709 13:02:18.178843 (+ 68354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:18.217779 (+ 38936us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:18.217809 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:18.217897 (+    88us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:18.217929 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.218004 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.218153 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:18.220853  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1393) took 2036 ms. Trace:
I20260709 13:02:18.219336  1564 rpcz_store.cc:276] 0709 13:02:16.623152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.967172 (+1344020us) service_pool.cc:224] Handling call
0709 13:02:18.218854 (+251682us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:18.203856  3104 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:18.221282  1567 rpcz_store.cc:276] 0709 13:02:16.184834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.568406 (+1383572us) service_pool.cc:224] Handling call
0709 13:02:18.220553 (+652147us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b54001275d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:18.220789 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1459252480}
I20260709 13:02:18.229890  3225 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
W20260709 13:02:18.230286  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 105290us
I20260709 13:02:18.221985  3226 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:02:18.245787  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 120412us
I20260709 13:02:18.247965  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 1688) took 1375 ms. Trace:
I20260709 13:02:18.180546  3181 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:18.254701  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 47861.9us
I20260709 13:02:18.196902  3095 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:02:18.248389  1721 rpcz_store.cc:276] 0709 13:02:16.861505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.908564 (+ 47059us) service_pool.cc:224] Handling call
0709 13:02:17.826371 (+917807us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b58001574d0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248de10aa 00007f8248e63447 00007f8248e7bfe7 00007f8248e7d7de 00007f8248e8927c 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c
0709 13:02:18.237245 (+410874us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":975839488}
I20260709 13:02:18.268692  2345 raft_consensus.cc:1240] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:18.273907  3217 consensus_queue.cc:1059] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:02:18.276926  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 132830us
I20260709 13:02:18.281769  3161 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:18.283190  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:18.283639  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:18.284693  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:18.285977  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:18.285971  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:18.290956  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:02:18.293412  3196 consensus_meta.cc:287] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.647s	user 0.017s	sys 0.004s
I20260709 13:02:18.297092  3192 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:18.301975  3151 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:18.302594  3104 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:18.303828  1538 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:18.307164  1537 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:18.314656  1720 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:18.315090  1714 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 2: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:18.308652  1611 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:18.330847  1687 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:18.316709  1685 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:18.343462  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 81958us
I20260709 13:02:18.321902  3104 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:18.347249  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 90254.2us
W20260709 13:02:18.348605  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 44869.9us
W20260709 13:02:18.345916  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 89248.3us
I20260709 13:02:18.341508  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:18.350242  3181 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:18.355479  3181 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.355948  3181 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:18.358083  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:18.359270  1722 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.355058  2343 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:18.364255  3059 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:18.364893  3059 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.350517  3104 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.322387  3196 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:18.365331  3059 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.365911  3217 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:18.380440  3234 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:18.381106  3234 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:18.381541  3234 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:18.388725  3161 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:18.351464  3151 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:18.393839  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1389) took 2221 ms. Trace:
I20260709 13:02:18.395144  3151 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:18.394390  1573 rpcz_store.cc:276] 0709 13:02:16.171181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.109401 (+1938220us) service_pool.cc:224] Handling call
0709 13:02:18.119143 (+  9742us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:18.136515 (+ 17372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:18.136563 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:18.137345 (+   782us) log.cc:844] Serialized 26 byte log entry
0709 13:02:18.244671 (+107326us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:18.244712 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.244795 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.392487 (+147692us) raft_consensus.cc:1685] finished
0709 13:02:18.392508 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.392679 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":187,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":318,"thread_start_us":106738,"threads_started":1,"wal-append.queue_time_us":207205}
W20260709 13:02:18.397153  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 77947.3us
I20260709 13:02:18.401861  3224 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:18.396452  3188 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:18.416327  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1662) took 1258 ms. Trace:
I20260709 13:02:18.416647  3187 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.432296  1611 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:18.433815  1538 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:18.416898  1647 rpcz_store.cc:276] 0709 13:02:17.158248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.035818 (+877570us) service_pool.cc:224] Handling call
0709 13:02:18.036695 (+   877us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:18.037327 (+   632us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:18.037355 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:18.119654 (+ 82299us) log.cc:844] Serialized 564 byte log entry
0709 13:02:18.305455 (+185801us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:18.305491 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.305579 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.415994 (+110415us) raft_consensus.cc:1685] finished
0709 13:02:18.416016 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.416205 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":107971,"threads_started":1,"wal-append.queue_time_us":195238}
I20260709 13:02:18.385550  3217 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.438373  3151 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:02:18.442505  1612 connection.cc:256] RPC call timeout handler was delayed by -2.1839220120000391s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
I20260709 13:02:18.442759  3188 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:18.443249  3188 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.443686  3188 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.443789  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:18.417512  3187 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:18.445891  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.446671  3187 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:18.447827  2314 leader_election.cc:336] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.774s (SENT)
I20260709 13:02:18.448385  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:18.450156  3180 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:18.442556  3151 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:18.477118  3151 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:18.463385  1538 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:18.477285  1687 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:18.453724  1640 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:18.478631  3103 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:18.463450  3221 raft_consensus.cc:3055] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:18.479576  3192 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:18.479243  3103 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.480748  3103 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.478682  1538 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:18.488131  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:18.488829  1714 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:18.489602  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:18.490206  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:18.465794  3161 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:18.465262  3196 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:18.492000  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:18.492817  1721 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:18.473362  3217 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:18.479848  3221 raft_consensus.cc:740] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:18.495124  3168 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:18.496202  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:18.473611  3180 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:18.497792  1719 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 2: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:18.495509  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:18.496142  3168 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.499588  1716 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:18.494657  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:18.499605  3168 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:18.468683  1685 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:18.502282  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 129200us
I20260709 13:02:18.503137  1685 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:18.498349  3193 consensus_queue.cc:1059] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:02:18.497400  3221 consensus_queue.cc:260] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:18.510393  3214 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:18.510886  3214 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.511543  3214 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.479626  1687 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:18.515544  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1678) took 1122 ms. Trace:
I20260709 13:02:18.516585  1640 rpcz_store.cc:276] 0709 13:02:17.393093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.426494 (+ 33401us) service_pool.cc:224] Handling call
0709 13:02:18.515216 (+1088722us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b54002db254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:18.515443 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2219883776}
I20260709 13:02:18.518384  3161 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:18.518798  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.519104  3161 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:18.520244  3225 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:18.520771  3225 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:18.521164  3225 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:18.522410  3161 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:18.526814  1722 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:18.533808  3202 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:18.536197  1685 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:18.506455  3196 raft_consensus.cc:3055] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:18.539505  3196 raft_consensus.cc:740] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:18.540741  3196 consensus_queue.cc:260] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:18.542070  3095 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:18.543592  3186 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:18.544168  3186 raft_consensus.cc:697] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:18.498442  3180 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:18.548156  1537 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:18.552754  1714 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:18.508437  3221 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:18.556144  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 126976us
W20260709 13:02:18.559775  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 181377us
I20260709 13:02:18.562186  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:18.566603  3224 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:18.568006  1718 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:18.569924  3103 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:18.570529  3196 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:18.572485  3103 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:18.562716  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 53009.3us
I20260709 13:02:18.581908  3234 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:18.584566  3234 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:18.545164  3186 consensus_queue.cc:237] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:18.585362  3234 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:18.600297  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 120883us
I20260709 13:02:18.605852  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:18.578397  3103 raft_consensus.cc:2764] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 13:02:18.615901  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 137424us
W20260709 13:02:18.561280  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 390422us
I20260709 13:02:18.618518  1720 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.626783  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 109) took 1875 ms. Trace:
I20260709 13:02:18.564072  3181 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:18.633741  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:18.627323  1565 rpcz_store.cc:276] 0709 13:02:16.750998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.114327 (+1363329us) service_pool.cc:224] Handling call
0709 13:02:18.179347 (+ 65020us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:18.254171 (+ 74824us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:18.254204 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:18.287840 (+ 33636us) log.cc:844] Serialized 26 byte log entry
0709 13:02:18.353222 (+ 65382us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:18.625610 (+272388us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.625978 (+   368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.626053 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.626536 (+   483us) raft_consensus.cc:1685] finished
0709 13:02:18.626549 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.626704 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6837,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":9562,"thread_start_us":303530,"threads_started":2,"wal-append.queue_time_us":235330}
I20260709 13:02:18.644315  3248 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:18.645718  3181 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:18.629241  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:18.649763  1640 raft_consensus.cc:2393] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:18.650990  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:18.610715  3234 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:18.657429  1646 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:18.652964  3113 consensus_meta.cc:287] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.654s	user 0.010s	sys 0.004s
I20260709 13:02:18.670720  1658 tablet_copy_service.cc:342] P f5f4b2707a9647e5bb99e065068dea2b: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-58b222526a1a4b52bdcda825e15c46f9 received from {username='slave'} at 127.0.0.1:44424
I20260709 13:02:18.671857  1658 tablet_copy_service.cc:434] P f5f4b2707a9647e5bb99e065068dea2b: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-58b222526a1a4b52bdcda825e15c46f9 on tablet 58b222526a1a4b52bdcda825e15c46f9 with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:18.675266  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:18.685102  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 52168.6us
W20260709 13:02:18.713163  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 61056.9us
I20260709 13:02:18.647933  3180 raft_consensus.cc:697] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:18.715377  1538 leader_election.cc:400] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4.
W20260709 13:02:18.703600  3244 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:02:18.667131  3113 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:18.719792  1470 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47518 (request call id 91) took 1903 ms. Trace:
I20260709 13:02:18.725523  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:18.725131  1657 tablet_copy_service.cc:140] P f5f4b2707a9647e5bb99e065068dea2b: Received BeginTabletCopySession request for tablet b82d3b49001741eda872554a7ecb2db3 from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:44424)
I20260709 13:02:18.728879  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:18.754155  1646 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:18.754288  1657 tablet_copy_service.cc:161] P f5f4b2707a9647e5bb99e065068dea2b: Beginning new tablet copy session on tablet b82d3b49001741eda872554a7ecb2db3 from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:44424: session id = cf2f71592be8465fb18e36e15d0b2f89-b82d3b49001741eda872554a7ecb2db3
I20260709 13:02:18.755301  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:18.757030  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:18.726040  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:18.729915  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 96317.4us
I20260709 13:02:18.734333  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:18.735095  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:18.736023  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:18.761049  1644 raft_consensus.cc:2393] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:18.719913  2314 leader_election.cc:336] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.016s (SENT)
I20260709 13:02:18.737847  2312 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:18.724915  1470 rpcz_store.cc:276] 0709 13:02:16.812184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.812468 (+   284us) service_pool.cc:224] Handling call
0709 13:02:18.701995 (+1889527us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:17.702516 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:17.705667 (+  3151us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:17.705714 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:02:17.705732 (+    18us) tablet.cc:662] Decoding operations
0709 13:02:17.706487 (+   755us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:17.706857 (+   370us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:17.706911 (+    54us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:02:17.792550 (+ 85639us) tablet.cc:701] Row locks acquired
0709 13:02:17.792688 (+   138us) write_op.cc:260] PREPARE: finished
0709 13:02:17.793477 (+   789us) write_op.cc:270] Start()
0709 13:02:17.834201 (+ 40724us) write_op.cc:276] Timestamp: P: 1783602137793266 usec, L: 0
0709 13:02:17.834277 (+    76us) op_driver.cc:348] REPLICATION: starting
0709 13:02:17.939138 (+104861us) log.cc:844] Serialized 1686 byte log entry
0709 13:02:18.033292 (+ 94154us) op_driver.cc:464] REPLICATION: finished
0709 13:02:18.295325 (+262033us) write_op.cc:301] APPLY: starting
0709 13:02:18.295479 (+   154us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:18.303167 (+  7688us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:18.335710 (+ 32543us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:18.364874 (+ 29164us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:18.365218 (+   344us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:02:18.365296 (+    78us) write_op.cc:312] APPLY: finished
0709 13:02:18.483089 (+117793us) log.cc:844] Serialized 41 byte log entry
0709 13:02:18.483346 (+   257us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:18.483476 (+   130us) write_op.cc:454] Released schema lock
0709 13:02:18.483552 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:18.484164 (+   612us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:18.484212 (+    48us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":217032,"apply.run_cpu_time_us":5603,"apply.run_wall_time_us":219449,"num_ops":4,"prepare.queue_time_us":430747,"prepare.run_cpu_time_us":10588,"prepare.run_wall_time_us":243906,"replication_time_us":198850,"thread_start_us":621117,"threads_started":2}]]}
I20260709 13:02:18.742579  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:18.742751  3214 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:18.747483  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:18.798775  2340 raft_consensus.cc:2393] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:18.800148  1646 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:18.800630  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:18.801833  1717 raft_consensus.cc:2393] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:18.773754  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:18.802131  1648 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:18.775332  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:18.803558  1639 raft_consensus.cc:2393] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:18.778354  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 79265.9us
I20260709 13:02:18.804646  3095 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:18.805133  2312 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:18.796694  3161 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:18.809859  3234 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:18.810094  1718 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:18.673550  3059 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:18.810429  3161 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:18.811094  3161 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:18.812496  3059 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:18.814116  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:18.815305  1722 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:18.727387  3188 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:18.818035  3239 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:18.819152  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:18.819957  3188 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:18.820856  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 101015us
I20260709 13:02:18.821915  1538 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:18.822825  3180 consensus_queue.cc:237] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:18.824838  2340 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:18.826128  3246 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:18.826659  1647 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:18.827008  3246 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:18.827154  3188 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:18.827567  3246 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:18.827584  3188 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.828532  3188 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:18.792713  1720 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:18.823364  1538 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:18.845840  1538 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:18.847312  1538 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:18.848991  3059 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:18.849503  3059 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.850253  3059 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.850764  3234 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:18.851547  3234 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.852077  3234 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.800017  3214 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:18.848453  1538 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:18.879251  1657 tablet_copy_source_session.cc:215] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:18.879453  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:18.883126  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:18.889165  1538 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:18.876725  3103 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:18.903057  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:18.910530  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:18.902257  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:18.911365  1648 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
W20260709 13:02:18.903800  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 178413us
I20260709 13:02:18.905131  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:18.913009  1644 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 2: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:18.906091  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:18.906812  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:18.909096  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:02:18.916065  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.854707  3168 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:18.890130  3249 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:18.920627  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:18.926179  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:18.927177  1639 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:18.928057  3248 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:18.928310  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:18.928741  3248 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:18.929059  1722 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.929443  3248 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.930521  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:18.930958  1538 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:18.931216  1721 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.941259  1721 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
W20260709 13:02:18.940181  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 162571us
I20260709 13:02:18.940446  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:18.910317  3103 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:18.963891  1642 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:18.965245  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:18.966956  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:18.967247  3246 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:18.968442  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:18.923322  3249 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:02:18.961966  2313 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 2.062s (SENT) [suppressed 46 similar messages]
I20260709 13:02:18.984018  1722 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:18.985338  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:18.986012  1722 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:02:18.935779  3104 consensus_meta.cc:287] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.550s	user 0.015s	sys 0.000s
W20260709 13:02:18.926183  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 119640us
I20260709 13:02:18.997844  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:18.998821  1612 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:18.970207  3103 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.920174  3222 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 13:02:18.995690  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:19.000931  1687 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:18.969558  3246 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:18.990025  3249 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:19.005577  3250 log.cc:927] Time spent T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.227s	user 0.005s	sys 0.001s
I20260709 13:02:18.926493  3225 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:19.019292  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 57366.6us
W20260709 13:02:19.016193  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 85335.8us
W20260709 13:02:19.019852  1685 leader_election.cc:336] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.527s (SENT)
I20260709 13:02:18.989507  3161 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:18.990907  3104 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:19.009883  3168 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:19.003424  3188 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:19.039450  1685 leader_election.cc:336] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.580s (SENT)
W20260709 13:02:19.040234  1685 leader_election.cc:336] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.580s (SENT)
I20260709 13:02:19.030275  3113 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:19.026118  3151 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:19.048233  1685 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.652s (SENT)
I20260709 13:02:19.051823  1538 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:19.051748  2343 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:19.069986  2340 raft_consensus.cc:2393] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
W20260709 13:02:19.079365  3217 consensus_meta.cc:287] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.570s	user 0.011s	sys 0.004s
I20260709 13:02:19.080144  3217 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.069144  3103 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.082211  1715 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:19.085183  3188 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.085918  3188 raft_consensus.cc:2764] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:02:19.090827  3104 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:19.093979  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:19.105121  3258 log.cc:927] Time spent T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:02:19.079365  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:19.097476  1647 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:19.041678  3180 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:19.153785  1470 catalog_manager.cc:5697] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 1 to 2, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 2 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:19.116413  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:19.117542  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 504840us
I20260709 13:02:19.041352  3253 consensus_queue.cc:1059] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 13:02:19.126971  3249 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.200843  3253 raft_consensus.cc:3055] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:19.201349  3253 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:19.201853  3224 consensus_queue.cc:1059] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:19.202379  3249 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.110232  3239 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:19.135102  3225 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:19.203202  3224 raft_consensus.cc:3055] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:19.139040  1713 raft_consensus.cc:1240] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:19.142930  3103 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.145871  1720 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
W20260709 13:02:19.147500  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 92582.6us
I20260709 13:02:19.203682  3224 raft_consensus.cc:740] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:19.204411  3239 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:19.206149  3224 consensus_queue.cc:260] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:19.207490  1537 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:19.149967  3217 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:19.153888  3180 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:02:19.160580  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 201143us
I20260709 13:02:19.210007  3180 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:19.212078  1687 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:19.161223  3095 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:19.213902  3103 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:19.214383  3103 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.214874  3103 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.216012  3095 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:19.216591  3095 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:19.216998  3095 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.162150  3059 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.163295  3161 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.218839  1537 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:19.219799  3059 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.129357  3234 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.178035  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:02:19.114992  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 129788us
W20260709 13:02:19.184911  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 219060us
I20260709 13:02:19.222107  1722 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:19.207551  3224 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:19.224015  2314 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:19.225111  3161 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:19.225973  3262 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:19.225553  2314 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:19.227546  3262 raft_consensus.cc:697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:19.228134  3161 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:19.228632  3161 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:19.229034  3161 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:19.228688  3262 consensus_queue.cc:237] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.231462  3159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b82d3b49001741eda872554a7ecb2db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:19.232520  1537 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:19.233999  3259 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:19.234905  3259 raft_consensus.cc:697] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:19.235061  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:19.236234  3196 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:19.236320  3259 consensus_queue.cc:237] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.185254  3248 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:19.185941  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 137262us
I20260709 13:02:19.226380  1687 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:19.192463  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 136307us
I20260709 13:02:19.199043  1644 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.202530  3253 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:19.204555  1611 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:19.239658  1538 leader_election.cc:400] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3.
I20260709 13:02:19.240219  1538 leader_election.cc:403] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:19.242129  3196 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:19.242813  1538 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:19.191341  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 134523us
I20260709 13:02:19.245847  3188 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3.
I20260709 13:02:19.248075  1538 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:19.242832  3196 raft_consensus.cc:697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:19.249467  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:19.250593  1565 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:19.250680  3196 consensus_queue.cc:237] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.248978  3188 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:19.255954  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:19.259578  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:19.260680  1572 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:19.243777  1611 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:19.261123  3253 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.259951  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:19.262238  1567 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:19.263572  1611 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:19.264487  3181 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:19.264881  3181 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.264979  3234 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.265254  3181 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.266913  3151 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:19.268301  3151 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.268810  3151 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.273485  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:19.283764  1564 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.257323  1573 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:19.288542  1612 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:19.290053  3168 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:19.237766  1537 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:19.275206  3104 consensus_queue.cc:1175] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "066ee0109aba4465ba17fa65e14ef504" responder_term: 2 status { last_received { term: 2 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 2 index: 5 } }
W20260709 13:02:19.299032  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 65475.7us
I20260709 13:02:19.319828  1537 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:19.220078  1470 catalog_manager.cc:5697] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: config changed from index -1 to 2, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:19.301955  3217 raft_consensus.cc:2764] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 13:02:19.303772  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 77478.3us
I20260709 13:02:19.297492  2312 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:19.304737  3248 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.246197  3180 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.337736  3104 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:19.339345  3180 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.310000  1644 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
W20260709 13:02:19.312542  3266 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.072s	user 0.000s	sys 0.002s
I20260709 13:02:19.299746  3234 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:19.341434  3217 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:19.342217  3234 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:19.342892  3234 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:19.343343  3222 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:19.238462  3103 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.347656  3103 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.325512  1572 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:19.268926  1611 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:19.325179  1470 catalog_manager.cc:5697] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 1 to 2, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 2 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:19.331668  2340 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:19.321699  3059 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:19.343736  1687 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:19.323091  1567 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:19.358714  1470 catalog_manager.cc:5697] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 1 to 2, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:02:19.371594  3180 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:19.372329  3180 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.372864  3180 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:19.373968  3193 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:19.376096  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 107471us
I20260709 13:02:19.376458  3095 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.377051  1611 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:19.372160  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 124354us
I20260709 13:02:19.378803  1538 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:19.379721  3249 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:19.379878  3095 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:19.380332  3249 raft_consensus.cc:697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:19.361454  3059 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:19.381304  3249 consensus_queue.cc:237] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.382200  3059 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:19.383581  1470 catalog_manager.cc:5697] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:19.394341  1565 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:19.395216  1612 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:19.399121  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 55509.4us
I20260709 13:02:19.405615  1537 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:19.408809  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 43553.4us
I20260709 13:02:19.409734  1612 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:19.404652  3188 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:19.385347  3224 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:19.319355  3181 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.417881  1687 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:19.399369  3193 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:19.381892  3161 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:19.408834  3248 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:19.430136  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 53187us
I20260709 13:02:19.420266  3168 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:19.424741  3196 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:19.397948  3095 consensus_queue.cc:1059] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:02:19.402096  2312 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:19.423933  1564 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:19.431813  3224 raft_consensus.cc:2764] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:02:19.441501  1687 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:19.433387  3168 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.442767  2343 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: []
W20260709 13:02:19.440239  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 50404.3us
I20260709 13:02:19.472254  3151 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.442790  3253 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:19.451584  3168 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:19.514564  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 59382.8us
I20260709 13:02:19.500782  3181 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.504774  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:19.505761  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:19.516299  3217 raft_consensus.cc:2764] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:02:19.517848  1720 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:19.518745  1722 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:19.508432  1538 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:19.485170  3222 raft_consensus.cc:3055] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:19.520001  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:19.520361  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:19.525468  1714 raft_consensus.cc:2410] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:19.518476  3225 consensus_queue.cc:1059] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:02:19.527029  3225 raft_consensus.cc:3055] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:19.527379  3225 raft_consensus.cc:740] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:19.518026  3253 raft_consensus.cc:697] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:19.528263  3225 consensus_queue.cc:260] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:19.529055  3253 consensus_queue.cc:237] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:19.528975  1715 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:19.446259  3239 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:19.538285  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 93410.7us
I20260709 13:02:19.524955  3222 raft_consensus.cc:740] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:02:19.547415  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 118591us
W20260709 13:02:19.540728  3269 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:02:19.551273  1611 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:19.562867  1687 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:19.569314  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 116638us
I20260709 13:02:19.570134  3161 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.575057  3159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:55874 (request call id 237) took 5428 ms. Trace:
I20260709 13:02:19.575901  1687 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:19.575152  1611 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:19.577281  1687 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:19.529520  3225 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:19.469388  2312 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:19.584635  1538 leader_election.cc:400] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3.
I20260709 13:02:19.587960  1538 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:19.591954  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 182252us
I20260709 13:02:19.567893  3259 raft_consensus.cc:2764] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:02:19.594815  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:19.594070  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 123125us
I20260709 13:02:19.600646  2312 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:19.591691  3224 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:19.601432  3224 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.601895  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:19.601981  2312 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:19.598124  3095 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:19.590385  3217 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:19.569358  3222 consensus_queue.cc:260] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:19.593556  2344 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:19.614676  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:19.618132  1611 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:19.619411  3275 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:19.620061  3275 raft_consensus.cc:697] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:19.620314  3234 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.621037  3275 consensus_queue.cc:237] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.624011  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:19.596450  3196 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:19.625536  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:19.626286  1720 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.598012  3193 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:19.626781  3193 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.614890  3104 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:19.627192  3193 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.627532  3104 raft_consensus.cc:697] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:19.628623  3104 consensus_queue.cc:237] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:19.630494  3239 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.596555  3161 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:19.631276  3161 raft_consensus.cc:697] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:19.631359  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:19.606755  3253 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:19.632514  3253 raft_consensus.cc:697] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:19.632341  3161 consensus_queue.cc:237] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:19.634313  3221 consensus_queue.cc:1059] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:19.634166  3253 consensus_queue.cc:237] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:19.634639  3222 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:19.635229  3221 raft_consensus.cc:3055] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:19.635620  3221 raft_consensus.cc:740] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:19.636662  1687 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:19.636586  3221 consensus_queue.cc:260] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:19.637902  3113 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:19.638357  3113 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:19.638173  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:19.638824  3113 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.639228  1687 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:19.639708  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:19.634174  3186 consensus_queue.cc:1059] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:02:19.630494  3151 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.615406  3217 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.646124  3217 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.646225  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:19.647323  3180 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.593355  3181 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:19.648878  3181 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.649786  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:19.648303  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:19.651154  1713 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:19.601967  3224 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.651091  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:19.654873  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:19.655056  3259 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.655400  3221 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:19.655910  3259 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.657225  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:19.657996  1714 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:19.658311  3259 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.660063  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:19.575979  3159 rpcz_store.cc:276] 0709 13:02:14.146609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.151396 (+  4787us) service_pool.cc:224] Handling call
0709 13:02:14.213624 (+ 62228us) ts_tablet_manager.cc:934] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
0709 13:02:19.229371 (+5015747us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:02:19.401370 (+171999us) tablet_metadata.cc:665] Metadata flushed
0709 13:02:19.401497 (+   127us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:02:19.573008 (+171511us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:02:19.573860 (+   852us) tablet_replica.cc:206] Creating consensus instance
0709 13:02:19.574623 (+   763us) ts_tablet_manager.cc:989] Replica registered.
0709 13:02:19.574804 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":395,"tablet-copy.queue_time_us":60595,"thread_start_us":267961,"threads_started":2}
I20260709 13:02:19.661898  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:19.663162  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:19.677554  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:19.666800  1720 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:19.675235  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:19.676177  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:19.676678  3262 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.682821  1564 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:19.683118  3262 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.676676  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:19.686435  1714 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:19.599938  3248 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:19.679778  3259 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.693404  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:19.694731  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.695992  1567 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:02:19.680089  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 81019.5us
I20260709 13:02:19.682299  3181 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.703917  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:19.704264  1612 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:19.704381  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:19.707530  1565 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:19.649551  3180 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.714885  3181 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:19.716913  3181 raft_consensus.cc:697] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:19.718338  3181 consensus_queue.cc:237] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.722613  1612 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:19.723964  3151 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:19.680214  3249 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:02:19.729256  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 111227us
W20260709 13:02:19.730123  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 114469us
I20260709 13:02:19.730406  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:19.731302  1573 raft_consensus.cc:2410] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 2 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:19.729296  1612 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:19.701049  3248 raft_consensus.cc:697] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:19.745944  1538 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:19.734588  2344 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:19.751664  1612 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:19.752215  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:19.757781  1567 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:19.754827  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:19.688274  3059 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.745353  3186 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:19.701299  3259 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.770164  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:19.771385  1573 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:19.734733  3159 tablet_copy_client.cc:806] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:19.775856  1611 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:19.781355  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:19.782768  1572 raft_consensus.cc:2368] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3.
I20260709 13:02:19.785967  3151 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:19.786669  3151 raft_consensus.cc:697] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:19.788219  3151 consensus_queue.cc:237] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.758268  3276 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:02:19.793005  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 53486.6us
I20260709 13:02:19.760409  3262 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.738926  3249 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.771188  3186 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:19.806128  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:19.810591  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:19.811376  1567 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:19.816398  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 177264us
I20260709 13:02:19.806753  3279 raft_consensus.cc:3055] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:19.803033  3180 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:19.815142  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:19.814441  3234 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.822686  3059 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.808821  3186 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:19.809849  3159 tablet_copy_client.cc:670] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:19.795922  3168 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.826162  3279 raft_consensus.cc:740] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:19.848047  1573 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:02:19.862001  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.862883  3262 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.863885  3249 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.874074  1687 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:19.867686  1538 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:19.839591  3180 raft_consensus.cc:697] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:19.857239  3168 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.884946  3279 consensus_queue.cc:260] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.886760  3279 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.895156  3259 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:19.895301  3193 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:19.896618  3277 log.cc:927] Time spent T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:02:19.898039  3234 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:19.898981  3234 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.899174  1538 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:19.899794  3234 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:19.871212  3224 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.901942  3188 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:19.902355  1687 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:19.903329  3278 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:19.903484  1687 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:19.905272  3222 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:19.905695  3222 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:19.906083  3222 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:19.907150  1713 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:19.908510  3225 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:02:19.909839  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.903744  3224 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:19.919570  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.922744  3103 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:19.868101  1564 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:19.928575  3103 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:19.929625  2312 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:19.930663  3161 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:19.930847  3103 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:19.931308  3161 raft_consensus.cc:697] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:19.872020  3275 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:19.831662  3248 consensus_queue.cc:237] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.947600  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:19.950086  1572 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:19.953080  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 201503us
I20260709 13:02:19.954620  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:19.959952  1564 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:02:19.959880  3113 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:19.963974  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 191460us
I20260709 13:02:19.873809  3217 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.971351  1612 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:19.976406  1573 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:19.977389  1570 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:02:19.978470  1568 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:19.979149  1566 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:02:19.978735  1573 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:19.982133  3291 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:19.983009  3291 raft_consensus.cc:697] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:19.983587  3151 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:19.984163  3151 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.984158  3291 consensus_queue.cc:237] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:19.984568  3168 consensus_queue.cc:1059] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:19.985176  3151 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.986210  3283 raft_consensus.cc:3055] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:19.986853  3283 raft_consensus.cc:740] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:19.987758  3283 consensus_queue.cc:260] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:19.989168  1564 raft_consensus.cc:1240] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:19.932345  3161 consensus_queue.cc:237] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:19.989811  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:19.990867  3196 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:19.991041  1612 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:19.991287  3196 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:19.991707  3196 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.992400  3275 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:19.993278  3275 raft_consensus.cc:697] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:19.995278  3275 consensus_queue.cc:237] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:19.996387  3113 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:19.999549  1722 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:20.003633  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:20.004359  1566 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:20.004136  3151 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:20.005414  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.007398  3180 consensus_queue.cc:237] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:20.008781  3283 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:20.016885  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 99025.2us
I20260709 13:02:20.008949  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:20.012405  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:20.011296  3196 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:19.966113  3224 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.963434  3193 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:20.041230  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:20.047127  1722 raft_consensus.cc:2410] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:20.047652  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.048731  1573 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:20.050889  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:20.057451  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.058993  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:20.060009  1644 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:20.061614  1687 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:20.052626  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:20.063447  1714 raft_consensus.cc:2393] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:20.069017  3168 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.069926  3113 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:20.069774  3168 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:20.069270  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:20.079931  1685 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.520s (SENT)
I20260709 13:02:20.070412  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:20.099236  1685 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:20.070739  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:20.095497  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:20.094877  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.100929  1640 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:20.101119  1573 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.106487  1568 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:20.107694  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:20.108799  1713 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:20.069809  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:20.069832  3151 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:20.121699  1647 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:20.122481  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 105322us
W20260709 13:02:20.123064  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 66701.8us
I20260709 13:02:20.125394  1538 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:20.126086  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.126539  3259 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:20.126856  1568 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:20.127030  3259 raft_consensus.cc:697] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:20.127746  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.128304  1568 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.127934  3259 consensus_queue.cc:237] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:20.131438  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:20.132176  1640 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:20.130782  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.134639  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:20.135668  1720 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:20.141535  1572 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:20.046729  3225 raft_consensus.cc:697] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:20.050040  3276 raft_consensus.cc:697] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:20.163998  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.155225  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:20.156765  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:20.157126  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:20.156862  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:20.162272  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:20.143476  2340 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:20.166067  1643 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:20.071049  3284 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.166620  1639 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.166946  1646 raft_consensus.cc:2393] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:20.174177  1566 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:20.166467  1572 raft_consensus.cc:2393] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:20.207861  1573 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:20.182626  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:20.185331  3188 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:20.186517  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:20.148236  3113 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:20.187350  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:20.214843  1645 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.215307  1641 raft_consensus.cc:2393] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:20.187494  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:20.191371  3276 consensus_queue.cc:237] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:20.133160  3278 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:20.196973  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 154620us
W20260709 13:02:20.198213  3095 consensus_meta.cc:287] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.551s	user 0.012s	sys 0.002s
I20260709 13:02:20.133097  3217 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:20.208500  1568 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:20.184319  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.210098  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.225273  1573 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:20.171012  1570 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:20.060437  3224 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:20.216200  1640 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:20.220873  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:20.220906  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:20.226735  1685 leader_election.cc:336] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.666s (SENT)
I20260709 13:02:20.232501  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:20.224838  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.229158  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.165211  3288 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:20.242878  3288 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:20.243396  3293 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:20.245067  1687 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:20.246573  1566 raft_consensus.cc:2393] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:20.248276  1566 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:02:20.248639  1566 pending_rounds.cc:85] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 1
I20260709 13:02:20.248914  1566 pending_rounds.cc:107] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:02:20.248972  1572 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:20.230010  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.252648  1570 raft_consensus.cc:2393] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:20.200668  3239 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:20.231081  1571 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:20.238323  3292 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.238497  1569 raft_consensus.cc:2393] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:20.240463  3224 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:20.249212  1566 raft_consensus.cc:2889] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:20.250366  1722 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.213836  3284 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:20.368904  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:20.352209  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:20.284178  3292 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:20.377863  1643 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:20.379213  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:20.285941  3239 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:20.379927  1643 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:20.380111  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.249454  3217 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.132s
I20260709 13:02:20.381103  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:20.337987  3168 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:20.382275  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:20.316358  1572 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:20.392313  1566 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:02:20.380199  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 198584us
I20260709 13:02:20.259841  1645 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
W20260709 13:02:20.283272  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.315059  1568 raft_consensus.cc:1240] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:20.222671  3234 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:20.383555  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:20.360530  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:20.406495  1643 raft_consensus.cc:2393] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:20.229516  3095 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:20.260391  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.297254  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:20.390115  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.383313  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.331404  1640 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:20.414618  1646 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:20.339234  1570 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:20.414942  1571 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:20.410254  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:20.416139  1645 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:20.416607  1641 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:20.411741  1647 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:20.418097  1687 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:20.264336  1639 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:20.259696  3288 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:20.418143  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.422447  1722 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:20.422075  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.423072  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.430970  1568 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.427047  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:20.282331  3293 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:20.431607  1639 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.431994  1643 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:20.434252  1566 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:20.440268  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:20.444573  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.445935  1566 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:20.448719  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.452204  1571 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:20.423255  3291 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:02:20.438784  3279 consensus_meta.cc:287] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.552s	user 0.013s	sys 0.001s
W20260709 13:02:20.456521  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.465811  1572 raft_consensus.cc:1240] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:20.469308  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 94472.7us
I20260709 13:02:20.468365  1567 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:20.471393  1687 leader_election.cc:400] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3.
I20260709 13:02:20.471748  1687 leader_election.cc:403] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:20.473059  3095 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:20.472843  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.475805  1570 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:20.483441  1639 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:20.484807  1568 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:20.240486  3225 consensus_queue.cc:237] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:20.451625  3217 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:20.499405  1639 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:20.500824  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:20.501772  1720 raft_consensus.cc:2410] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:20.504297  1573 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:20.505625  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.506316  1573 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.508283  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:20.509367  3292 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:20.509771  1567 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:20.511391  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:20.513823  1564 raft_consensus.cc:1240] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:20.490381  3291 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:20.513234  1567 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:20.468498  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.464203  3293 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:20.515425  3293 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:20.517856  1570 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:20.491861  3224 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3.
I20260709 13:02:20.498423  1537 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:20.529870  1611 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:20.499747  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:20.497628  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:20.531229  1642 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:20.494980  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 98759us
I20260709 13:02:20.531680  1572 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:02:20.531859  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.532469  1685 leader_election.cc:336] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.804s (SENT)
W20260709 13:02:20.533071  1685 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.811s (SENT)
I20260709 13:02:20.541945  1611 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:20.543823  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:20.546020  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:20.539494  3180 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:20.560514  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:20.500073  3217 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:20.562045  3217 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:20.562840  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:20.563017  3222 consensus_meta.cc:287] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.656s	user 0.013s	sys 0.004s
I20260709 13:02:20.563402  1715 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:20.563540  3222 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:20.564134  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:20.564755  1715 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:20.565565  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:20.568293  1719 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:02:20.571738  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:20.574617  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:20.578945  1564 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:20.574739  1569 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:20.582566  3291 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:20.581763  1715 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:20.583756  1720 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:20.584288  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:20.585218  1715 raft_consensus.cc:2393] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:20.583854  1572 raft_consensus.cc:3055] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:20.585809  1572 raft_consensus.cc:740] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:20.586862  1572 consensus_queue.cc:260] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:20.588315  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:20.589004  1722 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.592279  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:20.596310  1716 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:02:20.596495  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 59215.6us
I20260709 13:02:20.597734  1714 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:20.593904  3196 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:20.599431  1716 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:20.600389  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:20.600893  3234 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:20.601889  3260 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:02:20.603732  3283 consensus_meta.cc:287] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.584s	user 0.012s	sys 0.000s
I20260709 13:02:20.604195  3222 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:20.601521  1717 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:20.604996  1644 raft_consensus.cc:2410] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 3 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 13:02:20.606251  3186 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.761s	user 0.013s	sys 0.001s
I20260709 13:02:20.606927  3186 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:20.555457  3289 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:20.610752  3260 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:02:20.613077  3293 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:20.614256  3284 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:20.616240  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 72494.2us
I20260709 13:02:20.617700  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:20.618623  1566 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:20.619796  1538 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:20.596589  1719 raft_consensus.cc:2410] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:20.588354  1572 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.632019  1538 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:20.633476  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:20.636019  1573 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:20.639119  3249 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:20.639709  3249 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.640046  3249 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.641669  3248 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:02:20.644322  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 102449us
I20260709 13:02:20.647261  1713 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:20.648115  1538 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:20.647578  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 114513us
I20260709 13:02:20.649883  1722 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:20.652583  1611 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:20.654421  1611 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:20.632999  3262 consensus_meta.cc:287] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.768s	user 0.010s	sys 0.007s
W20260709 13:02:20.656143  1685 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.917s (SENT)
I20260709 13:02:20.661659  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:20.659647  3313 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:20.661265  1611 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:20.661761  3284 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:20.671376  1611 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:20.667372  1612 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:20.596256  1715 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:20.656935  3288 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:20.682471  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:20.669826  3234 raft_consensus.cc:2955] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:02:20.691280  1572 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:02:20.693118  1572 pending_rounds.cc:85] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 1
I20260709 13:02:20.694828  1572 pending_rounds.cc:107] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:02:20.695811  1572 raft_consensus.cc:2889] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:20.660447  3248 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:02:20.700829  1685 leader_election.cc:336] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.925s (SENT)
I20260709 13:02:20.700495  1611 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:20.660584  3291 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:20.669726  3289 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:20.704121  3288 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:20.704542  3283 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:20.711818  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:20.664187  3262 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:20.693163  3284 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:20.692162  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:20.725768  1611 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:20.731245  1685 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:20.736582  1715 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:20.737762  1713 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:20.696926  3313 raft_consensus.cc:697] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:02:20.742697  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 94696.4us
I20260709 13:02:20.711989  3248 raft_consensus.cc:697] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:20.743549  1565 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:20.741770  1721 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:20.726503  3186 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:20.740259  3288 raft_consensus.cc:697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:20.738188  3193 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:20.754889  3193 raft_consensus.cc:697] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:20.754980  3313 consensus_queue.cc:237] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:20.756270  3248 consensus_queue.cc:237] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:20.756510  3193 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:20.763935  1573 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:20.751988  3181 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:02:20.768700  3181 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:02:20.770468  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45601.6us
I20260709 13:02:20.765944  1572 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:20.776022  1611 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:20.740420  3293 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:20.781224  3186 consensus_queue.cc:1059] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:20.782739  3293 raft_consensus.cc:3055] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:20.783341  3293 raft_consensus.cc:740] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:20.784260  1537 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:20.788611  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:20.739884  3283 raft_consensus.cc:697] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:20.750837  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:20.725200  3289 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:02:20.803464  1715 raft_consensus.cc:2393] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:20.784684  3293 consensus_queue.cc:260] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:20.798270  1611 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:20.741211  2340 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:20.792877  3287 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:02:20.809356  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:20.805362  3186 consensus_queue.cc:1059] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:02:20.810424  3283 consensus_queue.cc:237] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:20.811260  1564 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:20.827842  1639 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:20.816555  1538 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:20.755017  3288 consensus_queue.cc:237] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:20.771111  3291 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:20.823230  1642 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:20.807108  3275 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:20.836733  1646 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:20.826963  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:20.842182  1538 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:20.823802  3276 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.792411  3297 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 13:02:20.811208  3225 consensus_queue.cc:1059] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
W20260709 13:02:20.853327  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 64867.8us
I20260709 13:02:20.821781  3222 consensus_queue.cc:1059] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:02:20.860733  1537 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:20.836115  3224 raft_consensus.cc:3055] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:20.840104  3095 consensus_queue.cc:1059] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:02:20.887277  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:20.863140  3276 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:20.863219  3291 raft_consensus.cc:697] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:20.854817  3193 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:20.894165  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:20.895363  1715 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:20.823684  3310 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:02:20.843230  3293 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:20.906040  1714 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:20.906790  3317 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:20.853940  3186 consensus_queue.cc:1059] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
W20260709 13:02:20.912176  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 50162.1us
I20260709 13:02:20.802297  3181 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:20.918248  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:20.847926  3188 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:20.856117  3314 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:20.920809  1714 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:20.921860  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:20.922297  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:02:20.928481  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 67853.2us
I20260709 13:02:20.924091  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:20.929700  1537 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:20.930200  1716 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:20.929198  1721 raft_consensus.cc:2393] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 2: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:20.856791  3292 consensus_queue.cc:1059] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 13:02:20.936133  1537 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:20.942247  1537 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:20.945535  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:20.911160  3224 raft_consensus.cc:740] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:20.946122  1537 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:20.956808  1714 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:20.959620  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 161494us
I20260709 13:02:20.961527  1719 raft_consensus.cc:1275] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:20.927857  3180 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:20.968156  1611 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:20.867712  3217 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:20.965870  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:20.971683  1714 raft_consensus.cc:2376] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4. Re-sending same reply.
I20260709 13:02:20.945062  3262 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:20.974185  1611 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:20.950220  3317 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:02:20.946846  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 83349.8us
I20260709 13:02:20.979570  1611 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:20.950665  3314 raft_consensus.cc:697] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:20.982738  1611 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:20.983978  1721 raft_consensus.cc:1240] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:20.987181  1611 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:20.895347  3291 consensus_queue.cc:237] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:20.970705  3188 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:20.991267  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 45328.9us
I20260709 13:02:20.984994  2313 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:20.995529  1537 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:20.998251  1537 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:20.981971  3193 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.004261  3193 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:21.004873  3262 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.005976  3253 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:21.004140  2313 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:20.981518  3288 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:20.970182  3289 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:02:21.007808  1537 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:20.972798  3151 consensus_queue.cc:1059] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:02:20.987215  3275 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.023343  1537 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:20.978082  3161 consensus_queue.cc:1059] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:02:20.979851  1719 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:02:20.994643  3259 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.028854  1537 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:20.989933  3295 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:02:20.982043  3279 consensus_queue.cc:1059] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:02:21.051585  3225 raft_consensus.cc:3055] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:21.060063  3225 raft_consensus.cc:740] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:21.061041  3225 consensus_queue.cc:260] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:21.062716  3225 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:21.027943  3224 consensus_queue.cc:260] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:21.068276  3224 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:21.074440  3161 raft_consensus.cc:3055] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:21.075109  3161 raft_consensus.cc:740] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:21.076416  3161 consensus_queue.cc:260] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:21.017570  3262 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:21.048974  3180 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:21.029120  3188 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.057605  3159 tablet_copy_client.cc:538] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:21.078994  3217 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:21.079751  3217 consensus_queue.cc:1059] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:21.082176  3287 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:21.084757  3287 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:21.057672  3322 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:21.056228  3323 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.056480  3314 consensus_queue.cc:237] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:21.093153  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 180138us
W20260709 13:02:21.097247  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 68460.1us
W20260709 13:02:21.106019  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 263919us
I20260709 13:02:21.109023  1685 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:21.107144  1719 raft_consensus.cc:606] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:21.110888  1719 raft_consensus.cc:614] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:21.121076  3181 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:21.109963  3253 raft_consensus.cc:697] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:21.123075  3253 consensus_queue.cc:237] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:21.125068  3159 tablet_bootstrap.cc:492] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:02:21.126194  3329 raft_consensus.cc:3055] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:21.126678  3329 raft_consensus.cc:740] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:21.094365  3104 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.107836  3188 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.126940  3295 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:21.114727  1537 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:21.131789  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 127735us
I20260709 13:02:21.051126  3249 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:21.093909  3180 raft_consensus.cc:697] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:02:21.119609  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 132523us
I20260709 13:02:21.149137  1611 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:21.111672  3322 raft_consensus.cc:697] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:21.158419  3336 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:21.160041  3335 consensus_queue.cc:1059] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:21.154451  3323 consensus_queue.cc:1059] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:02:21.162345  3329 consensus_queue.cc:260] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:21.164043  3329 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:21.163980  3322 consensus_queue.cc:237] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:21.136557  3095 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.148401  3295 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:21.166710  3095 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:21.168641  3151 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.136325  3292 raft_consensus.cc:3055] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:21.169338  3151 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:21.169864  3292 raft_consensus.cc:740] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:21.171691  3249 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:21.172632  3295 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:21.172684  3249 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.162954  3323 raft_consensus.cc:3055] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:21.171010  3292 consensus_queue.cc:260] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:21.176004  1611 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:21.154048  3338 consensus_queue.cc:1059] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:02:21.190542  3249 consensus_queue.cc:1059] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:21.191768  3180 consensus_queue.cc:237] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:21.193470  3151 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:21.154281  3181 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.185101  3161 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:21.142244  3193 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:21.197227  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.198642  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 67369us
W20260709 13:02:21.199900  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 72504.6us
I20260709 13:02:21.214767  3328 raft_consensus.cc:3055] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:21.196622  3323 raft_consensus.cc:740] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:21.215224  3289 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.214809  3334 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:21.242134  1639 raft_consensus.cc:1275] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:21.243460  3330 raft_consensus.cc:3055] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:02:21.277916  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.278968  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:21.245198  1647 raft_consensus.cc:1275] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:21.246201  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:21.230019  3291 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:02:21.229704  3283 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.252351  3326 raft_consensus.cc:3055] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:21.295059  3326 raft_consensus.cc:740] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:21.296655  3326 consensus_queue.cc:260] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:21.253180  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1156) took 1403 ms. Trace:
I20260709 13:02:21.297941  1565 rpcz_store.cc:276] 0709 13:02:19.850150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.850436 (+   286us) service_pool.cc:224] Handling call
0709 13:02:19.851061 (+   625us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:20.751415 (+900354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.252769 (+501354us) spinlock_profiling.cc:243] Waited 892 ms on lock 0x7b54002233d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:21.253079 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2051928448}
I20260709 13:02:21.255977  1641 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:21.262488  3323 consensus_queue.cc:260] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:21.227208  3292 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:21.265460  1570 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7501558aa93542308c5fc758446387df"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:02:21.267351  3104 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:02:21.272680  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49988.8us
I20260709 13:02:21.273757  1538 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:21.308578  1611 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:21.280339  1643 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:21.280347  1612 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:21.279048  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.310576  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.284803  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.311810  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.287959  3222 raft_consensus.cc:3055] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:21.230726  3328 raft_consensus.cc:740] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:21.252015  3239 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:21.304031  1570 raft_consensus.cc:606] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:21.349819  1570 raft_consensus.cc:614] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:21.347803  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:21.351020  1639 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:21.357892  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:21.358918  1641 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 1.
I20260709 13:02:21.275269  3330 raft_consensus.cc:740] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:21.364068  1640 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:21.292121  3288 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:02:21.364830  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 91248.3us
I20260709 13:02:21.375847  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:21.313622  3338 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.397485  1722 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:21.282070  3193 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.084s
I20260709 13:02:21.315316  3222 raft_consensus.cc:740] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:21.347129  3168 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.400007  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:21.356794  3181 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:21.408885  1714 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 2: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:21.348968  3095 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:21.375754  3323 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:21.276081  3289 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:21.410580  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:21.303952  3283 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:21.291141  3336 consensus_queue.cc:1059] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.113s
I20260709 13:02:21.432017  1713 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:21.436985  3188 consensus_queue.cc:1048] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:21.402631  3288 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.374909  3151 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 13:02:21.463653  1648 raft_consensus.cc:1275] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:02:21.429608  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 119067us
W20260709 13:02:21.458736  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.466960  3288 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:21.414252  3341 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.477705  3333 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.479033  3339 consensus_queue.cc:1048] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
I20260709 13:02:21.450480  3328 consensus_queue.cc:260] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:21.459810  3181 raft_consensus.cc:697] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:21.509555  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:02:21.544330  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 60957.1us
I20260709 13:02:21.451341  3234 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.073s
I20260709 13:02:21.408068  3338 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:21.521306  3336 consensus_queue.cc:1243] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer 066ee0109aba4465ba17fa65e14ef504 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:02:21.569697  3276 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:21.605367  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 460776us
I20260709 13:02:21.572700  3279 raft_consensus.cc:3055] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:21.566268  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:21.565677  1568 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:21.494218  1641 raft_consensus.cc:2955] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:02:21.544350  2344 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:21.607156  1537 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:21.591770  1639 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:21.556457  3328 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:21.601070  2313 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:21.611732  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.591645  3159 tablet_bootstrap.cc:700] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Time spent opening tablet: real 0.209s	user 0.002s	sys 0.001s
W20260709 13:02:21.647392  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 46461.9us
I20260709 13:02:21.599171  3095 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.658098  1645 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:21.639799  1537 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:21.661484  1537 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:21.663343  1537 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:21.665980  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:21.613446  3279 raft_consensus.cc:740] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:21.675362  1537 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:21.675577  1571 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:02:21.679159  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.679183  2347 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:21.680281  1568 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:21.548988  3334 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.091s
I20260709 13:02:21.695359  1646 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:21.611711  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:21.609242  3104 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:02:21.643424  3322 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.658243  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:21.678107  3248 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:21.678505  3341 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.676759  3253 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:21.746975  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.688939  3313 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.754809  1685 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:21.755963  3279 consensus_queue.cc:260] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:21.499387  3330 consensus_queue.cc:260] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:21.759315  3327 raft_consensus.cc:3055] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:21.779763  1716 raft_consensus.cc:1275] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:21.553894  3333 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:21.787045  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:21.790163  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:21.681879  3260 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:02:21.782014  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:21.791869  1716 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:21.782133  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:21.805027  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:21.809219  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:21.672657  3095 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:21.816234  1720 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:21.816823  1717 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:21.771873  3259 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.823000  3288 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:21.825982  3262 raft_consensus.cc:476] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Not starting pre-election -- already a leader
I20260709 13:02:21.826891  3259 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:21.815788  3330 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:21.829102  3259 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:21.752504  3339 consensus_queue.cc:1048] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
W20260709 13:02:21.829870  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.831861  3343 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.289s	user 0.000s	sys 0.001s
I20260709 13:02:21.823073  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:21.894186  1537 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:21.896454  1721 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 2: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:02:21.833554  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.785526  3327 raft_consensus.cc:740] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:21.846233  1572 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:21.816543  3279 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:21.736092  3188 consensus_queue.cc:1059] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 13:02:21.821077  3234 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.787019  3322 consensus_queue.cc:1059] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.077s
W20260709 13:02:21.901170  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 146413us
W20260709 13:02:21.868099  3224 consensus_meta.cc:287] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.799s	user 0.014s	sys 0.002s
I20260709 13:02:21.902374  1647 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:21.821596  3289 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:21.898556  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:21.907793  1648 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:02:21.866132  3161 consensus_meta.cc:287] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.641s	user 0.011s	sys 0.002s
W20260709 13:02:21.906461  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 225080us
I20260709 13:02:21.958925  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:21.771019  3181 consensus_queue.cc:237] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:21.903321  3259 raft_consensus.cc:2764] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:02:21.970052  3151 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:02:21.976598  1645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:39910
I20260709 13:02:21.976972  3338 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:21.977325  3331 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:21.980149  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:21.909397  3297 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.981832  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:21.965610  3346 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.097s	user 0.000s	sys 0.003s
I20260709 13:02:22.002853  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:21.983311  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.983472  1645 raft_consensus.cc:606] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Received request to transfer leadership
I20260709 13:02:22.000360  1647 raft_consensus.cc:2393] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 2: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:02:22.004215  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.004994  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.005669  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:22.007184  1647 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:22.008000  3340 raft_consensus.cc:3055] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:22.008587  3340 raft_consensus.cc:740] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:22.004119  1645 raft_consensus.cc:614] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:22.014111  3297 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:22.015753  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:22.016387  3297 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:22.017097  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:22.022521  1567 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:22.025056  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:22.017949  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:22.018607  3333 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:22.037745  1572 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:02:22.017863  3344 log.cc:927] Time spent T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.386s	user 0.002s	sys 0.000s
I20260709 13:02:21.959790  3234 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:21.998519  3336 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:21.992123  3262 raft_consensus.cc:3055] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:21.998186  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:22.023536  3352 log.cc:927] Time spent T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
W20260709 13:02:22.022608  3357 log.cc:927] Time spent T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:02:22.054386  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:22.056425  1643 consensus_meta.cc:287] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.696s	user 0.008s	sys 0.008s
I20260709 13:02:22.059116  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:22.009410  1647 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:22.002100  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:22.021544  3360 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.073138  1643 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:22.073447  1572 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:22.075650  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:22.076591  1643 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:22.076206  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:22.074750  3360 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:22.070847  1720 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
W20260709 13:02:22.072268  3323 consensus_meta.cc:287] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.655s	user 0.008s	sys 0.004s
I20260709 13:02:22.079332  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:22.080407  1720 raft_consensus.cc:2410] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:22.080466  3360 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:22.081555  1648 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:22.082005  1720 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:22.074689  2340 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
W20260709 13:02:22.083211  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 99989.8us
I20260709 13:02:22.021260  3350 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:22.085165  1647 raft_consensus.cc:2393] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:22.084954  1686 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:22.086905  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:22.026657  3368 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.050868  3293 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.084726  1720 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:22.093621  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:22.057093  3336 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.100466  1641 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:22.097235  1572 raft_consensus.cc:2376] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3. Re-sending same reply.
I20260709 13:02:22.058440  3262 raft_consensus.cc:740] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:22.104453  1537 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:22.109145  1647 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:22.116039  1643 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:22.116449  1645 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:22.123282  3355 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:02:22.128540  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 43887.2us
W20260709 13:02:22.128530  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.022320  3331 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.081s
W20260709 13:02:22.130165  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.028914  3367 consensus_queue.cc:1059] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.083s
I20260709 13:02:22.110536  3336 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:22.024323  3371 consensus_queue.cc:1048] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.093s
I20260709 13:02:22.134752  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:22.143463  1647 raft_consensus.cc:1240] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:22.226395  1572 raft_consensus.cc:1240] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:02:22.229319  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.229936  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.230443  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.230829  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.233863  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.133864  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:22.267117  1645 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:22.110733  3293 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:22.048125  3340 consensus_queue.cc:260] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:22.121564  3180 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.104s
I20260709 13:02:22.014129  3327 consensus_queue.cc:260] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:22.145675  3360 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:22.284219  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.287119  1569 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 1. Current term is 2. Ops: []
W20260709 13:02:22.290695  2344 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.623s	user 0.007s	sys 0.004s
I20260709 13:02:22.291499  2344 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:22.223934  3355 raft_consensus.cc:697] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:22.299185  1572 raft_consensus.cc:1240] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:22.100520  3350 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:22.306046  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:22.306942  1568 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:22.307473  1645 raft_consensus.cc:2410] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 3 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 13:02:22.310563  3374 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.214s	user 0.002s	sys 0.000s
I20260709 13:02:22.312753  3095 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:22.313867  3381 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.314628  3381 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:22.291483  3359 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.292726  3360 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.317045  3359 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:22.317973  3360 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:22.110420  3368 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:22.261147  3380 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:22.339534  3186 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.340469  3186 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:22.355886  1639 raft_consensus.cc:1240] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:22.312819  3180 raft_consensus.cc:3055] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:22.329797  3340 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:22.337239  2343 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:22.338213  3327 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:22.392206  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.262115  3262 consensus_queue.cc:260] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:22.418110  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:22.385756  3292 consensus_meta.cc:287] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.066s	user 0.014s	sys 0.001s
I20260709 13:02:22.488723  1643 raft_consensus.cc:1275] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:02:22.411986  3345 log.cc:927] Time spent T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.808s	user 0.002s	sys 0.009s
I20260709 13:02:22.477432  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:22.508508  1647 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:22.512987  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:22.513833  1645 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:22.383100  3180 raft_consensus.cc:740] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:22.520753  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:22.524032  1713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.033s	sys 0.091s
W20260709 13:02:22.524509  1713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.034s	sys 0.092s
I20260709 13:02:22.482827  3331 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.492552  3104 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:22.541431  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.493026  3371 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:22.506431  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:22.512840  3365 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.548427  1645 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:22.527722  3262 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:22.527472  3234 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:22.575472  3359 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:22.576181  3188 raft_consensus.cc:3055] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:22.576831  3188 raft_consensus.cc:740] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:22.416371  3380 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:22.579125  1645 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:22.562852  1716 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:22.596376  1717 raft_consensus.cc:1275] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:22.597044  1685 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:22.568341  3292 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.604387  1685 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:22.608351  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.546556  3331 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:22.608738  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 232135us
I20260709 13:02:22.587455  3326 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:22.579336  1564 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:22.610925  1645 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:22.548375  3104 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:22.636226  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 105) took 1385 ms. Trace:
I20260709 13:02:22.638331  1639 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:02:22.610649  3359 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:22.637022  1644 rpcz_store.cc:276] 0709 13:02:21.235384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.235582 (+   198us) service_pool.cc:224] Handling call
0709 13:02:21.243077 (+  7495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.243531 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.243561 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.244086 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 13:02:21.383864 (+139778us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:21.383922 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.384014 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:22.348538 (+964524us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:02:22.609689 (+261151us) raft_consensus.cc:1685] finished
0709 13:02:22.609724 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:22.620407 (+ 10683us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2178,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":215,"spinlock_wait_cycles":43352448,"thread_start_us":139373,"threads_started":1,"wal-append.queue_time_us":232609}
I20260709 13:02:22.551594  3371 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:22.578361  3365 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:22.644543  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.622852  3330 consensus_meta.cc:287] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.793s	user 0.014s	sys 0.002s
I20260709 13:02:22.610447  3385 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:22.603693  3234 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.545625  3355 consensus_queue.cc:237] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:22.662882  1639 raft_consensus.cc:1240] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:22.607077  3292 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:22.660673  3389 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.682547  1538 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:22.685972  3388 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.083s	user 0.002s	sys 0.000s
I20260709 13:02:22.692615  3181 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.660013  3359 raft_consensus.cc:697] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:22.673051  3385 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:22.664145  3293 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:22.701960  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:22.605212  3188 consensus_queue.cc:260] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:22.708649  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:22.712538  1639 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:22.587106  3180 consensus_queue.cc:260] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:22.696103  3330 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:22.704973  3279 consensus_meta.cc:287] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.796s	user 0.006s	sys 0.009s
I20260709 13:02:22.682691  3313 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:02:22.709851  3217 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:22.697154  3234 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:22.730866  3393 log.cc:927] Time spent T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:02:22.733709  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:22.735380  3222 consensus_queue.cc:260] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:22.739435  1612 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:22.740181  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:22.707653  3385 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:22.659149  3376 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 13:02:22.763337  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:22.764138  1714 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:22.765163  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:22.765800  3222 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:22.766621  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:22.766882  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:22.769917  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.770128  1714 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:22.698048  3315 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 13:02:22.766916  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:22.769513  1640 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:22.769464  1644 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:02:22.782044  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 83255.8us
I20260709 13:02:22.784008  1538 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:22.709120  3181 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:22.736719  3368 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:22.701385  3389 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:22.797456  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.789023  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:22.806635  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.756671  3188 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:22.729264  3330 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:22.866755  1645 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:02:22.826579  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.871534  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 143510us
I20260709 13:02:22.882045  2314 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:22.746587  3217 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:22.805194  3180 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:22.886114  1639 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:22.810817  3262 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:22.808382  3284 raft_consensus.cc:3055] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:22.898212  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:22.878185  3334 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:22.784750  3333 consensus_queue.cc:1059] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.114s
W20260709 13:02:22.911998  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 172657us
I20260709 13:02:22.913586  1721 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:02:22.902640  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:22.914613  1639 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:22.899498  3262 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.895998  3104 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:22.899730  3284 raft_consensus.cc:740] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:22.805574  3359 consensus_queue.cc:237] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:22.809868  3355 consensus_queue.cc:1048] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.100s
I20260709 13:02:22.879897  3331 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:22.927597  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 197997us
I20260709 13:02:22.930604  1644 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:22.930984  1645 raft_consensus.cc:1275] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:22.914888  3378 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:22.933864  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:22.934186  3378 raft_consensus.cc:697] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:22.935504  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:22.934314  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:22.944602  2312 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:22.913308  3181 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:22.946231  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:22.924536  3262 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:22.951082  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:22.946676  3287 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:22.961814  1716 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:02:22.971263  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:22.974864  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:22.983727  1572 raft_consensus.cc:2410] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:22.978705  1612 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:22.985049  1639 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
W20260709 13:02:22.955365  3384 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.275s	user 0.005s	sys 0.000s
I20260709 13:02:22.979113  1566 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:22.982479  1716 pending_rounds.cc:85] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 1
I20260709 13:02:23.002231  1716 pending_rounds.cc:107] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:02:22.948151  3168 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:02:22.952435  3313 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:22.994845  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:23.022266  1644 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:22.999857  1687 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:22.982228  3287 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.002732  1716 raft_consensus.cc:2889] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:22.978045  3249 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.965236  3315 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:23.007295  3371 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:23.013360  3330 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:22.959568  3389 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:23.015507  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:23.001539  3376 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.996698  3292 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:23.001408  3284 consensus_queue.cc:260] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:22.936357  3378 consensus_queue.cc:237] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:23.039882  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:23.040627  3403 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:23.013398  3336 consensus_meta.cc:287] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.784s	user 0.011s	sys 0.004s
I20260709 13:02:23.003197  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:23.046295  3287 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:22.938527  3333 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.058s
W20260709 13:02:23.024014  3328 consensus_meta.cc:287] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.378s	user 0.014s	sys 0.006s
I20260709 13:02:22.999748  3331 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:22.996233  3326 consensus_queue.cc:1048] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
W20260709 13:02:23.059299  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 59527.2us
I20260709 13:02:23.059751  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:23.058817  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:23.068081  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:23.093279  1639 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:23.048377  3329 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:23.058954  3390 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.274s	user 0.002s	sys 0.000s
I20260709 13:02:23.085333  3331 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:23.086515  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:23.089721  1721 raft_consensus.cc:606] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:23.110468  1721 raft_consensus.cc:614] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:23.047760  1645 raft_consensus.cc:2410] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 3 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:02:23.095358  3349 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:23.096815  3196 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.100785  1570 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:23.120538  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1190) took 1164 ms. Trace:
I20260709 13:02:23.121047  1570 rpcz_store.cc:276] 0709 13:02:21.956067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.111108 (+155041us) service_pool.cc:224] Handling call
0709 13:02:23.100176 (+989068us) spinlock_profiling.cc:243] Waited 5.93 ms on lock 0x7b5800011198. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60e9c 00007f8245d60a5b 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:02:23.100502 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.120159 (+ 19657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.120456 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14184960}
I20260709 13:02:23.102871  1573 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:02:23.123052  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1442) took 1525 ms. Trace:
I20260709 13:02:23.123513  1573 rpcz_store.cc:276] 0709 13:02:21.598207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.598650 (+   443us) service_pool.cc:224] Handling call
0709 13:02:23.102118 (+1503468us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b5800011198. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60eee 0000000000375939 00007f824ddd41e0 00007f824de9d0c4 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:02:23.102355 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.122719 (+ 20364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.122999 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36163968}
I20260709 13:02:23.089402  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1448) took 1306 ms. Trace:
I20260709 13:02:23.124866  1567 rpcz_store.cc:276] 0709 13:02:21.783084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.783202 (+   118us) service_pool.cc:224] Handling call
0709 13:02:21.814182 (+ 30980us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.039533 (+225351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.088944 (+1049411us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b54001b4254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:23.089295 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":478624640}
W20260709 13:02:23.114094  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:23.126765  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:23.127210  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:23.127571  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:23.127916  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.076731  3284 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:23.128845  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:23.138142  1644 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:23.050455  3313 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:23.157505  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:23.163573  3360 consensus_meta.cc:287] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.845s	user 0.016s	sys 0.000s
I20260709 13:02:23.164351  3360 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:23.115147  3259 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.133113  3331 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:23.165220  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 137) took 1890 ms. Trace:
I20260709 13:02:23.073303  3336 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:23.176281  1565 rpcz_store.cc:276] 0709 13:02:21.308887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.347687 (+ 38800us) service_pool.cc:224] Handling call
0709 13:02:21.382431 (+ 34744us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:21.425739 (+ 43308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.425790 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:21.447842 (+ 22052us) log.cc:844] Serialized 52 byte log entry
0709 13:02:21.678419 (+230577us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:21.713383 (+ 34964us) log.cc:844] Serialized 14 byte log entry
0709 13:02:21.890160 (+176777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.890314 (+   154us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:23.142811 (+1252497us) spinlock_profiling.cc:243] Waited 3.17 ms on lock 0x7b18001e5740. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:02:23.164846 (+ 22035us) raft_consensus.cc:1685] finished
0709 13:02:23.164960 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.165139 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82131,"prepare.queue_time_us":20317,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":58,"raft.queue_time_us":157332,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":7534,"spinlock_wait_cycles":7287552,"thread_start_us":303610,"threads_started":2,"wal-append.queue_time_us":365002}
I20260709 13:02:23.145295  3371 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.115432  3328 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.179836  3371 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:23.180014  3328 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:23.180809  3336 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:23.179046  3395 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.249s	user 0.003s	sys 0.000s
I20260709 13:02:23.196475  1687 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:23.056463  3249 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:23.215847  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1431) took 1460 ms. Trace:
I20260709 13:02:23.050387  3315 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:23.078835  3376 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:23.173702  3168 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.153227  3349 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.218756  1713 rpcz_store.cc:276] 0709 13:02:21.740031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.740174 (+   143us) service_pool.cc:224] Handling call
0709 13:02:21.787456 (+ 47282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.894838 (+107382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.894872 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.896085 (+  1213us) log.cc:844] Serialized 26 byte log entry
0709 13:02:22.527561 (+631476us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:22.527641 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.527778 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:23.199799 (+672021us) raft_consensus.cc:1685] finished
0709 13:02:23.199823 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.199955 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":204656,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":19114,"thread_start_us":647945,"threads_started":2,"wal-append.queue_time_us":631826}
I20260709 13:02:23.211859  3328 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:23.144742  3196 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:23.224745  3327 consensus_meta.cc:287] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.740s	user 0.016s	sys 0.001s
I20260709 13:02:23.237927  3349 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:23.243952  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:23.244954  1713 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:23.245954  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 104) took 1172 ms. Trace:
I20260709 13:02:23.251124  1713 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:23.246738  1715 rpcz_store.cc:276] 0709 13:02:22.073267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.336926 (+263659us) service_pool.cc:224] Handling call
0709 13:02:22.337334 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:22.337855 (+   521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:22.337897 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:22.366734 (+ 28837us) log.cc:844] Serialized 26 byte log entry
0709 13:02:22.542634 (+175900us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:22.542695 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.542796 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:23.245595 (+702799us) raft_consensus.cc:1685] finished
0709 13:02:23.245665 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.245867 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":178787,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":6737,"thread_start_us":170302,"threads_started":1,"wal-append.queue_time_us":284981}
I20260709 13:02:23.229483  3336 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:23.252888  1722 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:23.273996  3196 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:23.199011  3259 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:23.263115  3371 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:23.239267  3168 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:23.308609  3283 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:23.310631  3350 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:23.316921  1569 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
W20260709 13:02:23.319113  3188 consensus_meta.cc:287] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.509s	user 0.010s	sys 0.003s
I20260709 13:02:23.322046  3292 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.325987  3292 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:23.328383  3292 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:23.329780  3360 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:23.317751  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 74384.9us
W20260709 13:02:23.319217  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 437367us
I20260709 13:02:23.338961  2314 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:23.344270  3403 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:23.344863  3403 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.345252  3403 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:23.357788  1717 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.008s	sys 0.094s
W20260709 13:02:23.358469  1717 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.008s	sys 0.094s
I20260709 13:02:23.323825  1715 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:23.382014  3313 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:23.300510  3376 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:23.363883  3360 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.451140  3314 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:23.451946  3314 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:23.462541  3404 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:23.483474  1717 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:23.463318  3404 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:23.563598  3381 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:23.505748  3405 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.517756  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 106) took 2253 ms. Trace:
I20260709 13:02:23.508222  3376 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.529160  3234 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:23.527745  2343 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55890 (request call id 184) took 1692 ms. Trace:
I20260709 13:02:23.525485  3295 raft_consensus.cc:3055] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:23.535255  3315 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:23.551045  3314 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:23.556350  3168 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:23.440917  3360 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:23.574681  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:23.526664  3180 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.633s	user 0.009s	sys 0.007s
I20260709 13:02:23.581956  3159 tablet_bootstrap.cc:492] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:02:23.461491  3222 consensus_meta.cc:287] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.695s	user 0.012s	sys 0.001s
I20260709 13:02:23.582863  1642 rpcz_store.cc:276] 0709 13:02:21.264347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.454009 (+189662us) service_pool.cc:224] Handling call
0709 13:02:21.454676 (+   667us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.498569 (+ 43893us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.498619 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.499291 (+   672us) log.cc:844] Serialized 26 byte log entry
0709 13:02:22.084344 (+585053us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:22.084392 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.517169 (+1432777us) spinlock_profiling.cc:243] Waited 43.2 ms on lock 0x7b54001a9854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:23.517227 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:23.517313 (+    86us) raft_consensus.cc:1685] finished
0709 13:02:23.517326 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.517622 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12775,"raft.queue_time_us":3110,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":11380,"spinlock_wait_cycles":99458944,"thread_start_us":460788,"threads_started":1,"wal-append.queue_time_us":523992,"wal-append.run_cpu_time_us":2024,"wal-append.run_wall_time_us":1014976}
I20260709 13:02:23.596163  3292 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.621268  3404 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:23.480366  3287 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:23.622381  3360 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:23.633028  1718 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:23.604269  3295 raft_consensus.cc:740] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:23.598376  3405 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:23.646258  3295 consensus_queue.cc:260] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:23.651871  3295 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:23.601747  2343 rpcz_store.cc:276] 0709 13:02:21.835851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.888608 (+ 52757us) service_pool.cc:224] Handling call
0709 13:02:22.296636 (+408028us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.474163 (+177527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.527306 (+1053143us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b54003f84d0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:23.527590 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":753617792}
I20260709 13:02:23.637115  3292 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:23.658944  3259 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:23.601476  3376 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:23.657123  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:23.637245  3262 consensus_meta.cc:287] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.651s	user 0.013s	sys 0.004s
I20260709 13:02:23.670262  3292 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:23.670485  3222 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.682457  3360 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.687028  3217 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:23.683894  3334 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.770s	user 0.013s	sys 0.007s
I20260709 13:02:23.714043  3159 tablet_bootstrap.cc:492] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:02:23.714108  3381 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:23.715026  3404 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.741393  3360 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:23.744894  3405 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:23.742312  3329 consensus_meta.cc:287] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.633s	user 0.014s	sys 0.003s
I20260709 13:02:23.687916  3262 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:23.756146  3404 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:23.727574  3222 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:23.754251  1720 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.803s	user 0.015s	sys 0.025s
I20260709 13:02:23.746531  3334 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:23.757112  3360 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:23.787307  1720 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.017s	sys 0.026s
W20260709 13:02:23.791657  1612 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.022s	sys 0.014s
W20260709 13:02:23.795279  1612 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.023s	sys 0.014s
W20260709 13:02:23.770635  3351 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.004s	sys 0.015s
W20260709 13:02:23.791630  3385 consensus_meta.cc:287] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.018s	user 0.018s	sys 0.000s
W20260709 13:02:23.761871  3333 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.009s	sys 0.009s
I20260709 13:02:23.777204  3329 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:23.789183  3159 ts_tablet_manager.cc:1403] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 2.664s	user 0.057s	sys 0.026s
I20260709 13:02:23.755021  3381 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:23.794866  3359 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.018s	sys 0.019s
W20260709 13:02:23.798867  3351 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.005s	sys 0.017s
I20260709 13:02:23.776315  3404 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:23.817873  1611 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.009s	sys 0.037s
W20260709 13:02:23.836746  1611 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.009s	sys 0.037s
W20260709 13:02:23.809667  3333 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.009s	sys 0.010s
W20260709 13:02:23.810598  3359 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.018s	sys 0.019s
I20260709 13:02:23.845363  1717 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:02:23.845947  1717 pending_rounds.cc:85] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 2
I20260709 13:02:23.846207  1717 pending_rounds.cc:107] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:02:23.846504  1717 raft_consensus.cc:2889] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:23.849475  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:23.850332  1715 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:23.803472  3385 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:23.853840  3196 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.013s	sys 0.012s
I20260709 13:02:23.840718  3287 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:23.865206  1685 leader_election.cc:336] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.998s (SENT)
W20260709 13:02:23.865495  1572 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.016s	sys 0.013s
W20260709 13:02:23.866104  1572 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.016s	sys 0.013s
W20260709 13:02:23.867741  2312 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.842s	user 0.005s	sys 0.018s
W20260709 13:02:23.869824  2312 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.005s	sys 0.019s
W20260709 13:02:23.837771  1564 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.000s	sys 0.028s
W20260709 13:02:23.853763  3378 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.012s	sys 0.012s
W20260709 13:02:23.870679  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 850427us
W20260709 13:02:23.863234  1536 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.007s	sys 0.019s
W20260709 13:02:23.875705  1536 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.007s	sys 0.020s
I20260709 13:02:23.882071  1713 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:23.883500  2312 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:23.869153  3359 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.854642  3404 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.876960  1611 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:23.875100  1564 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.001s	sys 0.028s
W20260709 13:02:23.874428  3378 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.012s	sys 0.013s
I20260709 13:02:23.866196  3376 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:23.874738  3334 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:23.925208  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:23.870052  3196 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.013s	sys 0.012s
I20260709 13:02:23.883705  3168 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:23.935042  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 58259.8us
I20260709 13:02:23.943176  1611 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:23.923782  3181 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.855s	user 0.016s	sys 0.040s
W20260709 13:02:23.902853  3104 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.005s	sys 0.039s
I20260709 13:02:23.951712  3196 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:23.937274  3404 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:23.952580  3196 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:23.954406  3404 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:23.904141  3339 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.955010  3404 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.955232  3196 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:23.952458  3323 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.956102  3196 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.903683  3315 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.956314  3323 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:23.955358  3339 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:23.951783  3104 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.006s	sys 0.040s
I20260709 13:02:23.955559  3404 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:23.948792  3181 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.883s	user 0.016s	sys 0.042s
I20260709 13:02:23.960947  3196 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:23.963189  2315 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.004s	sys 0.021s
W20260709 13:02:24.000756  2315 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.974s	user 0.004s	sys 0.021s
I20260709 13:02:23.943454  3334 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:23.986727  1614 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.920s	user 0.031s	sys 0.047s
W20260709 13:02:24.006759  1614 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.953s	user 0.032s	sys 0.047s
I20260709 13:02:23.955041  3410 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:23.927872  3234 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.930287  3405 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.933785  3239 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:24.020383  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1441) took 1369 ms. Trace:
I20260709 13:02:23.925691  3407 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:23.962029  3408 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
W20260709 13:02:23.996929  1685 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.840s (SENT) [suppressed 24 similar messages]
I20260709 13:02:23.885582  3291 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260709 13:02:23.941116  3351 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:02:23.997064  3284 consensus_meta.cc:287] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.863s	user 0.020s	sys 0.000s
I20260709 13:02:23.917732  3359 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:24.016866  3354 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 2.124s	user 0.005s	sys 0.005s
W20260709 13:02:24.006649  3406 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:02:23.998479  3262 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:24.031806  1722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 1743) took 2074 ms (client timeout 2126 ms). Trace:
I20260709 13:02:24.034610  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1746) took 1916 ms. Trace:
I20260709 13:02:24.036577  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1455) took 1093 ms. Trace:
W20260709 13:02:23.994033  3367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.621s	user 0.008s	sys 0.043s
I20260709 13:02:24.021013  1720 rpcz_store.cc:276] 0709 13:02:22.650863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.910487 (+259624us) service_pool.cc:224] Handling call
0709 13:02:22.911925 (+  1438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:22.912359 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:22.912401 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:22.922727 (+ 10326us) log.cc:844] Serialized 26 byte log entry
0709 13:02:23.795935 (+873208us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:23.795978 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.796879 (+   901us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:24.020069 (+223190us) raft_consensus.cc:1685] finished
0709 13:02:24.020096 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.020296 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1220,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":4337,"thread_start_us":843067,"threads_started":1,"wal-append.queue_time_us":953629}
I20260709 13:02:23.951788  3168 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:24.008347  1471 catalog_manager.cc:5697] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 1 to 3, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
W20260709 13:02:24.057324  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 60438.4us
I20260709 13:02:24.028445  3365 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:24.039739  3180 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.069399  1572 rpcz_store.cc:276] 0709 13:02:22.942991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.136224 (+193233us) service_pool.cc:224] Handling call
0709 13:02:23.184920 (+ 48696us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:23.191161 (+  6241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:23.191712 (+   551us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:23.192915 (+  1203us) log.cc:844] Serialized 26 byte log entry
0709 13:02:23.866579 (+673664us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:23.866641 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.866743 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:24.036326 (+169583us) raft_consensus.cc:1685] finished
0709 13:02:24.036348 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.036508 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5609,"raft.queue_time_us":16551,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":3611,"thread_start_us":672960,"threads_started":1,"wal-append.queue_time_us":840808}
I20260709 13:02:24.041798  3407 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.069380  1716 rpcz_store.cc:276] 0709 13:02:22.118284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.599445 (+481161us) service_pool.cc:224] Handling call
0709 13:02:22.604983 (+  5538us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.038313 (+433330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.033679 (+995366us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b5400407c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:24.034363 (+   684us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":819361664}
W20260709 13:02:24.067602  1722 rpcz_store.cc:269] 0709 13:02:21.957684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.960918 (+  3234us) service_pool.cc:224] Handling call
0709 13:02:24.031505 (+2070587us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540014ffd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:24.031696 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2882771200}
I20260709 13:02:24.023798  3411 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:23.998481  3315 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:24.081287  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1746) took 1155 ms. Trace:
I20260709 13:02:24.042972  3381 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:24.073549  3367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.701s	user 0.009s	sys 0.044s
I20260709 13:02:24.062122  3222 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:24.021004  3234 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:24.037446  3405 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:24.102319  1643 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:24.039537  3239 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:24.111840  3349 consensus_meta.cc:287] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.867s	user 0.005s	sys 0.013s
I20260709 13:02:24.074312  3284 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:24.062927  3351 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:24.120280  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.103327  1645 rpcz_store.cc:276] 0709 13:02:22.925523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.111750 (+186227us) service_pool.cc:224] Handling call
0709 13:02:23.150883 (+ 39133us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:23.151338 (+   455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:23.151566 (+   228us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:23.171066 (+ 19500us) log.cc:844] Serialized 26 byte log entry
0709 13:02:23.725798 (+554732us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:23.725861 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.725962 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:24.080926 (+354964us) raft_consensus.cc:1685] finished
0709 13:02:24.080961 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.081209 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":127538,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":137,"thread_start_us":496995,"threads_started":1,"wal-append.queue_time_us":655261}
I20260709 13:02:24.121088  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1743) took 1200 ms. Trace:
I20260709 13:02:24.127281  1640 rpcz_store.cc:276] 0709 13:02:22.920245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.920383 (+   138us) service_pool.cc:224] Handling call
0709 13:02:22.923678 (+  3295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:22.924028 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:22.924058 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:22.924548 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:02:23.379779 (+455231us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:23.379830 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.380715 (+   885us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:24.117202 (+736487us) raft_consensus.cc:1685] finished
0709 13:02:24.117236 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.120995 (+  3759us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2941,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":8573,"thread_start_us":452827,"threads_started":1,"wal-append.queue_time_us":862684}
I20260709 13:02:24.097529  3407 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:24.134765  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 96) took 2982 ms. Trace:
I20260709 13:02:24.139544  3367 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:24.140604  3367 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:24.123631  3328 consensus_queue.cc:1059] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:02:24.099653  3180 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:24.135385  1719 rpcz_store.cc:276] 0709 13:02:21.144075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.144313 (+   238us) service_pool.cc:224] Handling call
0709 13:02:21.155107 (+ 10794us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.224418 (+ 69311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.224455 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.227083 (+  2628us) log.cc:844] Serialized 26 byte log entry
0709 13:02:21.608212 (+381129us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:21.608292 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.608403 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:24.013410 (+2405007us) spinlock_profiling.cc:243] Waited 45.8 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:02:24.119566 (+106156us) raft_consensus.cc:1685] finished
0709 13:02:24.119598 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.126193 (+  6595us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80727,"raft.queue_time_us":51044,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":5676,"spinlock_wait_cycles":105242496,"thread_start_us":313425,"threads_started":1,"wal-append.queue_time_us":528522}
W20260709 13:02:24.162093  3413 log.cc:927] Time spent T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.148s	user 0.001s	sys 0.001s
W20260709 13:02:24.169617  3295 consensus_meta.cc:287] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.517s	user 0.016s	sys 0.000s
I20260709 13:02:24.164872  3385 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:24.132892  3349 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:24.185209  1612 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:24.137269  3284 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:24.187844  1612 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:24.194279  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 251238us
I20260709 13:02:24.181089  3323 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:24.188927  3416 log.cc:927] Time spent T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.191s	user 0.000s	sys 0.002s
W20260709 13:02:24.190640  3403 consensus_meta.cc:287] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.845s	user 0.012s	sys 0.007s
I20260709 13:02:24.200446  1639 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:02:24.202445  1611 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:24.210002  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1737) took 2497 ms. Trace:
I20260709 13:02:24.197316  3291 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:24.193965  3196 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:24.196514  3181 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:24.210711  1646 rpcz_store.cc:276] 0709 13:02:21.712609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.712795 (+   186us) service_pool.cc:224] Handling call
0709 13:02:21.752808 (+ 40013us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.753424 (+   616us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.753466 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:23.953252 (+2199786us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b2c000120e8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824777ea11 00007f824778e372 00007f8248ec0a01 00007f8248e8543a 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2
0709 13:02:23.953790 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 13:02:24.189799 (+236009us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:24.189874 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.191330 (+  1456us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:24.209511 (+ 18181us) raft_consensus.cc:1685] finished
0709 13:02:24.209540 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.209886 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33030,"raft.queue_time_us":60618,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":27649,"spinlock_wait_cycles":52427520,"thread_start_us":235433,"threads_started":1,"wal-append.queue_time_us":253137}
I20260709 13:02:24.199720  3404 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:24.204727  3339 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:24.226670  3405 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:24.201289  3411 consensus_queue.cc:1059] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
W20260709 13:02:24.250275  3429 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:02:24.251212  3411 raft_consensus.cc:3055] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:24.251739  3411 raft_consensus.cc:740] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:02:24.254329  3417 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.152s	user 0.001s	sys 0.001s
I20260709 13:02:24.255185  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1745) took 1330 ms. Trace:
I20260709 13:02:24.255635  1639 rpcz_store.cc:276] 0709 13:02:22.925220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.106335 (+181115us) service_pool.cc:224] Handling call
0709 13:02:23.106828 (+   493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.206598 (+1099770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.254851 (+ 48253us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54001ff4d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:24.255116 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2514476416}
I20260709 13:02:24.256381  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1451) took 1316 ms. Trace:
I20260709 13:02:24.252689  3411 consensus_queue.cc:260] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:24.219839  3181 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.257323  3181 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:24.257628  3411 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:24.227761  3239 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:24.230224  3422 log.cc:927] Time spent T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:02:24.256943  1564 rpcz_store.cc:276] 0709 13:02:22.940063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.946995 (+  6932us) service_pool.cc:224] Handling call
0709 13:02:22.987233 (+ 40238us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:23.049250 (+ 62017us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:23.050288 (+  1038us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:23.148064 (+ 97776us) log.cc:844] Serialized 26 byte log entry
0709 13:02:23.937551 (+789487us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:23.943744 (+  6193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.968859 (+ 25115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:24.256115 (+287256us) raft_consensus.cc:1685] finished
0709 13:02:24.256150 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.256310 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4186,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":17530,"thread_start_us":692072,"threads_started":1,"wal-append.queue_time_us":845874}
I20260709 13:02:24.230835  3424 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:24.231343  3351 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:24.231573  3349 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:24.214784  3323 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:02:24.232239  3315 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:24.233666  3159 raft_consensus.cc:359] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:24.267139  3159 raft_consensus.cc:740] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:02:24.268136  3159 consensus_queue.cc:260] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:24.271795  3159 ts_tablet_manager.cc:1434] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.441s	user 0.011s	sys 0.000s
W20260709 13:02:24.275290  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.236539  3367 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:24.238058  3359 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:24.238303  3168 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:24.279428  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1459) took 1294 ms. Trace:
I20260709 13:02:24.280000  1569 rpcz_store.cc:276] 0709 13:02:22.985447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.086166 (+100719us) service_pool.cc:224] Handling call
0709 13:02:24.279161 (+1192995us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b5400013e54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:24.279308 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":426129280}
I20260709 13:02:24.216212  3403 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:24.247097  3180 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:24.284336  3403 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:24.286407  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1449) took 1568 ms. Trace:
I20260709 13:02:24.294194  1713 rpcz_store.cc:276] 0709 13:02:22.717687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.259065 (+541378us) service_pool.cc:224] Handling call
0709 13:02:23.259402 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.884955 (+625553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.286050 (+401095us) spinlock_profiling.cc:243] Waited 622 ms on lock 0x7b54004079d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:24.286310 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1431563648}
I20260709 13:02:24.224516  1611 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:24.323428  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 98902.4us
W20260709 13:02:24.322331  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 75463.6us
I20260709 13:02:24.331414  1612 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:24.331786  1470 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38662 (request call id 206) took 5460 ms. Trace:
I20260709 13:02:24.338725  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:24.340806  1687 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:24.343050  1687 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:24.343256  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:24.346556  1687 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:24.349331  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:24.354131  1647 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:24.329264  3434 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.332954  1470 rpcz_store.cc:276] 0709 13:02:18.864408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.864633 (+   225us) service_pool.cc:224] Handling call
0709 13:02:24.321738 (+5457105us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:19.564139 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:19.564792 (+   653us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:19.564847 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:02:19.564867 (+    20us) tablet.cc:662] Decoding operations
0709 13:02:19.565728 (+   861us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:19.565860 (+   132us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:19.565888 (+    28us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:02:19.566483 (+   595us) tablet.cc:701] Row locks acquired
0709 13:02:19.566502 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:02:19.566665 (+   163us) write_op.cc:270] Start()
0709 13:02:19.566802 (+   137us) write_op.cc:276] Timestamp: P: 1783602139566609 usec, L: 0
0709 13:02:19.566825 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:02:19.615559 (+ 48734us) log.cc:844] Serialized 2210 byte log entry
0709 13:02:19.932019 (+316460us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b180038b180. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:02:20.282211 (+350192us) op_driver.cc:464] REPLICATION: finished
0709 13:02:20.567557 (+285346us) write_op.cc:301] APPLY: starting
0709 13:02:20.567675 (+   118us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:20.567906 (+   231us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:20.567940 (+    34us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:20.568889 (+   949us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:20.569111 (+   222us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:02:20.569179 (+    68us) write_op.cc:312] APPLY: finished
0709 13:02:20.569467 (+   288us) log.cc:844] Serialized 47 byte log entry
0709 13:02:20.632787 (+ 63320us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:20.647148 (+ 14361us) write_op.cc:454] Released schema lock
0709 13:02:20.658135 (+ 10987us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:20.710074 (+ 51939us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:20.712414 (+  2340us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5109,"reported_tablets":12,"spinlock_wait_cycles":4864,"child_traces":[["op",{"apply.queue_time_us":253291,"apply.run_cpu_time_us":3795,"apply.run_wall_time_us":125443,"num_ops":5,"prepare.queue_time_us":143374,"prepare.run_cpu_time_us":32179,"prepare.run_wall_time_us":345974,"raft.queue_time_us":154131,"raft.run_cpu_time_us":11018,"raft.run_wall_time_us":256776,"replication_time_us":651899,"spinlock_wait_cycles":5557248,"thread_start_us":502837,"threads_started":4,"wal-append.queue_time_us":250644,"wal-append.run_cpu_time_us":11793,"wal-append.run_wall_time_us":1641723}]]}
I20260709 13:02:24.350795  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:24.354624  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:24.355700  2345 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:24.355964  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:24.356572  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:24.325493  3168 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:24.329151  3186 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:24.371467  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.372928  1713 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:24.374634  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:24.382242  1641 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:24.349253  3359 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:24.349040  3412 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:24.352182  3284 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:24.351018  3314 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:24.352982  3366 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.351576  3367 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:24.395845  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:24.490161  3367 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.490461  2344 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:24.491117  3367 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:24.382895  3262 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:24.492209  3262 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:24.495136  3262 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:24.497203  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:24.506254  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:24.507383  1572 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:24.344347  3407 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:24.387617  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:24.382558  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:24.334443  3323 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:02:24.353157  3365 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.396294  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:24.373291  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:24.334041  3349 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:02:24.286039  3181 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:24.477541  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:24.344383  3424 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:02:24.407871  1644 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:24.355731  2343 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 LEARNER]: Advancing to term 4
I20260709 13:02:24.400518  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:24.377576  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:24.394917  3376 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:24.485886  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:24.487269  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:24.402354  3186 consensus_queue.cc:1059] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.093s
I20260709 13:02:24.509585  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:24.543473  2345 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:02:24.484748  1646 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:24.545591  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:24.528954  1648 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:24.526463  1647 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:24.529342  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:24.534484  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:24.535159  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 146) took 1593 ms. Trace:
I20260709 13:02:24.597235  1568 rpcz_store.cc:276] 0709 13:02:22.941930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.943172 (+  1242us) service_pool.cc:224] Handling call
0709 13:02:22.964868 (+ 21696us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.214005 (+249137us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:23.214069 (+    64us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:23.214129 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:23.214181 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.534939 (+1320758us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b5400305f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:24.534992 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.535106 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":572284672}
I20260709 13:02:24.529342  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:24.535789  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:24.484412  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:24.387599  3360 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:24.399319  3359 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.388795  3378 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.607877  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:24.531497  2341 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:24.612043  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:24.613237  1573 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:24.537009  3095 raft_consensus.cc:3055] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:24.380199  3434 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:24.524353  3407 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:24.531944  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:24.536145  3239 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 13:02:24.600512  1641 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:24.515426  3380 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:24.525921  3314 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:24.615710  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:24.616477  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:24.402987  3412 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.624534  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:24.619202  3095 raft_consensus.cc:740] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:24.600209  3334 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:24.389395  3414 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:24.489332  3366 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:24.639760  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:24.530121  3365 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:24.610491  1644 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:24.640198  2345 raft_consensus.cc:1217] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:24.613379  3359 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:24.639171  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 96425.4us
I20260709 13:02:24.644630  1567 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:24.650586  1720 raft_consensus.cc:1240] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:24.630645  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:24.651053  1715 raft_consensus.cc:1275] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:24.650063  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:24.653151  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:25.077757  1716 raft_consensus.cc:2410] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 13:02:24.648129  1568 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:24.594842  3376 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:25.078783  1685 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:25.079198  1641 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:24.655133  3380 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:24.658250  3186 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:25.080607  3380 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:25.080772  3186 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:24.658802  3382 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.083338  3382 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:24.667531  3314 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:24.669247  3291 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:25.083847  3380 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:25.084511  3291 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:25.084781  3380 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.660514  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:24.673517  3181 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:24.670794  3095 consensus_queue.cc:260] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:24.893744  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.895273  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:24.651824  3385 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.652057  3412 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:24.653174  3224 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.623364  3378 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:24.892804  1657 tablet_copy_service.cc:342] P f5f4b2707a9647e5bb99e065068dea2b: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-b82d3b49001741eda872554a7ecb2db3 received from {username='slave'} at 127.0.0.1:44424
I20260709 13:02:25.012820  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:25.005159  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 143) took 2652 ms. Trace:
I20260709 13:02:25.058362  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1740) took 3304 ms. Trace:
I20260709 13:02:25.060181  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:25.061323  1573 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:25.068522  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:25.083729  3186 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:24.777961  3426 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:24.903221  3434 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:25.089407  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:25.201040  1639 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:25.087878  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57572 (request call id 4) took 2294 ms. Trace:
I20260709 13:02:25.114143  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:25.124042  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:25.131497  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:25.190392  1645 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:25.203358  1722 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:25.190501  1644 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:24.920173  2343 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEARNER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:25.117831  1572 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:24.901425  3411 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.191905  1718 rpcz_store.cc:276] 0709 13:02:21.754315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.755157 (+   842us) service_pool.cc:224] Handling call
0709 13:02:21.808496 (+ 53339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.636646 (+1828150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.057926 (+1421280us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b5400299ad4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:25.058284 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4195505536}
I20260709 13:02:25.195293  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:25.016933  3367 consensus_meta.cc:287] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.525s	user 0.017s	sys 0.005s
I20260709 13:02:25.218137  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:24.978156  3297 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.085537  3382 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:25.116865  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.229434  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.231410  1567 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:25.224195  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:24.912709  3329 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.208142  1566 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:25.113008  1717 raft_consensus.cc:2410] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:24.893437  3317 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:25.036187  3239 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.209123  1571 rpcz_store.cc:276] 0709 13:02:22.353351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.358684 (+  5333us) service_pool.cc:224] Handling call
0709 13:02:22.382382 (+ 23698us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:22.473599 (+ 91217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:22.473707 (+   108us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:24.270837 (+1797130us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b2c00000658. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824777ea11 00007f824778e372 00007f8248ec0a01 00007f8248e8543a 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2
0709 13:02:24.272391 (+  1554us) log.cc:844] Serialized 52 byte log entry
0709 13:02:24.579527 (+307136us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:24.804725 (+225198us) log.cc:844] Serialized 14 byte log entry
0709 13:02:24.806397 (+  1672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.886156 (+ 79759us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:24.886274 (+   118us) raft_consensus.cc:1685] finished
0709 13:02:25.004702 (+118428us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.005019 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124105,"prepare.queue_time_us":81348,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":111033,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":22441,"spinlock_wait_cycles":26451328,"thread_start_us":459447,"threads_started":2,"wal-append.queue_time_us":380413}
I20260709 13:02:25.201821  1721 rpcz_store.cc:276] 0709 13:02:22.775957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.893877 (+117920us) service_pool.cc:224] Handling call
0709 13:02:25.069966 (+2176089us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b540029a9d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e7f853 00007f824de4d60a 00007f82473d37d4 00007f82473d3762 00007f82473d3691 00007f82473d360c 00007f82473d28c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:25.070118 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":403394432}
I20260709 13:02:25.205936  3364 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.116406  3366 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:25.199699  3181 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.116494  3365 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:25.217664  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.320827  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.224292  3411 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:25.197563  1657 tablet_copy_service.cc:434] P f5f4b2707a9647e5bb99e065068dea2b: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-b82d3b49001741eda872554a7ecb2db3 on tablet b82d3b49001741eda872554a7ecb2db3 with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:25.342283  3411 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:25.121472  3249 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:25.201861  3259 consensus_queue.cc:1059] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.099s
I20260709 13:02:25.278194  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:25.212397  3186 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:25.085497  3380 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:25.225414  3095 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:25.366773  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:25.368554  1717 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:25.011195  3414 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:25.320998  1573 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:25.257577  1639 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:02:25.326659  3181 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:25.218696  3426 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:25.328702  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:25.385785  1721 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:25.304087  3239 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:25.305332  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.348445  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1119) took 2583 ms. Trace:
I20260709 13:02:25.355120  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.218696  3224 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:25.378651  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1212) took 1003 ms. Trace:
I20260709 13:02:25.402612  1573 rpcz_store.cc:276] 0709 13:02:24.375429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.217240 (+841811us) service_pool.cc:224] Handling call
0709 13:02:25.378464 (+161224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:25.218873  3385 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:25.231273  3367 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:25.229835  3276 consensus_queue.cc:1059] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:02:25.407820  2345 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:25.398115  1643 rpcz_store.cc:276] 0709 13:02:22.764611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.766185 (+  1574us) service_pool.cc:224] Handling call
0709 13:02:24.030168 (+1263983us) spinlock_profiling.cc:243] Waited 2.76 ms on lock 0x7b5800003098. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:02:24.030544 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.119658 (+ 89114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.347934 (+1228276us) spinlock_profiling.cc:243] Waited 71.4 ms on lock 0x7b54000a69d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:25.348343 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170507008}
I20260709 13:02:25.405073  1571 raft_consensus.cc:2393] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 2: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:25.409989  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1219) took 1031 ms. Trace:
I20260709 13:02:25.410461  1571 rpcz_store.cc:276] 0709 13:02:24.378316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.354681 (+976365us) service_pool.cc:224] Handling call
0709 13:02:25.409898 (+ 55217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:25.406002  1568 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:25.302134  3329 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:25.362772  3186 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:25.240540  3297 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:25.306823  3317 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:25.318347  3364 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:25.422405  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1207) took 1049 ms. Trace:
I20260709 13:02:25.388430  3382 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:25.425215  1567 rpcz_store.cc:276] 0709 13:02:24.373259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.232182 (+858923us) service_pool.cc:224] Handling call
0709 13:02:25.422320 (+190138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:25.425709  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:25.439657  1717 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:02:25.417555  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:25.439533  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:25.440959  1722 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:25.441134  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1205) took 1068 ms. Trace:
I20260709 13:02:25.441530  1573 rpcz_store.cc:276] 0709 13:02:24.372751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.413714 (+1040963us) service_pool.cc:224] Handling call
0709 13:02:25.441069 (+ 27355us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:25.428068  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.442953  1571 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:25.443572  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1204) took 1071 ms. Trace:
I20260709 13:02:25.443949  1571 rpcz_store.cc:276] 0709 13:02:24.372501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.423633 (+1051132us) service_pool.cc:224] Handling call
0709 13:02:25.443498 (+ 19865us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:25.388600  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 332614us
I20260709 13:02:25.452329  1571 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:02:25.469166  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 390446us
I20260709 13:02:25.407160  2346 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEARNER]: Advancing to term 5
I20260709 13:02:25.464399  1646 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
W20260709 13:02:25.476557  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.468238  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1217) took 1090 ms. Trace:
I20260709 13:02:25.477635  1571 rpcz_store.cc:276] 0709 13:02:24.377555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.444760 (+1067205us) service_pool.cc:224] Handling call
0709 13:02:25.445209 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:25.467814 (+ 22605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.468156 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:25.453158  1644 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:25.471066  1567 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:25.422710  3186 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:25.483198  1573 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:25.484265  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1203) took 1112 ms. Trace:
I20260709 13:02:25.488828  1567 rpcz_store.cc:276] 0709 13:02:24.372243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.439105 (+1066862us) service_pool.cc:224] Handling call
0709 13:02:25.470875 (+ 31770us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:25.483840 (+ 12965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.484171 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:25.491556  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1214) took 1115 ms. Trace:
I20260709 13:02:25.492085  1573 rpcz_store.cc:276] 0709 13:02:24.376195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.442366 (+1066171us) service_pool.cc:224] Handling call
0709 13:02:25.482947 (+ 40581us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:25.491250 (+  8303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.491494 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:25.464960  3410 raft_consensus.cc:3055] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:25.464008  3382 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:25.503930  3410 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:02:25.541868  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 319891us
I20260709 13:02:25.546577  1612 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:25.548091  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 350630us
I20260709 13:02:25.556172  1538 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:25.551002  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 236609us
I20260709 13:02:25.553021  1645 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:02:25.592059  1538 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:25.618495  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.533732  3378 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:25.622273  1573 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:25.621387  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.518007  3291 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:25.601471  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 55039.3us
I20260709 13:02:25.644380  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.645311  1567 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:25.617136  3259 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:25.600986  3315 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.601032  3249 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.601127  3365 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.649837  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 118) took 1013 ms. Trace:
I20260709 13:02:25.650312  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:25.654368  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 62461us
I20260709 13:02:25.633893  3224 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:25.658775  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.661312  1567 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:25.659782  1573 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:25.664397  1538 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:25.674707  3224 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:25.675947  3234 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:25.676568  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.678247  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.678740  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:25.680383  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.637804  3456 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:25.638990  3378 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.640585  3454 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.649870  3426 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:25.705546  1566 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:25.655108  3259 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.717909  3259 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:25.681842  3365 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:25.721921  3297 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:25.723790  3291 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:25.723976  3380 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:25.684158  3249 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:25.704120  3239 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:25.726625  3249 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:25.690915  3464 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.727322  3454 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:25.727988  3464 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:25.729884  3378 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:25.690445  3224 consensus_queue.cc:1243] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer cf2f71592be8465fb18e36e15d0b2f89 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:02:25.699816  3396 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:25.731916  3396 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:25.732347  3396 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:25.690183  3181 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:25.714282  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.661922  3315 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:25.738464  1573 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:25.740689  3315 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:25.657032  3276 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:25.740160  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:25.743541  3329 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:25.703912  3462 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:02:25.745692  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.746533  1564 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:25.723829  3365 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:25.747119  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1195) took 1377 ms. Trace:
I20260709 13:02:25.747622  3365 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.741442  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.749027  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1194) took 1379 ms. Trace:
I20260709 13:02:25.749408  1569 rpcz_store.cc:276] 0709 13:02:24.369332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.372328 (+  2996us) service_pool.cc:224] Handling call
0709 13:02:25.741192 (+1368864us) spinlock_profiling.cc:243] Waited 5.85 ms on lock 0x7b5800011198. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60eee 0000000000375939 00007f824ddd41e0 00007f824de9d0c4 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:02:25.748960 (+  7768us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13454720}
I20260709 13:02:25.750183  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:25.750682  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.750941  1717 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:25.751487  1569 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:25.752313  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 1754) took 1091 ms. Trace:
I20260709 13:02:25.740140  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.743579  1715 raft_consensus.cc:2410] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:25.716468  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1213) took 1340 ms. Trace:
I20260709 13:02:25.753729  3315 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:02:25.766934  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 1756) took 1104 ms. Trace:
I20260709 13:02:25.640156  3410 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:25.793610  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 169603us
I20260709 13:02:25.814787  1566 rpcz_store.cc:276] 0709 13:02:24.375772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.645629 (+269857us) service_pool.cc:224] Handling call
0709 13:02:25.716364 (+1070735us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:25.815088  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:25.817123  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:02:25.818399  1687 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.585s (SENT)
I20260709 13:02:25.818960  1566 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:25.819401  3359 consensus_meta.cc:287] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.702s	user 0.010s	sys 0.001s
I20260709 13:02:25.778347  3365 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:25.752926  1717 rpcz_store.cc:276] 0709 13:02:24.660994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.749729 (+1088735us) service_pool.cc:224] Handling call
0709 13:02:25.752227 (+  2498us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:25.819082  1687 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:25.819916  3359 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:25.820601  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 136339us
I20260709 13:02:25.733281  3378 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:25.813278  1715 rpcz_store.cc:276] 0709 13:02:24.661622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.739891 (+1078269us) service_pool.cc:224] Handling call
0709 13:02:25.766328 (+ 26437us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:25.825616  1573 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:25.830585  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:25.831645  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:25.833979  1565 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1199) took 1463 ms (client timeout 1535 ms). Trace:
W20260709 13:02:25.834587  1565 rpcz_store.cc:269] 0709 13:02:24.370741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.372010 (+  1269us) service_pool.cc:224] Handling call
0709 13:02:25.831461 (+1459451us) spinlock_profiling.cc:243] Waited 77.3 ms on lock 0x7b5800011198. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:02:25.833853 (+  2392us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177893760}
I20260709 13:02:25.722416  3364 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:25.842471  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:25.763939  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:25.712383  3414 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:25.848237  1538 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:25.764328  3469 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:25.843261  1717 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:25.806562  3464 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:25.849182  1714 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:25.830094  3095 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:25.857419  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.779639  3367 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:25.861557  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 195895us
I20260709 13:02:25.866072  1565 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:25.855597  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 1766) took 1191 ms. Trace:
I20260709 13:02:25.836033  3291 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.852768  3410 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:25.712138  3456 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:25.882333  1716 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:25.872816  1566 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:25.882035  3291 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:25.883718  3365 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:25.883915  3470 log.cc:927] Time spent T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:02:25.868482  3378 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:02:25.901065  3386 raft_consensus.cc:3055] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:25.871835  3461 consensus_queue.cc:1059] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:02:25.870303  1717 rpcz_store.cc:276] 0709 13:02:24.664051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.821980 (+1157929us) service_pool.cc:224] Handling call
0709 13:02:25.855522 (+ 33542us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:25.891786  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:25.892333  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:25.903359  1720 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:25.903693  2349 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:25.904352  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:25.894038  3095 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.894209  3186 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:25.895138  3376 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:25.895589  3291 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:25.926577  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:25.927603  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.928789  1565 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
W20260709 13:02:25.930229  1519 auto_rebalancer.cc:893] Removing replace marker failed: Replica f5f4b2707a9647e5bb99e065068dea2b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
W20260709 13:02:25.930766  1519 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:02:25.931063  1519 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:02:25.933691  3454 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:25.935025  1565 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:25.902722  3461 raft_consensus.cc:3055] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:25.936076  3461 raft_consensus.cc:740] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:25.936939  3461 consensus_queue.cc:260] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:25.938584  1565 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:25.939816  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.940526  1566 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:25.896204  3385 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:25.897039  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:25.897351  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:25.898102  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:25.898620  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:25.913579  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 1764) took 1250 ms. Trace:
I20260709 13:02:25.949818  1720 rpcz_store.cc:276] 0709 13:02:24.663576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.891339 (+1227763us) service_pool.cc:224] Handling call
0709 13:02:25.913512 (+ 22173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:25.949690  1713 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:25.915109  1717 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:25.916109  1686 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:25.918532  1573 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:02:25.915742  2313 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:02:25.924021  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.901904  1715 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:25.747643  1564 rpcz_store.cc:276] 0709 13:02:24.369655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.383234 (+ 13579us) service_pool.cc:224] Handling call
0709 13:02:25.745451 (+1362217us) spinlock_profiling.cc:243] Waited 62.1 ms on lock 0x7b5800011198. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:02:25.747046 (+  1595us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142718336}
I20260709 13:02:25.957260  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:25.938302  3461 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:25.960855  1687 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.695s (SENT)
I20260709 13:02:25.944245  3456 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:25.946852  1572 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1208) took 1573 ms (client timeout 1605 ms). Trace:
I20260709 13:02:25.901609  3386 raft_consensus.cc:740] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:25.866016  3414 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:25.981624  1568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1215) took 1605 ms (client timeout 2118 ms). Trace:
I20260709 13:02:25.948648  1571 raft_consensus.cc:1275] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:25.959117  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:25.938475  3475 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.121s	user 0.002s	sys 0.000s
I20260709 13:02:25.968828  3276 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.961745  1687 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:25.991972  3276 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:25.939065  3396 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:25.994143  1687 leader_election.cc:336] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.703s (SENT)
I20260709 13:02:25.994598  1687 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:25.996066  3224 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:02:25.966908  1572 rpcz_store.cc:269] 0709 13:02:24.373686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.508934 (+135248us) service_pool.cc:224] Handling call
0709 13:02:25.946590 (+1437656us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b5400305a54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:25.946765 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1210352896}
I20260709 13:02:25.952124  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 1767) took 1287 ms. Trace:
I20260709 13:02:25.968734  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:25.996307  3276 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:25.997330  1713 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:25.998137  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 1761) took 1335 ms. Trace:
I20260709 13:02:25.998592  1713 rpcz_store.cc:276] 0709 13:02:24.662877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.951478 (+1288601us) service_pool.cc:224] Handling call
0709 13:02:25.998020 (+ 46542us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:26.000118  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:26.000885  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 114899us
I20260709 13:02:25.969624  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:25.969910  3333 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:26.002223  1687 leader_election.cc:336] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.736s (SENT)
I20260709 13:02:25.948220  3095 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:26.002702  1687 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:26.003104  2343 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 LEARNER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 4. Current term is 5. Ops: [2.3-4.4]
I20260709 13:02:26.003588  3328 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:26.004623  3095 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:25.970680  3259 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:25.978210  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:25.978199  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:25.979635  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:26.007169  1570 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1200) took 1636 ms (client timeout 1922 ms). Trace:
I20260709 13:02:26.007759  3359 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:26.007577  1570 rpcz_store.cc:269] 0709 13:02:24.371143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.371903 (+   760us) service_pool.cc:224] Handling call
0709 13:02:25.979384 (+1607481us) spinlock_profiling.cc:243] Waited 81.9 ms on lock 0x7b5800011198. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:02:26.007106 (+ 27722us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188273408}
W20260709 13:02:25.982316  1568 rpcz_store.cc:269] 0709 13:02:24.376747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.228733 (+851986us) service_pool.cc:224] Handling call
0709 13:02:25.981381 (+752648us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b5400013954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:25.981542 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":399790976}
I20260709 13:02:25.983610  1720 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:26.008905  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:26.009701  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:26.009913  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:25.982200  3414 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.010921  3414 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.010955  1572 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 1.
I20260709 13:02:25.659497  1639 rpcz_store.cc:276] 0709 13:02:24.636046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.384247 (+748201us) service_pool.cc:224] Handling call
0709 13:02:25.384648 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:25.384878 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:25.419168 (+ 34290us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.649391 (+230223us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:25.649456 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:25.649510 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.649581 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.649755 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":38058,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":69,"raft.queue_time_us":512,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":8898,"thread_start_us":229812,"threads_started":1}
I20260709 13:02:25.972499  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:26.008919  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:26.012789  1717 rpcz_store.cc:276] 0709 13:02:24.664312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.904026 (+1239714us) service_pool.cc:224] Handling call
0709 13:02:25.951751 (+ 47725us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:26.014236  1687 leader_election.cc:336] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.722s (SENT)
I20260709 13:02:26.014724  1687 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:25.996832  3224 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:25.954154  3186 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:26.008662  1567 raft_consensus.cc:2393] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:26.010272  1720 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:25.957808  3403 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:25.962162  3385 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:25.948751  3412 consensus_meta.cc:287] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.718s	user 0.014s	sys 0.000s
I20260709 13:02:25.992847  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 116) took 1371 ms. Trace:
I20260709 13:02:26.015687  1713 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 1.
I20260709 13:02:26.027439  1720 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:26.049379  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 1762) took 1386 ms. Trace:
I20260709 13:02:26.047972  1640 rpcz_store.cc:276] 0709 13:02:24.621486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.222206 (+600720us) service_pool.cc:224] Handling call
0709 13:02:25.222718 (+   512us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:25.618994 (+396276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:25.619030 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:25.640149 (+ 21119us) log.cc:844] Serialized 52 byte log entry
0709 13:02:25.822283 (+182134us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:25.822351 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.822422 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:25.992416 (+169994us) raft_consensus.cc:1685] finished
0709 13:02:25.992443 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.992766 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10190,"raft.queue_time_us":120090,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":125,"thread_start_us":433169,"threads_started":2,"wal-append.queue_time_us":154522}
I20260709 13:02:26.049744  1720 rpcz_store.cc:276] 0709 13:02:24.663116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.009453 (+1346337us) service_pool.cc:224] Handling call
0709 13:02:26.049323 (+ 39870us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:26.027997  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.050853  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.052183  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.053200  1720 raft_consensus.cc:2393] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:02:26.026619  1687 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.761s (SENT)
I20260709 13:02:26.027629  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:26.054666  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.055536  1720 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:26.056464  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:26.056488  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:26.057729  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.057863  1570 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:26.058537  1720 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:26.059671  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.060664  1720 raft_consensus.cc:2410] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:26.061622  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.062454  1720 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:26.062213  2313 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:26.063687  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.063939  3333 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:26.064337  1720 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:26.064585  3333 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.065217  3333 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.059506  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:26.067389  1570 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:26.068552  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:26.065307  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.071388  1720 raft_consensus.cc:2410] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:26.075021  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:26.075367  1687 leader_election.cc:336] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.791s (SENT)
I20260709 13:02:26.076017  1687 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:26.076592  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.077467  1639 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:26.076592  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:26.079761  1572 raft_consensus.cc:2410] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:26.079879  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.080564  1717 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:26.027118  3456 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:26.025899  3403 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.082517  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.027357  3463 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:26.084401  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:26.084838  1720 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:26.015003  1564 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:26.085618  1570 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:26.086143  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:26.070748  3385 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.088091  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:26.092386  1720 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.088551  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.097388  1713 raft_consensus.cc:2410] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:02:26.097178  1687 leader_election.cc:336] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.829s (SENT)
I20260709 13:02:26.098872  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.099166  1687 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:26.099756  1713 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:26.100776  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.102443  1568 raft_consensus.cc:1240] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:26.104210  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.104905  1717 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:26.081352  3224 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:02:26.115880  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.117071  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.117798  1644 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:26.118643  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.119485  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.120456  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.120666  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.122014  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:26.122749  2344 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:26.124526  1648 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:26.125689  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.126250  1648 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:26.127071  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.127574  1648 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:26.085597  3403 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:26.094816  3463 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:26.018805  3386 consensus_queue.cc:260] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:26.136288  3386 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.110514  3456 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:26.139349  3452 log.cc:927] Time spent T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.687s	user 0.003s	sys 0.000s
I20260709 13:02:26.096455  3385 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:26.144299  3396 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:26.082553  3474 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:02:26.117436  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.121588  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.147171  1641 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:26.115724  2314 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:26.147287  1643 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:26.121572  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.070101  3412 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:26.152697  1642 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:26.153169  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.157335  1717 raft_consensus.cc:2410] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:26.157248  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.159334  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.159667  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.160707  1642 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:26.162091  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.163622  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:26.101555  1713 raft_consensus.cc:2410] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:26.166236  1720 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
W20260709 13:02:26.170665  1687 leader_election.cc:336] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.535s (SENT)
W20260709 13:02:26.179839  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.182015  1642 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:26.183307  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:26.184103  1642 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:26.183921  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:26.186177  1687 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.910s (SENT)
I20260709 13:02:26.188105  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.188405  1687 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:26.189272  1641 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:26.192824  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.195807  1648 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
W20260709 13:02:26.200914  1687 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.921s (SENT)
I20260709 13:02:26.137661  3463 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:26.204063  1645 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:26.205161  1717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:02:26.206107  1717 raft_consensus.cc:606] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:26.206596  1717 raft_consensus.cc:614] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:26.202291  1687 leader_election.cc:336] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.926s (SENT)
I20260709 13:02:26.207826  1687 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:26.210703  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.221248  3456 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:26.189105  3287 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:26.181257  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 65659.9us
I20260709 13:02:26.207800  3224 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:26.158527  3186 consensus_queue.cc:1059] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:02:26.245090  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.258631  1713 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:26.208194  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:02:26.260425  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 187159us
I20260709 13:02:26.213480  3359 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:02:26.269290  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.299413  3414 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:26.300251  1687 leader_election.cc:336] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.951s (SENT)
I20260709 13:02:26.301049  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.318346  1641 raft_consensus.cc:2393] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:26.319514  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.320554  1641 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:26.304371  1643 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
W20260709 13:02:26.317481  1687 leader_election.cc:336] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.998s (SENT)
I20260709 13:02:26.304572  3259 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:26.305147  1642 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:26.325165  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.307130  3385 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:26.308378  1714 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 110) took 1974 ms (client timeout 1835 ms). Trace:
W20260709 13:02:26.310665  1611 connection.cc:256] RPC call timeout handler was delayed by -1.9023325450000357s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1628s for a response
I20260709 13:02:26.328171  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:26.328717  1611 connection.cc:256] RPC call timeout handler was delayed by -1.7844325450000156s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1759s for a response
I20260709 13:02:26.257302  3287 raft_consensus.cc:697] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:02:26.330188  1714 rpcz_store.cc:269] 0709 13:02:24.334360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.334485 (+   125us) service_pool.cc:224] Handling call
0709 13:02:26.307973 (+1973488us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b5400247554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:26.308225 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3430603008}
I20260709 13:02:26.322697  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.323633  1687 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:26.295230  3474 consensus_queue.cc:1048] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:02:26.327138  1713 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:26.329398  1611 connection.cc:256] RPC call timeout handler was delayed by -1.3524325449999992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2239s for a response
I20260709 13:02:26.331226  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:26.312098  1537 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:26.329349  1641 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:26.314976  1717 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:26.328167  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.360097  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.361171  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1758) took 1699 ms. Trace:
W20260709 13:02:26.362993  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 50963.6us
I20260709 13:02:26.364737  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:26.365794  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:26.368394  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 50949.2us
I20260709 13:02:26.364318  1717 rpcz_store.cc:276] 0709 13:02:24.662153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.228342 (+1566189us) service_pool.cc:224] Handling call
0709 13:02:26.314536 (+ 86194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.360851 (+ 46315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.361103 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:26.370883  1714 raft_consensus.cc:3055] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:02:26.371038  1687 leader_election.cc:336] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.026s (SENT)
I20260709 13:02:26.366019  1537 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:26.372301  1714 raft_consensus.cc:740] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:26.366894  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.372124  1687 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:26.332773  3276 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:02:26.377563  1687 leader_election.cc:336] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.086s (SENT)
I20260709 13:02:26.337357  3359 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:26.385560  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.381572  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.335371  2344 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:26.362277  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1759) took 1699 ms. Trace:
I20260709 13:02:26.378360  1611 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:26.389669  1717 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:26.388669  1639 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:26.389495  1713 rpcz_store.cc:276] 0709 13:02:24.662408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.326352 (+1663944us) service_pool.cc:224] Handling call
0709 13:02:26.326866 (+   514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.361967 (+ 35101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.362182 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:26.362301  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:26.391667  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.391815  1642 raft_consensus.cc:2376] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2. Re-sending same reply.
I20260709 13:02:26.392042  1713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:02:26.392699  1639 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:26.373483  1714 consensus_queue.cc:260] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:26.394407  3400 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:26.393831  3481 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:26.395351  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.395895  3276 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:26.361377  3382 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:26.379544  3463 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:26.398312  3463 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.398809  3412 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:26.398976  3463 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:26.359184  2314 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:26.402026  3287 consensus_queue.cc:237] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:26.402519  3385 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:26.380352  3186 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:26.406914  3414 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:26.389868  1641 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:26.365091  3259 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:26.410418  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:26.411815  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:26.411913  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:26.413146  1713 raft_consensus.cc:606] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:26.395377  1714 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:26.414335  1641 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:26.413839  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:26.416898  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1768) took 1752 ms. Trace:
I20260709 13:02:26.417263  1639 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:26.390091  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.413403  1643 raft_consensus.cc:2376] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3. Re-sending same reply.
I20260709 13:02:26.415926  1713 raft_consensus.cc:614] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:26.415962  1537 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:26.417583  1717 rpcz_store.cc:276] 0709 13:02:24.664619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.379203 (+1714584us) service_pool.cc:224] Handling call
0709 13:02:26.380101 (+   898us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.416355 (+ 36254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.416821 (+   466us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:26.419260  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:26.420197  3462 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:26.420361  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:26.444075  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.444758  1643 raft_consensus.cc:2410] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:26.444943  1641 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
W20260709 13:02:26.420629  3482 log.cc:927] Time spent T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:02:26.420928  3276 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:26.421394  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:26.421938  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 264304us
I20260709 13:02:26.453775  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:26.430382  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.454726  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.456496  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.385043  3334 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.445816  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.446301  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.461192  1643 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:26.461933  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.462273  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.463876  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:26.446447  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.449604  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:26.457360  1722 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:02:26.464318  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.465770  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.466702  1722 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:26.468055  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.468873  1722 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 1.
I20260709 13:02:26.471467  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.472967  1722 raft_consensus.cc:2410] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:26.474021  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.474967  1722 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:26.474736  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.475178  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:26.476140  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.476600  1643 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:26.476759  1641 raft_consensus.cc:2376] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3. Re-sending same reply.
I20260709 13:02:26.477035  1722 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:26.481217  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.438074  3253 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:26.483033  1722 raft_consensus.cc:2410] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:26.469965  3400 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:26.450716  3474 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:26.484359  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:26.419070  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 87545.6us
I20260709 13:02:26.488325  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:26.485370  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.490423  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:26.491896  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:26.484464  3410 consensus_meta.cc:287] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.591s	user 0.012s	sys 0.004s
I20260709 13:02:26.500597  1644 raft_consensus.cc:2410] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:26.495365  3469 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:26.501292  3469 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:26.482107  3414 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.488490  1565 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:26.491901  3400 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.505539  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:26.478762  3334 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:26.509025  1612 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:26.509192  1714 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 13:02:26.506269  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:26.497375  3461 consensus_meta.cc:287] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.530s	user 0.010s	sys 0.000s
I20260709 13:02:26.447623  3403 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:26.522558  1648 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:26.507673  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.511117  1714 pending_rounds.cc:85] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 2
I20260709 13:02:26.511546  1572 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:26.523819  1714 pending_rounds.cc:107] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260709 13:02:26.508034  3414 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:26.518646  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:26.510540  3400 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:26.518962  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:26.523262  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:26.528555  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:26.464999  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.524379  1714 raft_consensus.cc:2889] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:26.529698  1714 pending_rounds.cc:107] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:02:26.530009  1685 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:26.530709  1714 raft_consensus.cc:2889] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:26.508458  3410 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.528921  1722 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:26.536156  3410 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:26.537987  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:26.539332  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:26.539906  3317 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:26.540853  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:26.543320  3414 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:26.519508  3463 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:26.531962  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1763) took 1868 ms. Trace:
I20260709 13:02:26.533815  1640 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:26.533964  1685 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:26.546550  3385 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:26.547101  3385 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.546649  1685 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:26.505321  3333 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:26.528280  3461 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:26.535315  3186 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:26.550194  1685 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:26.550686  1572 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:02:26.550827  1714 rpcz_store.cc:276] 0709 13:02:24.663353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.333019 (+1669666us) service_pool.cc:224] Handling call
0709 13:02:26.370738 (+ 37719us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.531616 (+160878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.531887 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:26.553675  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.547915  3385 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:26.566375  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.550004  3359 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:26.577602  1713 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:26.580663  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:26.581034  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:26.581272  1713 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.581678  1714 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.577626  1642 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:26.531280  3462 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:26.583293  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.579042  1722 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:26.591333  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.593727  1722 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:26.594969  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.583866  1685 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:26.546594  3463 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:26.601913  1685 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:26.550402  1564 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:26.603847  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 128487us
I20260709 13:02:26.558881  3186 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.604310  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.608091  1642 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:26.616027  1713 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:26.617825  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:26.589134  3382 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:26.643049  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:26.652278  1642 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:26.593959  3462 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:26.652982  1714 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:26.594393  3456 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 13:02:26.604602  3253 raft_consensus.cc:697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:26.577531  3461 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:26.655296  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 146350us
I20260709 13:02:26.664530  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:26.595125  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 99063us
I20260709 13:02:26.675673  1714 raft_consensus.cc:2410] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:26.674345  1612 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:26.608642  3095 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:26.680297  1612 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:26.651229  3382 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.685813   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:26.612298  3186 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:26.658500  3462 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.691514  1643 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.694876  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:26.696974  1718 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:26.701594  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:26.713025  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:26.703018  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1769) took 1487 ms. Trace:
I20260709 13:02:26.701849  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:26.714128  1642 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:26.715843  1718 rpcz_store.cc:276] 0709 13:02:25.216090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.696398 (+1480308us) service_pool.cc:224] Handling call
0709 13:02:26.696768 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.702666 (+  5898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.702945 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:26.716789  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:26.718777  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:26.139468  3365 consensus_queue.cc:1175] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "130bc5ff39b847b1b3204626f284f838" responder_term: 3 status { last_received { term: 2 index: 2 } last_committed_idx: 2 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:26.719436  1642 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:26.721344  1642 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:26.723011  1642 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 4. Current term is 5. Ops: [2.3-4.4]
I20260709 13:02:26.726302  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 147) took 1033 ms. Trace:
I20260709 13:02:26.726781  1573 rpcz_store.cc:276] 0709 13:02:25.694291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.771387 (+ 77096us) service_pool.cc:224] Handling call
0709 13:02:26.726088 (+954701us) spinlock_profiling.cc:243] Waited 8.55 ms on lock 0x7b54001275d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:26.726233 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19655040}
I20260709 13:02:26.728235  2345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 268) took 1590 ms. Trace:
I20260709 13:02:26.728636  2345 rpcz_store.cc:276] 0709 13:02:25.137586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.201482 (+ 63896us) service_pool.cc:224] Handling call
0709 13:02:26.728017 (+1526535us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b54000cc954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:26.728139 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":403987840}
I20260709 13:02:26.730063  3253 consensus_queue.cc:237] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:26.731288  3403 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:26.731469  1720 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:26.732836  1720 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:26.730104  3334 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:26.736860  1646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1766) took 2156 ms (client timeout 2261 ms). Trace:
W20260709 13:02:26.737321  1646 rpcz_store.cc:269] 0709 13:02:24.580079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.602681 (+ 22602us) service_pool.cc:224] Handling call
0709 13:02:26.736607 (+2133926us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b54000a73d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:26.736758 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":892475776}
I20260709 13:02:26.739629  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1463) took 1144 ms. Trace:
I20260709 13:02:26.732257  3385 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:26.742177  1471 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47518 (request call id 92) took 7579 ms. Trace:
I20260709 13:02:26.743091  3461 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:26.743482  3168 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:26.743705  3461 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:26.743966  3168 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.744119  3461 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.744416  3168 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:26.744535  3461 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.747524  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:26.749596  2349 raft_consensus.cc:2393] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:26.740206  1567 rpcz_store.cc:276] 0709 13:02:25.595349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.679751 (+ 84402us) service_pool.cc:224] Handling call
0709 13:02:26.739393 (+1059642us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b540041c8d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:26.739560 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":752416640}
W20260709 13:02:26.757366  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.743185  3469 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:26.749315   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:26.691280  3382 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.676683  3181 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:26.701994  3410 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:26.777323  1643 raft_consensus.cc:2410] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:26.757691  1572 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:26.786695  1714 raft_consensus.cc:1275] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:02:26.785591  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 183772us
I20260709 13:02:26.791288  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:26.792309  1714 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:26.791584  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:26.798316  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:02:26.799324  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 193828us
I20260709 13:02:26.790766  2344 raft_consensus.cc:2368] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6.
I20260709 13:02:26.802946  1611 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:26.836884  1611 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:26.839533  1611 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:26.818215  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.795395  1685 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:26.834081  1718 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 6: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:26.777920  3434 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.742619  1471 rpcz_store.cc:276] 0709 13:02:19.163258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.163545 (+   287us) service_pool.cc:224] Handling call
0709 13:02:26.742090 (+7578545us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:25.010204 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:25.010934 (+   730us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:25.010979 (+    45us) write_op.cc:435] Acquired schema lock
0709 13:02:25.010996 (+    17us) tablet.cc:662] Decoding operations
0709 13:02:25.011415 (+   419us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:25.011469 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:25.011499 (+    30us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:02:25.011710 (+   211us) tablet.cc:701] Row locks acquired
0709 13:02:25.011727 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:02:25.011913 (+   186us) write_op.cc:270] Start()
0709 13:02:25.012036 (+   123us) write_op.cc:276] Timestamp: P: 1783602145011861 usec, L: 0
0709 13:02:25.012058 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:02:25.013263 (+  1205us) log.cc:844] Serialized 599 byte log entry
0709 13:02:25.973806 (+960543us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x7b1800315580. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:02:26.134422 (+160616us) op_driver.cc:464] REPLICATION: finished
0709 13:02:26.486614 (+352192us) write_op.cc:301] APPLY: starting
0709 13:02:26.487815 (+  1201us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:26.487922 (+   107us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:26.492215 (+  4293us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:26.494470 (+  2255us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:26.494920 (+   450us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:02:26.494986 (+    66us) write_op.cc:312] APPLY: finished
0709 13:02:26.495212 (+   226us) log.cc:844] Serialized 23 byte log entry
0709 13:02:26.495413 (+   201us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:26.495502 (+    89us) write_op.cc:454] Released schema lock
0709 13:02:26.495542 (+    40us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:26.495986 (+   444us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:26.496013 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3265821,"reported_tablets":5,"spinlock_wait_cycles":9472,"child_traces":[["op",{"apply.queue_time_us":319981,"apply.run_cpu_time_us":2679,"apply.run_wall_time_us":13558,"num_ops":1,"prepare.queue_time_us":685032,"prepare.run_cpu_time_us":49969,"prepare.run_wall_time_us":633353,"raft.queue_time_us":157744,"raft.run_cpu_time_us":22572,"raft.run_wall_time_us":111803,"replication_time_us":1122119,"spinlock_wait_cycles":5034240,"thread_start_us":1056087,"threads_started":4,"wal-append.queue_time_us":642845}]]}
I20260709 13:02:26.845300  1611 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:26.791443  3456 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:26.868522  1687 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:26.788689  3181 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:26.882465  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 271229us
I20260709 13:02:26.873601  3497 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:26.890995  3497 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.891783  3497 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:26.896157  3456 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.896950  3456 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:26.897938  3502 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:26.900198  3456 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:02:26.900833  3456 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:26.901198  3456 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:26.901552  3456 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:26.884284  3335 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:02:26.909746  3335 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:26.910689  3335 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:26.911751  3335 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:26.914644  3500 log.cc:927] Time spent T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:26.856288  3410 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:02:26.878396  3385 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:26.943254  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 112) took 1879 ms. Trace:
I20260709 13:02:26.920883  3333 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:26.863534  3434 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:26.943922  1719 rpcz_store.cc:276] 0709 13:02:25.063634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.422468 (+1358834us) service_pool.cc:224] Handling call
0709 13:02:26.422801 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.423077 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:26.520219 (+ 97142us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.942718 (+422499us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.942804 (+    86us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:26.942877 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.942979 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.943167 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":171356,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":222253,"raft.run_cpu_time_us":946,"raft.run_wall_time_us":5271,"thread_start_us":517188,"threads_started":3,"wal-append.queue_time_us":267381}
I20260709 13:02:26.898820  3502 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:26.956033  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1478) took 1040 ms. Trace:
I20260709 13:02:26.957813  1719 rpcz_store.cc:276] 0709 13:02:25.915444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.951318 (+1035874us) service_pool.cc:224] Handling call
0709 13:02:26.952051 (+   733us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.952433 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:26.952484 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.955557 (+  3073us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:26.955651 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.955751 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.955948 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:26.957152  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 276914us
I20260709 13:02:26.982394  1612 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:26.948163  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:27.010403  1612 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:27.034327  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.945329  3495 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:26.959695  3502 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:27.038875  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 170654us
I20260709 13:02:27.044072  1687 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:27.040542  1537 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:26.969762  3504 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:27.005295  3359 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:27.058622  1687 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:27.042671  3495 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:27.067004  1537 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:27.079953  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:27.029676  3095 raft_consensus.cc:697] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:27.015934  3382 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:27.061153  3385 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:27.062361  3469 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:27.089421  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:27.080556  3495 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:27.123346  1611 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:27.061307  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:02:27.130404  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 1753) took 2474 ms (client timeout 2170 ms). Trace:
W20260709 13:02:27.130883  1716 rpcz_store.cc:269] 0709 13:02:24.655829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.678312 (+1022483us) service_pool.cc:224] Handling call
0709 13:02:27.130190 (+1451878us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54000230d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:27.130324 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":458566016}
I20260709 13:02:27.050818  3497 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:27.125281  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 114) took 2036 ms. Trace:
I20260709 13:02:27.126410  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1782) took 1009 ms. Trace:
I20260709 13:02:27.137285  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:27.107026  1715 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 1760) took 2444 ms (client timeout 2559 ms). Trace:
W20260709 13:02:27.140067  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 89338.6us
I20260709 13:02:27.108534  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:27.147182  1715 rpcz_store.cc:269] 0709 13:02:24.662636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.830271 (+1167635us) service_pool.cc:224] Handling call
0709 13:02:27.106575 (+1276304us) spinlock_profiling.cc:243] Waited 68.3 ms on lock 0x7b54003a2fd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:27.106949 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156985600}
I20260709 13:02:27.061411  3471 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:27.133333  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:27.173710  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 378484us
I20260709 13:02:27.174304  1716 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:27.149027  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:27.175946  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:27.178344  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:27.179244  1716 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:27.147535  1687 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:27.133613  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:27.145742  1721 rpcz_store.cc:276] 0709 13:02:25.089362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.660295 (+1570933us) service_pool.cc:224] Handling call
0709 13:02:26.686557 (+ 26262us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:26.948661 (+262104us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:26.948693 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:26.958679 (+  9986us) log.cc:844] Serialized 26 byte log entry
0709 13:02:27.021948 (+ 63269us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:27.022007 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.022113 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:27.125042 (+102929us) raft_consensus.cc:1685] finished
0709 13:02:27.125062 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.125205 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":207940,"raft.run_cpu_time_us":974,"raft.run_wall_time_us":973,"thread_start_us":324447,"threads_started":2,"wal-append.queue_time_us":132691}
I20260709 13:02:27.092803  1537 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:27.181115  1646 raft_consensus.cc:2376] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3. Re-sending same reply.
I20260709 13:02:27.154070  1644 rpcz_store.cc:276] 0709 13:02:26.117213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.118424 (+  1211us) service_pool.cc:224] Handling call
0709 13:02:27.126186 (+1007762us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b54000fffd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:27.126344 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":876341504}
I20260709 13:02:27.158893  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:27.183192  1572 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 6: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:27.160028  1573 raft_consensus.cc:1240] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:27.161734  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:27.185539  1566 raft_consensus.cc:2376] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5. Re-sending same reply.
I20260709 13:02:27.187436  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:27.059809  3509 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:27.159212  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:27.146735  1647 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:27.125792  3314 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:27.197396  1647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1771) took 1225 ms (client timeout 1576 ms). Trace:
I20260709 13:02:27.190413  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:27.197402  1567 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 6: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:02:27.199491  1647 rpcz_store.cc:269] 0709 13:02:25.971525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.971749 (+   224us) service_pool.cc:224] Handling call
0709 13:02:27.108136 (+1136387us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b5800003098. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:02:27.197294 (+ 89158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23802240}
I20260709 13:02:27.200850  1566 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:27.201180  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 108519us
I20260709 13:02:27.199066  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:27.202013  1537 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:27.202312  1568 raft_consensus.cc:2376] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4. Re-sending same reply.
W20260709 13:02:27.203671  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 46516.4us
I20260709 13:02:27.204869  1722 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:27.205956  1715 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:27.211462  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1484) took 1133 ms. Trace:
I20260709 13:02:27.211967  1715 rpcz_store.cc:276] 0709 13:02:26.078905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.205199 (+1126294us) service_pool.cc:224] Handling call
0709 13:02:27.205605 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:27.211082 (+  5477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.211390 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:27.115224  3411 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.208271  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:27.137539  3497 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:27.214680  1573 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:27.213824  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:27.215126  1570 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:27.217259  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:27.215502  1537 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:27.218520  1568 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:27.179976  3359 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:27.219216  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:27.194456  3471 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:27.261945  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 114995us
W20260709 13:02:27.262373  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.145862  3385 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:27.218864  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:27.274981  1567 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:27.095301  3181 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:27.218127  3514 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:02:27.196271  3287 consensus_queue.cc:1059] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:02:27.223637  3515 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:27.196985  3509 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.223830  3462 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:27.224131  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:27.209399  1538 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:27.265534  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 255668us
I20260709 13:02:27.319926  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:27.320815  3462 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:27.269568  1566 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:27.321208  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:27.263005  1687 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:27.278812  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:02:27.322317  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 59431.1us
I20260709 13:02:27.324371  3504 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:27.324873  3504 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.325220  3504 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.325541  1687 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:27.220180  3474 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:27.326840  3474 raft_consensus.cc:697] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:27.328227  3474 consensus_queue.cc:237] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:27.328356  3335 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:27.222874  3180 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:27.322274  3315 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:27.330004  3180 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:27.330690  3180 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:27.319692  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 110420us
W20260709 13:02:27.311003  3186 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.603s	user 0.011s	sys 0.002s
I20260709 13:02:27.220064  3463 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:27.309280  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:27.219820  3396 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.116s
I20260709 13:02:27.338220  1687 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:27.338579  1572 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:02:27.339411  3359 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:27.339820  3359 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.340252  3359 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.322368  3104 raft_consensus.cc:3055] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:27.342105  3104 raft_consensus.cc:740] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:27.343000  3104 consensus_queue.cc:260] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:27.344388  3104 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.346283  3463 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:27.282878  3513 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:27.347453  3463 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:27.311004  3512 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:27.348433  3512 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:27.349349  3512 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:27.339813  1687 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:27.352543  3510 raft_consensus.cc:2764] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:02:27.311563  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 116) took 1351 ms. Trace:
W20260709 13:02:27.356671  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.355329  1719 rpcz_store.cc:276] 0709 13:02:25.960519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.959882 (+999363us) service_pool.cc:224] Handling call
0709 13:02:26.961106 (+  1224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.961410 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:27.012495 (+ 51085us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.310721 (+298226us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:27.310795 (+    74us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:27.310841 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.311022 (+   181us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.311499 (+   477us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88714,"prepare.queue_time_us":64166,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":88865,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":85,"thread_start_us":208765,"threads_started":1}
I20260709 13:02:27.321153  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:27.333088  1538 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:27.302812  3181 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:27.224201  3095 consensus_queue.cc:237] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:27.303432  3515 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.318307  3509 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.271660  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:27.323431  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:27.389074  1573 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:27.389492  1570 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
W20260709 13:02:27.304776  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 128978us
I20260709 13:02:27.390789  1685 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:27.347625  3513 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.393780  3513 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.331915  3335 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:27.397907  3510 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:27.398777  3510 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:27.399226  3510 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:27.310273  3382 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:27.198650  3314 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.224628  3469 consensus_queue.cc:1059] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.136s
I20260709 13:02:27.224808  3365 consensus_queue.cc:1059] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.137s
I20260709 13:02:27.392959  1685 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:27.402184  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 274346us
I20260709 13:02:27.359259  1687 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:27.359004  1612 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:27.360668  1538 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:27.280381  3385 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.386577  3180 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:27.323475  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:27.263965  3471 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:27.444924  1685 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:27.446410  1685 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:27.448823  1685 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:27.403301  1611 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:27.450299  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 47079.3us
I20260709 13:02:27.451869  1611 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:27.455660  3516 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:27.459010  1611 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:27.388387  1716 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
W20260709 13:02:27.373785  3461 consensus_meta.cc:287] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.629s	user 0.014s	sys 0.002s
W20260709 13:02:27.434852  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 75778.1us
I20260709 13:02:27.461254  1716 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:27.461910  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1776) took 1034 ms. Trace:
I20260709 13:02:27.461542  1687 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:27.462352  1716 rpcz_store.cc:276] 0709 13:02:26.427162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.460567 (+1033405us) service_pool.cc:224] Handling call
0709 13:02:27.460906 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:27.461622 (+   716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.461849 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:27.463621  1687 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:27.435920  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 77039.4us
I20260709 13:02:27.464527  1685 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:27.384357  3339 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:27.384860  3520 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:27.385774  3181 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.303536  3497 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.142s
I20260709 13:02:27.465332  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:27.470712  3315 raft_consensus.cc:3055] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:27.487640  3180 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:27.407250  3314 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.478710  3516 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:27.488806  3516 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.472941  3410 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:27.494266  3410 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.494715  3410 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.384357  3186 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:27.498348  3186 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:27.472949  3461 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:27.498716  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1770) took 1543 ms. Trace:
I20260709 13:02:27.499270  3186 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:27.499454  1645 rpcz_store.cc:276] 0709 13:02:25.955448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.955644 (+   196us) service_pool.cc:224] Handling call
0709 13:02:25.956100 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.205049 (+248949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.498314 (+1293265us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b5400096d54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:27.498619 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":569667840}
I20260709 13:02:27.500391  3186 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:27.402369  3515 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:27.446729  3335 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:27.357532  3434 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:27.486949  3315 raft_consensus.cc:740] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:27.481093  1567 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:27.504328  3498 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:27.457516  3382 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:27.336786  3511 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:27.340207  3502 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:27.471217  3462 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:27.553997  3462 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.554543  3462 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.471415  3339 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.470563  3506 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:27.500509  3461 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:27.521541  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 349228us
I20260709 13:02:27.432344  3507 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
W20260709 13:02:27.509330  1564 consensus_meta.cc:287] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.856s	user 0.013s	sys 0.002s
W20260709 13:02:27.509140  3400 consensus_meta.cc:287] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.954s	user 0.017s	sys 0.000s
I20260709 13:02:27.489509  3516 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.445363  3504 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:27.557809  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 98861.9us
I20260709 13:02:27.599646  2312 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:27.500981  3186 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:27.523891  3338 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.586s	user 0.012s	sys 0.012s
W20260709 13:02:27.567643  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 348494us
I20260709 13:02:27.519383  3181 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.525225  3520 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:27.520494  3334 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:27.604892  3338 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.667s	user 0.015s	sys 0.012s
W20260709 13:02:27.609251  3168 consensus_meta.cc:287] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.864s	user 0.015s	sys 0.002s
I20260709 13:02:27.508859  3469 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:02:27.578409  3253 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.822s	user 0.013s	sys 0.012s
I20260709 13:02:27.603221  1611 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:27.603125  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 242554us
W20260709 13:02:27.614151  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:27.618017  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 152770us
I20260709 13:02:27.656138  1612 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:27.561067  3521 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:27.631600  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 167159us
I20260709 13:02:27.634188  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1487) took 1475 ms. Trace:
I20260709 13:02:27.569787  3339 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.632280  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55890 (request call id 189) took 3221 ms. Trace:
I20260709 13:02:27.669669  1538 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:27.592917  1564 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
W20260709 13:02:27.573992  3456 consensus_meta.cc:287] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.667s	user 0.011s	sys 0.003s
I20260709 13:02:27.637181  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:27.605351  3186 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.580711  3506 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:27.664250  1685 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:27.663282  1719 rpcz_store.cc:276] 0709 13:02:26.159079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.360737 (+1201658us) service_pool.cc:224] Handling call
0709 13:02:27.361237 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:27.361556 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:27.402992 (+ 41436us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.633660 (+230668us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:27.633744 (+    84us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:27.633822 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.633916 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.634100 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":41574,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":12267,"raft.queue_time_us":870,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":11150,"spinlock_wait_cycles":25887488,"thread_start_us":228602,"threads_started":1,"wal-append.queue_time_us":150192}
W20260709 13:02:27.678373  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 75148.3us
I20260709 13:02:27.682248  1572 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:02:27.681300  1612 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:02:27.654264  3253 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.013s	sys 0.017s
I20260709 13:02:27.698477  1685 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:27.699538  1611 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:27.647672  3469 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:27.703413  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 66371.7us
I20260709 13:02:27.551563  3498 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:27.711696  1611 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:27.668233  3521 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.713299  1537 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:27.632195  3314 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:27.636951  3287 consensus_queue.cc:1059] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
W20260709 13:02:27.634295  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 170583us
I20260709 13:02:27.613371  3400 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:27.724488  1565 raft_consensus.cc:1240] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:27.698100  3520 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:27.692266  3412 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:27.605446  3315 consensus_queue.cc:260] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:27.735697  2313 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:27.657019  3168 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:27.734094  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1462) took 2139 ms. Trace:
I20260709 13:02:27.711875  3511 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:27.744567  1571 rpcz_store.cc:276] 0709 13:02:25.594945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.595154 (+   209us) service_pool.cc:224] Handling call
0709 13:02:25.948073 (+352919us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:25.983034 (+ 34961us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.733869 (+1750835us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b5400305a50. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:27.734031 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":810734464}
I20260709 13:02:27.691313  3506 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.746819  1644 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:27.746838  1573 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:02:27.700506  1564 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1237) took 1212 ms (client timeout 1545 ms). Trace:
I20260709 13:02:27.747494  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:27.714864  3355 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:27.716159  3497 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:27.752715  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:27.720026  3396 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:27.753121  1639 raft_consensus.cc:2368] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 4. Current term is 5.
I20260709 13:02:27.753141  1646 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:27.739557  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1469) took 1909 ms. Trace:
I20260709 13:02:27.722280  3507 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:27.720870  3382 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:27.732748  3525 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:27.714224  3469 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:27.723101  3521 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:27.758424  3495 consensus_meta.cc:287] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.632s	user 0.012s	sys 0.001s
I20260709 13:02:27.746068  3518 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:27.764814  1720 rpcz_store.cc:276] 0709 13:02:25.830256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.733551 (+903295us) service_pool.cc:224] Handling call
0709 13:02:26.751642 (+ 18091us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.751859 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:26.779200 (+ 27341us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.336590 (+557390us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x7b180027eda0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a2523a 00007f8248e8fa2a
0709 13:02:27.737960 (+401370us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:27.738040 (+    80us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:27.738103 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.738211 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.739482 (+  1271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11360,"prepare.queue_time_us":27205,"prepare.run_cpu_time_us":169,"prepare.run_wall_time_us":3493,"spinlock_wait_cycles":5046016,"thread_start_us":389587,"threads_started":1}
I20260709 13:02:27.754263  3541 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.796064  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:27.797716  1720 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:27.700241  3456 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:27.730265  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:27.769412  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:27.771409  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:27.716572  3531 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:27.749960  3519 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 13:02:27.779543  1687 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:27.771746  3434 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:27.784044  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:27.713997  3403 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:02:27.790410  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 69471.9us
I20260709 13:02:27.845486  1538 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:27.747037  3461 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:02:27.847052  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 67624.6us
W20260709 13:02:27.799780  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 86578.3us
I20260709 13:02:27.738442  3335 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 13:02:27.712604  3526 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 13:02:27.769526  3180 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:27.806577  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:27.759450  3412 raft_consensus.cc:697] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:27.809172  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:27.877969  1720 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:27.879184  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:27.879801  1720 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:27.880716  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:27.772364  3536 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.090s
I20260709 13:02:27.763496  3396 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:27.783645  3509 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:27.786795  3525 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.790864  3511 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.848407  1537 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:27.848505  1687 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:27.903018  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 54561.5us
W20260709 13:02:27.753614  1564 rpcz_store.cc:269] 0709 13:02:26.481408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.509926 (+ 28518us) service_pool.cc:224] Handling call
0709 13:02:27.693811 (+1183885us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:27.854410  1721 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:27.796898  3507 raft_consensus.cc:697] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:02:27.864569  1640 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1190) took 1569 ms (client timeout 1900 ms). Trace:
W20260709 13:02:27.946949  1640 rpcz_store.cc:269] 0709 13:02:26.294982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.461471 (+166489us) service_pool.cc:224] Handling call
0709 13:02:27.864268 (+1402797us) spinlock_profiling.cc:243] Waited 70.6 ms on lock 0x7b54000a64d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:27.864472 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":162318720}
I20260709 13:02:27.948630  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:27.950251  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:27.870966  1612 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:02:27.951937  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 81383.7us
I20260709 13:02:27.873505  1713 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
W20260709 13:02:27.890467  3522 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:02:27.796756  3382 raft_consensus.cc:697] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:27.766388  3315 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:27.742645  3520 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.897099  3412 consensus_queue.cc:237] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:27.876439  3461 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:27.898227  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:27.904021  1687 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:27.804085  3481 consensus_queue.cc:1059] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.078s
I20260709 13:02:27.807651  3518 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:27.855062  3531 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.856184  3502 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:28.010083  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1155) took 1856 ms (client timeout 1761 ms). Trace:
I20260709 13:02:27.910393   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:27.876894  3180 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.847358  1569 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
W20260709 13:02:27.945070  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 96758.6us
I20260709 13:02:27.868281  1538 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:27.959838  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:27.800390  3495 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:27.953193  1612 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:27.961542  1611 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:27.962050  1640 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:28.032737  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 111) took 2972 ms. Trace:
I20260709 13:02:28.022164  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:28.036722  1537 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:28.038023  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 76612.6us
W20260709 13:02:28.010697  1648 rpcz_store.cc:269] 0709 13:02:26.153774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.196802 (+ 43028us) service_pool.cc:224] Handling call
0709 13:02:28.009845 (+1813043us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b5400098b54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:28.009970 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":713626496}
I20260709 13:02:28.040035  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1485) took 1912 ms. Trace:
I20260709 13:02:27.986752  3497 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:27.982497  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:28.056466  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:27.846925  3539 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:27.903028  3396 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:27.998030  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 94109.6us
I20260709 13:02:28.070459  1645 raft_consensus.cc:2393] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:28.079999  1687 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:27.901944  3403 raft_consensus.cc:697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:27.915289  3525 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:27.682364  2346 rpcz_store.cc:276] 0709 13:02:24.388809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.484094 (+ 95285us) service_pool.cc:224] Handling call
0709 13:02:25.310151 (+826057us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:25.808599 (+498448us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:25.808871 (+   272us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:25.809012 (+   141us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:25.809070 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.809160 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.593167 (+1784007us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b5400549050. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:27.609793 (+ 16626us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1859,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":122,"spinlock_wait_cycles":1715337984}
I20260709 13:02:28.041575  3180 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:28.030809  3400 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:28.049865  3481 raft_consensus.cc:3055] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:02:28.044818  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 176786us
I20260709 13:02:28.049661  1644 raft_consensus.cc:2393] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:27.818210  3541 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:28.112702  3540 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:28.112986  3359 consensus_meta.cc:287] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.772s	user 0.012s	sys 0.000s
I20260709 13:02:28.113543  3540 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.114054  3540 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:28.113729  3359 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.114289  3365 raft_consensus.cc:3055] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:28.114583  3541 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:28.051339  1537 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:28.114872  3365 raft_consensus.cc:740] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:27.968796  3504 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:28.057367  1573 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:02:28.058259  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 105208us
I20260709 13:02:28.036388  1717 rpcz_store.cc:276] 0709 13:02:25.060565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.419558 (+1358993us) service_pool.cc:224] Handling call
0709 13:02:27.139524 (+719966us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:27.139865 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:27.209614 (+ 69749us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.607541 (+397927us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:27.607599 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:27.607647 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.607723 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.032426 (+424703us) spinlock_profiling.cc:243] Waited 720 ms on lock 0x7b54000230d0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:28.032647 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":70069,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":60,"raft.queue_time_us":33271,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"spinlock_wait_cycles":1655095168,"thread_start_us":397311,"threads_started":2,"wal-append.queue_time_us":244320}
I20260709 13:02:27.979803  3474 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:28.123413  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:28.064291  3314 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:28.011729  3224 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:27.990461  3520 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:28.001300  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.086616  3498 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:28.088632  3276 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:27.947319  3501 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:28.059361  3512 consensus_meta.cc:287] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.709s	user 0.009s	sys 0.006s
I20260709 13:02:28.149211  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:28.150200  1573 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.911990  3335 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:28.101927  3461 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:28.056790  3524 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:27.972112  3471 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:28.108563  3385 consensus_meta.cc:287] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.638s	user 0.008s	sys 0.000s
W20260709 13:02:28.110098  3542 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
W20260709 13:02:28.111133  3530 log.cc:927] Time spent T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 13:02:27.907120  3511 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:28.111516  3495 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:28.029906  3531 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.112296  3550 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:28.049872  3334 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:28.115782  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:28.041985  3505 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:28.059317  3497 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:28.126639  1640 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:28.001224  3547 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:28.132758  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:28.232623  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:28.119254  3104 consensus_meta.cc:287] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.771s	user 0.015s	sys 0.001s
I20260709 13:02:28.138188  3541 consensus_queue.cc:1059] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:02:28.291771  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:28.061668  3502 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 13:02:28.103541  3481 raft_consensus.cc:740] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:02:28.148038  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 68131.5us
I20260709 13:02:28.132452  3546 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:28.128861  3474 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.141101  3168 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:28.183038  3535 log.cc:927] Time spent T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.629s	user 0.004s	sys 0.003s
I20260709 13:02:28.137115  3400 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:02:28.149753  3224 raft_consensus.cc:697] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:28.222361  1644 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:28.179543  3534 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.302s	user 0.002s	sys 0.000s
W20260709 13:02:28.224903  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 181181us
I20260709 13:02:28.158679  3524 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.155404  3552 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:28.187245  3462 consensus_meta.cc:287] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.632s	user 0.013s	sys 0.000s
I20260709 13:02:28.152145  3365 consensus_queue.cc:260] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.178249  3456 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:28.301764  1687 leader_election.cc:336] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.957s (SENT)
I20260709 13:02:28.117146  3507 consensus_queue.cc:237] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.299391  3095 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:28.237056  3497 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:28.173161  3385 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.159359  3512 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:28.231273  3334 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.290663  3550 raft_consensus.cc:697] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:28.140173  3382 consensus_queue.cc:237] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:28.312507  3541 raft_consensus.cc:3055] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:28.154533  3501 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.187525  3403 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:28.154592  3314 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:28.302474  3546 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.431856  3546 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:28.311550  3474 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:28.434691  3518 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:28.348682  3411 consensus_meta.cc:287] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.128s	user 0.007s	sys 0.006s
I20260709 13:02:28.315441  3509 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:28.348106  3524 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:28.313220  3461 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
W20260709 13:02:28.426577  3558 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
W20260709 13:02:28.429298  3513 consensus_meta.cc:287] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.034s	user 0.009s	sys 0.005s
I20260709 13:02:28.454478  3513 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.423782  3547 raft_consensus.cc:697] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:28.431902  3511 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:28.348376  3545 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.270s	user 0.001s	sys 0.001s
W20260709 13:02:28.462116  3551 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:02:28.464283  3501 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:28.466820  3525 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:28.450057  3548 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 13:02:28.372983  3541 raft_consensus.cc:740] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:02:28.444278  3515 consensus_meta.cc:287] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.923s	user 0.009s	sys 0.005s
I20260709 13:02:28.454911  3520 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.474512  3515 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.476115  3525 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:28.476454  3556 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:02:28.453526  3531 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:28.476955  3550 consensus_queue.cc:237] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.478816  3515 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:28.484697  3541 consensus_queue.cc:260] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.486667  3541 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:28.157141  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 105915us
I20260709 13:02:28.222260  1570 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:02:28.315129  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 328568us
I20260709 13:02:28.240186  1573 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:28.144272  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:28.348791  1687 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:28.426160  3224 consensus_queue.cc:237] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.359000  3276 raft_consensus.cc:697] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:28.429816  3314 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.622416  3314 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:28.623127  3186 consensus_meta.cc:287] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.912s	user 0.013s	sys 0.004s
I20260709 13:02:28.623950  3186 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:28.371040  3359 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:28.626134  1717 raft_consensus.cc:2393] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 4: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:28.374457  3365 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:28.431098  3385 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:28.429301  3463 consensus_meta.cc:287] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.070s	user 0.012s	sys 0.000s
W20260709 13:02:28.630375  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 316424us
I20260709 13:02:28.359743  3462 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.641803  3567 consensus_queue.cc:1059] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:28.643921  3567 raft_consensus.cc:3055] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:28.644558  3567 raft_consensus.cc:740] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:28.371479  3481 consensus_queue.cc:260] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.649456  3481 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:28.649943  3359 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.651152  3359 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.652364  3520 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:28.605067  3334 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:28.544004  1716 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: []
W20260709 13:02:28.653950  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.544926  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1475) took 2682 ms. Trace:
I20260709 13:02:28.439774  3471 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:28.569625  3557 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:28.532954  1646 raft_consensus.cc:2393] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:28.654672  1718 rpcz_store.cc:276] 0709 13:02:25.862247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.867767 (+1005520us) service_pool.cc:224] Handling call
0709 13:02:26.868717 (+   950us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:27.081549 (+212832us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:27.081577 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:27.147405 (+ 65828us) log.cc:844] Serialized 26 byte log entry
0709 13:02:27.530712 (+383307us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:27.530766 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.530862 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:28.544321 (+1013459us) raft_consensus.cc:1685] finished
0709 13:02:28.544365 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.544703 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":80797,"raft.run_cpu_time_us":1355,"raft.run_wall_time_us":9077,"thread_start_us":515654,"threads_started":2,"wal-append.queue_time_us":579660}
W20260709 13:02:28.541584  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.539685  1569 raft_consensus.cc:2393] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:28.450584  3411 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:28.543025  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 241281us
W20260709 13:02:28.527479  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 396006us
I20260709 13:02:28.606604  1640 raft_consensus.cc:2393] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:28.614380  3513 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:28.524899  1537 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:28.644609  3462 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:28.541945  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1786) took 2063 ms. Trace:
I20260709 13:02:28.366955  3095 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.622133  3531 consensus_queue.cc:1059] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:28.631493  3463 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.665726  1642 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:02:28.688455  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:28.678393  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 154074us
W20260709 13:02:28.680876  3315 consensus_meta.cc:287] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.693s	user 0.013s	sys 0.001s
I20260709 13:02:28.620739  3276 consensus_queue.cc:237] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.697036  3557 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:28.698485  3557 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:28.696762  3471 raft_consensus.cc:2764] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:02:28.489451  3547 consensus_queue.cc:237] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.721398  1573 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:28.718364  1644 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:28.730057  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:28.731880  3525 consensus_queue.cc:1059] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:28.733042  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:28.732270  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:28.734022  3525 raft_consensus.cc:3055] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:28.734562  3525 raft_consensus.cc:740] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:28.734644  3509 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:28.735951  1612 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:28.736054  3525 consensus_queue.cc:260] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.682932  1719 rpcz_store.cc:276] 0709 13:02:26.478016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.688381 (+1210365us) service_pool.cc:224] Handling call
0709 13:02:27.688926 (+   545us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:27.698352 (+  9426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:27.698380 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:27.698886 (+   506us) log.cc:844] Serialized 26 byte log entry
0709 13:02:27.860177 (+161291us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:27.890042 (+ 29865us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.890496 (+   454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.890589 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:28.026181 (+135592us) spinlock_profiling.cc:243] Waited 8.64 ms on lock 0x7b180044e7e0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:02:28.540795 (+514614us) raft_consensus.cc:1685] finished
0709 13:02:28.541082 (+   287us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.541530 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39760,"prepare.queue_time_us":31190,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"raft.queue_time_us":21610,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":44549,"spinlock_wait_cycles":19862912,"thread_start_us":106152,"threads_started":1,"wal-append.queue_time_us":400658}
I20260709 13:02:28.737313  3400 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:28.739418  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:28.739689  3463 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:28.740530  1719 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:28.741374  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1493) took 1024 ms. Trace:
I20260709 13:02:28.742151  3550 raft_consensus.cc:3055] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:28.743148  3550 raft_consensus.cc:740] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:28.743027  1537 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:28.743726  3410 consensus_meta.cc:287] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.187s	user 0.015s	sys 0.003s
I20260709 13:02:28.744263  3410 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:28.744980  3181 consensus_meta.cc:287] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.097s	user 0.012s	sys 0.013s
I20260709 13:02:28.745167  1537 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:28.745687  3181 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.748245  3181 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:28.748467  3095 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:28.748497  3531 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:28.749473  3531 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.750064  3531 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:28.750435  3561 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:28.751318  3561 raft_consensus.cc:697] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:28.742097  1719 rpcz_store.cc:276] 0709 13:02:27.716907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.738734 (+1021827us) service_pool.cc:224] Handling call
0709 13:02:28.741256 (+  2522us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:28.754350  1644 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:28.739454  1612 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:28.743992  3525 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:28.754272  3562 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:02:28.745252  3516 consensus_meta.cc:287] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.121s	user 0.017s	sys 0.004s
I20260709 13:02:28.753021  3561 consensus_queue.cc:237] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.779801  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:28.755832  3557 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:28.785691  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:28.786511  1647 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:28.789345  3411 consensus_queue.cc:1059] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:02:28.790450  3462 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:28.766444  3513 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:28.793370  3410 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:28.712262  3567 consensus_queue.cc:260] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.766386  3095 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.801016  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:28.802894  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:28.806370  3546 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:28.778163  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 501890us
I20260709 13:02:28.749433  3497 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.816160  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:28.814597  3550 consensus_queue.cc:260] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.816954  3516 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:28.828785  3510 consensus_meta.cc:287] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.429s	user 0.007s	sys 0.017s
I20260709 13:02:28.884908  3510 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:28.887971  1720 raft_consensus.cc:2393] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:28.896414  1538 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:28.897066  2342 raft_consensus.cc:2368] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6.
I20260709 13:02:28.831336  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:28.809314  3474 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:28.834075  3506 consensus_meta.cc:287] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.047s	user 0.007s	sys 0.015s
W20260709 13:02:28.835153  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.852136  1568 raft_consensus.cc:2393] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:28.866204  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:28.872717  3521 consensus_meta.cc:287] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.064s	user 0.013s	sys 0.010s
I20260709 13:02:28.873620  3536 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:28.875526  3095 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:28.876904  3339 consensus_meta.cc:287] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.185s	user 0.014s	sys 0.011s
I20260709 13:02:28.907428  1644 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 3: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:02:28.880261  3553 log.cc:927] Time spent T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.566s	user 0.000s	sys 0.001s
I20260709 13:02:28.809333  3355 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:28.846222  3462 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:02:28.915293  3575 log.cc:927] Time spent T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.102s	user 0.000s	sys 0.002s
I20260709 13:02:28.915804  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:28.916682  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 134) took 1115 ms. Trace:
I20260709 13:02:28.917124  1644 rpcz_store.cc:276] 0709 13:02:27.795616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.841695 (+1046079us) service_pool.cc:224] Handling call
0709 13:02:28.911229 (+ 69534us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:28.915654  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:28.922300  3339 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:28.923830  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 184606us
I20260709 13:02:28.935402  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:28.881366  3524 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.912963  3462 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:28.920572  3355 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:28.933385  3540 consensus_meta.cc:287] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.792s	user 0.018s	sys 0.005s
I20260709 13:02:28.912459  3525 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.928681  3186 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:28.941135  1567 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:28.953950  1612 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:28.902343  3506 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.927500  3314 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.966428  3536 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:28.971863  3315 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:28.990667  3315 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.991174  3315 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:28.991153  3516 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:28.978758  3546 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:28.912729  3521 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:28.998538  1719 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 3: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:02:28.983065  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.982183  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 1783) took 2548 ms. Trace:
I20260709 13:02:28.955615  3181 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.998005  3095 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:28.999797  1713 rpcz_store.cc:276] 0709 13:02:26.433672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.617407 (+183735us) service_pool.cc:224] Handling call
0709 13:02:28.981568 (+2364161us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54002990d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:28.981920 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2885702912}
W20260709 13:02:28.991926  3396 consensus_meta.cc:287] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.859s	user 0.021s	sys 0.003s
I20260709 13:02:28.953073  3513 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:29.007956  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.008213  3181 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:29.009512  3524 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:29.009820  3536 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:29.008039  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:29.011224  2344 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 6: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:29.003932  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 121) took 1231 ms. Trace:
I20260709 13:02:29.003890  1538 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:29.011781  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:29.014066  1719 rpcz_store.cc:276] 0709 13:02:27.769668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.855335 (+1085667us) service_pool.cc:224] Handling call
0709 13:02:29.000944 (+145609us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.016465  1641 raft_consensus.cc:2393] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:29.016340  2314 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.183s (SENT) [suppressed 48 similar messages]
I20260709 13:02:29.017704  1567 raft_consensus.cc:2393] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:28.989148  3525 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:29.041029  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1198) took 1297 ms. Trace:
I20260709 13:02:29.039395  3339 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:29.020578  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1801) took 1234 ms. Trace:
I20260709 13:02:29.021375  3474 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:29.022156  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 93945.9us
I20260709 13:02:29.043033  3525 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:29.023110  3533 raft_consensus.cc:3055] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:29.023607  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1794) took 1267 ms. Trace:
I20260709 13:02:29.043733  3533 raft_consensus.cc:740] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:29.043788  1640 rpcz_store.cc:276] 0709 13:02:27.755955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.232302 (+476347us) service_pool.cc:224] Handling call
0709 13:02:29.023343 (+791041us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b5400099054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:29.023540 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":851465472}
I20260709 13:02:29.045360  3533 consensus_queue.cc:260] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:29.047148  3533 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:29.047274  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 120) took 1280 ms. Trace:
I20260709 13:02:29.047978  1717 rpcz_store.cc:276] 0709 13:02:27.766788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.291382 (+524594us) service_pool.cc:224] Handling call
0709 13:02:29.046581 (+755199us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b5400211454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:29.047178 (+   597us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":765543168}
I20260709 13:02:29.048373  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:29.022559  3567 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:29.050132  3576 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:29.024904  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1789) took 1277 ms. Trace:
I20260709 13:02:29.015172  3513 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:29.054711  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:29.053872  1643 rpcz_store.cc:276] 0709 13:02:27.746906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.748389 (+  1483us) service_pool.cc:224] Handling call
0709 13:02:27.865733 (+117344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:27.876089 (+ 10356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:27.876120 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:27.884813 (+  8693us) log.cc:844] Serialized 26 byte log entry
0709 13:02:28.298965 (+414152us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:28.299038 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.299121 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:28.536221 (+237100us) raft_consensus.cc:1685] finished
0709 13:02:28.536264 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.024311 (+488047us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b54000a64d0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:29.024558 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":454,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":12708,"spinlock_wait_cycles":268586752,"thread_start_us":387786,"threads_started":1,"wal-append.queue_time_us":541905}
I20260709 13:02:29.055685  1713 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:29.056581  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:29.057143  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:28.960664  3540 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:29.058061  1713 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 1.
I20260709 13:02:29.059331  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:29.059752  3540 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:29.060236  1713 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:29.026799  3521 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:29.027810  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:29.041544  1647 rpcz_store.cc:276] 0709 13:02:27.743846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.743976 (+   130us) service_pool.cc:224] Handling call
0709 13:02:29.040748 (+1296772us) spinlock_profiling.cc:243] Waited 975 ms on lock 0x7b54002db254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:29.040949 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2241523456}
I20260709 13:02:29.062328  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:29.063040  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:29.063838  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:29.064275  1647 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:29.028142  3180 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.878s	user 0.015s	sys 0.009s
I20260709 13:02:29.066023  1719 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:29.021190  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:29.019711  3396 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:29.068015  1643 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:29.066215  3180 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:29.068723  1564 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:29.071097  1643 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:29.071234  3314 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:29.071700  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:29.067600  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1494) took 1348 ms. Trace:
I20260709 13:02:29.072728  1718 raft_consensus.cc:2376] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4. Re-sending same reply.
I20260709 13:02:29.072785  1719 rpcz_store.cc:276] 0709 13:02:27.718802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.047980 (+1329178us) service_pool.cc:224] Handling call
0709 13:02:29.067352 (+ 19372us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b5400246654. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:29.067502 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38454528}
I20260709 13:02:29.074769  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:29.076004  1719 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 2: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:29.076052  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1505) took 1033 ms. Trace:
W20260709 13:02:29.077935  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:29.078140  1639 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1174) took 2845 ms (client timeout 1957 ms). Trace:
W20260709 13:02:29.078617  1639 rpcz_store.cc:269] 0709 13:02:26.220741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.449168 (+228427us) service_pool.cc:224] Handling call
0709 13:02:29.066141 (+2616973us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54002dc154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:29.066579 (+   438us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2950335360}
I20260709 13:02:29.080379  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:29.081449  1639 raft_consensus.cc:2393] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:29.031076  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1788) took 1284 ms. Trace:
I20260709 13:02:29.065379  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:29.095697  3546 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.097074  3546 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:29.103245  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:29.104425  3461 raft_consensus.cc:3055] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:29.104686  3506 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:29.106953  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1503) took 1068 ms. Trace:
I20260709 13:02:29.107520  2344 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:29.107451  1570 rpcz_store.cc:276] 0709 13:02:28.038649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.041523 (+  2874us) service_pool.cc:224] Handling call
0709 13:02:28.072290 (+ 30767us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:28.530635 (+458345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.106321 (+575686us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54004074d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:29.106556 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341113600}
W20260709 13:02:29.109185  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.032760  1714 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:02:29.012326  3410 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:29.117017  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1784) took 2679 ms. Trace:
I20260709 13:02:29.117429  3181 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:29.117486  1716 rpcz_store.cc:276] 0709 13:02:26.437427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.463396 (+1025969us) service_pool.cc:224] Handling call
0709 13:02:27.463718 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.661402 (+1197684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.096402 (+435000us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54003002d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:29.116932 (+ 20530us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2482794368}
I20260709 13:02:29.118476  3382 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.119567  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:29.120472  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:29.033928  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:29.034492  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1492) took 1696 ms. Trace:
I20260709 13:02:29.037235  3510 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:29.037724  3355 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:28.040678  1715 rpcz_store.cc:276] 0709 13:02:26.127416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.213447 (+1086031us) service_pool.cc:224] Handling call
0709 13:02:27.216123 (+  2676us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:27.219932 (+  3809us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:27.633101 (+413169us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.039661 (+406560us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:28.039718 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:28.039769 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.039846 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.039973 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":61582,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":109,"raft.queue_time_us":2896,"thread_start_us":760408,"threads_started":2}
W20260709 13:02:29.040055  3469 consensus_meta.cc:287] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.239s	user 0.011s	sys 0.014s
I20260709 13:02:29.046027  1640 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: []
W20260709 13:02:29.023865  3554 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.710s	user 0.002s	sys 0.000s
I20260709 13:02:28.991670  3531 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:29.027374  3537 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:29.071982  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1795) took 1315 ms. Trace:
W20260709 13:02:29.029495  1722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 1779) took 2598 ms (client timeout 3240 ms). Trace:
I20260709 13:02:29.082417  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:29.041551  3512 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:29.042541  1645 rpcz_store.cc:276] 0709 13:02:27.784709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.872566 (+ 87857us) service_pool.cc:224] Handling call
0709 13:02:29.018421 (+1145855us) spinlock_profiling.cc:243] Waited 72.5 ms on lock 0x7b54001fe0d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:29.018931 (+   510us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166740352}
I20260709 13:02:29.082464  3396 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:29.134249  1722 rpcz_store.cc:269] 0709 13:02:26.430550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.594677 (+164127us) service_pool.cc:224] Handling call
0709 13:02:29.029029 (+2434352us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b540020ab54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:29.029263 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1384038784}
W20260709 13:02:29.031512  3549 log.cc:927] Time spent T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 1.044s	user 0.002s	sys 0.004s
I20260709 13:02:29.047150  1644 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:29.144328  1566 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:29.029986  3497 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:29.115192  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1500) took 1373 ms. Trace:
I20260709 13:02:29.117074  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:29.093597  1642 rpcz_store.cc:276] 0709 13:02:27.746618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.748286 (+  1668us) service_pool.cc:224] Handling call
0709 13:02:27.755065 (+  6779us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:28.693129 (+938064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.030717 (+337588us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b54001ff4d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:29.031012 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2068063616}
I20260709 13:02:29.159452  1713 rpcz_store.cc:276] 0709 13:02:27.741852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.058875 (+1317023us) service_pool.cc:224] Handling call
0709 13:02:29.115099 (+ 56224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.104935  3461 raft_consensus.cc:740] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:29.109577  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:02:29.122874  1646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1798) took 1365 ms (client timeout 1769 ms). Trace:
I20260709 13:02:29.119225  3382 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:29.127964  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:29.124292  1572 rpcz_store.cc:276] 0709 13:02:27.272769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.346260 (+ 73491us) service_pool.cc:224] Handling call
0709 13:02:27.681869 (+335609us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:27.690660 (+  8791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.935091 (+1244431us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b5400190fd0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:28.969129 (+ 34038us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":770659072}
I20260709 13:02:29.133762  1643 rpcz_store.cc:276] 0709 13:02:27.756523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.070088 (+1313565us) service_pool.cc:224] Handling call
0709 13:02:29.070569 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.071577 (+  1008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.071792 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.106675  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 117) took 3112 ms. Trace:
I20260709 13:02:29.137322  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1791) took 1381 ms. Trace:
I20260709 13:02:29.175089  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1790) took 1397 ms. Trace:
I20260709 13:02:29.176082  1640 rpcz_store.cc:276] 0709 13:02:27.747422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.045273 (+1297851us) service_pool.cc:224] Handling call
0709 13:02:29.045756 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.127158 (+ 81402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.129232 (+  2074us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.171612  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:29.179078  1570 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:29.076740  1571 rpcz_store.cc:276] 0709 13:02:28.042251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.162754 (+120503us) service_pool.cc:224] Handling call
0709 13:02:28.163087 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.530977 (+367890us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:28.532065 (+  1088us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.615269 (+ 83204us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:28.615354 (+    85us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:28.615424 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.075209 (+459785us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b54002ba054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:29.075268 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.075624 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43158,"prepare.queue_time_us":26572,"prepare.run_cpu_time_us":196,"prepare.run_wall_time_us":30515,"raft.queue_time_us":57967,"raft.run_cpu_time_us":470,"raft.run_wall_time_us":472,"spinlock_wait_cycles":843767168,"thread_start_us":26695,"threads_started":1,"wal-append.queue_time_us":27929}
W20260709 13:02:29.141963  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.176234  1644 rpcz_store.cc:276] 0709 13:02:27.747201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.938663 (+1191462us) service_pool.cc:224] Handling call
0709 13:02:29.006132 (+ 67469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.144504 (+138372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.144777 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.143375  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:29.147303  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1474) took 3294 ms. Trace:
I20260709 13:02:29.129747  3506 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.178496  1639 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:29.178124  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:29.173983  1646 rpcz_store.cc:269] 0709 13:02:27.757474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.783654 (+ 26180us) service_pool.cc:224] Handling call
0709 13:02:29.086508 (+1302854us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b5400097254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:29.122791 (+ 36283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1629404032}
I20260709 13:02:29.183884  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:29.190137  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:29.136981  3469 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:29.181790  1714 rpcz_store.cc:276] 0709 13:02:25.852614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.802380 (+949766us) service_pool.cc:224] Handling call
0709 13:02:29.031687 (+2229307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.109929 (+ 78242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.146912 (+ 36983us) spinlock_profiling.cc:243] Waited 2.23 s on lock 0x7b540020ab50. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:29.147122 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5125680896}
I20260709 13:02:29.244421  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:29.179781  1721 rpcz_store.cc:276] 0709 13:02:25.993867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.181296 (+1187429us) service_pool.cc:224] Handling call
0709 13:02:27.208434 (+ 27138us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:27.924451 (+716017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.106099 (+1181648us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b5400212d54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:29.106468 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1485224832}
I20260709 13:02:29.199980  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:29.254508  1719 raft_consensus.cc:2410] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:29.206135  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 165) took 1524 ms. Trace:
I20260709 13:02:29.134658  3339 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:29.208242  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 263679us
W20260709 13:02:29.210994  3578 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:02:29.221174  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:29.257318  1713 raft_consensus.cc:2393] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:29.244917  1722 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:29.255460  1565 rpcz_store.cc:276] 0709 13:02:27.616229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.727599 (+111370us) service_pool.cc:224] Handling call
0709 13:02:27.750906 (+ 23307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:27.827930 (+ 77024us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:27.827976 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:27.828029 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:27.828057 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.123423 (+1295366us) spinlock_profiling.cc:243] Waited 76.8 ms on lock 0x7b5400012054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:29.127574 (+  4151us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.140634 (+ 13060us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176544640}
I20260709 13:02:29.227531  3510 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.228164  2313 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:29.198611  1568 raft_consensus.cc:2393] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:29.237361  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:29.228641  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 166) took 1222 ms. Trace:
W20260709 13:02:29.229115  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.229952  1717 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:29.230607  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1492) took 1514 ms. Trace:
I20260709 13:02:29.231139  3315 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:29.231756  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 130) took 1444 ms. Trace:
I20260709 13:02:29.233246  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1507) took 1190 ms. Trace:
I20260709 13:02:29.262995  1720 rpcz_store.cc:276] 0709 13:02:27.715885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.880464 (+164579us) service_pool.cc:224] Handling call
0709 13:02:29.230371 (+1349907us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400209c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:29.230518 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2314210176}
I20260709 13:02:29.264673  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:29.267050  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.236662  3462 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:29.238250  3546 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:29.239785  3531 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:29.240667  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.216871  3537 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:29.243404  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:29.184340  1647 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:29.244424  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:29.261536  1569 raft_consensus.cc:2393] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:29.263900  1573 rpcz_store.cc:276] 0709 13:02:28.042941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.535046 (+492105us) service_pool.cc:224] Handling call
0709 13:02:28.535718 (+   672us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.722131 (+186413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.232939 (+510808us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54004065d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:29.233188 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":362499840}
I20260709 13:02:29.235494  3355 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:29.264328  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:29.273556  1715 raft_consensus.cc:2393] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:29.281548  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:29.263819  1641 rpcz_store.cc:276] 0709 13:02:27.787026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.982149 (+195123us) service_pool.cc:224] Handling call
0709 13:02:29.231521 (+1249372us) spinlock_profiling.cc:243] Waited 957 ms on lock 0x7b5400097754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:29.231687 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2201174400}
W20260709 13:02:29.296993  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 68979.2us
I20260709 13:02:29.302066  1643 raft_consensus.cc:2393] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:29.302703  1716 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:29.293272  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:29.299777  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1792) took 1544 ms. Trace:
I20260709 13:02:29.276614  1538 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:29.304453  1647 rpcz_store.cc:276] 0709 13:02:27.755347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.065419 (+1310072us) service_pool.cc:224] Handling call
0709 13:02:29.122685 (+ 57266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.299449 (+176764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.299701 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.306123  1642 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:29.307199  1566 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:29.308434  1648 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:29.310529  1718 raft_consensus.cc:2376] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4. Re-sending same reply.
I20260709 13:02:29.311654  1641 raft_consensus.cc:1240] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:29.312067  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:29.312170  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:29.313998  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:29.315239  1714 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:29.315472  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:29.316313  2349 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:02:29.315269  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:29.319123  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 129) took 1532 ms. Trace:
I20260709 13:02:29.313297  1718 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:29.316291  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:29.316687  1639 raft_consensus.cc:1240] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:29.321283  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 127) took 1535 ms. Trace:
I20260709 13:02:29.321614  1639 rpcz_store.cc:276] 0709 13:02:27.786230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.256870 (+1470640us) service_pool.cc:224] Handling call
0709 13:02:29.265669 (+  8799us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.321016 (+ 55347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.321224 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.331899  1639 raft_consensus.cc:1275] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:29.332176  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1499) took 1590 ms. Trace:
I20260709 13:02:29.332582  1713 rpcz_store.cc:276] 0709 13:02:27.741463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.331161 (+1589698us) service_pool.cc:224] Handling call
0709 13:02:29.331511 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.331740 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.331767 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:29.331857 (+    90us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:29.331885 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.331941 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.332097 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.321967  3403 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:29.308230  3567 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:29.323030  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:29.335244  3567 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:29.336408  1715 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:29.323511  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:29.339177  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:29.312382  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 131) took 1525 ms. Trace:
I20260709 13:02:29.340724  1641 rpcz_store.cc:276] 0709 13:02:27.787271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.302183 (+1514912us) service_pool.cc:224] Handling call
0709 13:02:29.311373 (+  9190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.312115 (+   742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.312302 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.327865  2349 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:02:29.262024  1564 rpcz_store.cc:276] 0709 13:02:28.006555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.030348 (+ 23793us) service_pool.cc:224] Handling call
0709 13:02:28.030708 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.069269 (+1038561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.228278 (+159009us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400013954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:29.228566 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2386424192}
I20260709 13:02:29.319648  1640 rpcz_store.cc:276] 0709 13:02:27.786757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.298821 (+1512064us) service_pool.cc:224] Handling call
0709 13:02:29.318309 (+ 19488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.318701 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.318734 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:29.318810 (+    76us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:29.318852 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.318922 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.319042 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.329233  1643 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:29.343063  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:29.343549  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 132) took 1556 ms. Trace:
I20260709 13:02:29.344276  1643 rpcz_store.cc:276] 0709 13:02:27.787530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.327806 (+1540276us) service_pool.cc:224] Handling call
0709 13:02:29.328744 (+   938us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.343206 (+ 14462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.343475 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.344732  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:02:29.345841  1685 leader_election.cc:336] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.605s (SENT)
I20260709 13:02:29.320447  3510 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:29.347481  1685 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:29.348913  1567 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:02:29.349291  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.357064  2342 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:29.327476  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:02:29.359670  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 122468us
I20260709 13:02:29.360845  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:29.361931  3546 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:29.316183  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:29.316458  1645 raft_consensus.cc:1240] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:29.339334  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:29.346585  1717 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:29.363164  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 133) took 1575 ms. Trace:
I20260709 13:02:29.364773  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:02:29.364892  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 45410.6us
I20260709 13:02:29.279778  3550 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:29.316471  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1489) took 2825 ms. Trace:
I20260709 13:02:29.369699  1647 rpcz_store.cc:276] 0709 13:02:27.787786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.305369 (+1517583us) service_pool.cc:224] Handling call
0709 13:02:29.362315 (+ 56946us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.362766 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.362792 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:29.362836 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:29.362886 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.362959 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.363092 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.365384  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1800) took 1600 ms. Trace:
I20260709 13:02:29.384374  3540 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:29.383987  3567 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:29.383057  1645 rpcz_store.cc:276] 0709 13:02:27.765130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.195197 (+1430067us) service_pool.cc:224] Handling call
0709 13:02:29.307357 (+112160us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.365073 (+ 57716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.365314 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.333683  3403 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:29.395711  3403 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:29.398157  1564 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:29.275520  3469 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:29.381510  1714 rpcz_store.cc:276] 0709 13:02:26.491430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.270064 (+2778634us) service_pool.cc:224] Handling call
0709 13:02:29.314723 (+ 44659us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.316048 (+  1325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.316361 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.270027  3512 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.308542  1611 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:29.273020  3461 consensus_queue.cc:260] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
W20260709 13:02:29.589085  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.266919  3506 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:29.399158  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:02:29.469921  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 159905us
W20260709 13:02:29.663211  2314 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.048s	sys 0.131s
I20260709 13:02:29.445123  1648 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:02:29.650184  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:29.651991  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 259687us
I20260709 13:02:29.458314  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:02:29.440335  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 163961us
W20260709 13:02:29.441833  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 96221.2us
I20260709 13:02:29.490026  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1794) took 2389 ms. Trace:
W20260709 13:02:29.590399  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:29.666494  2314 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.048s	sys 0.132s
W20260709 13:02:29.679733  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 663586us
I20260709 13:02:29.417724  1570 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:29.363204  2341 raft_consensus.cc:2393] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:29.499495  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:29.696878  1716 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:29.697582  1573 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
W20260709 13:02:29.679010  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:29.698920  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.400095  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:29.364658  3510 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:29.678467  1717 rpcz_store.cc:276] 0709 13:02:26.988431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.345779 (+2357348us) service_pool.cc:224] Handling call
0709 13:02:29.346382 (+   603us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.377528 (+ 31146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.377853 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.715780  1642 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:29.440757  3339 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:29.597173  3355 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:29.448906  3537 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:29.400317  3382 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:29.459022  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:29.720201  1571 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:29.407801  3315 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:29.601058  3180 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:29.723784  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:29.438988  2344 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:29.421317  3359 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:29.424094  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:29.735149  1537 leader_election.cc:400] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6.
I20260709 13:02:29.789520  1537 leader_election.cc:403] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:02:29.736958  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 437416us
I20260709 13:02:29.739387  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:29.749709  1567 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:29.791067  1566 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:29.791860  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:29.473726  1468 catalog_manager.cc:5697] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 1 to 2, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:29.792336  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:29.752352  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:29.793313  1566 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:29.722704  3339 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.722980  3512 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:29.765429  3462 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:29.766319  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:29.830612  1714 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
W20260709 13:02:29.769726  1685 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.802s (SENT)
I20260709 13:02:29.772473  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:29.745539  3510 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.793692  1573 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:29.759897  3567 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:29.794610  3537 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.795341  3469 consensus_queue.cc:1059] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:29.795150  1468 catalog_manager.cc:5697] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 1 to 2, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:29.796581  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:29.799031  3382 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.749709  2314 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:29.799749  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:29.803951  3540 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:29.808197  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:29.812772  3506 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:29.818238  3359 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.819061  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:29.745537  3355 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:29.754287  3410 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.864607  3410 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:29.866128  3403 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:29.818333  1717 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:29.840123  1715 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
W20260709 13:02:29.860749  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 91148.6us
I20260709 13:02:29.871555  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:29.840864  1718 consensus_meta.cc:287] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.521s	user 0.015s	sys 0.002s
I20260709 13:02:29.845873  1568 raft_consensus.cc:2393] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:29.846937  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:29.876442  1714 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:29.876646  2342 raft_consensus.cc:2393] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 2: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:29.878221  1568 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:02:29.879009  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 149786us
I20260709 13:02:29.880172  1718 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:29.834077  3339 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:29.848526  3540 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:29.854417  3359 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:29.858737  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 86393.7us
I20260709 13:02:29.855733  1571 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:29.856590  1570 raft_consensus.cc:2393] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:29.889421  1570 raft_consensus.cc:1275] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:29.890888  1570 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:02:29.854889  3382 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:29.894018  1685 leader_election.cc:336] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.895s (SENT)
I20260709 13:02:29.896050  3382 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:29.857092  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.899006  3540 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:29.899914  1685 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.922s (SENT)
I20260709 13:02:29.900493  1685 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:29.857266  3531 raft_consensus.cc:3055] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:29.902066  3531 raft_consensus.cc:740] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:29.902283  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 122) took 2124 ms. Trace:
I20260709 13:02:29.902690  1713 rpcz_store.cc:276] 0709 13:02:27.778365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.334343 (+1555978us) service_pool.cc:224] Handling call
0709 13:02:29.336144 (+  1801us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.572226 (+236082us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:29.572284 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:29.572706 (+   422us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:29.572804 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.902003 (+329199us) spinlock_profiling.cc:243] Waited 31.5 ms on lock 0x7b5400247554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:29.902050 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.902219 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":131,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":56157,"spinlock_wait_cycles":72553216}
I20260709 13:02:29.903383  3531 consensus_queue.cc:260] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:29.904789  3531 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:29.906554  3359 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:29.848976  3510 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:29.859190  3315 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:29.858707  1468 catalog_manager.cc:5697] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 2 to 3, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:29.840646  3180 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.851970  3567 raft_consensus.cc:697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:02:29.862811  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 194158us
W20260709 13:02:29.863226  3533 consensus_meta.cc:287] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.816s	user 0.013s	sys 0.002s
I20260709 13:02:29.851528  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:29.874090  3537 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:29.914417  1571 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:29.920733  1468 catalog_manager.cc:5697] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 1 to 3, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:29.860811  1569 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 2: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:29.856354  3469 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6.
I20260709 13:02:29.917017  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:29.920883  3315 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.925114  1573 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:29.930367  1570 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:29.930569  1538 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:29.930251  1717 raft_consensus.cc:2393] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:29.919778  3512 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:29.944686  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:29.945797  3253 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:29.946661  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:29.940409  1714 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:29.947525  1468 catalog_manager.cc:5697] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 1 to 2, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:29.948258  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:29.948873  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:29.949774  1718 raft_consensus.cc:2393] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:29.945824  3403 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:29.949106  1714 raft_consensus.cc:2393] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:29.955968  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:29.965452  1572 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:29.951682  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:29.948258  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:29.970715  1715 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:29.987613  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:29.979663  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:29.979851  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:30.004441  1573 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:30.006595  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 76125.4us
I20260709 13:02:30.010131  3469 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:29.924762  3180 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:30.012693  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:30.013410  1714 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:30.014019  2348 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:30.016063  3180 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:30.018182  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:30.020763  1718 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:29.939944  3382 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.957388  3540 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:30.030831  1572 raft_consensus.cc:1240] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:29.956986  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:30.029410  1714 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:02:30.031584  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:30.037895  1721 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.032s	sys 0.059s
W20260709 13:02:30.043206  1721 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.033s	sys 0.059s
I20260709 13:02:29.932168  3315 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:29.979100  3355 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:30.048900  1714 raft_consensus.cc:1275] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:02:30.052305  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.061429  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1809) took 1014 ms. Trace:
I20260709 13:02:30.062976  1714 rpcz_store.cc:276] 0709 13:02:29.036714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.041891 (+1005177us) service_pool.cc:224] Handling call
0709 13:02:30.042380 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.050630 (+  8250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.050916 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.072675  1717 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:30.076802  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1804) took 1041 ms. Trace:
I20260709 13:02:30.078223  1717 rpcz_store.cc:276] 0709 13:02:29.035230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.022867 (+987637us) service_pool.cc:224] Handling call
0709 13:02:30.059458 (+ 36591us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:30.074453 (+ 14995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.076172 (+  1719us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:30.079510  1722 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.016s	sys 0.072s
W20260709 13:02:30.080265  1722 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.016s	sys 0.072s
W20260709 13:02:30.084596  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:30.093833  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.097189  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 128) took 2310 ms. Trace:
W20260709 13:02:30.103211  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 203361us
I20260709 13:02:30.105479  1715 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:30.063776  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:30.113895  1715 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:02:30.118794  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 69139.8us
I20260709 13:02:30.119753  1714 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:02:30.128203  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.139925  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1811) took 1083 ms. Trace:
I20260709 13:02:30.140924  1714 rpcz_store.cc:276] 0709 13:02:29.037394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.098093 (+1060699us) service_pool.cc:224] Handling call
0709 13:02:30.119534 (+ 21441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:30.120208 (+   674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.120447 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.102051  3461 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:30.161595  1717 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:30.142521  3410 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:30.166989  1537 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:30.171957  1715 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:30.142771  1572 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:40642
I20260709 13:02:30.169323  1714 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:30.174065  3253 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:30.160910  1538 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:30.183118  1648 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:02:30.181465  1572 raft_consensus.cc:606] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Received request to transfer leadership
I20260709 13:02:30.183851  1648 pending_rounds.cc:85] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 5
I20260709 13:02:30.099273  3382 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:30.195591  1718 raft_consensus.cc:2410] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:30.190155  1648 pending_rounds.cc:107] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:02:30.207595  1648 raft_consensus.cc:2889] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:30.119171  3567 consensus_queue.cc:237] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:30.160794  3339 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:30.197006  3510 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:30.243084  1717 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:30.246590  1719 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.823s	user 0.021s	sys 0.050s
W20260709 13:02:30.242843  1685 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.961s (SENT)
W20260709 13:02:30.251808  1719 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.021s	sys 0.051s
I20260709 13:02:30.263734  1567 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:02:30.265367  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 105544us
I20260709 13:02:30.276373  1714 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:30.279790   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:30.285874  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.287353  3181 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.287968  3181 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:30.292533  3355 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:30.293907  1565 raft_consensus.cc:2410] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:30.273752  1685 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:30.304284  2312 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.011s	sys 0.060s
W20260709 13:02:30.306991  2312 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.011s	sys 0.061s
I20260709 13:02:30.308311  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1245) took 1077 ms. Trace:
W20260709 13:02:30.309077  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 628871us
I20260709 13:02:30.309199  1565 rpcz_store.cc:276] 0709 13:02:29.231167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.443748 (+212581us) service_pool.cc:224] Handling call
0709 13:02:29.900073 (+456325us) spinlock_profiling.cc:243] Waited 56.2 ms on lock 0x7b5800011198. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:02:29.944401 (+ 44328us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b5800011198. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60e9c 00007f8245d60a5b 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:02:30.307965 (+363564us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b54002b8754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:30.308227 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":964333440}
W20260709 13:02:30.312120  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.315824  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:02:30.316068  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.318941  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:30.279094  3537 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:30.305099  3531 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:30.310822  3315 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:30.304390  3510 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:02:30.340533  1720 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.000s	sys 0.026s
W20260709 13:02:30.353861  1720 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.000s	sys 0.027s
I20260709 13:02:30.356060  3510 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:30.318117  3359 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:30.356077  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.014s	sys 0.023s
W20260709 13:02:30.357514  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.015s	sys 0.023s
I20260709 13:02:30.357990  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1790) took 3858 ms. Trace:
I20260709 13:02:30.339254  3531 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:02:30.368854  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 144140us
W20260709 13:02:30.362088  1611 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.021s	sys 0.041s
W20260709 13:02:30.377235  1611 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.021s	sys 0.042s
W20260709 13:02:30.377620  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 586847us
I20260709 13:02:30.380203  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:30.385569  1718 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:30.383257  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:30.388942  1714 raft_consensus.cc:2393] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:30.097716  1646 rpcz_store.cc:276] 0709 13:02:27.786495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.258074 (+1471579us) service_pool.cc:224] Handling call
0709 13:02:29.278482 (+ 20408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.279535 (+  1053us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:29.640147 (+360612us) log.cc:844] Serialized 14 byte log entry
0709 13:02:30.081856 (+441709us) log.cc:844] Serialized 14 byte log entry
0709 13:02:30.082224 (+   368us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:30.082313 (+    89us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:30.096731 (+ 14418us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.096834 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.097124 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":495609,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":68,"raft.queue_time_us":12472,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":29103,"spinlock_wait_cycles":15485312,"thread_start_us":765003,"threads_started":2}
I20260709 13:02:30.369971  3181 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:30.381211  3531 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:30.383617  3339 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:30.463835  1565 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:30.479497  3533 consensus_queue.cc:1059] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:30.382423  3253 consensus_queue.cc:1059] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:02:30.482699  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1527) took 1238 ms. Trace:
I20260709 13:02:30.483562  3410 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.484207  1568 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:02:30.483306  1567 rpcz_store.cc:276] 0709 13:02:29.243947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.791252 (+547305us) service_pool.cc:224] Handling call
0709 13:02:30.482432 (+691180us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b54001b4754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:30.482637 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1082005760}
I20260709 13:02:30.486888  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1247) took 1255 ms. Trace:
W20260709 13:02:30.497012  1716 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.020s	sys 0.004s
W20260709 13:02:30.497694  1716 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.020s	sys 0.004s
W20260709 13:02:30.512709  1468 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.016s	sys 0.013s
W20260709 13:02:30.471051  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 64404.7us
I20260709 13:02:30.488636  1568 rpcz_store.cc:276] 0709 13:02:29.231881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.879372 (+647491us) service_pool.cc:224] Handling call
0709 13:02:30.483695 (+604323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:30.485058 (+  1363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.486690 (+  1632us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b54001b4750. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:30.486822 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1389046528}
W20260709 13:02:30.532083  1468 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.017s	sys 0.013s
W20260709 13:02:30.519207  2313 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.978s	user 0.048s	sys 0.094s
I20260709 13:02:30.527443  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:30.531682  3590 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.124s	user 0.003s	sys 0.000s
I20260709 13:02:30.462641  3315 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.535974  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:02:30.533502  1569 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.000s	sys 0.024s
I20260709 13:02:30.484548  3410 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:30.543411  3595 log.cc:927] Time spent T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.138s	user 0.003s	sys 0.000s
I20260709 13:02:30.361479  1720 rpcz_store.cc:276] 0709 13:02:26.499126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.277992 (+2778866us) service_pool.cc:224] Handling call
0709 13:02:29.278638 (+   646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.279308 (+   670us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:29.718199 (+438891us) log.cc:844] Serialized 14 byte log entry
0709 13:02:30.357480 (+639281us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:30.357557 (+    77us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:30.357626 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.357739 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.357910 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":802021,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":6791,"thread_start_us":1018405,"threads_started":2}
I20260709 13:02:30.545061  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 124) took 1708 ms. Trace:
I20260709 13:02:30.584964  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:30.537977  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1789) took 4039 ms. Trace:
I20260709 13:02:30.547251  1717 raft_consensus.cc:2393] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:30.554157  1569 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.001s	sys 0.024s
W20260709 13:02:30.576064  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:30.538621  2313 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.052s	sys 0.094s
I20260709 13:02:30.586010  1718 raft_consensus.cc:2410] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:02:30.612968  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 47060us
I20260709 13:02:30.543277  3253 raft_consensus.cc:3055] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:30.653842  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:30.608999  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:02:30.612274  3583 log.cc:927] Time spent T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.062s	user 0.001s	sys 0.001s
W20260709 13:02:30.668562  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:30.634972  3590 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.071s	user 0.001s	sys 0.001s
I20260709 13:02:30.663800  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:30.683326  1720 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:02:30.683493  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 522853us
I20260709 13:02:30.689250  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:30.690429  2348 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
W20260709 13:02:30.680704  1566 consensus_meta.cc:287] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.844s	user 0.016s	sys 0.012s
I20260709 13:02:30.694504  1566 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:30.698865  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1522) took 1456 ms. Trace:
W20260709 13:02:30.701687  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.561450  3315 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:30.705150  1713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.044s	sys 0.099s
I20260709 13:02:30.646781  3253 raft_consensus.cc:740] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:30.706208  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1801) took 1728 ms. Trace:
W20260709 13:02:30.706830  1713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.044s	sys 0.100s
W20260709 13:02:30.706910  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.704416  1566 rpcz_store.cc:276] 0709 13:02:29.242374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.792054 (+549680us) service_pool.cc:224] Handling call
0709 13:02:30.698768 (+906714us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.586867  1721 rpcz_store.cc:276] 0709 13:02:26.498801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.274043 (+2775242us) service_pool.cc:224] Handling call
0709 13:02:29.274749 (+   706us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.275768 (+  1019us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.275835 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:29.276705 (+   870us) log.cc:844] Serialized 26 byte log entry
0709 13:02:30.044123 (+767418us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:30.534473 (+490350us) log.cc:844] Serialized 14 byte log entry
0709 13:02:30.534931 (+   458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.535034 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:30.535091 (+    57us) raft_consensus.cc:1685] finished
0709 13:02:30.535106 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.537884 (+  2778us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":127894,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":33,"raft.queue_time_us":19552,"raft.run_cpu_time_us":504,"raft.run_wall_time_us":23689,"thread_start_us":1219506,"threads_started":2,"wal-append.queue_time_us":1064294}
I20260709 13:02:30.700768  1611 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:30.720569  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:30.734730  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:30.736303  1721 raft_consensus.cc:2393] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:30.744940  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:30.746343  1717 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:30.749262  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:30.750747  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 296809us
I20260709 13:02:30.727633  3410 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:30.719385  3510 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:30.769702  1720 raft_consensus.cc:2393] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 3: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:30.771890  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:30.773586  1721 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:30.762938  1685 leader_election.cc:336] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.158s (SENT)
I20260709 13:02:30.776029  1685 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:30.736956  3546 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.010s	sys 0.037s
I20260709 13:02:30.779924  3359 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:02:30.780093  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 149523us
I20260709 13:02:30.790727  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:30.792399  1718 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:30.793470  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1802) took 1814 ms. Trace:
I20260709 13:02:30.794378  1716 rpcz_store.cc:276] 0709 13:02:28.978605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.730559 (+751954us) service_pool.cc:224] Handling call
0709 13:02:29.782075 (+ 51516us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.782847 (+   772us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.782889 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:29.783795 (+   906us) log.cc:844] Serialized 26 byte log entry
0709 13:02:30.498361 (+714566us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:30.549269 (+ 50908us) log.cc:844] Serialized 14 byte log entry
0709 13:02:30.574065 (+ 24796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.574194 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:30.793055 (+218861us) raft_consensus.cc:1685] finished
0709 13:02:30.793118 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.793357 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53123,"prepare.queue_time_us":43210,"prepare.run_cpu_time_us":158,"prepare.run_wall_time_us":17922,"raft.queue_time_us":18038,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":7960,"thread_start_us":713800,"threads_started":1,"wal-append.queue_time_us":620486}
W20260709 13:02:30.781777  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 614859us
I20260709 13:02:30.778942  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:30.800696  1717 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:30.814307  1612 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:30.817553  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 330835us
I20260709 13:02:30.817852  3382 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:30.782505  3546 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.010s	sys 0.038s
I20260709 13:02:30.728654  3469 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:02:30.779789  3537 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:30.836005  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.789628  3591 raft_consensus.cc:3055] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:30.788980  3359 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:30.808007  2342 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:02:30.848498  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 144559us
I20260709 13:02:30.821082  3592 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:30.585125  1722 rpcz_store.cc:276] 0709 13:02:28.836770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.422317 (+585547us) service_pool.cc:224] Handling call
0709 13:02:29.434448 (+ 12131us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.436052 (+  1604us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:30.080855 (+644803us) log.cc:844] Serialized 14 byte log entry
0709 13:02:30.544519 (+463664us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:30.544604 (+    85us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:30.544673 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.544774 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.544976 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2562,"prepare.queue_time_us":735474,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":7832,"raft.run_cpu_time_us":802,"raft.run_wall_time_us":29099,"thread_start_us":1029927,"threads_started":2}
I20260709 13:02:30.857774  1685 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:30.840631  3382 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:30.876266  3597 log.cc:927] Time spent T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:02:30.843690  3591 raft_consensus.cc:740] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:30.877497  3604 consensus_queue.cc:1059] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:30.859217  3592 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.879210  3592 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:30.880136  1568 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:30.881017  3588 consensus_queue.cc:1059] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:30.862581  3606 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:30.883455  3588 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:30.859977  3533 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.891144  3533 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:30.892763  3461 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.730s	user 0.005s	sys 0.012s
I20260709 13:02:30.880019  3604 raft_consensus.cc:3055] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:30.864619  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:30.910049  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 169) took 1440 ms. Trace:
I20260709 13:02:30.865381  3253 consensus_queue.cc:260] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:30.916083  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 80412.1us
W20260709 13:02:30.878767  3594 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
W20260709 13:02:30.917331  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 116407us
W20260709 13:02:30.919011  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 61339.6us
I20260709 13:02:30.923946  1685 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:30.910501  1569 rpcz_store.cc:276] 0709 13:02:29.502493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.923390 (+420897us) service_pool.cc:224] Handling call
0709 13:02:29.942118 (+ 18728us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:30.010069 (+ 67951us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.010109 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:30.010766 (+   657us) log.cc:844] Serialized 26 byte log entry
0709 13:02:30.587177 (+576411us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:30.587225 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.587299 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:30.909785 (+322486us) raft_consensus.cc:1685] finished
0709 13:02:30.909800 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.909981 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67137,"raft.queue_time_us":68414,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":141,"thread_start_us":575771,"threads_started":1,"wal-append.queue_time_us":668943}
W20260709 13:02:30.940337  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 126106us
I20260709 13:02:30.944171  3593 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:30.944814  2343 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:30.904537  3604 raft_consensus.cc:740] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:30.907034  3180 consensus_queue.cc:1059] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:02:30.915277  3510 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:30.908682  3382 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:30.969918  3382 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:30.957558  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1806) took 1921 ms. Trace:
I20260709 13:02:30.947867  3315 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:30.896143  3599 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 13:02:30.709295  1719 rpcz_store.cc:276] 0709 13:02:28.978261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.422325 (+444064us) service_pool.cc:224] Handling call
0709 13:02:29.422697 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.423040 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:30.274631 (+851591us) log.cc:844] Serialized 14 byte log entry
0709 13:02:30.705476 (+430845us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:30.705611 (+   135us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:30.705790 (+   179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.705954 (+   164us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.706123 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":722160,"prepare.run_cpu_time_us":528,"prepare.run_wall_time_us":528,"raft.queue_time_us":3150,"thread_start_us":1196897,"threads_started":2}
I20260709 13:02:30.922547  3181 consensus_queue.cc:1059] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:02:30.991694  3181 raft_consensus.cc:3055] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:30.992400  3181 raft_consensus.cc:740] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:30.970578  1713 rpcz_store.cc:276] 0709 13:02:29.035851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.024392 (+988541us) service_pool.cc:224] Handling call
0709 13:02:30.031659 (+  7267us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:30.101960 (+ 70301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.102015 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:30.122329 (+ 20314us) log.cc:844] Serialized 52 byte log entry
0709 13:02:30.710396 (+588067us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:30.710438 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.710643 (+   205us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:30.957288 (+246645us) raft_consensus.cc:1685] finished
0709 13:02:30.957310 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.957493 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17117,"raft.run_cpu_time_us":420,"raft.run_wall_time_us":40768,"thread_start_us":532027,"threads_started":1,"wal-append.queue_time_us":784265}
I20260709 13:02:30.995293  3181 consensus_queue.cc:260] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:30.987466  1537 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:30.978348  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:30.989775  1716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:02:31.003301  1716 raft_consensus.cc:606] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:31.003821  1716 raft_consensus.cc:614] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:31.003456  3510 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:31.005326  3593 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:31.006126  1611 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:31.005998  3510 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:31.007539  3410 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:31.007290  3591 consensus_queue.cc:260] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:31.010407  3538 consensus_queue.cc:1059] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:31.011579  3591 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:31.012516  3533 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:30.977710  3607 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:31.005306  3610 consensus_queue.cc:1175] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "130bc5ff39b847b1b3204626f284f838" responder_term: 5 status { last_received { term: 4 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:31.007194  3461 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:31.016760  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.022953  3531 consensus_meta.cc:287] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.558s	user 0.019s	sys 0.002s
I20260709 13:02:31.022822  3604 consensus_queue.cc:260] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:31.023484  3531 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:31.024753  3604 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:31.016139  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:30.972599  1717 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:31.014067  3607 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:31.027837  3510 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:31.029188  3382 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:31.016317  3461 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.030993  3382 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:31.040781  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 89980.9us
I20260709 13:02:31.013218  3533 consensus_queue.cc:1059] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:31.040336  3461 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:31.059806  3181 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:31.062767  3605 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:31.057446  3531 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:31.118075  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 131692us
I20260709 13:02:31.118075  1565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7501558aa93542308c5fc758446387df"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:02:31.119275  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:31.121114  3531 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:31.121677  3531 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:31.122057  3531 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:31.110813  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1515) took 1334 ms. Trace:
I20260709 13:02:31.124562  1715 rpcz_store.cc:276] 0709 13:02:29.776469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.184905 (+408436us) service_pool.cc:224] Handling call
0709 13:02:30.201483 (+ 16578us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:30.202240 (+   757us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.202283 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:30.227736 (+ 25453us) log.cc:844] Serialized 26 byte log entry
0709 13:02:30.648225 (+420489us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:30.648300 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.649133 (+   833us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:31.110578 (+461445us) raft_consensus.cc:1685] finished
0709 13:02:31.110600 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.110749 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":586,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":12167,"thread_start_us":382744,"threads_started":1,"wal-append.queue_time_us":616103}
I20260709 13:02:31.053995  3603 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.128517  3603 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:31.131254  3618 raft_consensus.cc:3055] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:31.135134  3603 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:31.073407  3617 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.094923  3359 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:31.139210  3617 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:31.077512  3339 consensus_queue.cc:1059] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 13:02:31.035126  3510 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:31.143775  1565 raft_consensus.cc:606] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:31.149415  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 99003.9us
I20260709 13:02:31.149127  1565 raft_consensus.cc:614] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:31.151228  1537 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:31.151794  1611 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:31.132057  3618 raft_consensus.cc:740] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:02:31.153216  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.070950  3599 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.156904  1612 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:31.160432  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 99391.2us
W20260709 13:02:31.170526  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 88336.2us
I20260709 13:02:31.176563  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:31.178010  1569 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:02:31.179764  3315 raft_consensus.cc:3055] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:31.180274  3315 raft_consensus.cc:740] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:31.181671  3410 raft_consensus.cc:3055] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:31.182256  3410 raft_consensus.cc:740] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:31.183265  3410 consensus_queue.cc:260] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:31.156178  3382 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:31.191028  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:31.177089  1715 pending_rounds.cc:85] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 5
I20260709 13:02:31.204033  1716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:02:31.185529  3606 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:31.185323  3410 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:31.218779  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:31.218778  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:31.243085  2343 raft_consensus.cc:2376] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3. Re-sending same reply.
I20260709 13:02:31.244536  1647 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:31.181322  3315 consensus_queue.cc:260] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:31.199767  3592 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:31.248776  3315 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:31.221226  3607 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:31.221846  3462 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:31.222040  3616 log.cc:927] Time spent T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.117s	user 0.003s	sys 0.000s
I20260709 13:02:31.221590  3538 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:31.227952  1715 pending_rounds.cc:107] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:02:31.255627  1715 raft_consensus.cc:2889] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:31.255118  3462 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:31.228538  1716 raft_consensus.cc:606] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:31.258781  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.221581  3623 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:31.246608  3382 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:31.243129  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:31.204249  3621 consensus_queue.cc:1059] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.085s
I20260709 13:02:31.258555  1716 raft_consensus.cc:614] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:31.240463  1566 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:31.204172  3253 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:31.264884  1686 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:31.176306  3603 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:31.284241  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.202150  3510 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:31.286019  1648 raft_consensus.cc:2410] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:02:31.296624  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 139831us
I20260709 13:02:31.224432  3617 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:31.289250  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:31.300776  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 149722us
I20260709 13:02:31.312491  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:31.312978  1537 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:31.313428  1644 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:31.313889  3625 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.314597  3625 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:31.318749  1716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:02:31.319414  1716 raft_consensus.cc:606] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:31.319768  1716 raft_consensus.cc:614] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:31.320849  3605 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:31.321353  2312 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:31.285717  3462 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:31.322854  2312 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:31.326031  3624 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:31.326567  3624 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.327054  3624 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:31.328744  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:31.330025  1645 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 3: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:31.312424  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:31.330454  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:31.340658  1566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:02:31.305387  3537 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:31.341621  1566 raft_consensus.cc:606] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:31.342078  1566 raft_consensus.cc:614] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:31.342151  3537 raft_consensus.cc:697] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:31.343272  1647 raft_consensus.cc:2410] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:31.198608  3618 consensus_queue.cc:260] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:31.229861  3599 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:31.298210  3623 raft_consensus.cc:697] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:31.357594  3623 consensus_queue.cc:237] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:31.358526  3599 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:31.364477  3567 raft_consensus.cc:3055] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:31.365166  3567 raft_consensus.cc:740] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:31.325374  3461 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:31.367882  3461 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:31.368130  1612 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:31.370121  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:31.302075  3382 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:31.374756  3592 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:31.376536  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:31.378993  3339 consensus_queue.cc:1059] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:31.380865  3315 consensus_queue.cc:1059] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:31.380510  3469 consensus_queue.cc:1048] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:31.374020   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:31.391299   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:31.384121  3618 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:31.396601  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:31.355247  1646 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:31.402516  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:31.373440  3382 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:31.414352  1645 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:31.354776  3603 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:31.373122  3567 consensus_queue.cc:260] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:31.420737  1612 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:31.425896  3614 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:31.426601  3180 raft_consensus.cc:3055] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:31.409157  3607 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:31.430727  1537 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:31.435302  3614 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:31.437295  3614 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:31.436690  3410 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:31.456293  1612 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:31.462702  1612 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:31.463068  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:31.427064  3180 raft_consensus.cc:740] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:31.465124  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:31.462265  3621 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:31.472585  3625 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:31.427994  3461 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:31.476534  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 45989.3us
I20260709 13:02:31.477423  1645 raft_consensus.cc:2410] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:31.343315  3537 consensus_queue.cc:237] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:31.494949  3180 consensus_queue.cc:260] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:31.495420  3607 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.499532  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:31.502957  1646 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:31.413705  3531 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:31.511387  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:31.516790  1715 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:31.520982  1718 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:31.512291  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:31.516297  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:31.519831  1720 raft_consensus.cc:1275] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:31.522188  3510 consensus_queue.cc:1059] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:31.522362  1716 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:02:31.523387  3510 raft_consensus.cc:3055] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:31.523784  3510 raft_consensus.cc:740] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:31.523838  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:31.524796  3510 consensus_queue.cc:260] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:31.525269  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:31.526093  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:31.526630  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:31.526132  3410 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.492957  3604 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:31.528967  3510 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:31.491029  3603 raft_consensus.cc:697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:31.515247  3461 raft_consensus.cc:697] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 5 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:31.517966  3592 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:31.539170  3567 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:31.518517  3607 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:31.540531  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:31.541472  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:31.523859  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:31.543107  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:31.569975  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:31.575320  1717 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:31.573474  1722 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:31.544370  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:31.518231  3339 raft_consensus.cc:3055] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:31.574311  1648 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:31.579088  1714 raft_consensus.cc:2410] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 13:02:31.547796  3315 raft_consensus.cc:993] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504: : Instructing follower 130bc5ff39b847b1b3204626f284f838 to start an election
I20260709 13:02:31.549139  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:31.582890  1644 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:31.527486  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:31.548902  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:31.560987  1715 raft_consensus.cc:1275] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:31.522287  1647 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
W20260709 13:02:31.562955  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 100419us
W20260709 13:02:31.564746  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.565418  1565 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:31.526213  3180 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:31.583877  1645 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:31.587205  1612 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:31.576880  1719 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:31.546870  2313 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:31.560134  3605 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.565495  3604 raft_consensus.cc:697] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:31.578598  3339 raft_consensus.cc:740] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:31.579515  3592 raft_consensus.cc:697] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:31.574122  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:31.577054  3410 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:31.576738  1640 raft_consensus.cc:2376] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4. Re-sending same reply.
I20260709 13:02:31.598925  1612 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:31.600888  3617 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.618646  3617 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:31.620931  3621 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:02:31.603552  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 56836.6us
I20260709 13:02:31.613854  3591 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:31.621191  3592 consensus_queue.cc:237] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:31.619206  3461 consensus_queue.cc:237] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:31.620076  1612 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:31.600265  3605 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:31.631690  3181 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:31.634095  1612 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:31.636410  1643 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:31.639052  1612 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:31.608269  3550 raft_consensus.cc:1081] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Signalling peer 130bc5ff39b847b1b3204626f284f838 to start an election
I20260709 13:02:31.582060  3603 consensus_queue.cc:237] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:31.617373  3623 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:31.642220  3469 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.710145  3469 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:31.712724  3469 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:31.715451  3617 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:31.718444  3605 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:31.718533  3624 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:31.665791  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.670030  1685 leader_election.cc:336] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.545s (SENT)
I20260709 13:02:31.621657  3621 raft_consensus.cc:697] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:31.625845  3591 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.741822  1685 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:31.639273  3531 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:31.637609  3410 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:31.638156  3181 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:31.752609  1718 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:31.633036  3604 consensus_queue.cc:237] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:31.755086  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 116106us
I20260709 13:02:31.763557  3359 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:31.774528  3624 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:31.748041  3623 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:31.752058  3591 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:31.784634  3614 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:31.805576  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:31.783390  1612 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:31.808221  1641 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:31.809104  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 139187us
W20260709 13:02:31.810948  1685 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.598s (SENT)
I20260709 13:02:31.811782  1685 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:31.807976  1720 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:31.638021  3339 consensus_queue.cc:260] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:31.843122  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 61884.1us
I20260709 13:02:31.852522  1612 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:31.856310  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:31.863992  1569 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:31.893411  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:31.895187  1568 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:31.833091  3625 consensus_queue.cc:1059] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:02:31.824611  3359 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:31.902267  3359 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:31.891460  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:31.910146  3181 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:31.911012  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:31.910557  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:31.912645  1640 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:31.911676  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:31.914040  1639 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:31.912680  1646 raft_consensus.cc:2393] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:31.793460  3621 consensus_queue.cc:237] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:31.915926  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:31.919803  1643 raft_consensus.cc:2393] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:31.924243  3614 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:31.842702  3605 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.941707  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:31.943647  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:31.940761  1687 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:31.846953  3623 raft_consensus.cc:697] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 5 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:31.948825  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:31.949237  1568 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:02:31.947428  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:31.951102  1568 pending_rounds.cc:85] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 1
I20260709 13:02:31.964195  1568 pending_rounds.cc:107] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:02:31.965068  1568 raft_consensus.cc:2889] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:31.891742  3537 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:02:31.975960  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 98721.7us
W20260709 13:02:31.977068  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.977886  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 166992us
I20260709 13:02:32.012130  2312 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:02:32.021155  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.026875  1570 raft_consensus.cc:1275] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:02:32.027806  1570 pending_rounds.cc:85] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 2
I20260709 13:02:32.028115  1570 pending_rounds.cc:107] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:02:32.028869  1570 raft_consensus.cc:2889] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:32.030546  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.031383  1572 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:31.959676  3510 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:31.957155  3592 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:32.052610  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:32.077234  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:32.078480  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.080070  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:32.219219  1567 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:31.936820  3605 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:31.974996  3469 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.975244  3550 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:31.976558  3181 raft_consensus.cc:697] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:32.095980  1646 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:02:31.956938  3410 consensus_queue.cc:1059] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.090s
I20260709 13:02:32.096287  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:32.014982  3603 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:32.095943  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:02:32.113940  3180 consensus_meta.cc:287] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.524s	user 0.006s	sys 0.007s
I20260709 13:02:32.106531  3624 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.115801  3461 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.119009  3533 raft_consensus.cc:3055] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:32.076690  3625 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:32.125603  3359 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:32.137303  3635 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.204s	user 0.000s	sys 0.004s
I20260709 13:02:32.142230  3537 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:32.144194  3645 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:32.145879  3339 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:32.148406  2342 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:02:32.174554  3639 log.cc:927] Time spent T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.065s	user 0.002s	sys 0.000s
W20260709 13:02:32.179294  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 155948us
W20260709 13:02:32.179844  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.053833  3510 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:31.996549  1685 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.857s (SENT)
I20260709 13:02:32.304363  1685 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:31.976105  2343 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
W20260709 13:02:32.185763  3630 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.133s	user 0.001s	sys 0.001s
I20260709 13:02:32.208415  1569 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
W20260709 13:02:32.195966  3627 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.109s	user 0.001s	sys 0.003s
I20260709 13:02:32.210269  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.220885  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:02:32.220441  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 368059us
I20260709 13:02:32.231602  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:32.312523  1570 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:32.312979  1572 raft_consensus.cc:2410] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:32.239286  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.246847  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.314437  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.315304  1566 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:32.315900  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.316659  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.209163  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.252534  1643 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:32.254045  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.298759  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.318794  1566 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:32.276983  3625 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.245925  3469 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:32.273445  3533 raft_consensus.cc:740] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:32.307957  1687 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:32.320272  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:32.321568  1569 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:32.284057  3359 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.312848  1573 raft_consensus.cc:2376] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5. Re-sending same reply.
I20260709 13:02:32.320549  1612 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:32.325291  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:32.325382  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 328953us
I20260709 13:02:32.330744  1566 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:32.331787  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.319186  1565 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:32.338368  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:32.340729  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:32.343116  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:32.343608  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.344632  1568 raft_consensus.cc:2410] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:32.347162  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.306231  3510 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:32.349673  1572 raft_consensus.cc:2393] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:32.350931  1570 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:32.277047  3624 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:32.276474  3461 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:32.276594  3603 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:32.280236  3550 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:32.355984  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.360950  3469 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:32.352607  1687 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:32.364346  3650 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:32.397891  1571 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 2: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:32.393426  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:02:32.398612  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 46091.8us
I20260709 13:02:32.400549  1687 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:32.371129  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:32.320703  3625 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:32.401875  1566 raft_consensus.cc:2376] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3. Re-sending same reply.
I20260709 13:02:32.380803  1569 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:32.403398  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.403446  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.404225  1572 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:32.406555  1687 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:32.418128  1568 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:32.274279  3623 consensus_queue.cc:237] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:32.402061  3537 raft_consensus.cc:2764] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:02:32.432099   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:32.441948  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.421203  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.422246  1687 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:32.424125  2312 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:32.450596  1567 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:02:32.419178  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 382255us
W20260709 13:02:32.425045  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 247015us
W20260709 13:02:32.451563  2314 leader_election.cc:336] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.183s (SENT)
I20260709 13:02:32.452047  2314 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:32.373607  3605 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:32.455583  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:32.341881  3359 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:32.421128  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.449625  1572 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:32.431468  3645 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:32.497108  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.521435  1565 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:32.378237  3533 consensus_queue.cc:260] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:32.464954  3614 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:32.470746  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 48561.6us
I20260709 13:02:32.526516  1687 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:32.471400  3652 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:32.490056  3636 log.cc:927] Time spent T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.303s	user 0.000s	sys 0.002s
I20260709 13:02:32.410274  3649 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:32.488677  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.463514  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.374200  3181 consensus_queue.cc:237] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:32.534389  3645 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:32.531002  1567 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:32.529032  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.475163  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:32.532389  1687 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:32.496567  3615 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:32.532861  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 77402.1us
W20260709 13:02:32.527838  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 207359us
I20260709 13:02:32.580786  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.535665  1573 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:32.581512  1567 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
W20260709 13:02:32.582340  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 50061.3us
I20260709 13:02:32.504695  3567 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:32.581463  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:32.538662  2312 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:32.583293  1567 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:32.498977  3253 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:32.583510  1572 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:32.588742  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.589109  1612 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:32.590651  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 193295us
W20260709 13:02:32.612213  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.590404  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:02:32.608605  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 157074us
I20260709 13:02:32.528585  3461 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:32.525527  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:32.626166  1568 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:02:32.627377  1568 pending_rounds.cc:85] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 1
I20260709 13:02:32.627704  1568 pending_rounds.cc:107] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:02:32.615370  1687 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:02:32.628626  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 46486.9us
I20260709 13:02:32.628865  1568 raft_consensus.cc:2889] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:32.464318  3621 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.133s
I20260709 13:02:32.539280  3180 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:32.616120  1572 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:02:32.579422  3645 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.535725  3649 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:32.638165  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:32.539573  3614 raft_consensus.cc:697] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:32.530356  3652 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:32.580571  3550 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:32.641706  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 52676.1us
I20260709 13:02:32.679338  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:32.634255  3653 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:32.641996  3461 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.643038  3660 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:32.642074  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.657847  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:32.664778  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 120179us
I20260709 13:02:32.681340  3651 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:02:32.682232  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.716877  1611 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:32.690054  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.718259  1571 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:32.695818  1612 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:32.628984  3605 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:02:32.673799  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 58526us
I20260709 13:02:32.724282  1687 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:32.680646  1565 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:32.659092  3645 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:32.736042  1687 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:32.657934  3603 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:32.743476  1571 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:02:32.613001  3537 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:32.738991  3651 raft_consensus.cc:697] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:02:32.739571  3654 log.cc:927] Time spent T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 13:02:32.648639  3649 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:32.598127  3567 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:02:32.748821  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.658569  1564 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:32.677275  3652 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:32.751353  3537 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.723642  3253 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.716217  3660 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.760619  3650 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:32.773669  3651 consensus_queue.cc:237] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:32.613384  3615 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:32.736459  3382 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:32.806774  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 90037.6us
W20260709 13:02:32.807727  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 71922us
I20260709 13:02:32.818418  1687 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:32.818907  1611 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:32.777287  3655 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:32.789556  3653 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:32.777814  3567 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:32.735080  3461 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:32.828536  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 133063us
I20260709 13:02:32.815536  3537 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:32.824790  3591 consensus_meta.cc:287] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.990s	user 0.012s	sys 0.008s
I20260709 13:02:32.817684  3650 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:32.820752  3621 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.827962  3614 consensus_queue.cc:237] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:32.820297  3382 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:32.817502  3660 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:32.827539  3661 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:32.834080  3339 consensus_meta.cc:287] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.002s
I20260709 13:02:32.875303  1569 raft_consensus.cc:2410] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:32.819582  3253 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:32.877879  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 48724.1us
I20260709 13:02:32.845432  3181 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:32.887979  3359 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:32.844468  3650 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:32.851524  3382 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:32.898177  1612 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:32.856791  3591 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:32.904204  3665 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:32.910614  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 92294.6us
I20260709 13:02:32.911854  1687 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:32.839282  3653 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:02:32.852365  3621 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:32.891264  3181 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.945875  3181 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:32.946189  3668 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.947014  3668 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:32.919409  3359 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:32.949872  3359 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:32.920440  3665 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:32.950384  3359 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:32.951493  3315 raft_consensus.cc:1064] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:02:32.919350  3461 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:32.937299  2343 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 247) took 1440 ms. Trace:
W20260709 13:02:32.938514  3657 log.cc:927] Time spent T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 13:02:32.959879  3621 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:32.962092  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:32.962263  3591 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:32.966163  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:32.968472  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:32.978170  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:32.976812  1715 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
W20260709 13:02:32.979557  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 67832.2us
I20260709 13:02:32.981298  1687 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:32.980810  1720 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:32.992238  1685 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:32.959458  2343 rpcz_store.cc:276] 0709 13:02:31.497278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.504195 (+  6917us) service_pool.cc:224] Handling call
0709 13:02:31.548674 (+ 44479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.313354 (+764680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.936760 (+623406us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b5400712854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:32.937185 (+   425us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":741256960}
I20260709 13:02:32.999377  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:33.003562  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 105620us
I20260709 13:02:33.011977  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:33.013295  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 194431us
I20260709 13:02:33.025008  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:33.003542  1719 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:42276
I20260709 13:02:32.996934  3665 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:33.029969  1611 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:33.031296  1612 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:33.032193  3623 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:33.036953  3621 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:02:33.037859  1611 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:33.035748  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.043809  3666 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:33.044589  3339 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.045198  3339 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:33.047950  3339 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:33.048887  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:33.049741  3669 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:02:33.020113  3670 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:33.052472  1717 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:33.050905  1537 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:33.059139  3339 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:33.059656  3339 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.060024  3339 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:33.062294  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:33.070760  1715 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:33.051703  3670 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.078425  3670 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:33.083817  3531 consensus_queue.cc:1048] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:33.086586  3615 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:33.088320  3607 consensus_meta.cc:287] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.507s	user 0.018s	sys 0.002s
I20260709 13:02:33.088750  3607 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:33.090826  3253 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:33.081045  3670 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:33.094055  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:33.095049  3510 consensus_meta.cc:287] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.714s	user 0.013s	sys 0.006s
I20260709 13:02:33.134845  3603 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.136540  3670 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:33.137107  3603 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:33.139884  3660 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:33.141019  3603 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:33.100304  3537 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:33.101102  3462 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:33.102020  3625 consensus_meta.cc:287] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.683s	user 0.011s	sys 0.004s
I20260709 13:02:33.142359  3462 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.142923  3462 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:33.142549  3625 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:33.143141  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:33.143810  3640 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.934s	user 0.005s	sys 0.018s
I20260709 13:02:33.144198  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:33.144831  1715 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:33.018610  3604 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.146634  3604 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:33.149226  3604 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:33.105849  3653 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:33.106011  3668 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:33.080961  3339 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:33.150002  3653 raft_consensus.cc:697] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:33.106603  3649 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:33.151201  3653 consensus_queue.cc:237] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:33.106675  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 75505.3us
I20260709 13:02:33.154704  3624 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:33.143826  1720 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:33.109763  3607 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:33.110124  3461 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:33.157544  3461 raft_consensus.cc:697] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:33.156314  3625 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:33.159068  3461 consensus_queue.cc:237] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:33.159853  3537 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:33.160375  3648 raft_consensus.cc:3055] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:33.161114  3648 raft_consensus.cc:740] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:02:33.111135  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 71131.8us
I20260709 13:02:33.170517  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:33.171396  1569 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:33.113057  3591 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.174067  3591 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:33.175987  3533 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:33.178287  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:33.179029  1566 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:02:33.179411  3625 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:33.108121  3666 raft_consensus.cc:697] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:33.180335  2312 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:33.181432  3624 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:33.181223  3666 consensus_queue.cc:237] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:33.181978  3624 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.182492  3624 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:33.173221  3339 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:33.183323  3591 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:33.180166  3649 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:33.185905  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.112622  3315 consensus_queue.cc:237] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 3, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:33.120666  3650 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:33.189837  3650 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:33.193996  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:33.194748  1566 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:33.111835  3359 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:33.122450  3667 log.cc:927] Time spent T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.127s	user 0.000s	sys 0.005s
I20260709 13:02:33.122474  3531 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.196578  3531 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:33.128599  1570 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:33.135301  3510 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:33.198246  3648 consensus_queue.cc:260] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:33.123620  3567 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:33.150087  3668 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:33.200399  1714 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:33.200313  3668 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.201366  3668 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:33.201522  3567 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:33.099603  1537 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:33.169721  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:33.203137  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 103691us
W20260709 13:02:33.108801  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 71094.7us
I20260709 13:02:33.203583  1565 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:33.204849  1537 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:33.204921  1611 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:33.207412  3604 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:33.207970  3607 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:33.209259  3607 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.209380  3623 consensus_queue.cc:1059] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:33.209950  3607 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:33.209820  3605 consensus_queue.cc:1048] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:33.210985  3621 raft_consensus.cc:3055] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:33.211424  3621 raft_consensus.cc:740] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:33.212354  3621 consensus_queue.cc:260] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:33.208284  3604 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.213186  3605 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:33.215039  3605 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.216063  3605 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:33.199515  3510 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:33.213748  3673 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:02:33.218164  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:33.218930  1565 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:33.214080  3604 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:33.214143  1537 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:33.221230  1537 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:33.213409  3621 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:33.199533  3531 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:33.225354  3658 consensus_queue.cc:1059] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:33.225934  3339 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:33.227383  3339 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.227836  3339 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:33.234162  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:33.235569  1566 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:33.119490  3645 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:33.121762  3181 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:33.196795  1569 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:33.244553  3531 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:33.244731  2344 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Advancing to term 4
I20260709 13:02:33.246759  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:33.248037  1570 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:02:33.244820  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1541) took 1048 ms. Trace:
W20260709 13:02:33.251703  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.244818  1565 raft_consensus.cc:1275] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:02:33.253868  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 261725us
I20260709 13:02:33.256026  1611 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:33.259371  1611 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:33.250054  1567 rpcz_store.cc:276] 0709 13:02:32.196702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.610346 (+413644us) service_pool.cc:224] Handling call
0709 13:02:32.610870 (+   524us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:32.617184 (+  6314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:32.617213 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:32.665763 (+ 48550us) log.cc:844] Serialized 52 byte log entry
0709 13:02:32.815500 (+149737us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:32.815544 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.815616 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:33.071258 (+255642us) spinlock_profiling.cc:243] Waited 5.86 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:02:33.244467 (+173209us) raft_consensus.cc:1685] finished
0709 13:02:33.244489 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.244754 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":101,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":14460,"spinlock_wait_cycles":13478144,"thread_start_us":77073,"threads_started":1,"wal-append.queue_time_us":135358}
I20260709 13:02:33.262372  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:33.265525  2312 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:02:33.272495  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 291263us
I20260709 13:02:33.278323  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:33.276615  1611 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:33.280191  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:33.281222  2348 raft_consensus.cc:2393] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:33.282541  1687 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:33.241122  3410 raft_consensus.cc:3055] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:33.293063  2313 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:33.300874  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.269414  3673 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:33.302541  1720 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:02:33.302579  1685 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.048s (SENT)
I20260709 13:02:33.280191  2312 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:33.320751  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:33.322991  3625 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:02:33.324354  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 44277.6us
I20260709 13:02:33.280573  3648 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:33.280704  3664 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:33.327875  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:33.342398  1714 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:33.328462  1685 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:33.260702  3603 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:02:33.330142  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:33.338070  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 106010us
I20260709 13:02:33.338416  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:33.338125  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:33.337563  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:33.352726  1716 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:33.351622  1715 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:02:33.352289  1721 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:33.355695  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:33.287824  3623 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:33.328959  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:33.299247  3410 raft_consensus.cc:740] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:33.296118  3567 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:33.359273  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:33.324975  3673 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.360643  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:33.365410  1717 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 6: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:33.296828  3649 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:33.367237  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:33.272473  3510 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:02:33.324064  3665 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.319967  3591 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:33.358316  1537 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:33.375195  1640 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:33.391539  1715 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:33.351203  3664 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.401988  3664 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:33.402927  2340 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:33.370642  3658 consensus_queue.cc:1048] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:02:33.403988  1717 raft_consensus.cc:1275] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:33.405733  3645 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:33.346787  3382 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:33.376277  3359 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:33.370864  3651 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:33.376434  1471 catalog_manager.cc:5697] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 3 to 4, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 4 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:33.374951  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 81406.3us
I20260709 13:02:33.375769  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:33.378588  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:33.379287  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:33.437423  1646 raft_consensus.cc:2410] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:33.438676  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:33.440649  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:33.378162  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:33.383055  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:33.442229  1648 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:33.377506  3649 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.384066  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:33.292775  3650 consensus_queue.cc:1059] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:02:33.337211  3592 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.347314  3655 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.393554  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:33.375497  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:33.346915  3653 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.453282  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:33.365038  3623 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.363030  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:33.404251  1721 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:33.409792  1719 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:33.376511  3673 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:33.388720  3537 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:02:33.367324  3567 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.422646  1717 raft_consensus.cc:1275] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:02:33.423967  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 65781.1us
I20260709 13:02:33.323606  3625 raft_consensus.cc:697] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:33.443888  1642 raft_consensus.cc:2393] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:33.435094  1471 catalog_manager.cc:5697] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:02:33.445904  1646 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:33.453188  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:33.455885  1647 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:33.443051  3649 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:33.397712  3665 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:33.471822  1468 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38666 (request call id 212) took 14114 ms (client timeout 15000 ms). Trace:
I20260709 13:02:33.528349  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:33.529022  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:33.739285  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 373150us
W20260709 13:02:34.320852  3670 consensus_meta.cc:287] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.174s	user 0.016s	sys 0.002s
I20260709 13:02:34.321271  3670 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:34.321938  3670 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:34.325263  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.478803  3623 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:33.753499  1471 catalog_manager.cc:5697] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 3 to 4, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 4 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:33.745970  3567 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:33.680485  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.520112  3653 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
W20260709 13:02:33.597659  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 47274.2us
I20260709 13:02:33.861541  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:33.769155  3382 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:33.884714  3531 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.006s	sys 0.005s
W20260709 13:02:33.947214  2315 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.010s	sys 0.040s
I20260709 13:02:33.477169  3592 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:33.472306  3655 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:34.359091  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:34.363201  1685 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:34.364210  1471 catalog_manager.cc:5697] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 1 to 2, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:33.732472  3676 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.378592  3676 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:34.379897  1565 raft_consensus.cc:1240] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:34.301998  3625 consensus_queue.cc:237] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:34.303349  3680 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:34.303975  3359 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:34.304826  3605 consensus_meta.cc:287] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.088s	user 0.014s	sys 0.022s
I20260709 13:02:33.433142  3674 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:34.375249  3531 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.098s	user 0.010s	sys 0.006s
I20260709 13:02:34.305402  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:33.473055  1715 raft_consensus.cc:1275] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:02:34.307225  3533 consensus_meta.cc:287] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 1.131s	user 0.013s	sys 0.007s
I20260709 13:02:34.306860  1570 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:34.307811  2314 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.029s (SENT) [suppressed 53 similar messages]
I20260709 13:02:33.450865  3645 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:34.309067  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:34.309820  3181 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:33.885192  1566 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.005s	sys 0.007s
W20260709 13:02:34.486642  1566 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.208s	user 0.011s	sys 0.016s
W20260709 13:02:34.310954  3462 consensus_meta.cc:287] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.167s	user 0.024s	sys 0.033s
W20260709 13:02:34.312306  3621 consensus_meta.cc:287] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.090s	user 0.014s	sys 0.005s
I20260709 13:02:34.309296  1719 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:34.315737  3607 consensus_meta.cc:287] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.104s	user 0.015s	sys 0.006s
W20260709 13:02:34.315970  3652 consensus_meta.cc:287] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.498s	user 0.019s	sys 0.036s
I20260709 13:02:34.317592  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:34.318923  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:34.516110  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 51340.5us
I20260709 13:02:34.319240  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:34.519220  1642 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:34.322024  1468 rpcz_store.cc:269] 0709 13:02:19.330266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.330534 (+   268us) service_pool.cc:224] Handling call
0709 13:02:33.444671 (+14114137us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:30.844088 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:30.905769 (+ 61681us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:30.905824 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:02:30.905862 (+    38us) tablet.cc:662] Decoding operations
0709 13:02:31.002018 (+ 96156us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:31.002144 (+   126us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:31.002176 (+    32us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:02:31.003205 (+  1029us) tablet.cc:701] Row locks acquired
0709 13:02:31.003273 (+    68us) write_op.cc:260] PREPARE: finished
0709 13:02:31.003496 (+   223us) write_op.cc:270] Start()
0709 13:02:31.003667 (+   171us) write_op.cc:276] Timestamp: P: 1783602151003412 usec, L: 0
0709 13:02:31.003711 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 13:02:31.004450 (+   739us) log.cc:844] Serialized 2085 byte log entry
0709 13:02:31.724786 (+720336us) op_driver.cc:464] REPLICATION: finished
0709 13:02:32.238244 (+513458us) write_op.cc:301] APPLY: starting
0709 13:02:32.238374 (+   130us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:32.238882 (+   508us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:32.269400 (+ 30518us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:32.283105 (+ 13705us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:32.410244 (+127139us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:02:32.425912 (+ 15668us) write_op.cc:312] APPLY: finished
0709 13:02:32.480497 (+ 54585us) log.cc:844] Serialized 47 byte log entry
0709 13:02:32.480760 (+   263us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:32.480904 (+   144us) write_op.cc:454] Released schema lock
0709 13:02:32.480989 (+    85us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:32.481522 (+   533us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:32.481555 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":17,"spinlock_wait_cycles":4864,"child_traces":[["op",{"apply.queue_time_us":483701,"apply.run_cpu_time_us":9564,"apply.run_wall_time_us":248770,"num_ops":5,"prepare.queue_time_us":758065,"prepare.run_cpu_time_us":36331,"prepare.run_wall_time_us":383345,"raft.queue_time_us":339936,"raft.run_cpu_time_us":57074,"raft.run_wall_time_us":364646,"replication_time_us":635920,"thread_start_us":1140203,"threads_started":4,"wal-append.queue_time_us":219257,"wal-append.run_cpu_time_us":82027,"wal-append.run_wall_time_us":2305048}]]}
I20260709 13:02:34.341572  1641 raft_consensus.cc:2410] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:34.366336  1685 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:34.233603  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.001s	sys 0.009s
W20260709 13:02:34.367956  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.368846  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.368919  3650 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:34.337066  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.336180  1537 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:34.373127  2315 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.944s	user 0.012s	sys 0.042s
W20260709 13:02:34.524468  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 188499us
I20260709 13:02:34.374799  1648 raft_consensus.cc:2410] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:34.525764  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1816) took 1136 ms. Trace:
I20260709 13:02:34.527055  1648 rpcz_store.cc:276] 0709 13:02:33.389750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.443061 (+ 53311us) service_pool.cc:224] Handling call
0709 13:02:34.525688 (+1082627us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.527721  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:34.528162  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:34.528339  1642 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:34.529261  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:34.376662  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.538255  1648 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:34.538498  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.705334  3410 consensus_queue.cc:260] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:34.384486  1471 catalog_manager.cc:5697] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 3 to 4, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 4 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:34.305012  3624 consensus_meta.cc:287] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 1.122s	user 0.029s	sys 0.015s
W20260709 13:02:34.381400  3677 log.cc:927] Time spent T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:02:34.450054  1716 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:34.451257   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:34.452976  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 105764us
W20260709 13:02:34.519744  3339 consensus_meta.cc:287] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.291s	user 0.013s	sys 0.008s
I20260709 13:02:34.414626  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:34.555905  1717 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
W20260709 13:02:34.455090  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 147288us
I20260709 13:02:34.483357  1570 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:34.483815  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1531) took 1183 ms. Trace:
W20260709 13:02:34.350744  3649 consensus_meta.cc:287] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.599s	user 0.013s	sys 0.002s
W20260709 13:02:34.395792  3679 log.cc:927] Time spent T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.713s	user 0.003s	sys 0.009s
W20260709 13:02:34.325495  2344 consensus_meta.cc:287] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 1.080s	user 0.006s	sys 0.010s
W20260709 13:02:34.361521  3664 consensus_meta.cc:287] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.958s	user 0.010s	sys 0.001s
W20260709 13:02:34.332414  3604 consensus_meta.cc:287] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.105s	user 0.016s	sys 0.000s
I20260709 13:02:34.386849  3680 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:34.539745  3688 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.387347  3676 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:34.508472  1573 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.060s	user 0.004s	sys 0.011s
W20260709 13:02:34.520989  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.002s	sys 0.009s
I20260709 13:02:34.522572  1647 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:34.532379  1538 leader_election.cc:336] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.529s (SENT)
W20260709 13:02:34.329097  3648 consensus_meta.cc:287] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.992s	user 0.009s	sys 0.003s
I20260709 13:02:34.431874  3674 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:34.576050  3603 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:34.540021  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1818) took 1131 ms. Trace:
I20260709 13:02:34.477905  3695 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.541944  1471 catalog_manager.cc:5697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:34.542497  1537 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:34.688714  1644 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:34.431730  3531 raft_consensus.cc:697] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:34.451377  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1542) took 1015 ms. Trace:
I20260709 13:02:34.696703  1715 rpcz_store.cc:276] 0709 13:02:33.435663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.436049 (+   386us) service_pool.cc:224] Handling call
0709 13:02:33.456066 (+ 20017us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.450797 (+994731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.451304 (+   507us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.322494  3670 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:34.559504  1722 rpcz_store.cc:276] 0709 13:02:33.300534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.300690 (+   156us) service_pool.cc:224] Handling call
0709 13:02:33.301911 (+  1221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.324112 (+ 22201us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:33.531489 (+207377us) log.cc:844] Serialized 14 byte log entry
0709 13:02:34.483342 (+951853us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:34.483403 (+    61us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:34.483459 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.483533 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.483715 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1023697,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":41,"raft.queue_time_us":96,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":31546,"thread_start_us":339703,"threads_started":2}
I20260709 13:02:34.492138  3655 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:34.575239  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 201) took 1106 ms. Trace:
I20260709 13:02:34.701463  1570 rpcz_store.cc:276] 0709 13:02:33.452621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.452834 (+   213us) service_pool.cc:224] Handling call
0709 13:02:34.559394 (+1106560us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.413208  3359 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:34.699049  1471 catalog_manager.cc:5697] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:34.572028  3649 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:34.483816  3645 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:34.618088  1538 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:34.623781  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:34.730192  1716 raft_consensus.cc:2393] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:34.413512  3605 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:34.340591  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:34.731427  1721 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:34.525547  3462 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:34.525317  3607 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:34.482131  3382 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:34.616700  1573 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.168s	user 0.005s	sys 0.012s
I20260709 13:02:34.617815  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:34.522857  3652 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:34.683408  1646 rpcz_store.cc:276] 0709 13:02:33.408844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.317036 (+908192us) service_pool.cc:224] Handling call
0709 13:02:34.539937 (+222901us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:34.699844  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 157474us
I20260709 13:02:34.700182  1643 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:34.808233  1537 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:34.711925  2314 leader_election.cc:336] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.537s (SENT)
I20260709 13:02:34.811085  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:34.713721  2312 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:02:34.812296  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 99252.2us
I20260709 13:02:34.616401  3680 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:34.618825  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:34.710418  3695 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:34.572932  3339 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:34.557753  3624 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:34.561416  3650 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:34.727522  1720 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:34.609899  3696 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:34.732590  1471 catalog_manager.cc:5697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:34.609822  3693 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.696070  3666 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.800812  1642 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:34.801209  1639 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:34.611181  3682 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.267s
I20260709 13:02:34.683494  3665 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:34.801563  1570 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:34.675351  3688 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:34.720247  2344 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEARNER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:34.816365  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:02:34.811504  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 99640.7us
I20260709 13:02:34.819417  3690 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:34.820502  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:34.821521  3382 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:34.823737  1640 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 5. Ops: []
W20260709 13:02:34.829397  3700 log.cc:927] Time spent T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:02:34.736456  3697 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:34.613935  3604 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:34.830049  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:34.843092  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:34.846114  1565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:40642
W20260709 13:02:34.837244  3659 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.016s	sys 0.030s
I20260709 13:02:34.839658  3649 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:34.864811  1719 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:34.611729  3664 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:34.597991  3683 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.164s
I20260709 13:02:34.695953  3603 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:35.225148  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 858871us
I20260709 13:02:35.260224  3666 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:35.261221  3659 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.528s	user 0.021s	sys 0.035s
I20260709 13:02:34.834925  3670 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:34.848203  3695 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:35.261953  3603 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:35.248931  3645 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:35.262946  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:35.092820  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.263736  1564 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:02:35.264667  3673 consensus_meta.cc:287] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.380s	user 0.010s	sys 0.005s
I20260709 13:02:35.265708  1715 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:02:34.838003  3531 consensus_queue.cc:237] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:35.239673  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1543) took 1789 ms. Trace:
I20260709 13:02:35.266880  1718 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 7. Ops: []
I20260709 13:02:35.268247  1685 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:35.268170  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:35.272543  1565 raft_consensus.cc:606] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Received request to transfer leadership
W20260709 13:02:35.258059  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 725721us
W20260709 13:02:35.278434  1648 consensus_meta.cc:287] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.719s	user 0.033s	sys 0.025s
W20260709 13:02:34.843932  3668 consensus_meta.cc:287] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.642s	user 0.020s	sys 0.005s
I20260709 13:02:35.282723  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:35.455185  1648 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:35.456351  1642 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:35.249516  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1555) took 2016 ms. Trace:
I20260709 13:02:34.845765  3315 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:35.457161  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1257) took 1132 ms. Trace:
I20260709 13:02:35.459558  1570 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:35.496148  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:02:35.458881  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 190720us
W20260709 13:02:34.923960  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.457206  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1246) took 1136 ms. Trace:
I20260709 13:02:35.472699  1572 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:34.937501  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:35.460736  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.318987  3664 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:34.904362  1569 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:02:34.832010  1747 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:38281 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:38281 timed out after 15.000s (SENT)
I20260709 13:02:35.336444  3592 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:35.497905  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.498733  1685 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:35.498518  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 140) took 1190 ms. Trace:
I20260709 13:02:35.512580  1715 rpcz_store.cc:276] 0709 13:02:34.307790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.735541 (+427751us) service_pool.cc:224] Handling call
0709 13:02:35.498433 (+762892us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.863726  3614 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:35.498967  1642 rpcz_store.cc:276] 0709 13:02:34.321124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.259598 (+938474us) service_pool.cc:224] Handling call
0709 13:02:35.457069 (+197471us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:35.497411  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:35.496472  1648 rpcz_store.cc:276] 0709 13:02:34.324277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.529021 (+204744us) service_pool.cc:224] Handling call
0709 13:02:35.456992 (+927971us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:35.520126  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:35.520707  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:35.457206  1573 rpcz_store.cc:276] 0709 13:02:33.232792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.232939 (+   147us) service_pool.cc:224] Handling call
0709 13:02:33.245927 (+ 12988us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:33.394181 (+148254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.394273 (+    92us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:33.396192 (+  1919us) log.cc:844] Serialized 26 byte log entry
0709 13:02:34.806772 (+1410580us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:34.806832 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.806937 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:35.249189 (+442252us) raft_consensus.cc:1685] finished
0709 13:02:35.249223 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:35.249431 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":652120,"raft.queue_time_us":435937,"raft.run_cpu_time_us":12689,"raft.run_wall_time_us":656718,"thread_start_us":1481282,"threads_started":2,"wal-append.queue_time_us":1226637}
I20260709 13:02:35.520126  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:35.522121  1642 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.455304  1714 rpcz_store.cc:276] 0709 13:02:33.450711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.450917 (+   206us) service_pool.cc:224] Handling call
0709 13:02:33.521342 (+ 70425us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:33.521876 (+   534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.521920 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:34.314908 (+792988us) log.cc:844] Serialized 52 byte log entry
0709 13:02:34.536700 (+221792us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:34.536745 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.536818 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:35.239200 (+702382us) raft_consensus.cc:1685] finished
0709 13:02:35.239233 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:35.239479 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":801014,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":143,"thread_start_us":221097,"threads_started":1,"wal-append.queue_time_us":304121}
I20260709 13:02:35.514384  1646 raft_consensus.cc:2410] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
W20260709 13:02:35.515985  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.248618  3462 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:35.266517  3604 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:35.257364  3615 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.513556  1685 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:35.524910  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:35.532986  1565 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:35.533793  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1833) took 1152 ms. Trace:
I20260709 13:02:34.841898  3623 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:35.528770  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 271337us
I20260709 13:02:34.900422  3652 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:34.831455  3674 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:35.534224  1646 rpcz_store.cc:276] 0709 13:02:34.381247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.807743 (+426496us) service_pool.cc:224] Handling call
0709 13:02:35.533708 (+725965us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.829537  2342 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:35.535953  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1839) took 1059 ms. Trace:
I20260709 13:02:35.502929  3339 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:35.557767  1648 rpcz_store.cc:276] 0709 13:02:34.476347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.520440 (+1044093us) service_pool.cc:224] Handling call
0709 13:02:35.535827 (+ 15387us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:35.503759  3683 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:35.503459  3603 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:35.547649  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:35.496526  3605 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:35.551277  1642 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:34.923576  3676 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.537300  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:35.640820  1568 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:35.496443  3708 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:35.282615  3697 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:35.514042  3624 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:35.480877  3673 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:35.529395  3670 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:35.661873  3670 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:35.662369  3670 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:35.611740  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:35.663445  1648 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:35.664602  3707 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 13:02:35.668268  3673 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:35.647629  3697 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:35.669567  3709 log.cc:927] Time spent T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:02:35.674569  3462 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.676259  3462 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:35.622118  3603 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:35.678033  3603 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:35.683570  3670 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
W20260709 13:02:35.689000  3680 consensus_meta.cc:287] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.829s	user 0.006s	sys 0.003s
I20260709 13:02:35.480127  3668 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:35.691004  2341 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 LEARNER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 13:02:35.561973  1537 connection.cc:256] RPC call timeout handler was delayed by -2.2783137220000071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:02:35.677528  1568 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:35.623201  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:35.622540  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:35.282629  3693 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:35.692258  1646 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.692342  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:35.693338  1568 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.695199  3693 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:35.696286  1713 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:35.654771  3676 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:35.698304  3668 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:35.607136  3683 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:35.700333  3676 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:35.692664  1573 raft_consensus.cc:2393] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:35.702603  1646 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
W20260709 13:02:35.703328  1646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1251) took 1380 ms (client timeout 1800 ms). Trace:
I20260709 13:02:35.703661  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1527) took 2405 ms. Trace:
I20260709 13:02:35.704239  1713 rpcz_store.cc:276] 0709 13:02:33.298031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.298188 (+   157us) service_pool.cc:224] Handling call
0709 13:02:33.298717 (+   529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.696887 (+2398170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.703412 (+  6525us) spinlock_profiling.cc:243] Waited 2.4 s on lock 0x7b540020a154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:35.703598 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5513869696}
I20260709 13:02:35.706123  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:35.706637  1572 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:35.622517  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:35.559422  1685 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:35.707024  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 58090.7us
I20260709 13:02:35.536526  3614 raft_consensus.cc:697] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:35.558753  3649 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:35.644055  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 189126us
W20260709 13:02:35.711921  1538 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.542s (SENT)
W20260709 13:02:35.712018  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 152673us
I20260709 13:02:35.710718  1568 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:02:35.649027  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:35.713557  1570 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.713619  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:35.650964  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:35.530732  3650 consensus_queue.cc:1059] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.093s
I20260709 13:02:35.529716  3695 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 13:02:35.530895  3645 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.093s
I20260709 13:02:35.565568  3604 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.643757  3664 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:02:35.734290  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:35.641016  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:35.690435  3697 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:35.639348  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1255) took 1315 ms (client timeout 1689 ms). Trace:
W20260709 13:02:35.741353  1642 rpcz_store.cc:269] 0709 13:02:34.323813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.519831 (+1196018us) service_pool.cc:224] Handling call
0709 13:02:35.639260 (+119429us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:35.742406  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:35.708001  1565 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:35.608007  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:35.743212  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.749591  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.708073  2314 leader_election.cc:336] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.232s (SENT)
I20260709 13:02:35.701237  3688 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:35.703680  1646 rpcz_store.cc:269] 0709 13:02:34.322509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.557943 (+1235434us) service_pool.cc:224] Handling call
0709 13:02:35.703257 (+145314us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:35.753916  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:35.712754  1538 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:35.714757  1715 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:35.756129  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1547) took 1439 ms. Trace:
I20260709 13:02:35.756450  1715 rpcz_store.cc:276] 0709 13:02:34.316757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.609507 (+1292750us) service_pool.cc:224] Handling call
0709 13:02:35.756064 (+146557us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:35.722347  3649 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:35.716329  2312 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:35.734318  3650 raft_consensus.cc:3055] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:35.760162  3650 raft_consensus.cc:740] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:35.761853  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:35.762631  1715 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:35.742599  3339 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.560102  3615 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:35.778255  3339 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:35.780023  3615 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:35.701259  3683 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:35.725366  3462 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:35.709147  3660 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:35.726289  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:02:35.781565  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 69692.1us
W20260709 13:02:35.782428  1538 leader_election.cc:336] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.571s (SENT)
I20260709 13:02:35.737793  1570 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
W20260709 13:02:35.783107  1612 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:44424 dropped due to backpressure: service queue is full with 50 items
W20260709 13:02:35.783380  1538 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.646s (SENT)
I20260709 13:02:35.739310  1648 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:35.783923  1538 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:35.784832  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1256) took 1460 ms (client timeout 1800 ms). Trace:
I20260709 13:02:35.784893  3688 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:02:35.785303  1648 rpcz_store.cc:269] 0709 13:02:34.324046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.559603 (+1235557us) service_pool.cc:224] Handling call
0709 13:02:35.784761 (+225158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:35.739638  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:35.743716  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1559) took 2489 ms. Trace:
I20260709 13:02:35.750660  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:35.743276  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1835) took 1361 ms (client timeout 1744 ms). Trace:
W20260709 13:02:35.792737  1642 rpcz_store.cc:269] 0709 13:02:34.381931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.742143 (+1360212us) service_pool.cc:224] Handling call
0709 13:02:35.743163 (+  1020us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:35.794023  3660 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:35.798308  3462 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:35.788594  1566 rpcz_store.cc:276] 0709 13:02:33.252634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.271725 (+ 19091us) service_pool.cc:224] Handling call
0709 13:02:33.272117 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:34.487281 (+1215164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:34.487327 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:34.519414 (+ 32087us) log.cc:844] Serialized 26 byte log entry
0709 13:02:34.928254 (+408840us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:34.928308 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.928405 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:35.741528 (+813123us) raft_consensus.cc:1685] finished
0709 13:02:35.741551 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:35.741752 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":364991,"raft.queue_time_us":577995,"raft.run_cpu_time_us":9777,"raft.run_wall_time_us":493592,"thread_start_us":1529587,"threads_started":2,"wal-append.queue_time_us":830099}
I20260709 13:02:35.799233  3462 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:35.802850  1713 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:35.744026  3382 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:35.751567  1573 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:40642
I20260709 13:02:35.748981  1714 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:35.722164  3676 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:35.713446  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:35.744588  3567 consensus_meta.cc:287] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.348s	user 0.012s	sys 0.011s
I20260709 13:02:35.553035  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 284) took 2298 ms. Trace:
W20260709 13:02:35.764766  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 48560.5us
I20260709 13:02:35.765836  3614 consensus_queue.cc:237] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:35.766495  1646 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
W20260709 13:02:35.851182  1646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1830) took 1471 ms (client timeout 1936 ms). Trace:
W20260709 13:02:35.851325  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 138015us
W20260709 13:02:35.851596  1646 rpcz_store.cc:269] 0709 13:02:34.379995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.753602 (+1373607us) service_pool.cc:224] Handling call
0709 13:02:35.851122 (+ 97520us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:35.852063  1685 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:35.763161  3649 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:35.773941  3715 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:35.761763  3650 consensus_queue.cc:260] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:35.742456  3605 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:02:35.715487  3680 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:35.918607  3668 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:35.927222  3670 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:35.810564  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:35.928097  3668 raft_consensus.cc:697] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:35.830621  1537 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:35.826728  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 118661us
I20260709 13:02:35.918165  3382 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:35.929265  3668 consensus_queue.cc:237] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:35.815027  3694 log.cc:927] Time spent T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.462s	user 0.000s	sys 0.003s
W20260709 13:02:35.919610  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 67628.3us
I20260709 13:02:35.932552  1685 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:35.806224  3315 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:35.528924  3696 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:35.834591  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 51251.5us
I20260709 13:02:35.742929  3713 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:35.843238  1573 raft_consensus.cc:606] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Received request to transfer leadership
I20260709 13:02:35.843293  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 143) took 1019 ms. Trace:
I20260709 13:02:35.844450  1715 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:35.847157  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:35.853147  2312 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:35.783624  1612 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:44424 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1843)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1844)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1846)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1250)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1855)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1845)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1253)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1842)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1840)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1254)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1828)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1847)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1854)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1841)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1849)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1850)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1851)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1853)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1865)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1857)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1856)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1249)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1861)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1859)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 139)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1868)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1869)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1821)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1823)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1824)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1826)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1247)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1248)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1827)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1829)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1832)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1837)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1838)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1852)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1867)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 140)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 141)
I20260709 13:02:35.927441  3650 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:35.930039  3603 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:35.743783  3604 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:35.934695  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 104175us
W20260709 13:02:35.956200  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.956755  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:35.938674  1685 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:35.964198  1538 connection.cc:256] RPC call timeout handler was delayed by -1.4015110270000264s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1603s for a response
W20260709 13:02:35.869913  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.919209  3690 consensus_meta.cc:287] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.004s
I20260709 13:02:35.943567  1714 rpcz_store.cc:276] 0709 13:02:34.823999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.537112 (+713113us) service_pool.cc:224] Handling call
0709 13:02:35.843215 (+306103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:35.863770  3567 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:35.926607  3462 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:35.946756  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 93749.3us
I20260709 13:02:35.946949  3615 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:35.944234  1713 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:35.950897  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:35.979388  1570 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:02:35.979713  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:35.858362  3682 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:35.962857  1537 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:35.966197  1538 connection.cc:256] RPC call timeout handler was delayed by -1.3931315810000342s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1886s for a response
W20260709 13:02:35.949446  3681 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.897s	user 0.027s	sys 0.057s
I20260709 13:02:35.916682  3623 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:35.941991  3688 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:35.921586  3697 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:35.943071  3315 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:35.973016  1685 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:35.967834  3670 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:35.943424  3382 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:35.936895  3715 raft_consensus.cc:697] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:02:36.010550  1641 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1804) took 2657 ms (client timeout 2106 ms). Trace:
W20260709 13:02:36.011030  1641 rpcz_store.cc:269] 0709 13:02:33.354024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.354163 (+   139us) service_pool.cc:224] Handling call
0709 13:02:36.010311 (+2656148us) spinlock_profiling.cc:243] Waited 969 ms on lock 0x7b54001c0a54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:36.010471 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2227965696}
W20260709 13:02:35.984126  1538 connection.cc:256] RPC call timeout handler was delayed by -1.3805315810000707s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19s for a response
I20260709 13:02:35.985074  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:35.986944  1573 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:35.989375  1537 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:35.990020  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:35.998339  1612 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:44424 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1250)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1855)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1845)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1253)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1842)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1840)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1254)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1828)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1847)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1854)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1841)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1849)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1850)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1851)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1853)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1865)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1857)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1856)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1249)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1861)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 144)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1859)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 139)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 146)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1868)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1869)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1821)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1823)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1824)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1826)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1247)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1248)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1827)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1829)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1832)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1837)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1838)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1852)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1867)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 140)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 141)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 143)
I20260709 13:02:35.980334  1713 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.987628  3690 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:36.018957  1538 connection.cc:256] RPC call timeout handler was delayed by -1.349931581000078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19340000000000002s for a response
I20260709 13:02:36.019263  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1561) took 1661 ms. Trace:
I20260709 13:02:36.019683  1569 rpcz_store.cc:276] 0709 13:02:34.357700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.358804 (+  1104us) service_pool.cc:224] Handling call
0709 13:02:36.019049 (+1660245us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b54002b8c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:36.019209 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1244223744}
I20260709 13:02:36.020376  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:36.019382  1538 connection.cc:256] RPC call timeout handler was delayed by -1.3364315810000562s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19490000000000002s for a response
W20260709 13:02:36.021165  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1802) took 2690 ms (client timeout 1934 ms). Trace:
W20260709 13:02:36.021555  1643 rpcz_store.cc:269] 0709 13:02:33.330934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.331491 (+   557us) service_pool.cc:224] Handling call
0709 13:02:36.020998 (+2689507us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54001a9354. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:36.021101 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2305417472}
I20260709 13:02:36.022473  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:36.023016  1643 raft_consensus.cc:2410] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 13:02:36.023301  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1855) took 1161 ms. Trace:
I20260709 13:02:36.023476  1643 rpcz_store.cc:276] 0709 13:02:34.862062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.022292 (+1160230us) service_pool.cc:224] Handling call
0709 13:02:36.023264 (+   972us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:36.024178  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1809) took 2668 ms (client timeout 1529 ms). Trace:
W20260709 13:02:36.024437  1644 rpcz_store.cc:269] 0709 13:02:33.356242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.356432 (+   190us) service_pool.cc:224] Handling call
0709 13:02:36.024059 (+2667627us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54001f4854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:36.024129 (+    70us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3022841216}
I20260709 13:02:36.026701  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1813) took 2669 ms. Trace:
I20260709 13:02:36.026974  1640 rpcz_store.cc:276] 0709 13:02:33.357629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.321153 (+963524us) service_pool.cc:224] Handling call
0709 13:02:34.321592 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.265269 (+943677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.026545 (+761276us) spinlock_profiling.cc:243] Waited 502 ms on lock 0x7b54001f4354. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:36.026660 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1154344832}
I20260709 13:02:35.973254  3683 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:02:35.987270  3681 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.041s	user 0.029s	sys 0.057s
I20260709 13:02:36.005579  1645 raft_consensus.cc:2368] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6.
I20260709 13:02:36.008183  1685 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:36.003631  2312 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:36.002110  3615 raft_consensus.cc:697] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:35.996129  3720 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:36.070034  3720 raft_consensus.cc:697] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:36.070348  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:36.071216  1647 raft_consensus.cc:2393] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:36.071178  3720 consensus_queue.cc:237] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:36.058070  2343 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:36.032435  1714 raft_consensus.cc:2393] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:36.007647  3382 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:36.079857  3382 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:36.082350  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 204) took 1352 ms. Trace:
I20260709 13:02:36.082808  1567 rpcz_store.cc:276] 0709 13:02:34.730537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.740569 (+ 10032us) service_pool.cc:224] Handling call
0709 13:02:34.801450 (+ 60881us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.262864 (+461414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:35.262906 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:35.262964 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:35.263010 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.082056 (+819046us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b54001b5b54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:36.082113 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:36.082284 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1060451584}
I20260709 13:02:36.084731  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:36.085372  1716 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:36.087383  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1556) took 2854 ms. Trace:
I20260709 13:02:36.088083  1564 rpcz_store.cc:276] 0709 13:02:33.233396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.260936 (+ 27540us) service_pool.cc:224] Handling call
0709 13:02:33.342562 (+ 81626us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:35.454917 (+2112355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.086964 (+632047us) spinlock_profiling.cc:243] Waited 1.91 s on lock 0x7b54002242d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:36.087301 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4404351872}
I20260709 13:02:36.001768  3688 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:36.039068  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1539) took 2654 ms (client timeout 1846 ms). Trace:
I20260709 13:02:36.133773  3688 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:36.034031  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:36.135342  1722 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:02:36.038403  1641 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1250) took 1716 ms (client timeout 2099 ms). Trace:
W20260709 13:02:36.141849  1641 rpcz_store.cc:269] 0709 13:02:34.322266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.011863 (+1689597us) service_pool.cc:224] Handling call
0709 13:02:36.038317 (+ 26454us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.142949  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:36.037271  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.017125  3567 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:36.154309  3670 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:36.154722  3668 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.155261  3668 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:36.155546  3660 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:02:36.165215  3676 consensus_queue.cc:1059] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:36.047262  3661 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.033232  3695 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:36.048238  3462 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:36.048954  1645 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1807) took 2693 ms (client timeout 1542 ms). Trace:
W20260709 13:02:36.187703  1645 rpcz_store.cc:269] 0709 13:02:33.355151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.355335 (+   184us) service_pool.cc:224] Handling call
0709 13:02:36.048687 (+2693352us) spinlock_profiling.cc:243] Waited 2.64 s on lock 0x7b5400097c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:36.048880 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6083320192}
I20260709 13:02:36.189464  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:36.036533  3697 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:36.054205  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:35.980038  3713 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:36.055864  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:36.204857  1717 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:36.206274  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:36.207098  1717 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:36.058001  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.058525  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:36.060065  3339 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:36.060876  3607 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:36.061213  3696 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:36.062427  3615 consensus_queue.cc:237] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:36.065448  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 57343.8us
I20260709 13:02:36.216847  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:36.218341  1685 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:36.219427  1685 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:36.066987  3653 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:36.068708  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:36.071882  1647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1840) took 1510 ms (client timeout 1977 ms). Trace:
I20260709 13:02:36.078426  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:36.090157  1571 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:36.090656  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1837) took 3132 ms. Trace:
I20260709 13:02:36.091719  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:35.859724  2344 rpcz_store.cc:276] 0709 13:02:33.232513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.237952 (+  5439us) service_pool.cc:224] Handling call
0709 13:02:33.244520 (+  6568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.496440 (+2251920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.528686 (+ 32246us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:36.095438  3689 log.cc:927] Time spent T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 1.761s	user 0.009s	sys 0.019s
W20260709 13:02:36.100761  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.101913  3603 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:36.104707  3359 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:36.106140  1639 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1820) took 2636 ms (client timeout 1886 ms). Trace:
W20260709 13:02:36.314805  1639 rpcz_store.cc:269] 0709 13:02:33.469658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.470702 (+  1044us) service_pool.cc:224] Handling call
0709 13:02:36.105871 (+2635169us) spinlock_profiling.cc:243] Waited 419 ms on lock 0x7b54003566d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:36.106053 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":964833024}
I20260709 13:02:36.195742  3668 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:36.109093  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:36.051949  3715 consensus_queue.cc:237] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:36.110697  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:36.032687  3645 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.058099  3651 raft_consensus.cc:3055] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:36.325395  3651 raft_consensus.cc:740] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:36.121296  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:36.327521  1640 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1842) took 1589 ms (client timeout 1785 ms). Trace:
W20260709 13:02:36.327939  1640 rpcz_store.cc:269] 0709 13:02:34.738652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.027755 (+1289103us) service_pool.cc:224] Handling call
0709 13:02:36.327459 (+299704us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.328863  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:36.129799  3410 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:36.195353  3462 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:36.332671  3462 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:36.131286  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:36.334074  1721 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:36.216646  2340 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:36.131843  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:36.310945  3649 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:36.338311  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1845) took 1500 ms (client timeout 1646 ms). Trace:
W20260709 13:02:36.338696  1643 rpcz_store.cc:269] 0709 13:02:34.837489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.023853 (+1186364us) service_pool.cc:224] Handling call
0709 13:02:36.338247 (+314394us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.339651  3649 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:36.133780  1720 rpcz_store.cc:269] 0709 13:02:33.372096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.374178 (+  2082us) service_pool.cc:224] Handling call
0709 13:02:36.026252 (+2652074us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400212354. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:36.026332 (+    80us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2966402944}
I20260709 13:02:36.340590  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:36.137313  1719 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:36.341440  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:42276 (request call id 1841) took 3370 ms. Trace:
W20260709 13:02:36.130647  3684 log.cc:927] Time spent T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.772s	user 0.013s	sys 0.015s
I20260709 13:02:36.341804  1719 rpcz_store.cc:276] 0709 13:02:32.972013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.979353 (+  7340us) service_pool.cc:224] Handling call
0709 13:02:36.135953 (+3156600us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b5400212854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e7ca3d 00007f824de4ccd0 00007f82473d1254 00007f82473d11e2 00007f82473d1111 00007f82473d108c 00007f82473d0342 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:36.341354 (+205401us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":875286400}
I20260709 13:02:36.342379  3666 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:36.043327  1538 leader_election.cc:336] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.840s (SENT)
I20260709 13:02:36.343328  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:36.343777  1538 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:36.344069  1719 raft_consensus.cc:2393] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:02:36.344667  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 301423us
I20260709 13:02:36.344938  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:36.043313  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:36.198139  1713 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
W20260709 13:02:36.345597  1538 connection.cc:256] RPC call timeout handler was delayed by -1.1910892030000468s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1875s for a response
I20260709 13:02:36.345925  1644 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:36.346017  1538 connection.cc:256] RPC call timeout handler was delayed by -1.1454087910000226s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17620000000000002s for a response
I20260709 13:02:36.346181  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 138) took 2038 ms. Trace:
I20260709 13:02:36.068288  3604 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:36.347060  1713 rpcz_store.cc:276] 0709 13:02:34.307249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.979446 (+1672197us) service_pool.cc:224] Handling call
0709 13:02:36.345990 (+366544us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.348698  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:36.209271  1537 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:36.350083  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 140905us
W20260709 13:02:36.351601  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.354709  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:36.355547  3713 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:36.356353  3713 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:36.356443  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:36.347206  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:36.353482  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.357867  1648 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
W20260709 13:02:36.358579  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1843) took 1522 ms (client timeout 1529 ms). Trace:
W20260709 13:02:36.358925  1648 rpcz_store.cc:269] 0709 13:02:34.836624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.346947 (+1510323us) service_pool.cc:224] Handling call
0709 13:02:36.358517 (+ 11570us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.359367  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:36.359947  3659 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.175419  3614 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:36.362244  3614 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.362622  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:36.365839  3567 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.366585  3567 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:36.367736  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:36.369206  3567 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:36.196106  3697 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.227213  3676 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:36.422928  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:36.423651  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:36.424919  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1841) took 1750 ms (client timeout 2052 ms). Trace:
W20260709 13:02:36.425433  1648 rpcz_store.cc:269] 0709 13:02:34.674446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.406442 (+1731996us) service_pool.cc:224] Handling call
0709 13:02:36.424805 (+ 18363us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.413947  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:36.323450  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:36.320376  1717 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:02:36.326875  3651 consensus_queue.cc:260] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:36.339612  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:36.441666  1639 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1847) took 1603 ms (client timeout 1840 ms). Trace:
I20260709 13:02:36.371382  3680 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:36.372268  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:36.346375  1538 connection.cc:256] RPC call timeout handler was delayed by -1.1391164030000027s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1563s for a response
W20260709 13:02:36.442425  1639 rpcz_store.cc:269] 0709 13:02:34.838475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.315727 (+1477252us) service_pool.cc:224] Handling call
0709 13:02:36.441560 (+125833us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.174508  3687 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.196590  3661 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:36.357542  1713 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:36.380872  1646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1846) took 1543 ms (client timeout 1571 ms). Trace:
I20260709 13:02:36.365939  3462 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.384953  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:36.386112  1647 rpcz_store.cc:269] 0709 13:02:34.561132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.070009 (+1508877us) service_pool.cc:224] Handling call
0709 13:02:36.071813 (+  1804us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:36.294839  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.460394  1685 connection.cc:256] RPC call timeout handler was delayed by -1.9274506620000125s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2s for a response
W20260709 13:02:36.461886  1685 leader_election.cc:336] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.000s (SENT)
I20260709 13:02:36.462271  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:36.387339  1572 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:02:36.169878  1641 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:36.463788  1641 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1254) took 2140 ms (client timeout 2269 ms). Trace:
W20260709 13:02:36.464390  1641 rpcz_store.cc:269] 0709 13:02:34.323584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.142674 (+1819090us) service_pool.cc:224] Handling call
0709 13:02:36.463698 (+321024us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.329464  1640 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
W20260709 13:02:36.465792  1640 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1854) took 1625 ms (client timeout 1875 ms). Trace:
W20260709 13:02:36.466668  1640 rpcz_store.cc:269] 0709 13:02:34.840835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.328587 (+1487752us) service_pool.cc:224] Handling call
0709 13:02:36.465715 (+137128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.395705  1686 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:02:36.210219  1645 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1828) took 1831 ms (client timeout 2242 ms). Trace:
I20260709 13:02:36.468659  1537 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:36.464046  1639 raft_consensus.cc:2376] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2. Re-sending same reply.
W20260709 13:02:36.469135  1645 rpcz_store.cc:269] 0709 13:02:34.379234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.188962 (+1809728us) service_pool.cc:224] Handling call
0709 13:02:36.210155 (+ 21193us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:36.470445  1639 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1850) took 1630 ms (client timeout 1934 ms). Trace:
I20260709 13:02:36.472028  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:36.204644  3695 raft_consensus.cc:697] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:36.413235  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:36.506875  1570 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:36.507109  3695 consensus_queue.cc:237] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:36.508455  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:36.224426  3675 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:36.345917  3603 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:36.394961  2313 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:36.413362  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 141) took 2105 ms. Trace:
I20260709 13:02:36.360538  3659 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:36.435590  1564 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:36.324223  3645 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:36.362737  3614 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:36.442292  1643 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:36.443053  1538 connection.cc:256] RPC call timeout handler was delayed by -1.1005496190000486s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1936s for a response
W20260709 13:02:36.543030  1538 connection.cc:256] RPC call timeout handler was delayed by -1.0893892030000529s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1988s for a response
I20260709 13:02:36.357247  3713 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.174647  3670 raft_consensus.cc:697] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:36.436770  1565 raft_consensus.cc:2376] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5. Re-sending same reply.
I20260709 13:02:36.440047  1722 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
W20260709 13:02:36.448227  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.550905  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1848) took 1711 ms (client timeout 1886 ms). Trace:
I20260709 13:02:36.448077  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:36.449936  1644 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:36.553589  1537 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:36.554284  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1253) took 2231 ms (client timeout 2168 ms). Trace:
W20260709 13:02:36.554955  1644 rpcz_store.cc:269] 0709 13:02:34.323299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.024914 (+1701615us) service_pool.cc:224] Handling call
0709 13:02:36.554189 (+529275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.556486  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:36.557386  1644 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
W20260709 13:02:36.452653  1646 rpcz_store.cc:269] 0709 13:02:34.838110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.091405 (+1253295us) service_pool.cc:224] Handling call
0709 13:02:36.380792 (+289387us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.560101  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1864) took 1135 ms. Trace:
I20260709 13:02:36.560678  1644 rpcz_store.cc:276] 0709 13:02:35.424465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.555957 (+1131492us) service_pool.cc:224] Handling call
0709 13:02:36.559999 (+  4042us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.381898  1718 rpcz_store.cc:276] 0709 13:02:32.958152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.958329 (+   177us) service_pool.cc:224] Handling call
0709 13:02:32.958829 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.454691 (+2495862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.090324 (+635633us) spinlock_profiling.cc:243] Waited 2.31 s on lock 0x7b5400212854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:36.090593 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5307694976}
I20260709 13:02:36.453969  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:36.563472  1648 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:36.564136  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1849) took 1724 ms (client timeout 1900 ms). Trace:
I20260709 13:02:36.564921  1721 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:36.421599  3697 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:36.512145  3680 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:36.510146  3651 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:36.583951  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:36.584555  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:36.587567  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1866) took 1135 ms. Trace:
I20260709 13:02:36.588672  1644 rpcz_store.cc:276] 0709 13:02:35.452559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.561621 (+1109062us) service_pool.cc:224] Handling call
0709 13:02:36.587412 (+ 25791us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:36.564821  1648 rpcz_store.cc:269] 0709 13:02:34.839396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.434838 (+1595442us) service_pool.cc:224] Handling call
0709 13:02:36.564058 (+129220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.599458  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:36.552809  1643 rpcz_store.cc:269] 0709 13:02:34.839016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.339340 (+1500324us) service_pool.cc:224] Handling call
0709 13:02:36.550695 (+211355us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.530205  3591 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:36.462342  1685 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:36.223618  1642 raft_consensus.cc:2368] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6.
I20260709 13:02:36.611785  1537 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:36.612541  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1844) took 1775 ms (client timeout 1542 ms). Trace:
I20260709 13:02:36.461758  3315 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:36.612984  1642 rpcz_store.cc:269] 0709 13:02:34.837160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.068390 (+1231230us) service_pool.cc:224] Handling call
0709 13:02:36.612454 (+544064us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.614111  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:36.615243  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1249) took 2293 ms (client timeout 3046 ms). Trace:
I20260709 13:02:36.623066  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:36.465333  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:36.626736  1567 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:36.472117  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:36.628922  1641 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:36.471174  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:36.411512  1716 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:36.630740  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 1859) took 1018 ms. Trace:
I20260709 13:02:36.352583  1612 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:44424 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1841)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1849)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1850)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1851)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1853)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1865)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1857)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1856)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1249)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1861)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 144)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1859)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 149)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 139)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 146)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 151)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 147)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1868)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 148)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 155)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1869)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 158)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 150)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 152)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1821)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1823)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1824)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1826)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1247)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1248)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1827)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1829)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1832)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1837)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1838)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1852)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1867)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 140)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 141)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 143)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 153)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 154)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 156)
W20260709 13:02:36.445727  3689 log.cc:927] Time spent T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:02:36.656682  1722 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:02:36.668854  1641 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1853) took 1828 ms (client timeout 1988 ms). Trace:
W20260709 13:02:36.670650  1641 rpcz_store.cc:269] 0709 13:02:34.840504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.465303 (+1624799us) service_pool.cc:224] Handling call
0709 13:02:36.668744 (+203441us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.671314  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:36.554494  3653 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.672433  1644 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:36.629911  1645 raft_consensus.cc:2376] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4. Re-sending same reply.
W20260709 13:02:36.615738  1642 rpcz_store.cc:269] 0709 13:02:34.322026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.613786 (+2291760us) service_pool.cc:224] Handling call
0709 13:02:36.615152 (+  1366us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.673424  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1857) took 1451 ms. Trace:
I20260709 13:02:36.678224  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1862) took 1313 ms. Trace:
I20260709 13:02:36.678880  1644 rpcz_store.cc:276] 0709 13:02:35.367296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.589764 (+1222468us) service_pool.cc:224] Handling call
0709 13:02:36.678026 (+ 88262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.676767  1645 rpcz_store.cc:276] 0709 13:02:35.222290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.470834 (+1248544us) service_pool.cc:224] Handling call
0709 13:02:36.673351 (+202517us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:36.673465  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 211635us
W20260709 13:02:36.684125  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.480069  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 84456.8us
I20260709 13:02:36.685128  1685 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:36.686525  3359 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:36.481339  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:36.687935  1640 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1865) took 1243 ms (client timeout 1563 ms). Trace:
W20260709 13:02:36.688769  1640 rpcz_store.cc:269] 0709 13:02:35.444074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.467653 (+1023579us) service_pool.cc:224] Handling call
0709 13:02:36.687749 (+220096us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:36.471158  1639 rpcz_store.cc:269] 0709 13:02:34.839703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.443542 (+1603839us) service_pool.cc:224] Handling call
0709 13:02:36.470116 (+ 26574us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.486533  3676 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:36.492100  3382 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:36.460229  3687 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:36.509231  1570 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:36.704499  1648 raft_consensus.cc:2410] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:36.510619  1714 rpcz_store.cc:276] 0709 13:02:34.308094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.078112 (+1770018us) service_pool.cc:224] Handling call
0709 13:02:36.413275 (+335163us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:36.510800  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 116034us
W20260709 13:02:36.516990  1571 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1298) took 1706 ms (client timeout 2000 ms). Trace:
I20260709 13:02:36.584090  3603 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:36.520093  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:36.705150  1571 rpcz_store.cc:269] 0709 13:02:34.810897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.842146 (+ 31249us) service_pool.cc:224] Handling call
0709 13:02:36.516692 (+1674546us) spinlock_profiling.cc:243] Waited 801 ms on lock 0x7b54002baa54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:36.516900 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1842543360}
I20260709 13:02:36.672456  1564 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:36.706302  1647 raft_consensus.cc:2410] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:02:36.707005  1647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1851) took 1867 ms (client timeout 1949 ms). Trace:
I20260709 13:02:36.707180  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:36.707427  1647 rpcz_store.cc:269] 0709 13:02:34.839985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.459001 (+1619016us) service_pool.cc:224] Handling call
0709 13:02:36.706941 (+247940us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.708194  1639 raft_consensus.cc:2393] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:36.708501  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:36.709746  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.709950  1647 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:36.608143  3668 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:36.549167  1538 connection.cc:256] RPC call timeout handler was delayed by -1.0849128300000075s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18530000000000002s for a response
I20260709 13:02:36.450704  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:36.566771  1713 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:36.631759  1716 rpcz_store.cc:276] 0709 13:02:35.611982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.084438 (+472456us) service_pool.cc:224] Handling call
0709 13:02:36.630626 (+546188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.558414  3713 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:36.558873  3510 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:36.756798  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:36.758338  1571 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:36.468518  3462 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:36.683173  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:36.695917  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:36.697798  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:36.674170  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:36.708865  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 139) took 1035 ms. Trace:
I20260709 13:02:36.764998  1639 rpcz_store.cc:276] 0709 13:02:35.701877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.690319 (+988442us) service_pool.cc:224] Handling call
0709 13:02:36.708641 (+ 18322us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.533725  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:36.680673  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:36.767524  1715 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:36.478324  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:36.719230  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:36.768428  1641 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:36.769263  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1859) took 1501 ms. Trace:
I20260709 13:02:36.769400  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1858) took 1511 ms. Trace:
I20260709 13:02:36.769829  1644 rpcz_store.cc:276] 0709 13:02:35.267844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.680075 (+1412231us) service_pool.cc:224] Handling call
0709 13:02:36.769176 (+ 89101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.719652  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1856) took 1522 ms. Trace:
I20260709 13:02:36.563282  3688 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:36.749513  1537 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:36.747655  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:36.746809  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:36.672863  3683 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:36.712657  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1860) took 1381 ms. Trace:
I20260709 13:02:36.770145  1641 rpcz_store.cc:276] 0709 13:02:35.257876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.671850 (+1413974us) service_pool.cc:224] Handling call
0709 13:02:36.769325 (+ 97475us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.770613  1570 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:36.778576  1647 rpcz_store.cc:276] 0709 13:02:35.331310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.708204 (+1376894us) service_pool.cc:224] Handling call
0709 13:02:36.712586 (+  4382us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.779605  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:36.772172  1572 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:02:36.780140  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:36.781528  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:36.782852  1572 raft_consensus.cc:2393] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:36.771657  1648 rpcz_store.cc:276] 0709 13:02:35.196978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.598977 (+1401999us) service_pool.cc:224] Handling call
0709 13:02:36.719577 (+120600us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.783843  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:36.703467  3591 raft_consensus.cc:697] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:36.717787  3710 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:36.794298  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:36.798681  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:36.803747  1713 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:36.807267  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:36.808331  1564 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:36.808563  1722 raft_consensus.cc:1275] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:02:36.809551  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1549) took 2492 ms. Trace:
I20260709 13:02:36.810505  1722 rpcz_store.cc:276] 0709 13:02:34.317453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.792837 (+2475384us) service_pool.cc:224] Handling call
0709 13:02:36.796985 (+  4148us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:36.808953 (+ 11968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.809452 (+   499us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.813045  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:36.815148  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:36.815533  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:36.816047  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1869) took 1143 ms. Trace:
I20260709 13:02:36.816725  1568 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:36.702307  3650 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:02:36.820003  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.787065  1716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:02:36.787652  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:36.814090  1573 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:36.821698  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1868) took 1201 ms. Trace:
I20260709 13:02:36.822058  1641 rpcz_store.cc:276] 0709 13:02:35.620353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.778833 (+1158480us) service_pool.cc:224] Handling call
0709 13:02:36.821605 (+ 42772us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.703028  3676 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.789587  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:36.824098  1572 raft_consensus.cc:2393] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:36.774941  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:36.816396  1644 rpcz_store.cc:276] 0709 13:02:35.672672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.795918 (+1123246us) service_pool.cc:224] Handling call
0709 13:02:36.815987 (+ 20069us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.826903  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:36.827392  1641 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:36.817304  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:36.817806  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 136) took 3323 ms. Trace:
I20260709 13:02:36.764036  1646 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:36.760020  3603 raft_consensus.cc:697] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:02:36.818598  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 130957us
I20260709 13:02:36.822463  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:36.828522  1713 rpcz_store.cc:276] 0709 13:02:33.481591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.761908 (+3280317us) service_pool.cc:224] Handling call
0709 13:02:36.803442 (+ 41534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:36.804935 (+  1493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.805302 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.829717  1573 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:36.830381  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1861) took 1472 ms. Trace:
I20260709 13:02:36.830783  1646 rpcz_store.cc:276] 0709 13:02:35.358507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.655916 (+1297409us) service_pool.cc:224] Handling call
0709 13:02:36.830223 (+174307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.703193  3653 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:36.829767  1685 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:36.834926  1645 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:36.807621  2342 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55890 (request call id 206) took 3820 ms (client timeout 2022 ms). Trace:
I20260709 13:02:36.825031  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:36.763342  3668 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:36.815881  3683 raft_consensus.cc:697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:36.835525  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:36.898927  3462 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:37.271817  1573 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:36.968969  3713 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:37.202685  3729 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
I20260709 13:02:37.273154  1572 raft_consensus.cc:3055] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:37.273689  1570 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:02:37.213371  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.834954  3676 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:37.220100  3687 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:37.198840  3731 log.cc:927] Time spent T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
W20260709 13:02:37.207792  3410 consensus_meta.cc:287] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.878s	user 0.009s	sys 0.005s
W20260709 13:02:37.109418  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 279759us
W20260709 13:02:36.904971  1537 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.0.107.254:38281, user_credentials={real_user=slave}} blocked reactor thread for 50121.9us
I20260709 13:02:36.941375  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:37.279340  1569 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:37.279752  1537 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:37.009953  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:36.931754  1722 raft_consensus.cc:1240] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:36.816651  3670 consensus_queue.cc:237] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:37.282879  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 142) took 2601 ms. Trace:
I20260709 13:02:37.283272  1722 rpcz_store.cc:276] 0709 13:02:34.681092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.839991 (+2158899us) service_pool.cc:224] Handling call
0709 13:02:36.930995 (+ 91004us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.282558 (+351563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.282802 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:37.284264  1648 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 2.
I20260709 13:02:37.284871  1722 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:02:36.826582  3689 log.cc:927] Time spent T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:02:36.790786  3680 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:37.018823  3591 consensus_queue.cc:237] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:36.871176  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:36.899828  1644 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:37.289881  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1821) took 3688 ms. Trace:
I20260709 13:02:36.676841  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:37.199961  3730 log.cc:927] Time spent T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
W20260709 13:02:37.291698  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 144) took 1324 ms (client timeout 1537 ms). Trace:
I20260709 13:02:37.291965  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:02:37.292433  1642 rpcz_store.cc:269] 0709 13:02:35.967544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.675606 (+708062us) service_pool.cc:224] Handling call
0709 13:02:37.291622 (+616016us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.897940  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:37.273717  1572 raft_consensus.cc:740] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:37.263232  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:36.929570  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:37.296299  1564 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:36.870407  3691 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.296074  1572 consensus_queue.cc:260] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:37.296108  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 150) took 1120 ms. Trace:
I20260709 13:02:37.297140  3691 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:37.298067  1639 rpcz_store.cc:276] 0709 13:02:36.175464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.819316 (+643852us) service_pool.cc:224] Handling call
0709 13:02:37.296039 (+476723us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:37.299317  2343 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55890 (request call id 212) took 1192 ms. Trace:
W20260709 13:02:37.299530  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 210468us
I20260709 13:02:37.299762  2343 rpcz_store.cc:276] 0709 13:02:36.107462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.107673 (+   211us) service_pool.cc:224] Handling call
0709 13:02:36.108302 (+   629us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:36.108665 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:36.108699 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:36.110000 (+  1301us) log.cc:844] Serialized 564 byte log entry
0709 13:02:36.110160 (+   160us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:36.110201 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.110268 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:37.299083 (+1188815us) raft_consensus.cc:1685] finished
0709 13:02:37.299104 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.299256 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:37.275665  3687 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:36.871217  2342 rpcz_store.cc:269] 0709 13:02:32.968730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.977809 (+  9079us) service_pool.cc:224] Handling call
0709 13:02:36.769326 (+3791517us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b540065b3d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:36.788066 (+ 18740us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3444232960}
I20260709 13:02:37.301683  3687 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:37.305333  3687 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:37.307317  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:37.308131  3737 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.308928  3737 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:36.818768  3710 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:37.278550  1685 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:37.312963  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.313958  1685 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:37.297518  1572 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:02:37.273885  1713 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:37.315659  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1550) took 2998 ms. Trace:
I20260709 13:02:37.315958  1713 rpcz_store.cc:276] 0709 13:02:34.317949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.830102 (+2512153us) service_pool.cc:224] Handling call
0709 13:02:37.273377 (+443275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.315368 (+ 41991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.315606 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:37.329800  3670 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:37.330456  3670 raft_consensus.cc:697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:37.343331  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1850) took 1975 ms. Trace:
I20260709 13:02:37.344226  1572 raft_consensus.cc:2410] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:02:37.331771  3670 consensus_queue.cc:237] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:37.367579  1713 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:02:37.368644  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1855) took 2008 ms. Trace:
I20260709 13:02:37.318406  3695 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:37.290345  1644 rpcz_store.cc:276] 0709 13:02:33.601813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.826528 (+3224715us) service_pool.cc:224] Handling call
0709 13:02:36.899428 (+ 72900us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.289533 (+390105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.289816 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:37.281847  3683 consensus_queue.cc:237] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:37.321532  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:37.416735  1573 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:37.299367  3668 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:02:37.326167  3680 raft_consensus.cc:697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:37.272465  3734 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.142s
I20260709 13:02:37.327450  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 209) took 1473 ms. Trace:
I20260709 13:02:37.448227  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 158) took 1096 ms. Trace:
I20260709 13:02:37.450932  1566 rpcz_store.cc:276] 0709 13:02:35.854469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.858727 (+  4258us) service_pool.cc:224] Handling call
0709 13:02:35.859248 (+   521us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:35.933326 (+ 74078us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:36.318310 (+384984us) log.cc:844] Serialized 14 byte log entry
0709 13:02:36.721191 (+402881us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:36.777465 (+ 56274us) log.cc:844] Serialized 26 byte log entry
0709 13:02:36.777869 (+   404us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:36.777928 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.778227 (+   299us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:37.327165 (+548938us) raft_consensus.cc:1685] finished
0709 13:02:37.327190 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.327375 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":509298,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":74501,"raft.run_cpu_time_us":548,"raft.run_wall_time_us":26171,"thread_start_us":605199,"threads_started":2,"wal-append.queue_time_us":612706}
I20260709 13:02:37.328526  3687 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:37.343932  1722 rpcz_store.cc:276] 0709 13:02:35.310019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.284152 (+1974133us) service_pool.cc:224] Handling call
0709 13:02:37.284609 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:37.285324 (+   715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.285555 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:37.345532  1641 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:37.346921  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:37.463581  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1570) took 1019 ms. Trace:
I20260709 13:02:37.464043  1572 rpcz_store.cc:276] 0709 13:02:36.444057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.345872 (+901815us) service_pool.cc:224] Handling call
0709 13:02:37.463497 (+117625us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:37.340924  1538 leader_election.cc:336] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.563s (SENT)
I20260709 13:02:37.359227  1569 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
W20260709 13:02:37.316516  3651 consensus_meta.cc:287] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.677s	user 0.007s	sys 0.004s
W20260709 13:02:37.297466  3614 consensus_meta.cc:287] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.717s	user 0.011s	sys 0.000s
I20260709 13:02:37.502959  1718 raft_consensus.cc:2410] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:37.343466  3737 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:37.512992  1569 raft_consensus.cc:1240] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:37.513908  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1568) took 1094 ms. Trace:
I20260709 13:02:37.514307  1569 rpcz_store.cc:276] 0709 13:02:36.419932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.512416 (+1092484us) service_pool.cc:224] Handling call
0709 13:02:37.512767 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.513593 (+   826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.513845 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:37.516852  1572 raft_consensus.cc:1240] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:37.517673  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1567) took 1101 ms. Trace:
I20260709 13:02:37.518042  1572 rpcz_store.cc:276] 0709 13:02:36.416828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.516278 (+1099450us) service_pool.cc:224] Handling call
0709 13:02:37.516644 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.517367 (+   723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.517594 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:37.322045  3675 consensus_meta.cc:287] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.812s	user 0.014s	sys 0.000s
I20260709 13:02:37.300448  3691 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:37.326517  3742 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:02:37.347121  3724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.004s	sys 0.018s
I20260709 13:02:37.180033  3359 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:37.572151  1721 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:37.573163  1716 raft_consensus.cc:606] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:37.573652  1716 raft_consensus.cc:614] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:37.451757  1611 rpcz_store.cc:276] 0709 13:02:36.352195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.448148 (+1095953us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:02:37.580809  1611 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:44424 dropped due to backpressure: service queue is full with 50 items [suppressed 2 similar messages]
I20260709 13:02:37.369346  1713 rpcz_store.cc:276] 0709 13:02:35.360540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.329584 (+1969044us) service_pool.cc:224] Handling call
0709 13:02:37.367185 (+ 37601us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:37.368355 (+  1170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.368589 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:37.494006  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 214387us
W20260709 13:02:37.497888  2314 leader_election.cc:336] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.029s (SENT)
W20260709 13:02:37.582043  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 84226.3us
I20260709 13:02:36.900039  3382 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:36.941387  3661 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:37.502795  1722 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 7. Ops: []
I20260709 13:02:37.347136  3462 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.503129  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1823) took 3191 ms. Trace:
I20260709 13:02:37.599557  1641 rpcz_store.cc:276] 0709 13:02:34.312221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.343793 (+3031572us) service_pool.cc:224] Handling call
0709 13:02:37.345296 (+  1503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.502752 (+157456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.503043 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:37.339200  3688 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:37.054625  3645 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:37.501920  3614 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:37.511926  3739 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:37.529830  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.531828  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 190755us
W20260709 13:02:37.616580  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.540042  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 44222.6us
W20260709 13:02:37.553511  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.560124  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.568202  3724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.004s	sys 0.019s
W20260709 13:02:37.525882  3744 log.cc:927] Time spent T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 13:02:37.075519  3603 consensus_queue.cc:237] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:37.597797  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1856) took 2088 ms. Trace:
I20260709 13:02:37.622776  1722 rpcz_store.cc:276] 0709 13:02:35.510021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.501307 (+1991286us) service_pool.cc:224] Handling call
0709 13:02:37.502591 (+  1284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.596512 (+ 93921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.597725 (+  1213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:37.579577  3659 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:37.585366  3675 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:37.616659  1538 leader_election.cc:336] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.603s (SENT)
I20260709 13:02:37.618888  1685 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:37.620144  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 156171us
W20260709 13:02:37.598994  3697 consensus_meta.cc:287] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.987s	user 0.006s	sys 0.007s
I20260709 13:02:37.625609  1538 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:37.629614  1538 leader_election.cc:336] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.744s (SENT)
I20260709 13:02:37.626245  1685 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:37.627152  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.631322  2349 raft_consensus.cc:2368] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8.
W20260709 13:02:37.633973  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.643111  1720 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:37.645231  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:37.601255  3742 raft_consensus.cc:697] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:37.646404  1713 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:37.631371  3741 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.501951  3680 consensus_queue.cc:237] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:37.622121  3651 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.653584  1685 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:37.626497  3645 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.644680  3359 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.667263  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:37.663621  1685 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:37.648392  3688 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:37.683449  1685 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:37.647387  3749 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.650135  3733 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:37.613591  3462 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:37.624577  3715 consensus_queue.cc:1059] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:02:37.655375  3675 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:37.635502  3697 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:37.679774  3651 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:37.725278  3748 raft_consensus.cc:3055] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:37.725826  3748 raft_consensus.cc:740] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:37.621922  3739 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:37.727455  3748 consensus_queue.cc:260] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:37.728811  3739 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:37.708753  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 75173.3us
I20260709 13:02:37.686362  3537 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:37.743793  1538 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.886s (SENT)
I20260709 13:02:37.704983  3734 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:37.744014  1713 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
W20260709 13:02:37.746500  1538 leader_election.cc:336] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.900s (SENT)
I20260709 13:02:37.703660  3733 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.751097  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:37.753443  1722 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:37.703055  3732 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:37.749015  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:37.738824  3748 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:37.759814  1538 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.934s (SENT)
I20260709 13:02:37.760341  1538 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:37.747277  3734 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.701282  3359 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:37.764201  2314 leader_election.cc:336] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.805s (SENT)
I20260709 13:02:37.762501  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1857) took 2235 ms. Trace:
W20260709 13:02:37.756371  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.745245  3537 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:37.768339  1538 leader_election.cc:336] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.936s (SENT)
I20260709 13:02:37.685314  3645 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:37.765044  1713 rpcz_store.cc:276] 0709 13:02:35.527522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.748421 (+2220899us) service_pool.cc:224] Handling call
0709 13:02:37.761954 (+ 13533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.762158 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:37.762184 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:37.762224 (+    40us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:37.762261 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.762313 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.762445 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:37.769279  1538 leader_election.cc:336] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.949s (SENT)
I20260709 13:02:37.661921  3741 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:37.772193  1538 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:37.761458  3734 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:37.781215  1573 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:37.755502  3733 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:37.783908  1722 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:37.711714  3749 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:37.784749  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1572) took 1323 ms. Trace:
I20260709 13:02:37.785060  1569 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:02:37.765367  3688 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:37.788300  1573 rpcz_store.cc:276] 0709 13:02:36.461435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.321164 (+859729us) service_pool.cc:224] Handling call
0709 13:02:37.784685 (+463521us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:37.757459  3732 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.770907  3537 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:37.789853  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:37.801462  1573 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:37.795775  3651 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:37.827729  3687 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.828332  3687 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:37.820987  3755 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:02:37.833837  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.793463  3732 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:37.825757  3688 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:37.839382  1713 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:37.844825  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1858) took 2306 ms. Trace:
I20260709 13:02:37.846401  1713 rpcz_store.cc:276] 0709 13:02:35.538895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.787398 (+2248503us) service_pool.cc:224] Handling call
0709 13:02:37.837262 (+ 49864us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:37.844459 (+  7197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.844756 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:37.850442  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:37.818318  3758 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.772725  3742 consensus_queue.cc:237] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:37.845422  3676 consensus_meta.cc:287] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.569s	user 0.006s	sys 0.007s
I20260709 13:02:37.819411  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:02:37.863718  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 64717us
I20260709 13:02:37.819649  3614 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:37.852131  3758 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:37.903151  3359 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:37.904198  3758 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:37.904870  2347 raft_consensus.cc:1240] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:37.893124  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:37.896667  1537 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:37.852798  3755 consensus_queue.cc:1059] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:02:37.917865  1537 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:37.901768  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:37.927152  1566 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:37.904196  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:37.905174  2342 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:37.913921  3710 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:37.879751  3676 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:37.921594  3603 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:37.940910  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:37.945327  3713 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:37.956359  3603 raft_consensus.cc:697] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:37.962875  3749 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:37.964071  1717 raft_consensus.cc:2393] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:37.963164  3676 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:37.966826  2344 raft_consensus.cc:1240] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:37.967866  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:37.968677  1567 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
W20260709 13:02:37.970041  1570 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1574) took 1429 ms (client timeout 1788 ms). Trace:
W20260709 13:02:37.970549  1570 rpcz_store.cc:269] 0709 13:02:36.540891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.779312 (+238421us) service_pool.cc:224] Handling call
0709 13:02:37.969812 (+1190500us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b5400406fd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:37.969962 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1133465984}
I20260709 13:02:37.971846  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:37.972661  1570 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:37.974278  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1848) took 2692 ms. Trace:
I20260709 13:02:37.974967  3741 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:37.976333  3756 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:02:37.977200  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:37.981618  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:37.983101  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:37.980522  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:37.984422  1713 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 2.
I20260709 13:02:37.985896  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:37.987536  3757 log.cc:927] Time spent T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:02:37.987779  1713 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:02:37.981849  3603 consensus_queue.cc:237] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:37.991295  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:37.992153  1713 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:37.992201  3755 consensus_queue.cc:1059] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:37.992939  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:37.951576  3697 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:37.994385  3710 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:37.994711  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:37.995312  3710 raft_consensus.cc:697] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:37.995718  1570 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:37.930773  3645 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:38.003549  1685 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:38.004721  1685 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:37.939244  2341 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:37.922101  3688 consensus_queue.cc:1059] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:02:38.005612  1685 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:37.935364  3670 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.005927  3645 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:38.007442  3650 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:38.006090  3676 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:38.008420  3676 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.008562  3741 raft_consensus.cc:3055] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:38.007035  1567 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:02:38.007459  3645 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.008279  3650 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.007179  3670 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.009924  3645 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:38.010318  3650 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:38.012451  2312 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:38.009977  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:38.022971  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.974751  1721 rpcz_store.cc:276] 0709 13:02:35.282130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.018450 (+1736320us) service_pool.cc:224] Handling call
0709 13:02:37.018850 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.572769 (+553919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.974012 (+401243us) spinlock_profiling.cc:243] Waited 553 ms on lock 0x7b5400211954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:37.974211 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1271857408}
I20260709 13:02:38.025182  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:38.009244  3741 raft_consensus.cc:740] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:37.997076  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:38.027194  1570 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:38.030385  1685 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:37.994727  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:38.032095  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:38.033187  1565 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:38.034844  1713 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:38.009193  3676 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:37.993520  3755 raft_consensus.cc:3055] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:38.040699  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:38.038012  1685 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:37.997112  3710 consensus_queue.cc:237] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.042061  1721 raft_consensus.cc:2393] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:02:38.043344  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 274100us
W20260709 13:02:38.043998  1538 leader_election.cc:336] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.988s (SENT)
I20260709 13:02:38.044471  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 217) took 1333 ms. Trace:
I20260709 13:02:38.039469  3675 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:38.034278  3748 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:38.048152  1565 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:02:38.045341  1538 leader_election.cc:336] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.106s (SENT)
I20260709 13:02:38.049079  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 216) took 1389 ms. Trace:
I20260709 13:02:38.049304  1538 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:38.046177  1567 rpcz_store.cc:276] 0709 13:02:36.710938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.043655 (+1332717us) service_pool.cc:224] Handling call
0709 13:02:38.044001 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.044175 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:38.044198 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:38.044237 (+    39us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:38.044272 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.044321 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:38.044421 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:38.036796  3359 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:38.049764  1565 rpcz_store.cc:276] 0709 13:02:36.659891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.038067 (+1378176us) service_pool.cc:224] Handling call
0709 13:02:38.047773 (+  9706us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.048745 (+   972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.049007 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:38.050642  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.051842  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 218) took 1175 ms. Trace:
I20260709 13:02:38.056013  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:38.039708  3755 raft_consensus.cc:740] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:38.071848  1565 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:02:38.072916  1565 pending_rounds.cc:85] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 2
I20260709 13:02:38.057117  3651 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:38.073212  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:38.074040  3651 raft_consensus.cc:697] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:38.074834  1714 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:38.074045  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1861) took 2346 ms. Trace:
W20260709 13:02:38.058954  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.076712  3732 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.076220  1722 rpcz_store.cc:276] 0709 13:02:35.727566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.791149 (+2063583us) service_pool.cc:224] Handling call
0709 13:02:37.791470 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:37.791845 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:37.791872 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:37.803607 (+ 11735us) log.cc:844] Serialized 26 byte log entry
0709 13:02:38.032709 (+229102us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:38.032752 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.032823 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:38.073742 (+ 40919us) raft_consensus.cc:1685] finished
0709 13:02:38.073764 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:38.073958 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3734,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":6364,"thread_start_us":178796,"threads_started":1,"wal-append.queue_time_us":236428}
I20260709 13:02:38.078919  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:38.079574  1722 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:02:38.080720  1647 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:38.080085  3755 consensus_queue.cc:260] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.053232  3359 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.081920  3755 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:38.037340  3758 consensus_queue.cc:1059] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:02:38.084177  3650 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.066347  3734 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.114943  1639 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:02:38.115849  3734 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:38.067634  3687 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.116233  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1247) took 3794 ms. Trace:
I20260709 13:02:38.069887  3660 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:38.117898  1643 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 13:02:38.118758  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1826) took 3805 ms. Trace:
I20260709 13:02:38.119427  1643 rpcz_store.cc:276] 0709 13:02:34.313185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.117010 (+3803825us) service_pool.cc:224] Handling call
0709 13:02:38.117595 (+   585us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:38.118468 (+   873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.118699 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:38.048285  3748 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.056813  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1331) took 1610 ms. Trace:
I20260709 13:02:38.143754  3748 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:38.143884  1572 rpcz_store.cc:276] 0709 13:02:36.446964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.527350 (+1080386us) service_pool.cc:224] Handling call
0709 13:02:37.527748 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:37.600467 (+ 72719us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:37.600505 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:37.601109 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 13:02:37.750848 (+149739us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:37.854476 (+103628us) log.cc:844] Serialized 14 byte log entry
0709 13:02:37.854776 (+   300us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.855645 (+   869us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:38.056525 (+200880us) raft_consensus.cc:1685] finished
0709 13:02:38.056548 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:38.056732 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5697,"prepare.queue_time_us":147465,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":81,"raft.queue_time_us":27747,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":16144,"thread_start_us":253493,"threads_started":3,"wal-append.queue_time_us":186753}
I20260709 13:02:38.146445  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:38.147081  1572 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:38.082000  3359 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:38.088929  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1563) took 1705 ms. Trace:
I20260709 13:02:38.075333  3651 consensus_queue.cc:237] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.158840  3758 raft_consensus.cc:3055] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:38.116406  3676 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:38.159301  3758 raft_consensus.cc:740] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:38.092097  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55890 (request call id 213) took 1984 ms. Trace:
I20260709 13:02:38.094324  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1329) took 1666 ms. Trace:
I20260709 13:02:38.096427  1646 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:02:38.096937  1714 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:38.164399  1572 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
W20260709 13:02:38.098798  1715 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 1863) took 2369 ms (client timeout 1808 ms). Trace:
I20260709 13:02:38.162674  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1248) took 3841 ms. Trace:
I20260709 13:02:38.100140  1564 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:38.101068  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 1865) took 1991 ms. Trace:
I20260709 13:02:38.102181  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:38.098054  3733 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.061735  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:38.165599  1718 rpcz_store.cc:276] 0709 13:02:36.109867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.609447 (+499580us) service_pool.cc:224] Handling call
0709 13:02:38.100709 (+1491262us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b54002c6ad4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:38.100894 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1697051904}
I20260709 13:02:38.165380  1646 rpcz_store.cc:276] 0709 13:02:34.321779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.095473 (+3773694us) service_pool.cc:224] Handling call
0709 13:02:38.096163 (+   690us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:38.162388 (+ 66225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.162615 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:38.108538  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.160176  3758 consensus_queue.cc:260] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.096314  3732 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:38.055301  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:38.117084  1639 rpcz_store.cc:276] 0709 13:02:34.321417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.105425 (+3784008us) service_pool.cc:224] Handling call
0709 13:02:38.114636 (+  9211us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:38.115622 (+   986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.115912 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:38.055212  3670 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:38.081463  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1824) took 3768 ms. Trace:
I20260709 13:02:38.182062  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 161) took 1227 ms. Trace:
I20260709 13:02:38.182812  1642 rpcz_store.cc:276] 0709 13:02:36.954909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.101848 (+1146939us) service_pool.cc:224] Handling call
0709 13:02:38.181942 (+ 80094us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:38.097870  1640 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:38.121567  3687 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:38.123203  1722 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:02:38.191754  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:38.193162  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:38.198508  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:38.200089  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:02:38.125131  1719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 144) took 1933 ms (client timeout 1868 ms). Trace:
I20260709 13:02:38.127017  1565 pending_rounds.cc:107] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:02:38.200959  1686 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:38.130425  3315 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:38.201627  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:38.134899  1641 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: [2.2-3.3]
I20260709 13:02:38.202510  1642 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:38.138227  1685 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:38.203226  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1837) took 3820 ms. Trace:
I20260709 13:02:38.203078  3660 consensus_queue.cc:1059] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:02:38.203346  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 65321.1us
I20260709 13:02:38.204041  3615 raft_consensus.cc:3055] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:38.168476  3676 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:38.204519  3615 raft_consensus.cc:740] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:38.204821  3676 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:38.205216  3676 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.138732  3653 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:38.139554  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 94171.3us
I20260709 13:02:38.205472  3615 consensus_queue.cc:260] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:38.206162  1646 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
W20260709 13:02:38.206694  1538 leader_election.cc:336] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.186s (SENT)
I20260709 13:02:38.207312  1538 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:38.201002  1719 rpcz_store.cc:269] 0709 13:02:36.192201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.344700 (+152499us) service_pool.cc:224] Handling call
0709 13:02:38.124910 (+1780210us) spinlock_profiling.cc:243] Waited 919 ms on lock 0x7b5400247f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:38.125054 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2113129728}
W20260709 13:02:38.116534  1645 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1863) took 2720 ms (client timeout 2304 ms). Trace:
I20260709 13:02:38.208029  1565 raft_consensus.cc:2889] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:38.051388  3741 consensus_queue.cc:260] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.150350  1568 rpcz_store.cc:276] 0709 13:02:36.383437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.384617 (+  1180us) service_pool.cc:224] Handling call
0709 13:02:38.088651 (+1704034us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b54004060d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:38.088837 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":825102080}
I20260709 13:02:38.150817  1573 raft_consensus.cc:1275] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:38.158138  3739 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:38.162207  1569 rpcz_store.cc:276] 0709 13:02:36.428176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.526844 (+1098668us) service_pool.cc:224] Handling call
0709 13:02:37.784682 (+257838us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:37.787986 (+  3304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.093898 (+305912us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54001b6550. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:38.094063 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":592074112}
I20260709 13:02:38.104013  3645 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.032857  3537 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:38.165136  1715 rpcz_store.cc:269] 0709 13:02:35.729008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.945487 (+216479us) service_pool.cc:224] Handling call
0709 13:02:38.098592 (+2153105us) spinlock_profiling.cc:243] Waited 754 ms on lock 0x7b5400248954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:38.098723 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1733863680}
I20260709 13:02:38.226967  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:38.161984  2349 rpcz_store.cc:276] 0709 13:02:36.107977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.110443 (+  2466us) service_pool.cc:224] Handling call
0709 13:02:38.091794 (+1981351us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b5400712854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:38.091999 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3019457408}
I20260709 13:02:38.183638  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:38.184608  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1832) took 3803 ms. Trace:
I20260709 13:02:38.182546  1647 rpcz_store.cc:276] 0709 13:02:34.312557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.079992 (+3767435us) service_pool.cc:224] Handling call
0709 13:02:38.080499 (+   507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.081179 (+   680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.081402 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:38.229667  1647 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:38.161587  3661 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.174953  1537 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:38.180145  1717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 147) took 1966 ms (client timeout 2575 ms). Trace:
I20260709 13:02:38.184695  3767 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:38.233992  1717 rpcz_store.cc:269] 0709 13:02:36.213358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.439853 (+226495us) service_pool.cc:224] Handling call
0709 13:02:38.179546 (+1739693us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400211454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:38.180040 (+   494us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3243007744}
I20260709 13:02:38.236142  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:38.237301  1717 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:38.141258  3650 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:38.190675  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.055198  1567 rpcz_store.cc:276] 0709 13:02:36.876081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.050812 (+1174731us) service_pool.cc:224] Handling call
0709 13:02:38.051276 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.051475 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:38.051501 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:38.051546 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:38.051575 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.051642 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:38.051770 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:38.194082  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:38.142482  3339 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:38.194089  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:38.195451  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:38.196218  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:38.245668  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1867) took 2714 ms. Trace:
I20260709 13:02:38.195775  3758 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:38.198905  3733 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:38.205565  3676 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:38.140636  1570 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: []
W20260709 13:02:38.208482  1645 rpcz_store.cc:269] 0709 13:02:35.396414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.682671 (+1286257us) service_pool.cc:224] Handling call
0709 13:02:38.115816 (+1433145us) spinlock_profiling.cc:243] Waited 55 ms on lock 0x7b54001fe5d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:38.116429 (+   613us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126446464}
I20260709 13:02:38.260385  1645 raft_consensus.cc:1240] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:02:38.260825  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:38.261080  1564 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:38.203629  1642 rpcz_store.cc:276] 0709 13:02:34.382644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.183776 (+3801132us) service_pool.cc:224] Handling call
0709 13:02:38.184199 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.202974 (+ 18775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.203169 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:38.210670  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1829) took 3831 ms. Trace:
I20260709 13:02:38.127055  3510 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.221884  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:38.223897  1569 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:38.263219  1641 rpcz_store.cc:276] 0709 13:02:34.379619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.133968 (+3754349us) service_pool.cc:224] Handling call
0709 13:02:38.134533 (+   565us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:38.210346 (+ 75813us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.210587 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:38.266964  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 143) took 2316 ms. Trace:
W20260709 13:02:38.225746  1538 leader_election.cc:336] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.242s (SENT)
I20260709 13:02:38.268236  1642 rpcz_store.cc:276] 0709 13:02:35.951255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.265567 (+2314312us) service_pool.cc:224] Handling call
0709 13:02:38.266160 (+   593us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.266435 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:38.266492 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:38.266559 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:38.266609 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.266691 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:38.266878 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:38.228494  1640 rpcz_store.cc:276] 0709 13:02:34.380930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.097176 (+3716246us) service_pool.cc:224] Handling call
0709 13:02:38.097526 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.183981 (+ 86455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.184529 (+   548us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:38.231693  1643 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:38.227865  1572 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:38.271060  1640 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:38.272725  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57572 (request call id 10) took 2189 ms. Trace:
I20260709 13:02:38.273460  1716 rpcz_store.cc:276] 0709 13:02:36.062928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.748598 (+685670us) service_pool.cc:224] Handling call
0709 13:02:38.251861 (+1503263us) spinlock_profiling.cc:243] Waited 752 ms on lock 0x7b5400211954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e7f853 00007f824de4d60a 00007f82473d37d4 00007f82473d3762 00007f82473d3691 00007f82473d360c 00007f82473d28c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:38.252374 (+   513us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1729391488}
I20260709 13:02:38.202083  3687 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:38.275302  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 141) took 2511 ms. Trace:
W20260709 13:02:38.275776  1687 leader_election.cc:336] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.603s (SENT)
I20260709 13:02:38.276445  1687 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:38.232508  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:38.276985  1569 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:38.277416  1565 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:38.242828  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 67977.6us
W20260709 13:02:38.244932  1455 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 2.144s	user 0.034s	sys 0.043s
I20260709 13:02:38.278326  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 153) took 1927 ms. Trace:
I20260709 13:02:38.276140  1641 rpcz_store.cc:276] 0709 13:02:35.763780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.265102 (+2501322us) service_pool.cc:224] Handling call
0709 13:02:38.274030 (+  8928us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.274328 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:38.274374 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:38.274440 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:38.274683 (+   243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.274820 (+   137us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:38.275200 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:38.283905  1642 raft_consensus.cc:3055] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:38.284374  1642 raft_consensus.cc:740] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:38.246374  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:38.243945  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:02:38.247179  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 146) took 2036 ms (client timeout 1527 ms). Trace:
I20260709 13:02:37.581151  1611 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:44424 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1827)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1829)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1832)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1837)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1838)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1852)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1867)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 140)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 141)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 143)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 153)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 154)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 156)
I20260709 13:02:38.287005  1717 raft_consensus.cc:2393] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:02:38.287727  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:38.249385  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 148) took 2102 ms (client timeout 2232 ms). Trace:
I20260709 13:02:38.289228  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1852) took 3449 ms. Trace:
I20260709 13:02:38.223802  1644 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: []
W20260709 13:02:38.269235  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 43530.8us
I20260709 13:02:38.309638  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1827) took 3931 ms. Trace:
I20260709 13:02:38.227735  3766 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:38.280594  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 140) took 2538 ms. Trace:
I20260709 13:02:38.310088  1644 rpcz_store.cc:276] 0709 13:02:34.378309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.170487 (+3792178us) service_pool.cc:224] Handling call
0709 13:02:38.223571 (+ 53084us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.309353 (+ 85782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.309573 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:38.311384  1645 rpcz_store.cc:276] 0709 13:02:35.742692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.259820 (+2517128us) service_pool.cc:224] Handling call
0709 13:02:38.260153 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:38.280125 (+ 19972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.280530 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:38.312099  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:38.280588  3687 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.312511  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:38.313728  1644 raft_consensus.cc:2393] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:38.314411  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:38.308615  3733 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:02:38.289479  1648 rpcz_store.cc:269] 0709 13:02:36.147090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.783509 (+636419us) service_pool.cc:224] Handling call
0709 13:02:38.249190 (+1465681us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b5400183b54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:38.249317 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1131475200}
I20260709 13:02:38.319116  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:38.319995  1648 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:38.255299  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:38.291240  1641 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:38.286598  1720 rpcz_store.cc:269] 0709 13:02:36.211053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.340364 (+129311us) service_pool.cc:224] Handling call
0709 13:02:38.246979 (+1906615us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400212d54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:38.247104 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2981542784}
I20260709 13:02:38.322728  1572 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:38.323215  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:38.293294  1685 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:38.294405  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:38.325023  1717 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:38.326223  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:38.327664  1717 raft_consensus.cc:2410] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:38.292310  1573 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:02:38.295542  1612 connection.cc:256] RPC call timeout handler was delayed by -3.2285039430000211s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1592s for a response
I20260709 13:02:38.247184  3741 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:38.307664  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:38.249339  3315 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:38.246313  1647 rpcz_store.cc:276] 0709 13:02:35.531271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.228907 (+2697636us) service_pool.cc:224] Handling call
0709 13:02:38.229413 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.245412 (+ 15999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.245588 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29440}
I20260709 13:02:38.230078  1646 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:02:38.337919  1646 pending_rounds.cc:85] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 2
I20260709 13:02:38.338049  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:38.338330  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:38.338953  1714 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:38.339365  1647 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
W20260709 13:02:38.310002  1538 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.304s (SENT)
I20260709 13:02:38.282981  1640 rpcz_store.cc:276] 0709 13:02:36.350684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.270143 (+1919459us) service_pool.cc:224] Handling call
0709 13:02:38.270705 (+   562us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:38.277866 (+  7161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.278223 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:38.315529  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:38.340188  1538 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:38.308804  1643 rpcz_store.cc:276] 0709 13:02:34.840236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.212128 (+3371892us) service_pool.cc:224] Handling call
0709 13:02:38.231410 (+ 19282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.287238 (+ 55828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.289159 (+  1921us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:38.315979  1455 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 2.216s	user 0.035s	sys 0.043s
I20260709 13:02:38.227079  3739 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.314790  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:38.317853  1687 leader_election.cc:336] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.622s (SENT)
I20260709 13:02:38.343123  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:38.343680  1687 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:38.344367  1716 raft_consensus.cc:2393] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:38.340591  1645 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
W20260709 13:02:38.345491  1687 leader_election.cc:336] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.666s (SENT)
I20260709 13:02:38.322856  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 156) took 1971 ms. Trace:
I20260709 13:02:38.333556  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:38.337132  1719 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:38.346148  1687 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:38.347433  1687 leader_election.cc:336] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.689s (SENT)
I20260709 13:02:38.347885  1720 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: []
W20260709 13:02:38.341533  1455 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 2.24178e+06us
I20260709 13:02:38.317051  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:38.316205  1642 consensus_queue.cc:260] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:38.351003  1642 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:38.245129  3661 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.338304  1646 pending_rounds.cc:107] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:02:38.329978  1565 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:38.360074  1646 raft_consensus.cc:2889] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:38.347961  1687 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:38.248006  3767 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.349049  1641 rpcz_store.cc:276] 0709 13:02:36.351697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.283661 (+1931964us) service_pool.cc:224] Handling call
0709 13:02:38.290911 (+  7250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.322559 (+ 31648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.322796 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:38.363106  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:38.306555  3737 consensus_queue.cc:1059] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:02:38.353029  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1868) took 2242 ms. Trace:
I20260709 13:02:38.313285  3687 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:38.371601  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1838) took 3957 ms. Trace:
I20260709 13:02:38.364144  1644 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:38.372303  1646 rpcz_store.cc:276] 0709 13:02:34.414058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.207192 (+3793134us) service_pool.cc:224] Handling call
0709 13:02:38.229529 (+ 22337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.371154 (+141625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.371387 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12160}
W20260709 13:02:38.329113  1612 connection.cc:256] RPC call timeout handler was delayed by -3.0926039430000287s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1743s for a response
I20260709 13:02:38.294121  3510 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:38.378628  1716 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:38.379417  1648 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
W20260709 13:02:38.380276  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 63365.9us
I20260709 13:02:38.364414  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:38.381353  1537 leader_election.cc:400] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8.
I20260709 13:02:38.381748  1537 leader_election.cc:403] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:02:38.373864  1612 connection.cc:256] RPC call timeout handler was delayed by -3.0791039430000069s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1758s for a response
I20260709 13:02:38.371080  1720 rpcz_store.cc:276] 0709 13:02:36.110672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.347228 (+2236556us) service_pool.cc:224] Handling call
0709 13:02:38.347628 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.352634 (+  5006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.352945 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:38.375761  1538 connection.cc:256] RPC call timeout handler was delayed by -1.3506125929999939s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2743s for a response
I20260709 13:02:38.379428  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1866) took 2269 ms. Trace:
I20260709 13:02:38.383354  1716 rpcz_store.cc:276] 0709 13:02:36.110139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.345266 (+2235127us) service_pool.cc:224] Handling call
0709 13:02:38.378435 (+ 33169us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.379153 (+   718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.379363 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:38.383064  1538 leader_election.cc:336] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.661s (SENT)
I20260709 13:02:38.385555  1538 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:38.386757  1719 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:38.382691  1642 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:02:38.389812  1642 pending_rounds.cc:85] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 5
I20260709 13:02:38.390275  1642 pending_rounds.cc:107] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:02:38.390755  1642 raft_consensus.cc:2889] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:38.382153  1612 connection.cc:256] RPC call timeout handler was delayed by -3.0431039430000055s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17980000000000002s for a response
W20260709 13:02:38.391829  1612 connection.cc:256] RPC call timeout handler was delayed by -3.0026039429999969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18430000000000002s for a response
I20260709 13:02:38.391881  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 154) took 2040 ms. Trace:
W20260709 13:02:38.392777  1612 connection.cc:256] RPC call timeout handler was delayed by -2.9999039429999925s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18460000000000001s for a response
I20260709 13:02:38.399026  3739 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:38.344470  3720 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:38.343799  3650 consensus_queue.cc:1059] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 13:02:38.349838  3649 consensus_queue.cc:1059] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:02:38.344131  3651 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 13:02:38.349464  3734 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:02:38.392949  1642 rpcz_store.cc:276] 0709 13:02:36.350967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.283424 (+1932457us) service_pool.cc:224] Handling call
0709 13:02:38.283751 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.391570 (+107819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.391813 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:38.351089  3733 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:38.530900  3733 raft_consensus.cc:3055] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:38.531497  3733 raft_consensus.cc:740] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:38.532461  3733 consensus_queue.cc:260] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.471829  1564 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:38.471103  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:38.462530  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:38.389501  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1869) took 2278 ms. Trace:
I20260709 13:02:38.534422  1648 raft_consensus.cc:2376] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4. Re-sending same reply.
I20260709 13:02:38.534644  1719 rpcz_store.cc:276] 0709 13:02:36.110938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.350386 (+2239448us) service_pool.cc:224] Handling call
0709 13:02:38.385559 (+ 35173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.388930 (+  3371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.389392 (+   462us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:38.349628  3382 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:38.537551  1719 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:38.540043  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 151) took 1366 ms. Trace:
I20260709 13:02:38.540647  1719 rpcz_store.cc:276] 0709 13:02:37.173437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.536524 (+1363087us) service_pool.cc:224] Handling call
0709 13:02:38.537000 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.539538 (+  2538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.539793 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:38.417382  1720 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:38.542094  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 150) took 1830 ms. Trace:
W20260709 13:02:38.481578  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 77537.3us
I20260709 13:02:38.542443  1720 rpcz_store.cc:276] 0709 13:02:36.712055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.415717 (+1703662us) service_pool.cc:224] Handling call
0709 13:02:38.416709 (+   992us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.541824 (+125115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.542043 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:38.544317  3339 consensus_queue.cc:1059] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:38.544979  1720 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:38.545949  3603 raft_consensus.cc:3055] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:38.546432  3603 raft_consensus.cc:740] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:38.479974  3462 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:38.370158  3315 raft_consensus.cc:697] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:38.547482  1720 raft_consensus.cc:1275] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:38.427760  3359 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.547309  3603 consensus_queue.cc:260] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.547580  3315 consensus_queue.cc:237] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.548653  3603 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:38.542676  1719 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:38.504472  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:38.549484  3359 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:38.550047  1644 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:38.550544  1719 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:02:38.551412  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:38.551853  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:38.552157  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.437877  1722 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:38.511714  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.524976  3649 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.554337  3649 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:38.556059  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:38.556677  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.556775  3649 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:38.525784  3650 raft_consensus.cc:3055] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:38.557458  3650 raft_consensus.cc:740] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:38.558755  3650 consensus_queue.cc:260] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:38.553663  1685 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:38.561960  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 214494us
I20260709 13:02:38.418970  3697 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:02:38.438723  3688 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:38.417659  3746 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:38.577416  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:38.436937  3748 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:38.399014  3661 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:38.486821  1612 connection.cc:256] RPC call timeout handler was delayed by -2.7623039430000063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.211s for a response
I20260709 13:02:38.545198  3339 consensus_queue.cc:1059] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:38.545831  3604 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:38.358551  3732 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:38.557235  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:38.490360  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:38.562633  1687 leader_election.cc:336] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.800s (SENT)
I20260709 13:02:38.599408  1687 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:38.546633  3645 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:38.600688  1687 leader_election.cc:336] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.800s (SENT)
I20260709 13:02:38.601229  1687 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:38.491575  3537 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:38.562543  1685 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:38.584867  3760 raft_consensus.cc:3055] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:38.547425  3767 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:38.588114  1612 connection.cc:256] RPC call timeout handler was delayed by -2.757803942999999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21150000000000002s for a response
W20260709 13:02:38.608664  1612 connection.cc:256] RPC call timeout handler was delayed by -2.7452039430000355s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2129s for a response
W20260709 13:02:38.609042  1612 connection.cc:256] RPC call timeout handler was delayed by -2.5067039429999909s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23940000000000003s for a response
W20260709 13:02:38.609339  1612 connection.cc:256] RPC call timeout handler was delayed by -2.0711039430000255s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2878s for a response
I20260709 13:02:38.560743  3650 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:38.591238  3649 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:38.601545  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:38.590080  3733 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:38.599078  1538 connection.cc:256] RPC call timeout handler was delayed by -1.0268487849999701s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
I20260709 13:02:38.617929  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:38.618713  1648 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:38.585882  3382 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:38.562762  3510 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:38.607537  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:38.632526  1573 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:38.609606  1612 connection.cc:256] RPC call timeout handler was delayed by -1.8182039430000145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3159s for a response
W20260709 13:02:38.638005  1612 connection.cc:256] RPC call timeout handler was delayed by -1.7372039429999973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3249s for a response
I20260709 13:02:38.640662  1612 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:38.614002  1644 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:38.600416  1722 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
W20260709 13:02:38.612888  1687 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.099s (SENT)
I20260709 13:02:38.646557  1687 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:38.616693  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 54221.2us
I20260709 13:02:38.606595  3746 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:38.649359  3462 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:38.652220  3462 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:38.652758  3462 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.631657  3537 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:38.632076  3339 consensus_queue.cc:1059] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:02:38.627449  3766 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:38.627830  3732 raft_consensus.cc:697] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:38.664080  3732 consensus_queue.cc:237] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.609935  3359 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:38.608314  3760 raft_consensus.cc:740] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:38.670481  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:38.677038  1648 raft_consensus.cc:2376] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6. Re-sending same reply.
I20260709 13:02:38.677251  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:38.678406  1642 raft_consensus.cc:2376] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5. Re-sending same reply.
I20260709 13:02:38.673492  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:38.653182  3462 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:38.670575  1538 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:38.684778  1612 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:38.686987  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:38.680747  2340 raft_consensus.cc:2393] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 6: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:38.702910  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:38.687696  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:38.705037  1648 raft_consensus.cc:2376] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6. Re-sending same reply.
I20260709 13:02:38.704383  1642 raft_consensus.cc:2376] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5. Re-sending same reply.
I20260709 13:02:38.705282  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:38.705878  3767 consensus_queue.cc:1048] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:38.708441  1612 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:38.702090  1644 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
W20260709 13:02:38.700124  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 87276.5us
I20260709 13:02:38.696775  1573 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
W20260709 13:02:38.710333  1687 leader_election.cc:336] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.168s (SENT)
I20260709 13:02:38.703586  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:38.712141  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:38.710774  1687 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:38.713171  1573 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:38.711401  1641 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:38.628332  3745 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:02:38.716825  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:38.717132  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:38.718724  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:38.665212  3748 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:38.703915  3661 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:38.708212  3651 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.703341  3537 raft_consensus.cc:697] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:38.723418  3770 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:02:38.714776  3649 raft_consensus.cc:697] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:38.725172  3649 consensus_queue.cc:237] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.714696  3739 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:38.704037  3746 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:38.727975  1646 raft_consensus.cc:2368] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7.
I20260709 13:02:38.728943  1645 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:38.729058  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.730372  3774 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:38.723533  3651 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.726990  3687 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:38.751183  3676 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.756021  3651 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:38.757599  1713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:02:38.762645  3676 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
W20260709 13:02:38.767251  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.763435  3688 raft_consensus.cc:3055] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:38.742949  3784 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:02:38.790463  3688 raft_consensus.cc:740] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:38.760600  1715 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:02:38.757620  1721 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:38.794164  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1864) took 2987 ms. Trace:
I20260709 13:02:38.762645  1713 raft_consensus.cc:606] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:38.796383  1713 raft_consensus.cc:614] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:38.794979  1714 rpcz_store.cc:276] 0709 13:02:35.807609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.339528 (+2531919us) service_pool.cc:224] Handling call
0709 13:02:38.339806 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:38.458322 (+118516us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:38.599927 (+141605us) log.cc:844] Serialized 14 byte log entry
0709 13:02:38.713138 (+113211us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:38.729050 (+ 15912us) log.cc:844] Serialized 26 byte log entry
0709 13:02:38.729264 (+   214us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:38.729318 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.729419 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:38.793780 (+ 64361us) raft_consensus.cc:1685] finished
0709 13:02:38.793815 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:38.794066 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":235855,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":72949,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":171,"thread_start_us":288304,"threads_started":3,"wal-append.queue_time_us":165565}
I20260709 13:02:38.771282  3615 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:38.807111  3771 log.cc:927] Time spent T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.430s	user 0.004s	sys 0.000s
W20260709 13:02:38.776976  3775 log.cc:927] Time spent T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:02:38.778720  3670 raft_consensus.cc:3055] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:38.809358  3670 raft_consensus.cc:740] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:38.775497  3687 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:38.741312  3760 consensus_queue.cc:260] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.811000  3760 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:38.810624  3670 consensus_queue.cc:260] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.736959  3537 consensus_queue.cc:237] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:38.812463  3670 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:38.782810  3758 consensus_meta.cc:287] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.521s	user 0.013s	sys 0.000s
I20260709 13:02:38.816984  1720 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:02:38.817441  1720 pending_rounds.cc:85] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 6
I20260709 13:02:38.736014  3745 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:38.817688  1720 pending_rounds.cc:107] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:02:38.818059  1720 raft_consensus.cc:2889] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:38.818042  3745 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:38.792716  3688 consensus_queue.cc:260] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:38.828271  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.828989  3688 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:38.829553  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.835624  3697 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8.
I20260709 13:02:38.839270  3745 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:38.960425  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:38.965900  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 249747us
I20260709 13:02:38.971104  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1344) took 1176 ms. Trace:
W20260709 13:02:38.975337  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.968811  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:38.981526  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:38.981608  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:38.982595  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:38.982947  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:38.984685  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:38.988610  1612 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:38.988970  1647 raft_consensus.cc:1275] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:02:39.044631  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.992637  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:38.995836  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 325024us
I20260709 13:02:38.972224  1571 rpcz_store.cc:276] 0709 13:02:37.794514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.114736 (+320222us) service_pool.cc:224] Handling call
0709 13:02:38.115115 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:38.174392 (+ 59277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:38.174430 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:38.211218 (+ 36788us) log.cc:844] Serialized 26 byte log entry
0709 13:02:38.452062 (+240844us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:38.452122 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.452218 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:38.969975 (+517757us) raft_consensus.cc:1685] finished
0709 13:02:38.970032 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:38.970626 (+   594us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70738,"raft.queue_time_us":58995,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":9137,"thread_start_us":192400,"threads_started":1,"wal-append.queue_time_us":371172}
I20260709 13:02:39.008909  3537 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:39.011164  1687 leader_election.cc:336] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.269s (SENT)
I20260709 13:02:39.013509  3659 consensus_queue.cc:1048] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:39.015390  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:39.015640  1721 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:38.945122  1471 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47518 (request call id 93) took 7238 ms. Trace:
I20260709 13:02:39.021569  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1565) took 2637 ms. Trace:
I20260709 13:02:39.022408  3710 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:39.030211  3603 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:39.033084  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1577) took 2462 ms. Trace:
I20260709 13:02:39.048035  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 1834) took 4666 ms. Trace:
I20260709 13:02:39.054273  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:39.060976  1639 rpcz_store.cc:276] 0709 13:02:34.381578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.175181 (+3793603us) service_pool.cc:224] Handling call
0709 13:02:38.194408 (+ 19227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.385602 (+191194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:38.385656 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:38.385702 (+    46us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:38.385729 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.047354 (+661625us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b5400183154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:39.047436 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.047773 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":439213824}
I20260709 13:02:39.053826  1566 rpcz_store.cc:276] 0709 13:02:36.384163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.519445 (+1135282us) service_pool.cc:224] Handling call
0709 13:02:37.519769 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.928862 (+409093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.020799 (+1091937us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b54002ba554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:39.021296 (+   497us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":936381312}
I20260709 13:02:39.059959  1570 rpcz_store.cc:276] 0709 13:02:36.570738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.034912 (+1464174us) service_pool.cc:224] Handling call
0709 13:02:38.043031 (+  8119us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.259275 (+216244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.032097 (+772822us) spinlock_profiling.cc:243] Waited 97.4 ms on lock 0x7b54002ba054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:39.032780 (+   683us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223963520}
I20260709 13:02:39.078717  1640 raft_consensus.cc:2410] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:39.081221  1715 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:39.085968  1567 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:39.087457  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 76615.5us
I20260709 13:02:39.090466  1648 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:39.097993  1641 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
I20260709 13:02:39.100215  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:39.100935  1685 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:39.106251  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:39.036871  3660 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:02:39.109576  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:39.110009  1564 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:39.111539  3537 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:39.113101  3785 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.122985  3785 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:39.124042  3697 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.124933  3697 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:39.118983  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:39.134141  1641 raft_consensus.cc:2393] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:39.120507  3767 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:39.116314  1642 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:39.130120  3739 consensus_queue.cc:1059] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:39.113445  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:39.072898  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 457450us
I20260709 13:02:39.119714  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:39.138685  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 178122us
I20260709 13:02:39.116506  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:39.079048  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:39.135612  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:39.113688  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:39.133601  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:39.163645  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:39.164824  1645 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:39.043939  3743 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.124s
I20260709 13:02:39.186200  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:39.187785  1571 raft_consensus.cc:2368] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7.
I20260709 13:02:39.188169  3537 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:39.189112  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:39.192193  1566 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:39.205878  1571 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:40642
I20260709 13:02:39.207063  1571 raft_consensus.cc:606] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Received request to transfer leadership
I20260709 13:02:39.207166  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:39.208122  1648 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:39.210333  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:39.194844  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:39.210768  3603 consensus_queue.cc:1048] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:39.212451  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:39.193905  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:39.197897  3650 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:02:39.195363  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 94918.9us
I20260709 13:02:39.213917  3650 raft_consensus.cc:697] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:39.215277  1648 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:39.208724  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:39.219527  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:39.210209  3710 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:39.220685  1639 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:39.213195  1647 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:39.183185  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:39.210410  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 222455us
I20260709 13:02:39.276835  1566 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:39.262658  1646 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:39.207525  1571 raft_consensus.cc:614] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:39.261636  1567 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:39.318089  1564 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:39.238201  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:39.318711  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 93148.9us
I20260709 13:02:39.184404  1640 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:39.281414  1720 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:39.221372  3785 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:39.281601  1685 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:39.328310  1469 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38650 (request call id 195) took 19689 ms (client timeout 15000 ms). Trace:
I20260709 13:02:39.331712  1721 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:39.330117  1648 raft_consensus.cc:2410] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:39.342597  1572 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:02:39.344228  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.347329  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:39.347790  3697 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:39.330485  1469 rpcz_store.cc:269] 0709 13:02:19.622750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.623013 (+   263us) service_pool.cc:224] Handling call
0709 13:02:19.885898 (+262885us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b54002b0054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248def090 00007f8248e8df3a 00007f824ec441cc 00007f824ed4237e 00007f824d2032a4 00007f824d203232 00007f824d203161 00007f824d2030dc 00007f824d202392 00007f82471c048c 00007f82471bf8c6
0709 13:02:39.311687 (+19425789us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":863770,"reported_tablets":12,"spinlock_wait_cycles":596331904}
I20260709 13:02:39.349305  2312 leader_election.cc:400] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7.
I20260709 13:02:39.351886  3537 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.378377  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:39.376689  2312 leader_election.cc:403] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:02:39.357543  3615 consensus_meta.cc:287] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.553s	user 0.008s	sys 0.016s
I20260709 13:02:39.364686  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:39.368189  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.370936  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.376499  3710 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:39.377755  3537 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:39.376468  2341 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:39.391970  3785 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Advancing to term 7
W20260709 13:02:39.376397  3462 consensus_meta.cc:287] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.671s	user 0.011s	sys 0.011s
I20260709 13:02:39.284507  3680 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.113s
I20260709 13:02:39.235723  3795 consensus_queue.cc:1059] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.169s
I20260709 13:02:39.411963  1642 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:39.411290  1641 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:39.430151  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:39.432934  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:39.216144  3650 consensus_queue.cc:237] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:39.054323  1471 rpcz_store.cc:276] 0709 13:02:31.706479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.706968 (+   489us) service_pool.cc:224] Handling call
0709 13:02:32.222935 (+515967us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54002b0054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248def090 00007f8248e8df3a 00007f824ec441cc 00007f824ed4237e 00007f824d2032a4 00007f824d203232 00007f824d203161 00007f824d2030dc 00007f824d202392 00007f82471c048c 00007f82471bf8c6
0709 13:02:38.944685 (+6721750us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:35.801898 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:35.804609 (+  2711us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:35.804658 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:02:35.804680 (+    22us) tablet.cc:662] Decoding operations
0709 13:02:35.931324 (+126644us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:35.931415 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:35.931453 (+    38us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:02:35.932162 (+   709us) tablet.cc:701] Row locks acquired
0709 13:02:35.932185 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:02:35.932369 (+   184us) write_op.cc:270] Start()
0709 13:02:35.932493 (+   124us) write_op.cc:276] Timestamp: P: 1783602155932317 usec, L: 0
0709 13:02:35.932517 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:02:35.984251 (+ 51734us) log.cc:844] Serialized 3281 byte log entry
0709 13:02:37.551857 (+1567606us) op_driver.cc:464] REPLICATION: finished
0709 13:02:37.675324 (+123467us) write_op.cc:301] APPLY: starting
0709 13:02:37.675507 (+   183us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:37.675717 (+   210us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:37.675865 (+   148us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:37.722122 (+ 46257us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:37.722512 (+   390us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:02:37.722569 (+    57us) write_op.cc:312] APPLY: finished
0709 13:02:37.722912 (+   343us) log.cc:844] Serialized 65 byte log entry
0709 13:02:37.723134 (+   222us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:37.723269 (+   135us) write_op.cc:454] Released schema lock
0709 13:02:37.723328 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:37.723779 (+   451us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:37.723808 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":270059,"reported_tablets":13,"spinlock_wait_cycles":612062464,"child_traces":[["op",{"apply.queue_time_us":122586,"apply.run_cpu_time_us":4924,"apply.run_wall_time_us":48732,"num_ops":8,"prepare.queue_time_us":276449,"prepare.run_cpu_time_us":172357,"prepare.run_wall_time_us":761118,"raft.queue_time_us":712524,"raft.run_cpu_time_us":9960,"raft.run_wall_time_us":84589,"replication_time_us":1618938,"thread_start_us":1548807,"threads_started":4,"wal-append.queue_time_us":803830,"wal-append.run_cpu_time_us":26599,"wal-append.run_wall_time_us":1915167}]]}
I20260709 13:02:39.438967  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:39.447746  1648 raft_consensus.cc:2410] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:39.444954  1639 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 2.
W20260709 13:02:39.473587  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 267449us
I20260709 13:02:39.410836  3462 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:39.493834  1572 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:39.515364  1645 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:39.517596  1643 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 6: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:39.518590  1641 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:39.523005  1648 raft_consensus.cc:3055] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:39.524026  1648 raft_consensus.cc:740] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:02:39.528589  3688 consensus_meta.cc:287] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.692s	user 0.007s	sys 0.014s
I20260709 13:02:39.530596  1639 raft_consensus.cc:1275] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:39.512590  3739 raft_consensus.cc:3055] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:39.532402  3739 raft_consensus.cc:740] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:39.508973  3795 consensus_queue.cc:1059] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:02:39.536468  3537 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:39.539759  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.472405  3615 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
W20260709 13:02:39.552403  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.521322  2313 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:39.578725  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 223) took 1483 ms. Trace:
W20260709 13:02:39.556604  1573 consensus_meta.cc:287] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.713s	user 0.014s	sys 0.008s
W20260709 13:02:39.573299  1538 connection.cc:256] RPC call timeout handler was delayed by -1.012772640000037s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1785s for a response
I20260709 13:02:39.580317  1569 rpcz_store.cc:276] 0709 13:02:37.948991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.277984 (+328993us) service_pool.cc:224] Handling call
0709 13:02:38.278470 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.278859 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:38.296874 (+ 18015us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.097346 (+800472us) spinlock_profiling.cc:243] Waited 9.32 ms on lock 0x7b18003b3bc0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a2523a 00007f8248e8fa2a
0709 13:02:39.401680 (+304334us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:39.405473 (+  3793us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:39.413897 (+  8424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.423836 (+  9939us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.432126 (+  8290us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17759,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":2420,"raft.queue_time_us":48229,"raft.run_cpu_time_us":885,"raft.run_wall_time_us":6517,"spinlock_wait_cycles":21424896,"thread_start_us":211249,"threads_started":1,"wal-append.queue_time_us":1092222}
W20260709 13:02:39.631165  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 350120us
W20260709 13:02:39.635178  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.631446  1573 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
W20260709 13:02:39.637663  3670 consensus_meta.cc:287] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.744s	user 0.015s	sys 0.012s
W20260709 13:02:39.638497  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.639066  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.639833  3688 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.640435  3688 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:39.642912  3688 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:39.606608  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:39.646232  3720 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:39.646790  2340 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:39.661705  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.670224  1646 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:39.673997  3649 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:02:39.685995  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 183119us
I20260709 13:02:39.631476  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:39.634243  3660 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:39.684645  1611 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:39.693707  1538 leader_election.cc:336] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.541s (SENT)
I20260709 13:02:39.702627  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1355) took 1299 ms. Trace:
I20260709 13:02:39.660331  1648 consensus_queue.cc:260] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:39.706857  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 236940us
I20260709 13:02:39.707245  3462 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:39.708936  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.709792  3688 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:39.704001  1573 rpcz_store.cc:276] 0709 13:02:38.402918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.710573 (+307655us) service_pool.cc:224] Handling call
0709 13:02:39.696078 (+985505us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:39.684775  3649 raft_consensus.cc:697] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:02:39.718524  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 94150.1us
I20260709 13:02:39.712174  3688 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:39.703873  1538 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:39.692577  3537 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:39.731182  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.732227  1648 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:39.730396  2313 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.917s (SENT) [suppressed 55 similar messages]
W20260709 13:02:39.734874  1687 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.973s (SENT)
I20260709 13:02:39.720351  3660 raft_consensus.cc:697] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:39.744354  1611 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:39.753371  2314 leader_election.cc:336] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 3.751s (SENT)
I20260709 13:02:39.733898  3537 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:39.754457  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:39.755048  3801 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.073s	user 0.000s	sys 0.002s
I20260709 13:02:39.724829  3710 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:39.743945  1612 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:39.770891  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:39.774065  1713 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:39.774986  1714 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:02:39.777611  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1283) took 1061 ms. Trace:
I20260709 13:02:39.778247  1644 rpcz_store.cc:276] 0709 13:02:38.716502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.718161 (+  1659us) service_pool.cc:224] Handling call
0709 13:02:38.718543 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.017909 (+299366us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b1800031fc0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:02:39.168695 (+150786us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.168724 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:39.343582 (+174858us) log.cc:844] Serialized 26 byte log entry
0709 13:02:39.610678 (+267096us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:39.610717 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.626848 (+ 16131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:39.776965 (+150117us) raft_consensus.cc:1685] finished
0709 13:02:39.776990 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.777373 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4487,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":303,"spinlock_wait_cycles":670422912,"thread_start_us":356185,"threads_started":2,"wal-append.queue_time_us":374205}
W20260709 13:02:39.780659  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 50065.6us
W20260709 13:02:39.782706  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.783205  3649 consensus_queue.cc:237] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:39.789801  3661 raft_consensus.cc:2764] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:02:39.793862  3670 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.794740  3670 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:39.788682  1685 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:39.800480  1685 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:39.802670  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 88567.8us
I20260709 13:02:39.803399  1716 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:39.804364  3806 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:02:39.805295  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.806052  3796 consensus_queue.cc:1048] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:39.807094  3767 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:39.808245  3796 raft_consensus.cc:2764] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:02:39.808059  3739 consensus_queue.cc:260] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:39.804605  3462 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:39.809331  3462 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.809412  3670 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:39.809916  3462 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:39.810158  3670 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.774506  1715 raft_consensus.cc:1275] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:39.814320  3660 consensus_queue.cc:237] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:39.793452  3804 raft_consensus.cc:2955] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:02:39.882529  1572 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:39.804774  1612 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:39.883021  1565 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:02:39.898901  1648 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:39.769557  3537 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:39.913125  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 246343us
I20260709 13:02:39.830061  3739 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:39.921202  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.919884  1647 consensus_meta.cc:287] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.582s	user 0.017s	sys 0.001s
I20260709 13:02:39.922029  1647 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:39.932487  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:39.929522  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 124845us
I20260709 13:02:39.918452  3806 raft_consensus.cc:697] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:02:39.953203  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.959553  3799 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:39.961202  1612 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:39.962682  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 232329us
I20260709 13:02:39.926970  3670 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:39.975677  2314 leader_election.cc:336] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 3.435s (SENT)
I20260709 13:02:39.952525  3710 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:39.994582  1714 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:40.005813  3732 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:40.010517  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 276918us
I20260709 13:02:40.014266  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:40.015784  3806 consensus_queue.cc:237] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:40.015908  3680 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:40.017062  3670 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:40.000006  3650 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:40.000809  3537 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.001806  3710 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:40.031088  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 79563.6us
I20260709 13:02:40.036820  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1289) took 1058 ms. Trace:
W20260709 13:02:40.045888  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 111250us
I20260709 13:02:40.037436  1643 rpcz_store.cc:276] 0709 13:02:38.972862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.981888 (+  9026us) service_pool.cc:224] Handling call
0709 13:02:40.030527 (+1048639us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b540007f154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:40.030671 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1064279808}
I20260709 13:02:40.038959  1611 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:40.056969  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 93382.2us
I20260709 13:02:40.006486  3732 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:40.034132  3650 raft_consensus.cc:697] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:02:40.070725  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 43582.2us
I20260709 13:02:40.032509  3615 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:02:40.081846  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:40.083281  1714 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:40.088855  3797 log.cc:927] Time spent T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.168s	user 0.000s	sys 0.002s
I20260709 13:02:40.090348  3820 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:40.093778  3770 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:40.094726  3770 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.095299  3770 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:40.102315  3819 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:40.103494  3819 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.104074  3819 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:40.104625  3821 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:02:40.107038  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 54334.3us
I20260709 13:02:40.108492  1612 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:40.112056  3732 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:40.059401  3688 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:40.127398  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:40.107674  3821 consensus_queue.cc:1048] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
W20260709 13:02:40.129062  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 57101.6us
I20260709 13:02:40.040673  3537 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:40.112242  3650 consensus_queue.cc:237] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:40.132789  3680 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:40.134856  3799 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:40.137219  3537 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:40.132572  1537 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:40.059703  3462 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:40.157374  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 357323us
I20260709 13:02:40.154771  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:40.127408  2340 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
W20260709 13:02:40.162539  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.164698  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.166324  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.167454  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.169239  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:40.154201  3799 raft_consensus.cc:697] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:40.165482  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:40.201200  1721 raft_consensus.cc:2368] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7.
I20260709 13:02:40.203939  1612 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:40.198336  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.198797  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45946.4us
I20260709 13:02:40.165860  3796 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:40.222177  1566 raft_consensus.cc:1275] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:02:40.222932  3661 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:40.224538  1720 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:02:40.224288  3799 consensus_queue.cc:237] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:40.219895  1685 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:40.224311  1611 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:40.245993  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1362) took 1217 ms. Trace:
I20260709 13:02:40.246654  1568 rpcz_store.cc:276] 0709 13:02:38.954035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.178827 (+224792us) service_pool.cc:224] Handling call
0709 13:02:39.198182 (+ 19355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.198870 (+   688us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.198900 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:39.199566 (+   666us) log.cc:844] Serialized 26 byte log entry
0709 13:02:39.417814 (+218248us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:39.417895 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.417997 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.171106 (+753109us) raft_consensus.cc:1685] finished
0709 13:02:40.171130 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.171589 (+   459us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":135,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":144,"thread_start_us":217644,"threads_started":1,"wal-append.queue_time_us":532604}
W20260709 13:02:40.248301  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 44527.9us
I20260709 13:02:40.246582  3796 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:40.250461  3796 raft_consensus.cc:697] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:40.254030  3825 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:40.276155  1564 raft_consensus.cc:1275] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:02:40.246030  1519 auto_rebalancer.cc:893] Removing replace marker failed: Replica 066ee0109aba4465ba17fa65e14ef504 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } } pending_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:02:40.276569  3825 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:40.277069  1519 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:02:40.276990  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:40.277525  1519 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:02:40.277138  3825 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:40.280853  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:40.280859  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:40.258191  3827 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.286559  3827 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:40.287531  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 116676us
I20260709 13:02:40.230274  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:40.290831  3732 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:02:40.273937  3785 consensus_meta.cc:287] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.869s	user 0.017s	sys 0.003s
I20260709 13:02:40.293515  3785 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7.
I20260709 13:02:40.294353  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 229) took 1172 ms. Trace:
I20260709 13:02:40.294979  1565 rpcz_store.cc:276] 0709 13:02:39.121564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.161449 (+ 39885us) service_pool.cc:224] Handling call
0709 13:02:40.294040 (+1132591us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b54000f69d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:40.294212 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":466393984}
I20260709 13:02:40.250254  1612 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:40.267915  3828 log.cc:927] Time spent T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:40.252576  3796 consensus_queue.cc:237] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:40.274888  3659 raft_consensus.cc:3055] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:40.289490  3649 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:40.249254  3791 consensus_queue.cc:1059] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.093s
I20260709 13:02:40.297096  3462 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:40.294252  1611 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:40.304417  1714 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:02:40.335570  2313 leader_election.cc:400] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7.
W20260709 13:02:40.340059  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.341969  3767 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:40.344308  1685 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:40.345933  3661 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.346798  3661 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:40.348045  3829 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.348737  3829 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:40.339589  2313 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:40.358413  3830 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.358807  3817 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:40.359387  3830 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:40.359506  3817 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:40.362068  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.492975  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 118040us
I20260709 13:02:40.571684  1686 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:40.519163  1687 leader_election.cc:336] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.705s (SENT)
W20260709 13:02:40.553304  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 259206us
W20260709 13:02:40.509280  1538 leader_election.cc:336] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.785s (SENT)
I20260709 13:02:40.315388  3659 raft_consensus.cc:740] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:40.525667  3650 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:40.704118  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 203523us
I20260709 13:02:40.705196  3710 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:40.636147  3537 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:40.520222  3827 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:40.580930  1687 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:40.708359  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 191991us
W20260709 13:02:40.709211  1687 leader_election.cc:336] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 3.046s (SENT)
W20260709 13:02:40.596637  2314 leader_election.cc:336] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.575s (SENT)
I20260709 13:02:40.634503  1611 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:40.410173  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 118715us
W20260709 13:02:40.635596  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 64042.4us
I20260709 13:02:40.680294  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:40.706861  3660 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.709607  1687 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:40.711160  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 76861.6us
I20260709 13:02:40.710104  2314 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:40.721360  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:40.720522  3537 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:02:40.722884  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 126318us
W20260709 13:02:40.726526  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.515985  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 265863us
W20260709 13:02:40.418949  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.712442  1685 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:40.744091  3846 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:40.378006  3603 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.748594  2340 raft_consensus.cc:2955] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:02:40.758998  3603 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:40.761222  1568 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:40.750031  3660 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:40.762028  3710 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.763525  1565 raft_consensus.cc:1240] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:40.436291  3688 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.765344  1571 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:40.481422  3767 raft_consensus.cc:3055] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:02:40.747373  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.363899  3649 raft_consensus.cc:697] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:02:40.766749  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.767613  3649 consensus_queue.cc:237] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:40.747043  3659 consensus_queue.cc:260] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:40.768479  3670 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:40.768973  3659 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:40.770716  3739 consensus_queue.cc:1059] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:40.770615  3670 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:40.771678  3837 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.378875  3785 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:40.772292  3837 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:40.564766  3830 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:40.774024  3840 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.774842  3840 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:40.361922  3661 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:40.704172  3650 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:40.734584  3827 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:40.436594  3820 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.738300  3799 consensus_queue.cc:1059] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:02:40.762883  3710 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:40.764318  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:40.744982  3825 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:40.718205  3817 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:40.511615  3462 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:40.597831  3805 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.771260  3720 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:40.598814  3835 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:40.684659  3770 consensus_meta.cc:287] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.589s	user 0.008s	sys 0.011s
W20260709 13:02:40.677475  3804 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.664s	user 0.012s	sys 0.003s
W20260709 13:02:40.847860  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.762952  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:40.777213  1721 raft_consensus.cc:1275] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:40.779574  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:40.850226  1570 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:40.850698  1714 raft_consensus.cc:2410] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:02:40.834980  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 129948us
W20260709 13:02:40.792920  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.765905  3767 raft_consensus.cc:740] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:02:40.852020  1538 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.977s (SENT)
I20260709 13:02:40.683030  3680 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:40.853118  1538 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:40.779476  1686 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:40.853358  1537 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:40.773128  3785 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.564767  3732 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:40.859061  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 115142us
W20260709 13:02:40.860308  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 147990us
I20260709 13:02:40.772013  3670 raft_consensus.cc:697] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:40.518045  3829 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:40.740948  3795 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:40.761003  1468 catalog_manager.cc:5697] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:40.906301  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 197151us
I20260709 13:02:40.777446  3840 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:40.788259  3791 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:40.912406  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.782616  3660 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:40.916375  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:40.853752  3680 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:40.792273  3603 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:40.792663  3837 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:40.886129  3785 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:40.765555  3688 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:40.940289  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 78348.3us
I20260709 13:02:40.940925  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:40.942550  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:40.777617  3827 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:40.922935  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 71870.9us
I20260709 13:02:40.948086  2347 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:02:40.779850  2341 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:40.958364  3831 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:40.959007  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:40.963375  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:40.964262  1721 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:02:40.964447  3710 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:40.923393  3819 consensus_meta.cc:287] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.813s	user 0.018s	sys 0.003s
I20260709 13:02:40.951587  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:40.900811  3739 consensus_queue.cc:1243] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer 066ee0109aba4465ba17fa65e14ef504 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:02:40.951174  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.855733  3770 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:41.013891  1571 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 13:02:41.014626  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 274842us
I20260709 13:02:40.942027  3829 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:40.847726  3805 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:41.053886  1687 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.540s (SENT)
I20260709 13:02:40.860455  3835 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:40.918403  3720 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:40.923339  3820 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:40.900560  3767 consensus_queue.cc:260] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:41.014492  3791 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:41.167469  2312 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:41.228279  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:41.100930  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:41.357993  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:41.242887  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.426960  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.430629  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.359246  1715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:02:41.433683  1715 raft_consensus.cc:606] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:41.435081  1571 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:02:41.240197  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.404855  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 321790us
W20260709 13:02:41.358837  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 315717us
W20260709 13:02:41.247185  3854 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.205s	user 0.000s	sys 0.000s
I20260709 13:02:41.436164  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:41.442739  1564 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:41.289312  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:41.245990  3851 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:41.437291  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 264774us
I20260709 13:02:41.042630  3732 raft_consensus.cc:697] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:02:41.450471  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 350027us
I20260709 13:02:41.157795  1468 catalog_manager.cc:5697] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 1 to 5, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 5 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:41.451339  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 599767us
W20260709 13:02:41.455883  2314 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 3.778s (SENT)
I20260709 13:02:41.450275  1716 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:02:41.466744  1565 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:41.240958  3815 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.117779  3817 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:02:41.435545  3846 consensus_meta.cc:287] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.586s	user 0.009s	sys 0.004s
I20260709 13:02:41.469004  1569 raft_consensus.cc:1275] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:41.240299  3825 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:41.397439  3806 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.101338  3799 consensus_queue.cc:1243] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer 066ee0109aba4465ba17fa65e14ef504 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:02:41.446311  3739 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.117306  3670 consensus_queue.cc:237] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:41.476370  3720 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:41.442788  3603 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:41.478518  3732 consensus_queue.cc:237] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:41.480048  1468 catalog_manager.cc:5697] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 1 to 3, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 3 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
W20260709 13:02:41.481837  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 439456us
W20260709 13:02:41.482496  1538 leader_election.cc:336] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.052s (SENT)
I20260709 13:02:41.482700  1571 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:02:41.482981  3851 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:41.485191  3805 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:41.015713  3819 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:41.486414  3852 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.486173  3767 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:41.487669  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:41.487378  3815 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:41.488358  1568 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 13:02:41.490486  3785 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.539s	user 0.004s	sys 0.008s
I20260709 13:02:41.495734  3661 raft_consensus.cc:3055] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:41.499315  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:41.568744  1714 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:41.509464  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.570729  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.571390  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.511642  2314 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:41.521258  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.521425  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:41.465153  3829 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:41.485406  3650 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:41.471472  3688 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:41.564465  3816 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:41.560310  1468 catalog_manager.cc:5697] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: config changed from index 2 to 3, term changed from 1 to 3, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to <none>, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:02:41.524912  3799 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.513811  3846 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:41.607993  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.531270  3770 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:41.536083  3796 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.515857  3603 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:41.560586  3805 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:41.613193  3825 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:41.622131  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 139726us
I20260709 13:02:41.518806  3739 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:41.579519  3661 raft_consensus.cc:740] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:02:41.625558  1720 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.003s	sys 0.009s
W20260709 13:02:41.614653  3848 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:02:41.620543  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:41.632463  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:41.613171  3720 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:02:41.683917  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 219591us
I20260709 13:02:41.534682  3806 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:41.703254  1715 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:41.682570  1468 catalog_manager.cc:5697] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 2 to 3, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:02:41.704020  1721 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:41.693260  1537 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:41.606138  3785 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:41.574139  3852 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:41.725605  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.761679  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.683486  3827 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:41.686367  3837 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:02:41.794314  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 216494us
W20260709 13:02:41.724614  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 268821us
I20260709 13:02:41.694777  3537 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:02:41.829895  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 318981us
I20260709 13:02:41.819825  1468 catalog_manager.cc:5697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:41.634063  3829 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:41.624989  3830 consensus_queue.cc:1048] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.100s
I20260709 13:02:41.824965  1716 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:41.824903  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:41.849121  1468 catalog_manager.cc:5697] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 1 to 3, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 3 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:41.900193  1714 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
W20260709 13:02:41.917407  3856 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.228s	user 0.000s	sys 0.004s
W20260709 13:02:41.905741  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.859916  1685 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:41.955682  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 95876.4us
I20260709 13:02:41.956647  1685 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:41.870116  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.878666  3865 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:41.829298  3817 raft_consensus.cc:697] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:41.793263  3821 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:41.702369  3688 consensus_queue.cc:1048] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.125s
W20260709 13:02:41.962589  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.850322  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1910) took 1073 ms. Trace:
I20260709 13:02:41.713194  3720 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:41.682931  3796 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
W20260709 13:02:41.879274  3863 log.cc:927] Time spent T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.360s	user 0.000s	sys 0.001s
I20260709 13:02:41.713090  3825 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:41.926384  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.032632  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.725246  3860 consensus_queue.cc:1059] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.170s
I20260709 13:02:42.046648  3829 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:41.880975  3537 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:41.631395  3799 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:41.918720  3804 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.113s
W20260709 13:02:41.950927  1687 leader_election.cc:336] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.919s (SENT)
I20260709 13:02:41.953370  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1911) took 1175 ms. Trace:
W20260709 13:02:41.942718  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 166343us
I20260709 13:02:41.857179  3837 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:42.021656  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 192004us
I20260709 13:02:42.080714  1714 rpcz_store.cc:276] 0709 13:02:40.777971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.952212 (+1174241us) service_pool.cc:224] Handling call
0709 13:02:41.952751 (+   539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.952970 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.952998 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:41.953046 (+    48us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:41.953080 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.953137 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.953265 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:41.927999  3820 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:42.096067  2340 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 1.336s	user 0.015s	sys 0.001s
I20260709 13:02:42.036528  3815 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:42.046734  3603 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:42.038151  3688 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:41.969920  3649 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:42.047113  3860 consensus_queue.cc:1243] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:02:41.970676  3816 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:41.958600  3879 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.972497  3852 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:41.973152  3851 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:41.967152  3659 consensus_meta.cc:287] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.198s	user 0.012s	sys 0.003s
I20260709 13:02:41.971481  3739 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:41.968776  3817 consensus_queue.cc:237] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:42.043009  3720 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:42.050190  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.120119  3816 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:42.159885  3868 log.cc:927] Time spent T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:02:42.162472  3879 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:42.168111  3871 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.347s	user 0.001s	sys 0.001s
W20260709 13:02:42.168670  3680 consensus_meta.cc:287] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.216s	user 0.012s	sys 0.004s
I20260709 13:02:42.054263  1468 catalog_manager.cc:5697] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:41.879703  3661 consensus_queue.cc:260] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:42.075857  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 271679us
I20260709 13:02:42.004590  3821 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.009001  1715 rpcz_store.cc:276] 0709 13:02:40.777470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.804084 (+1026614us) service_pool.cc:224] Handling call
0709 13:02:41.849719 (+ 45635us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.849949 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.849975 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:41.850026 (+    51us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:41.850057 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.850118 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.850257 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:41.967475  3865 raft_consensus.cc:697] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:02:42.090229  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 139510us
I20260709 13:02:42.093992  3819 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:42.030293  3835 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:42.099789  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 165) took 1864 ms. Trace:
I20260709 13:02:42.099254  3688 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.265599  3688 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:41.970289  3869 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.069s
W20260709 13:02:42.099061  3870 log.cc:927] Time spent T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.215s	user 0.000s	sys 0.000s
I20260709 13:02:42.047151  3806 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:42.080523  3791 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:42.228582  3821 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:42.110095  1612 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.021s	sys 0.025s
W20260709 13:02:42.117654  1538 leader_election.cc:336] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.876s (SENT)
I20260709 13:02:42.100344  3846 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:42.293924  1612 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.021s	sys 0.025s
I20260709 13:02:42.103971  3649 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:42.148622  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 319709us
I20260709 13:02:42.321316  3649 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:42.179714  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.325228  2347 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 1 LEARNER]: Advancing to term 3
W20260709 13:02:42.328925  3844 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 1.247s	user 0.014s	sys 0.001s
W20260709 13:02:42.223299  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.231564  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 55378us
I20260709 13:02:42.330063  1538 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:42.238062  3799 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:42.165481  3879 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:42.239450  3785 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:42.331446  3835 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:42.172681  3816 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:42.160358  3796 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:42.241257  3865 consensus_queue.cc:237] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:42.245759  1720 rpcz_store.cc:276] 0709 13:02:40.212884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.225612 (+ 12728us) service_pool.cc:224] Handling call
0709 13:02:40.718626 (+493014us) spinlock_profiling.cc:243] Waited 7.82 ms on lock 0x7b580009db98. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:02:40.736484 (+ 17858us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.966165 (+229681us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.966209 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:41.046838 (+ 80629us) log.cc:844] Serialized 26 byte log entry
0709 13:02:41.688849 (+642011us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:41.688891 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.689155 (+   264us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.047852 (+358697us) raft_consensus.cc:1685] finished
0709 13:02:42.047872 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.077362 (+ 29490us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":172618,"raft.queue_time_us":112625,"raft.run_cpu_time_us":352,"raft.run_wall_time_us":68489,"spinlock_wait_cycles":17994112,"thread_start_us":571520,"threads_started":1,"wal-append.queue_time_us":906236}
W20260709 13:02:42.280253  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.294458  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 819941us
I20260709 13:02:42.308115  1468 catalog_manager.cc:5697] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 1 to 3, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 3 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.294350  3846 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:42.174381  3680 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:42.277472  3767 consensus_meta.cc:287] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.709s	user 0.008s	sys 0.000s
I20260709 13:02:42.357276  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:42.348549  1687 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.966s (SENT)
I20260709 13:02:42.423671  2347 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 LEARNER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:42.358494  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:42.422896  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 967144us
I20260709 13:02:42.357465  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:42.443784  1687 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:42.444543  1566 raft_consensus.cc:2410] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:42.442082  1569 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:02:42.444351  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 350900us
I20260709 13:02:42.452096  1565 raft_consensus.cc:1240] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:42.452306  1569 raft_consensus.cc:1240] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:42.442544  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1912) took 1615 ms. Trace:
I20260709 13:02:42.453378  1714 rpcz_store.cc:276] 0709 13:02:40.826956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.095041 (+1268085us) service_pool.cc:224] Handling call
0709 13:02:42.096683 (+  1642us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.096989 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:42.128251 (+ 31262us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.442086 (+313835us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:42.442149 (+    63us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:42.442223 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.442295 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.442473 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":618,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":243521,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":141,"thread_start_us":71108,"threads_started":1}
I20260709 13:02:42.461128  1568 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
W20260709 13:02:42.461879  3861 log.cc:927] Time spent T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.469s	user 0.005s	sys 0.000s
W20260709 13:02:42.461874  1538 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.979s (SENT)
I20260709 13:02:42.463330  1538 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:42.366123  3796 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.444105  1468 catalog_manager.cc:5697] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 1 to 3, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 3 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.370762  3537 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:42.445030  3837 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:42.450701  3688 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:42.504422  3688 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:42.509711  1564 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:42.511756  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.498225  3855 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.646s	user 0.002s	sys 0.005s
I20260709 13:02:42.514207  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1899) took 2339 ms. Trace:
I20260709 13:02:42.498876  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1903) took 1709 ms. Trace:
I20260709 13:02:42.515085  3770 raft_consensus.cc:3055] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:42.515587  3770 raft_consensus.cc:740] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:42.515441  1719 rpcz_store.cc:276] 0709 13:02:40.741450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.741693 (+   243us) service_pool.cc:224] Handling call
0709 13:02:40.767580 (+ 25887us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:40.952635 (+185055us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.971384 (+ 18749us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:41.053923 (+ 82539us) log.cc:844] Serialized 52 byte log entry
0709 13:02:41.534499 (+480576us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:41.622231 (+ 87732us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.635091 (+ 12860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.635187 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.288559 (+653372us) spinlock_profiling.cc:243] Waited 9.05 ms on lock 0x7b4800190158. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ddaeaa1 00007f824dda5b37 00007f824dda59ca 00007f824de9d26f 00007f824de9cf37 00007f824deb155c 00007f824deb1508 00007f824deb1488 00007f824deb1443 00007f824deb06ec 00007f824cddaa96
0709 13:02:42.423636 (+135077us) raft_consensus.cc:1685] finished
0709 13:02:42.450370 (+ 26734us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.450585 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":254365,"prepare.queue_time_us":101151,"prepare.run_cpu_time_us":4468,"prepare.run_wall_time_us":65454,"raft.queue_time_us":72978,"raft.run_cpu_time_us":21865,"raft.run_wall_time_us":592711,"spinlock_wait_cycles":20824320,"thread_start_us":550613,"threads_started":2,"wal-append.queue_time_us":703614}
I20260709 13:02:42.479861  3852 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.517819  3852 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:42.501854  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1904) took 1727 ms. Trace:
I20260709 13:02:42.520325  1718 rpcz_store.cc:276] 0709 13:02:40.756242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.756442 (+   200us) service_pool.cc:224] Handling call
0709 13:02:40.779986 (+ 23544us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:40.780509 (+   523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.780543 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:40.781332 (+   789us) log.cc:844] Serialized 52 byte log entry
0709 13:02:41.092423 (+311091us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:41.092475 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.092559 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.375313 (+1282754us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:02:42.463393 (+ 88080us) raft_consensus.cc:1685] finished
0709 13:02:42.463414 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.483291 (+ 19877us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":489,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"spinlock_wait_cycles":26275584,"thread_start_us":302833,"threads_started":1,"wal-append.queue_time_us":993752}
I20260709 13:02:42.516701  3770 consensus_queue.cc:260] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:42.523847  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1372) took 1555 ms. Trace:
I20260709 13:02:42.532401  1686 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:02:42.531028  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 65737.8us
W20260709 13:02:42.538427  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 81873.1us
I20260709 13:02:42.538304  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:42.541008  1564 raft_consensus.cc:2393] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:42.544914  1612 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:42.505481  3688 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:42.548343  1573 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:42.550902  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:42.549858  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:42.554020  1569 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:42.514698  1717 rpcz_store.cc:276] 0709 13:02:40.174211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.221370 (+ 47159us) service_pool.cc:224] Handling call
0709 13:02:40.672476 (+451106us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b580009db98. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:02:40.672729 (+   253us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:40.673461 (+   732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.673501 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:40.764593 (+ 91092us) log.cc:844] Serialized 52 byte log entry
0709 13:02:41.091257 (+326664us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:41.091316 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.091415 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.371134 (+1279719us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:02:42.513610 (+142476us) raft_consensus.cc:1685] finished
0709 13:02:42.513630 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.513836 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":150,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":186,"spinlock_wait_cycles":72644352,"thread_start_us":312618,"threads_started":1,"wal-append.queue_time_us":856867}
I20260709 13:02:42.557130  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:42.562637  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.556070  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1901) took 2341 ms. Trace:
W20260709 13:02:42.569291  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.574324  1571 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:42.538053  3860 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.578192  3661 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:42.583531  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 51290.6us
I20260709 13:02:42.564740  1713 rpcz_store.cc:276] 0709 13:02:40.214559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.284523 (+ 69964us) service_pool.cc:224] Handling call
0709 13:02:40.284969 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.302566 (+ 17597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.302599 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:40.362627 (+ 60028us) log.cc:844] Serialized 26 byte log entry
0709 13:02:40.576783 (+214156us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:40.577000 (+   217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.577085 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.328151 (+1751066us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:02:42.464223 (+136072us) raft_consensus.cc:1685] finished
0709 13:02:42.464253 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.555997 (+ 91744us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12202,"raft.queue_time_us":16852,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":111,"spinlock_wait_cycles":8339200,"thread_start_us":64927,"threads_started":1,"wal-append.queue_time_us":439466}
I20260709 13:02:42.492349  3796 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:42.556329  3462 raft_consensus.cc:3055] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:42.595012  3770 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:42.595881  3883 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.597921  3767 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:42.598685  3462 raft_consensus.cc:740] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:42.599951  3462 consensus_queue.cc:260] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:42.598197  3805 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:42.601848  3805 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.602304  3805 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:42.601424  3462 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:42.611256  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.612870  3867 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:42.616533  3894 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.616216  3882 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:02:42.618150  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 269654us
W20260709 13:02:42.618997  1687 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.200s (SENT)
W20260709 13:02:42.621407  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.611003  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 72159.4us
W20260709 13:02:42.628638  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.629438  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.596745  3883 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:42.556617  3660 consensus_queue.cc:1059] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:02:42.646575  3877 consensus_queue.cc:1059] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:02:42.647883  3660 raft_consensus.cc:3055] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:42.648428  3660 raft_consensus.cc:740] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:42.624141  3860 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:42.649605  3660 consensus_queue.cc:260] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:42.617280  3894 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:42.665127  3660 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:42.655223  3796 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:42.657335  3860 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:42.661818  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:02:42.666646  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.657742  1468 catalog_manager.cc:5697] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 2 to 3, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
W20260709 13:02:42.677122  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 48647.9us
I20260709 13:02:42.657743  3877 raft_consensus.cc:3055] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:02:42.663184  3899 log.cc:927] Time spent T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:02:42.660060  3852 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:42.686375  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:42.663003  3846 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:42.695312  1719 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:02:42.709513  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:42.717593  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:42.719363  1538 connection.cc:256] RPC call timeout handler was delayed by -2.403434395999966s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20840000000000003s for a response
W20260709 13:02:42.720396  1538 connection.cc:256] RPC call timeout handler was delayed by -2.3962343960000112s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20920000000000002s for a response
I20260709 13:02:42.676065  1468 catalog_manager.cc:5697] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.719492  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:02:42.687083  3795 consensus_meta.cc:287] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.765s	user 0.015s	sys 0.007s
I20260709 13:02:42.714761  1718 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:42.667327  3894 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:42.727264  1612 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:42.729079  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:42.730024  1721 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
W20260709 13:02:42.732882  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.734642  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:42.735252  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:42.735872  1566 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:42.736207  1564 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:42.736855  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:42.737723  1714 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:02:42.743723  1713 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:42.683281  3877 raft_consensus.cc:740] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:42.746486  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:42.730037  3795 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:42.748752  3849 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:42.673250  3649 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:42.715397  2340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 265) took 1946 ms. Trace:
I20260709 13:02:42.682920  3879 consensus_queue.cc:1048] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 13:02:42.753172  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:42.754849  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:42.657531  3816 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:42.759011  1468 catalog_manager.cc:5697] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 2 to 4, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 4 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.694083  3732 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:02:42.693595  3869 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:02:42.757954  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:42.761989  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:42.762600  1567 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:42.762244  1572 raft_consensus.cc:2410] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:42.683410  3860 consensus_queue.cc:1059] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:02:42.766115  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:42.534067  1570 rpcz_store.cc:276] 0709 13:02:40.968882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.969159 (+   277us) service_pool.cc:224] Handling call
0709 13:02:41.033431 (+ 64272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.034081 (+   650us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:41.367543 (+333462us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.485496 (+1117953us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b1800078280. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a2523a
0709 13:02:42.523454 (+ 37958us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:42.523518 (+    64us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:42.523575 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.523652 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.523788 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37666,"prepare.queue_time_us":583063,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":70,"raft.queue_time_us":42940,"spinlock_wait_cycles":49139072,"thread_start_us":525148,"threads_started":2,"wal-append.queue_time_us":1028488}
I20260709 13:02:42.769599  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:42.778102  1718 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:42.772037  1566 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:42.729981  3821 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:42.748771  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:02:42.783517  1538 leader_election.cc:336] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.762s (SENT)
I20260709 13:02:42.784089  1538 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:42.747987  3680 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:42.780433  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:42.790351  1612 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:42.792233  1567 raft_consensus.cc:2410] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:42.782617  1468 catalog_manager.cc:5697] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 1 to 3, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 3 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.755128  3796 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:42.805562  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:42.809396  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:42.796061  1468 catalog_manager.cc:5697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 2 to 4, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 4 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.812848  1569 raft_consensus.cc:2410] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:02:42.813763  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 46822.2us
I20260709 13:02:42.816634  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:42.820572  1713 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:42.823954  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:42.827073  1571 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:42.790462  3688 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:42.842760  1567 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:42.785001  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:42.789294  3825 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:42.846833  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.848642  1538 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.853s (SENT)
W20260709 13:02:42.900570  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.901984  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.902415  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.902776  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.903967  1537 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:42.896509  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:42.904917  1713 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:42.912973  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.912288  1537 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:42.916579  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.914459  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.919730  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.914568  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 76310.1us
W20260709 13:02:42.922947  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 99302.3us
I20260709 13:02:42.916174  1685 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:42.920141  1537 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:42.940061  1685 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:42.942229  3732 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:42.944590  3732 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.945232  3732 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:42.953025  3817 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:42.954419  3817 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.954902  3817 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:42.962491  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.859898  3795 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:42.965152  3825 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:42.905468  3877 consensus_queue.cc:260] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:43.012297  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:42.988957  3649 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:42.992733  3909 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:02:42.758785  2340 rpcz_store.cc:276] 0709 13:02:40.739268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.747785 (+  8517us) service_pool.cc:224] Handling call
0709 13:02:40.748199 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.748460 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:42.096757 (+1348297us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.568610 (+471853us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.628559 (+ 59949us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:42.638785 (+ 10226us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:42.645037 (+  6252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.676472 (+ 31435us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.685902 (+  9430us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":367476,"prepare.queue_time_us":387,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":77,"raft.queue_time_us":385698,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":9658,"thread_start_us":124537,"threads_started":2,"wal-append.queue_time_us":351546}
I20260709 13:02:42.918215  3688 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:43.029274  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:43.030421  1566 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 6: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:02:43.006345  1538 leader_election.cc:336] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.875s (SENT)
I20260709 13:02:43.023713  1685 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:43.003628  1537 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:42.928369  3680 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:42.922497  3796 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:43.001256  3869 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.050071  1571 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:42.930624  3852 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.096s
I20260709 13:02:43.040663  3720 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:43.078074  3869 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:43.070459  1685 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:43.024135  3825 raft_consensus.cc:697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:43.040694  3680 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.039041  3688 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:43.026268  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:02:43.119566  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 117083us
W20260709 13:02:43.115418  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 58866.3us
I20260709 13:02:43.073436  3537 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:43.062986  3883 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:43.130023  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.082430  3795 raft_consensus.cc:2764] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:02:43.083621  3877 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:43.140918  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 175110us
I20260709 13:02:43.087007  3720 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.143651  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:43.144997  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.144588  1567 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
W20260709 13:02:43.159585  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 56912.6us
I20260709 13:02:43.119822  3680 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:43.176349  3825 consensus_queue.cc:237] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:43.179363  3869 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:43.122687  3688 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:43.180724  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:43.182547  3914 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:02:43.182935  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.183817  1645 raft_consensus.cc:2393] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:43.184741  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.147748  3816 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:43.162351  3883 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.186422  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:43.186529  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.187209  3883 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:43.164233  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:43.190544  1647 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:43.191329  3883 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:43.191826  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.192507  1647 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:43.165077  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.165445  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:43.193953  1640 raft_consensus.cc:2376] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5. Re-sending same reply.
I20260709 13:02:43.194368  1644 raft_consensus.cc:2410] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:43.166239  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:43.166522  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:43.195140  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.164215  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.196642  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:43.196961  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.197484  1640 raft_consensus.cc:2410] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:43.198313  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:43.165086  1648 raft_consensus.cc:1275] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:43.146334  3720 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:43.211335  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.212327  1648 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:43.193624  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.195147  1641 raft_consensus.cc:2376] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7. Re-sending same reply.
I20260709 13:02:43.214725  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:43.215480  1641 raft_consensus.cc:2368] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5.
I20260709 13:02:43.216279  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.216925  1641 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:43.128684  3537 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.217991  3537 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:43.195480  1646 raft_consensus.cc:2376] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5. Re-sending same reply.
I20260709 13:02:43.199957  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:43.201273  3770 consensus_meta.cc:287] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.606s	user 0.010s	sys 0.006s
I20260709 13:02:43.125386  3830 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:43.199888  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:43.185521  1645 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:43.203116  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.234620  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:43.203359  3661 consensus_meta.cc:287] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.610s	user 0.007s	sys 0.007s
I20260709 13:02:43.213449  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:43.238626  1648 raft_consensus.cc:2393] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:43.222429  1713 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
W20260709 13:02:43.234872  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.202109  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:43.242501  1643 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:43.242101  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:43.244750  1641 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:43.219307  3907 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.400s	user 0.000s	sys 0.003s
I20260709 13:02:43.245328  1718 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:02:43.246279  1687 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:43.247995  3905 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.264815  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.248164  1644 raft_consensus.cc:2410] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:02:43.248941  3817 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:43.249424  1645 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:43.250126  1640 raft_consensus.cc:3055] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:43.251295  1647 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:43.273463  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:43.272342  1640 raft_consensus.cc:740] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:43.282300  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.283010  1648 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:43.299021  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:43.226591  3804 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 13:02:43.317297  1643 raft_consensus.cc:2368] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5.
I20260709 13:02:43.312175  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.317396  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:43.349453  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.309435  1642 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
W20260709 13:02:43.365202  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.397603  1571 raft_consensus.cc:1217] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[3.6-3.6]   Dedup: 3.6->[]
I20260709 13:02:43.425136  1571 raft_consensus.cc:1240] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:02:43.406858  1644 raft_consensus.cc:3055] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:02:43.408977  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 171119us
I20260709 13:02:43.426007  1644 raft_consensus.cc:740] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:43.408211  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:43.293170  3905 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:43.430141  1566 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:43.421591  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:43.398022  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:43.432597  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:43.402904  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 275243us
I20260709 13:02:43.436398  1687 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:02:43.439116  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:43.444885  1565 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.006s	sys 0.011s
W20260709 13:02:43.447623  1565 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.006s	sys 0.011s
I20260709 13:02:43.409150  1640 consensus_queue.cc:260] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:43.425148  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:43.461371  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.461392  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.466138  1645 raft_consensus.cc:2410] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:43.470938  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.470136  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.471865  1645 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:43.455216  1644 consensus_queue.cc:260] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:43.421661  3821 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:43.312404  3830 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:43.476459  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:43.473531  1639 raft_consensus.cc:2376] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4. Re-sending same reply.
W20260709 13:02:43.446727  3660 consensus_meta.cc:287] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.773s	user 0.006s	sys 0.006s
I20260709 13:02:43.479295  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:43.733186  3720 consensus_meta.cc:287] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.522s	user 0.016s	sys 0.001s
I20260709 13:02:43.814826  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.815850  1639 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:43.586498  1565 raft_consensus.cc:1240] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:43.516476  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:43.587467  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.510574  1644 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:43.561426  1648 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:43.741997  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1390) took 1002 ms. Trace:
W20260709 13:02:43.818519  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.501042  1567 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:02:43.669533  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:43.820560  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:43.499018  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 460165us
I20260709 13:02:43.453869  3649 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:43.822081  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.662367  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:43.815166  3720 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:43.752822  3791 raft_consensus.cc:3055] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:43.759979  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:43.554837  1687 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:43.703512  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:43.842538  1568 raft_consensus.cc:2410] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:02:43.716240  3680 consensus_meta.cc:287] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.542s	user 0.002s	sys 0.010s
I20260709 13:02:43.559355  2347 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:43.497325  1640 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:43.496464  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.816998  1643 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:43.820204  1572 rpcz_store.cc:276] 0709 13:02:42.739844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.783737 (+ 43893us) service_pool.cc:224] Handling call
0709 13:02:42.811268 (+ 27531us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.831115 (+ 19847us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.831152 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.852435 (+ 21283us) log.cc:844] Serialized 26 byte log entry
0709 13:02:43.435135 (+582700us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:43.435209 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.435559 (+   350us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:43.741692 (+306133us) raft_consensus.cc:1685] finished
0709 13:02:43.741720 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.741912 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22974,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":54059,"thread_start_us":428615,"threads_started":1,"wal-append.queue_time_us":827702}
I20260709 13:02:43.828297  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:43.829051  1571 raft_consensus.cc:2410] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:02:43.680019  3916 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.200s	user 0.002s	sys 0.001s
W20260709 13:02:43.588052  3918 log.cc:927] Time spent T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:02:43.837527  1642 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:43.851516  1645 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
W20260709 13:02:43.838130  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.846590  2341 raft_consensus.cc:2410] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:02:43.854136  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.854758  1642 raft_consensus.cc:2376] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5. Re-sending same reply.
I20260709 13:02:43.855705  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.856688  1642 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:43.857041  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:43.838204  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.842687  1566 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:43.529307  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.858696  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:43.733186  3805 consensus_meta.cc:287] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.130s	user 0.016s	sys 0.004s
I20260709 13:02:43.827554  3649 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:43.554975  3821 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.847260  1537 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:43.910805  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 63891.5us
I20260709 13:02:43.912366  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:43.911334  3821 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:43.846660  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:43.849942  1648 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:43.734964  3905 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:43.604173  3817 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:43.684731  3888 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.859978  1572 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:43.860414  1639 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:43.938365  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:43.938829  1639 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:02:43.939201  1639 pending_rounds.cc:85] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 2
I20260709 13:02:43.939500  1639 pending_rounds.cc:107] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:02:43.939819  1639 raft_consensus.cc:2889] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:43.635077  3830 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:43.941100  3830 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:43.599290  3913 log.cc:927] Time spent T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.348s	user 0.000s	sys 0.003s
I20260709 13:02:43.871030  1646 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:43.862419  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:43.949205  1642 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:43.862579  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 312573us
I20260709 13:02:43.859998  1647 raft_consensus.cc:2410] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:43.862288  1570 raft_consensus.cc:2393] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 6: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:43.876174  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:43.871214  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:43.939755  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:43.896445  3720 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:43.903838  1645 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:43.841678  3791 raft_consensus.cc:740] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:02:43.861577  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 432201us
W20260709 13:02:43.907666  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.916486  1537 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:43.932740  1641 consensus_meta.cc:287] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.661s	user 0.017s	sys 0.000s
I20260709 13:02:43.946902  1648 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:02:43.894958  3805 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:43.951354  1568 raft_consensus.cc:2410] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:43.957093  1641 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:43.850399  3879 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:43.957593  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.959467  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:43.959910  1566 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:02:43.962778  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:43.977034  1565 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:43.919520  3649 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:43.967429  2341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55890 (request call id 228) took 1216 ms. Trace:
I20260709 13:02:43.967571  3879 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:02:43.984778  3688 consensus_meta.cc:287] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.804s	user 0.011s	sys 0.003s
I20260709 13:02:43.985553  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1386) took 1247 ms. Trace:
W20260709 13:02:43.993281  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:43.953313  3732 consensus_meta.cc:287] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.007s	user 0.020s	sys 0.005s
I20260709 13:02:43.990088  3660 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.014052  1564 rpcz_store.cc:276] 0709 13:02:42.738219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.738350 (+   131us) service_pool.cc:224] Handling call
0709 13:02:42.743294 (+  4944us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.757433 (+ 14139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.757464 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.776614 (+ 19150us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.929207 (+152593us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:42.929248 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.929320 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:43.985288 (+1055968us) raft_consensus.cc:1685] finished
0709 13:02:43.985312 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.985481 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":170,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":11770,"thread_start_us":115705,"threads_started":1,"wal-append.queue_time_us":319221}
I20260709 13:02:43.975615  1641 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:43.975653  1571 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
W20260709 13:02:44.001343  3537 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.783s	user 0.010s	sys 0.006s
I20260709 13:02:43.974267  3817 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.010473  3879 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:43.848316  3680 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:44.030763  1643 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:43.998054  3896 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:43.994496  3720 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:44.007565  2341 rpcz_store.cc:276] 0709 13:02:42.735455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.742898 (+  7443us) service_pool.cc:224] Handling call
0709 13:02:43.945805 (+1202907us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400712854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:43.951544 (+  5739us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2406229248}
I20260709 13:02:44.033573  3537 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:44.035204  3830 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:43.952096  3888 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:44.003279  3865 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
W20260709 13:02:44.035396  3877 consensus_meta.cc:287] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.874s	user 0.017s	sys 0.000s
I20260709 13:02:44.060847  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1394) took 1282 ms. Trace:
W20260709 13:02:44.049491  3931 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.140s	user 0.000s	sys 0.005s
I20260709 13:02:44.065733  1643 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:44.064484  1573 rpcz_store.cc:276] 0709 13:02:42.778442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.786246 (+  7804us) service_pool.cc:224] Handling call
0709 13:02:42.811542 (+ 25296us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:42.882461 (+ 70919us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.883510 (+  1049us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:42.982128 (+ 98618us) log.cc:844] Serialized 52 byte log entry
0709 13:02:43.111190 (+129062us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:43.111234 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.111309 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:44.060403 (+949094us) raft_consensus.cc:1685] finished
0709 13:02:44.060457 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.060733 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11517,"raft.queue_time_us":60670,"raft.run_cpu_time_us":558,"raft.run_wall_time_us":39608,"thread_start_us":126884,"threads_started":1,"wal-append.queue_time_us":434082}
W20260709 13:02:44.034387  3825 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.010s	sys 0.019s
I20260709 13:02:44.049891  3720 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.033288  3660 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:44.016659  3688 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:44.078814  1642 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:44.050277  3896 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:02:44.081318  1538 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.863s	user 0.001s	sys 0.034s
I20260709 13:02:44.030177  3732 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:44.082862  1643 raft_consensus.cc:3055] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:44.036651  3817 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:44.102502  1643 raft_consensus.cc:740] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:02:44.103176  1538 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.885s	user 0.002s	sys 0.034s
I20260709 13:02:44.104003  3896 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:44.090260  3720 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:44.107962  3688 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:44.105960  1569 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: []
W20260709 13:02:44.124191  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 906229us
W20260709 13:02:44.080142  3825 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.010s	sys 0.020s
I20260709 13:02:44.136788  1643 consensus_queue.cc:260] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:44.136293  1687 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:44.141036  1643 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:44.142637  1642 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:44.144873  3660 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:44.076404  3791 consensus_queue.cc:260] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:44.147445  3791 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:44.103056  3805 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:44.152877  3830 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:44.156117  1644 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:02:44.156857  1644 pending_rounds.cc:85] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 2
I20260709 13:02:44.157204  1644 pending_rounds.cc:107] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:02:44.157502  1644 raft_consensus.cc:2889] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:44.158939  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1594) took 1139 ms. Trace:
I20260709 13:02:44.159648  1569 rpcz_store.cc:276] 0709 13:02:43.020013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.020167 (+   154us) service_pool.cc:224] Handling call
0709 13:02:43.441566 (+421399us) spinlock_profiling.cc:243] Waited 6.72 ms on lock 0x7b5800011198. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:02:43.441933 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.139772 (+697839us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.158639 (+ 18867us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b54001b4254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:44.158859 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1542150784}
I20260709 13:02:44.105809  3732 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:44.184962  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 74552.5us
I20260709 13:02:44.191464  3821 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:44.198645  1644 raft_consensus.cc:3055] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:44.198930  3537 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:44.199188  1644 raft_consensus.cc:740] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:44.200425  1644 consensus_queue.cc:260] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:44.201370  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.201467  1644 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:44.192987  3919 log.cc:927] Time spent T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.516s	user 0.004s	sys 0.003s
I20260709 13:02:44.196765  3825 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:44.219436  1642 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:44.228107  3817 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:44.229136  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.220508  1538 leader_election.cc:336] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.041s (SENT)
I20260709 13:02:44.229176  3879 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:44.238299  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.241456  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.242101  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:44.243273  1642 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:44.219028  3825 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:44.261422  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1925) took 1558 ms. Trace:
W20260709 13:02:44.265430  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 44987.7us
W20260709 13:02:44.268689  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.262202  1717 rpcz_store.cc:276] 0709 13:02:42.702582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.713870 (+ 11288us) service_pool.cc:224] Handling call
0709 13:02:42.717250 (+  3380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.748859 (+ 31609us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.748901 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.749493 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 13:02:43.076652 (+327159us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b180044d5e0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a250d1 00007f8248e40a0f
0709 13:02:43.246391 (+169739us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:43.246434 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.246525 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:44.030758 (+784233us) spinlock_profiling.cc:243] Waited 88.8 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:02:44.261041 (+230283us) raft_consensus.cc:1685] finished
0709 13:02:44.261073 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.261319 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1390,"raft.queue_time_us":16814,"raft.run_cpu_time_us":583,"raft.run_wall_time_us":8304,"spinlock_wait_cycles":206465152,"thread_start_us":169538,"threads_started":1,"wal-append.queue_time_us":823539}
I20260709 13:02:44.269358  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:44.272261  1642 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
W20260709 13:02:44.270704  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.263464  3825 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:44.284128  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:44.284838  1642 raft_consensus.cc:2376] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4. Re-sending same reply.
W20260709 13:02:44.298134  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 449372us
I20260709 13:02:44.268119  3680 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:44.299347  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.293321  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 391857us
I20260709 13:02:44.331184  3817 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:02:44.273504  2340 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.026s	sys 0.011s
I20260709 13:02:44.345139  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:44.347046  1642 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:44.352059  3537 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:44.353341  3936 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.354852  3936 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:44.356773  1566 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:44.329362  3888 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:44.380849  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.381619  2313 leader_election.cc:336] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.601s (SENT)
I20260709 13:02:44.368641  1644 raft_consensus.cc:1275] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:02:44.384182  1644 pending_rounds.cc:85] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 2
W20260709 13:02:44.385031  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.385582  1644 pending_rounds.cc:107] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:02:44.386240  1644 raft_consensus.cc:2889] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:44.384475  1538 leader_election.cc:336] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 3.004s (SENT)
I20260709 13:02:44.401823  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:44.402838  1642 raft_consensus.cc:2393] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:44.408492  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.398736  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:44.349401  2340 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.026s	sys 0.011s
I20260709 13:02:44.410754  1566 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:02:44.412251  2312 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:44.399667  3926 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.417961  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:44.426156  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.426115  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.429577  1640 consensus_meta.cc:287] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.585s	user 0.006s	sys 0.007s
I20260709 13:02:44.431861  1640 raft_consensus.cc:1275] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
W20260709 13:02:44.432278  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.364934  3928 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.012s	sys 0.012s
I20260709 13:02:44.432673  1640 pending_rounds.cc:85] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 1
I20260709 13:02:44.433192  1640 pending_rounds.cc:107] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:02:44.451488  1640 raft_consensus.cc:2889] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:44.391059  3941 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.465065  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:44.441289  1573 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:44.473148  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1326) took 1257 ms. Trace:
I20260709 13:02:44.453388  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:44.474367  1642 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
W20260709 13:02:44.443349  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 58930.8us
W20260709 13:02:44.452186  3928 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.013s	sys 0.013s
I20260709 13:02:44.414834  3936 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:44.476224  1644 raft_consensus.cc:2410] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:44.473925  1640 rpcz_store.cc:276] 0709 13:02:43.215661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.244699 (+ 29038us) service_pool.cc:224] Handling call
0709 13:02:43.249999 (+  5300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.472823 (+1222824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.473078 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.399708  3732 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.072s
I20260709 13:02:44.482831  1687 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:44.432163  3830 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:44.493281  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:44.508944  1642 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:44.512177  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.513866  3830 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:44.475257  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 93685.6us
I20260709 13:02:44.439714  3877 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:44.519245  3877 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.520000  3877 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:44.409814  3927 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:44.477475  3805 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.489039  3894 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:44.445282  3929 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 13:02:44.507580  1538 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 3.046s (SENT)
W20260709 13:02:44.524197  2314 leader_election.cc:336] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.059s (SENT)
I20260709 13:02:44.526046  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:44.528585  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:44.549860  1642 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
W20260709 13:02:44.503198  1537 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.047s	sys 0.078s
I20260709 13:02:44.551625  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:44.552469  2341 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:44.525358  3817 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:44.555794  3817 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:44.556277  3817 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:44.540679  3936 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:44.558586  2314 leader_election.cc:336] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.902s (SENT)
I20260709 13:02:44.558655  3936 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
W20260709 13:02:44.552277  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 100566us
I20260709 13:02:44.433395  3926 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:44.563426  1537 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.047s	sys 0.078s
W20260709 13:02:44.563833  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 647623us
I20260709 13:02:44.564559  1537 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:44.445487  3649 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:44.541117  1611 connection.cc:256] RPC call timeout handler was delayed by -1.2741893709999772s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
I20260709 13:02:44.572305  1538 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:44.576843  1611 connection.cc:256] RPC call timeout handler was delayed by -1.0617893709999748s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19720000000000001s for a response
I20260709 13:02:44.582345  1640 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:44.550006  3805 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:44.494498  3941 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:44.588034  3941 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:44.594286  3945 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.595158  3945 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:44.599200  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 121228us
I20260709 13:02:44.599963  3926 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:44.600466  1538 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 3.100s (SENT)
I20260709 13:02:44.600628  3926 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:44.601015  3926 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.601899  3944 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:44.602690  3944 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.603129  3944 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:44.562124  3927 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:44.565935  3879 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:44.605525  3944 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:44.613300  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:44.622323  1641 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
W20260709 13:02:44.617559  1538 leader_election.cc:336] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.583s (SENT)
I20260709 13:02:44.601400  3926 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:44.618302  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:44.625061  1648 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:44.618739  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:44.626296  1647 raft_consensus.cc:2376] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5. Re-sending same reply.
W20260709 13:02:44.618148  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 53692.4us
I20260709 13:02:44.627969  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:44.624208  1538 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.637s (SENT)
I20260709 13:02:44.614114  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:44.632491  1538 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:44.634442  1611 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:44.620996  1643 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:44.560257  2342 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:02:44.631759  1537 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:44.621987  1642 raft_consensus.cc:1275] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:44.638967  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:44.567262  3894 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:44.632913  1645 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
W20260709 13:02:44.609150  3933 log.cc:927] Time spent T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:02:44.639865  1644 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
W20260709 13:02:44.605592  3939 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:02:44.653273  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:44.639695  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:44.643697  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:44.654770  1646 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:44.655959  1640 raft_consensus.cc:2410] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:02:44.648810  2314 leader_election.cc:336] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.911s (SENT)
I20260709 13:02:44.656219  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:44.611719  3680 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.607980  3927 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:44.638273  3941 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:44.679569  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.690968  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.561565  3936 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:44.661185  1611 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:44.660861  3791 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:44.661923  3732 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:44.663331  3837 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:44.723976  3837 raft_consensus.cc:697] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:44.612341  3940 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.664018  3688 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:02:44.664467  3896 consensus_meta.cc:287] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.525s	user 0.011s	sys 0.003s
I20260709 13:02:44.638027  3944 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:44.671445  3680 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:44.649608  3943 raft_consensus.cc:2764] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:02:44.630492  3947 consensus_queue.cc:1059] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.087s
I20260709 13:02:44.718523  3945 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:44.719004  3805 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:44.719195  1687 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:44.726074  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 101876us
W20260709 13:02:44.751165  1538 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.771s (SENT)
I20260709 13:02:44.752292  1538 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:44.640928  3720 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:44.733918  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:44.759645  1640 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:44.733958  2313 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.964s (SENT) [suppressed 33 similar messages]
W20260709 13:02:44.753773  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.754849  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:44.763211  1646 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:44.742125  3944 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:44.756435  3680 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:44.758237  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.732748  3936 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:44.767710  1538 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.808s (SENT)
I20260709 13:02:44.770903  3805 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:44.771693  3941 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:44.741048  3896 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:44.773231  2341 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
W20260709 13:02:44.775063  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.781711  2340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 271) took 1072 ms. Trace:
I20260709 13:02:44.786808  1538 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:44.741503  3688 raft_consensus.cc:697] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:02:44.795605  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 76541.2us
I20260709 13:02:44.725426  3837 consensus_queue.cc:237] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:44.798010  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 140551us
W20260709 13:02:44.805559  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 144536us
I20260709 13:02:44.812093  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:44.812949  1611 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:44.813242  1685 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:44.790572  2342 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55890 (request call id 229) took 1893 ms (client timeout 2151 ms). Trace:
W20260709 13:02:44.808893  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.789494  3791 raft_consensus.cc:3055] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:44.785795  3944 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:44.818477  1640 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:44.824221  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:44.826850  2347 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:44.732528  3940 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:44.827778  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:44.830087  1713 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:44.824288  1564 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:44.830436  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:44.831461  1714 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:44.833068  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.834177  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.835320  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:44.836400  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 68742.6us
W20260709 13:02:44.837703  1538 leader_election.cc:336] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.978s (SENT)
I20260709 13:02:44.838667  1717 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:44.810158  3805 raft_consensus.cc:697] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:44.811060  3935 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:44.832864  1686 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:44.845081  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:44.814720  3951 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:44.844965  1538 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:44.824088  3537 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:44.875860  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:44.868932  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:44.877084  1611 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:44.876560  1720 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:44.877189  1568 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:44.875897  2347 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:02:44.880874  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:44.881660  1640 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:44.882644  1567 raft_consensus.cc:1275] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
I20260709 13:02:44.869350  3805 consensus_queue.cc:237] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:44.858513  1713 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:02:44.825371  3791 raft_consensus.cc:740] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:44.854553  1565 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
W20260709 13:02:44.862006  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 47617us
I20260709 13:02:44.887522  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:44.894081  1714 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:02:44.899899  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:44.900313  1572 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:44.876523  2313 leader_election.cc:336] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.911s (SENT)
I20260709 13:02:44.901165  2313 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:44.814517  3877 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:44.871232  3537 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.902810  3537 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:44.896725  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 61637.2us
I20260709 13:02:44.903878  1687 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:44.844184  3936 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:44.906030  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:44.912364  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.915040  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:44.916236  1718 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:44.901907  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:44.918252  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:44.919629  1565 raft_consensus.cc:2393] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:44.921286  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:44.932793  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:44.933523  1646 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:02:44.937937  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:44.938738  1567 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:44.939170  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:44.939850  1565 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:44.944932  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:44.945961  1640 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
W20260709 13:02:44.816565  2342 rpcz_store.cc:269] 0709 13:02:42.897506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.913581 (+ 16075us) service_pool.cc:224] Handling call
0709 13:02:44.790257 (+1876676us) spinlock_profiling.cc:243] Waited 933 ms on lock 0x7b5400600254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:44.790472 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2146112256}
I20260709 13:02:44.904570  3927 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:02:44.950654  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 113025us
I20260709 13:02:44.905040  3879 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:44.979583  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:44.980228  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 103305us
I20260709 13:02:44.980463  1714 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:44.934826  3865 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:45.008241  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:45.009752  1640 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:02:44.990972  1572 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:02:44.981348  1611 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:45.017994  3936 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:45.018828  3936 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.019377  3936 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:44.995080  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:45.042305  3947 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:02:44.979984  3950 log.cc:927] Time spent T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:02:44.937556  3953 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:45.070127  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:45.077486  1718 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:02:45.085042  1646 raft_consensus.cc:2410] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
W20260709 13:02:45.100445  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 89223.3us
W20260709 13:02:45.100790  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 50642.2us
W20260709 13:02:45.107486  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 203701us
W20260709 13:02:45.108292  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.102452  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:45.108487  1687 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:45.113128  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:44.897178  3935 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:45.114235  1714 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:02:45.123450  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 56614.7us
I20260709 13:02:45.139448  1567 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:45.141942  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:45.142448  1538 leader_election.cc:336] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.118s (SENT)
I20260709 13:02:45.080101  3953 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.147037  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:45.150216  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:45.168213  1567 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:45.162312  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:45.171017  1611 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:45.171762  1644 raft_consensus.cc:2393] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:45.162415  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:45.142683  1639 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:44.937937  3688 consensus_queue.cc:237] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:45.090732  3894 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:45.180480  1645 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:45.184098  1565 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:45.172360  1611 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:45.104334  3720 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:45.202765  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:45.189982  1646 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:02:45.208250  1611 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:45.212658  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:02:45.149984  3825 consensus_meta.cc:287] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.824s	user 0.018s	sys 0.001s
I20260709 13:02:45.215088  1640 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:45.216557  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:45.225139  1645 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:45.162701  3956 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:45.256127  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.227311  1537 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:45.185272  3953 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:45.302644  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 75420.6us
I20260709 13:02:45.307770  1537 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:45.218297  3947 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:45.201710  3945 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:45.214406  3896 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:45.320140  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.322325  1537 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:45.325295  3961 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:45.270507  3825 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:45.144224  3791 consensus_queue.cc:260] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:45.271260  3927 consensus_queue.cc:1059] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.126s
I20260709 13:02:44.800194  2340 rpcz_store.cc:276] 0709 13:02:43.710462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.710800 (+   338us) service_pool.cc:224] Handling call
0709 13:02:43.711547 (+   747us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:43.712891 (+  1344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:43.712928 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:43.713548 (+   620us) log.cc:844] Serialized 26 byte log entry
0709 13:02:44.451788 (+738240us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:44.465450 (+ 13662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.484436 (+ 18986us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:44.779604 (+295168us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b2400310a78. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245c1e9c4 00007f8245c1cacb 00007f8245dd7dc9 00007f8245dd7a55 00007f8245dd6fb0 00007f8248a4d488 00007f8248a4dbe8 00007f8248a25398 00007f8248a2076a 00007f8248a1fbb6
0709 13:02:44.781383 (+  1779us) raft_consensus.cc:1685] finished
0709 13:02:44.781411 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.781621 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":47211,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"spinlock_wait_cycles":43129600,"thread_start_us":695770,"threads_started":1,"wal-append.queue_time_us":735050}
W20260709 13:02:45.405448  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 297027us
I20260709 13:02:45.327526  1537 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:45.326675  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.405598  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:45.422048  1714 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:45.416877  1645 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:02:45.328533  3720 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:02:45.390249  3954 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.300s	user 0.000s	sys 0.002s
W20260709 13:02:45.404119  3958 log.cc:927] Time spent T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
I20260709 13:02:45.328791  3951 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:45.325966  3945 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.465819  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:45.328958  3961 raft_consensus.cc:697] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 6 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:02:45.476365  1685 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.006s	sys 0.009s
W20260709 13:02:45.491760  1685 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.006s	sys 0.009s
I20260709 13:02:45.479538  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:45.480757  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:45.491922  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.477517  1645 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:45.476359  3951 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.439248  3877 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:45.541528  1565 raft_consensus.cc:2393] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:45.542354  1639 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:45.542708  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 642902us
W20260709 13:02:45.543236  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.441107  3732 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:45.216085  3879 raft_consensus.cc:697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:45.476672  3894 raft_consensus.cc:3055] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:45.524797  3951 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:45.468343  3940 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:45.627221  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 205635us
I20260709 13:02:45.585798  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:45.637233  1644 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:45.578176  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:45.641047  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:45.597250  1716 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.008s	sys 0.013s
W20260709 13:02:45.644100  1716 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.009s	sys 0.013s
W20260709 13:02:45.613570  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 286126us
W20260709 13:02:45.648654  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:45.628669  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.568241  1714 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:45.493124  3945 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:45.639272  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 94004.3us
I20260709 13:02:45.524137  3830 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:45.490885  3649 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:45.552959  3960 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:45.689111  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:45.589175  3894 raft_consensus.cc:740] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:45.678510  1687 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:02:45.673503  3936 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.653s	user 0.016s	sys 0.003s
W20260709 13:02:45.720695  1568 consensus_meta.cc:287] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.842s	user 0.020s	sys 0.004s
W20260709 13:02:45.722728  1720 consensus_meta.cc:287] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.814s	user 0.014s	sys 0.000s
I20260709 13:02:45.724875  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:45.738209  1568 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:45.738543  1720 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
W20260709 13:02:45.726733  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 584329us
I20260709 13:02:45.738858  1721 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:45.740986  1538 leader_election.cc:336] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.196s (SENT)
I20260709 13:02:45.726192  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:45.742115  1640 raft_consensus.cc:2376] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5. Re-sending same reply.
I20260709 13:02:45.742420  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:45.743345  1713 raft_consensus.cc:2393] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:45.755483  1645 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:45.757208  1714 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:45.748957  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:45.753062  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.759359  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:45.760612  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.750118  1644 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:02:45.698063  3978 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.760027  1646 raft_consensus.cc:2376] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5. Re-sending same reply.
I20260709 13:02:45.614475  3732 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:45.772627  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:45.786612  1644 raft_consensus.cc:2410] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 13:02:45.724191  3791 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:45.728844  3947 raft_consensus.cc:697] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 5 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:45.790184  3894 consensus_queue.cc:260] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:45.792277  3894 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:45.794099  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 53185.9us
W20260709 13:02:45.794888  1538 leader_election.cc:336] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.234s (SENT)
I20260709 13:02:45.795258  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:45.796666  3537 consensus_meta.cc:287] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.889s	user 0.015s	sys 0.000s
W20260709 13:02:45.797046  1538 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.603s (SENT)
I20260709 13:02:45.768167  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:02:45.804327  3926 consensus_meta.cc:287] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.154s	user 0.018s	sys 0.000s
I20260709 13:02:45.805876  3830 consensus_queue.cc:1048] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:02:45.807813  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 170796us
I20260709 13:02:45.808784  1687 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:45.710216  3970 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:02:45.811254  3969 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:02:45.819901  3940 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:45.821794  3879 consensus_queue.cc:237] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:45.824155  3953 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:45.827208  3953 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:45.827345  3956 raft_consensus.cc:2764] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 13:02:45.828137  3953 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:45.828641  3953 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.752449  3936 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:45.843274  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 46254.5us
W20260709 13:02:45.856038  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.860973  1721 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
W20260709 13:02:45.861826  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:45.862600  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.744516  3960 raft_consensus.cc:697] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 6 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:02:45.867192  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:45.879730  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 71031.8us
I20260709 13:02:45.797236  3537 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:45.883458  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:45.887125  2314 leader_election.cc:336] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.524s (SENT)
I20260709 13:02:45.890375  1644 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:45.894258  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:45.914393  1646 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:02:45.906960  1687 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:45.829070  3953 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:45.805476  3926 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:45.882371  3970 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:45.941457  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:45.940378  3817 consensus_meta.cc:287] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.383s	user 0.014s	sys 0.010s
I20260709 13:02:45.829061  3978 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:45.812783  3961 consensus_queue.cc:237] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:45.948282  3962 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.536s	user 0.003s	sys 0.000s
W20260709 13:02:45.977470  3944 consensus_meta.cc:287] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.145s	user 0.017s	sys 0.003s
I20260709 13:02:45.879837  3965 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:45.998138  3978 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:46.003181  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1400) took 1164 ms. Trace:
I20260709 13:02:46.005420  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1399) took 1167 ms. Trace:
I20260709 13:02:46.023924  1573 rpcz_store.cc:276] 0709 13:02:44.838127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.838291 (+   164us) service_pool.cc:224] Handling call
0709 13:02:44.838668 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.861114 (+ 22446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.861153 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.898324 (+ 37171us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.227371 (+329047us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:45.227461 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.227715 (+   254us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.005180 (+777465us) raft_consensus.cc:1685] finished
0709 13:02:46.005201 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.005343 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16746,"raft.queue_time_us":43338,"raft.run_cpu_time_us":356,"raft.run_wall_time_us":11437,"thread_start_us":241857,"threads_started":1,"wal-append.queue_time_us":993341}
W20260709 13:02:46.006814  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 126126us
I20260709 13:02:46.010612  3989 raft_consensus.cc:2764] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
W20260709 13:02:46.011179  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.013303  3977 raft_consensus.cc:1064] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:02:46.016256  3935 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:46.016793  3732 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:46.016738  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:45.944690  3970 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:46.021696  1639 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:02:45.943570  3817 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:46.022745  1571 rpcz_store.cc:276] 0709 13:02:44.838529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.838683 (+   154us) service_pool.cc:224] Handling call
0709 13:02:44.860060 (+ 21377us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:45.120838 (+260778us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.121024 (+   186us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:45.121767 (+   743us) log.cc:844] Serialized 52 byte log entry
0709 13:02:45.724789 (+603022us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:45.724826 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.724963 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.002922 (+277959us) raft_consensus.cc:1685] finished
0709 13:02:46.002950 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.003115 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34595,"raft.queue_time_us":50543,"raft.run_cpu_time_us":427,"raft.run_wall_time_us":29171,"thread_start_us":594939,"threads_started":2,"wal-append.queue_time_us":816379}
I20260709 13:02:45.979751  3825 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:46.041499  1687 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:46.080543  1687 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:46.072350  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 374409us
W20260709 13:02:46.042845  1569 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.000s	sys 0.010s
W20260709 13:02:46.082741  1569 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.000s	sys 0.010s
W20260709 13:02:46.042495  1718 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.005s	sys 0.005s
W20260709 13:02:46.083490  1718 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.940s	user 0.006s	sys 0.005s
I20260709 13:02:46.073210  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:46.084738  1721 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
W20260709 13:02:46.043157  1611 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.004s	sys 0.006s
W20260709 13:02:46.085492  1611 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.004s	sys 0.006s
W20260709 13:02:46.085858  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 877685us
W20260709 13:02:46.087081  1538 connection.cc:256] RPC call timeout handler was delayed by -1.232197479999968s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22250000000000003s for a response
I20260709 13:02:46.089882  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:02:46.041468  3977 raft_consensus.cc:1068] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: Unable to promote non-voter cf2f71592be8465fb18e36e15d0b2f89: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:02:46.063876  3837 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.003s	sys 0.007s
I20260709 13:02:46.039158  3970 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:46.127799  1719 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:46.128031  1538 leader_election.cc:336] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.931s (SENT)
I20260709 13:02:46.075039  3935 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.138298  1538 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:46.141269  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 54836us
W20260709 13:02:46.151540  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 71085.1us
W20260709 13:02:46.168753  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.123649  3837 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.003s	sys 0.007s
W20260709 13:02:46.183095  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 98838.4us
I20260709 13:02:46.122378  3732 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:46.214076  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1948) took 1337 ms. Trace:
I20260709 13:02:46.214578  1718 rpcz_store.cc:276] 0709 13:02:44.876654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.098288 (+221634us) service_pool.cc:224] Handling call
0709 13:02:45.098778 (+   490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.099086 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:45.143192 (+ 44106us) log.cc:844] Serialized 14 byte log entry
0709 13:02:46.213533 (+1070341us) log.cc:844] Serialized 14 byte log entry
0709 13:02:46.213720 (+   187us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:46.213763 (+    43us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:46.213812 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.213882 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.214017 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":129085,"prepare.queue_time_us":79744,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":75,"thread_start_us":940351,"threads_started":1}
I20260709 13:02:46.217607  1687 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:46.184620  2313 leader_election.cc:336] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.524s (SENT)
I20260709 13:02:46.219787  2313 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:46.087401  3989 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:46.186074  1573 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:46.227193  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.227560  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:46.191112  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:46.036820  3944 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:46.181073  3720 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.002s	sys 0.009s
W20260709 13:02:46.194017  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 182231us
I20260709 13:02:46.204612  1685 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:46.184075  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.211930  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:46.245728  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:46.139554  3936 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:46.190002  3983 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.259115  1573 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:02:46.260183  1613 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.026s	sys 0.044s
W20260709 13:02:46.260692  1613 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.026s	sys 0.044s
I20260709 13:02:46.267942  1715 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
W20260709 13:02:46.271270  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.273995  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.247745  3720 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.002s	sys 0.010s
W20260709 13:02:46.260748  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 56564.8us
I20260709 13:02:46.277042  1566 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:46.429032  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:46.313493  3720 raft_consensus.cc:697] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:46.268483  1715 pending_rounds.cc:85] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 1
I20260709 13:02:46.254967  3830 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:46.458462  1715 pending_rounds.cc:107] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:02:46.460311  1715 raft_consensus.cc:2889] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:46.331274  4003 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:02:46.470185  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.258960  3537 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:46.220920  3935 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:46.259243  3982 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.312932  3956 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:46.354715  1685 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:46.507453  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 152890us
I20260709 13:02:46.211639  3977 consensus_queue.cc:1048] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.094s
W20260709 13:02:46.477767  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.466225  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:46.512578  1571 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:46.513491  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 286726us
I20260709 13:02:46.243713  3688 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:02:46.515596  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 387319us
I20260709 13:02:46.522903  1685 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:46.260846  3975 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.173s
I20260709 13:02:46.259693  3987 consensus_queue.cc:1059] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.223s
I20260709 13:02:46.040427  3965 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:46.481498  3953 consensus_meta.cc:287] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.000s
W20260709 13:02:46.478904  3976 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.490s	user 0.001s	sys 0.001s
W20260709 13:02:46.490787  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 306311us
W20260709 13:02:46.495389  3979 log.cc:927] Time spent T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.351s	user 0.000s	sys 0.005s
I20260709 13:02:46.014528  3960 consensus_queue.cc:237] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:46.467607  3926 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:46.466792  4000 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:46.531244  1538 leader_election.cc:336] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.078s (SENT)
W20260709 13:02:46.548969  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.549489  1538 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:46.536729  1685 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:46.295698  3983 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:46.375612  3936 consensus_queue.cc:1059] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.132s
I20260709 13:02:46.516197  3956 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.531898  3989 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:46.082090  3947 consensus_queue.cc:237] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:46.465692  3817 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:46.571775  1687 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:46.543380  3537 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:46.543815  3985 raft_consensus.cc:3055] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:46.573704  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 259) took 2166 ms. Trace:
W20260709 13:02:46.574322  1685 connection.cc:256] RPC call timeout handler was delayed by -1.6623102739999922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1637s for a response
I20260709 13:02:46.586041  3877 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:46.482142  4003 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:46.538303  3953 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:46.617414  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.618450  1538 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.187s (SENT)
W20260709 13:02:46.608601  4002 log.cc:927] Time spent T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 13:02:46.568531  3941 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:46.623070  3981 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:46.612287  3985 raft_consensus.cc:740] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:46.569404  3688 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:02:46.627641  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 55834.6us
I20260709 13:02:46.629081  3977 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:46.698889  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.630247  3994 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.269s	user 0.000s	sys 0.002s
W20260709 13:02:46.634505  3999 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:02:46.635319  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1939) took 1814 ms. Trace:
I20260709 13:02:46.708833  1719 rpcz_store.cc:276] 0709 13:02:44.821069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.821220 (+   151us) service_pool.cc:224] Handling call
0709 13:02:44.830606 (+  9386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.138523 (+1307917us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.634927 (+496404us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54002c60d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:46.635242 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2978407168}
W20260709 13:02:46.636422  3934 log.cc:927] Time spent T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:02:46.749719  1719 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:46.641965  3537 raft_consensus.cc:697] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:46.752112  3537 consensus_queue.cc:237] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:46.643766  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:46.754267  2340 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 6: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
W20260709 13:02:46.646224  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.525720  3982 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:46.650038  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.758463  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.656806  2314 leader_election.cc:336] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.964s (SENT)
I20260709 13:02:46.554123  3975 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:46.659806  1567 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:02:46.762097  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 105351us
I20260709 13:02:46.763849  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:46.764632  1567 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:46.762385  3975 raft_consensus.cc:697] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:46.767992  3982 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:46.767155  3975 consensus_queue.cc:237] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:46.589206  3877 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.770066  3877 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:46.772581  3877 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:46.761421  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.579424  1564 rpcz_store.cc:276] 0709 13:02:44.397034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.397168 (+   134us) service_pool.cc:224] Handling call
0709 13:02:44.420475 (+ 23307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.830718 (+410243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.563385 (+1732667us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b5400013954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:46.563795 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":928283520}
I20260709 13:02:46.562294  3956 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:46.667706  1571 raft_consensus.cc:2410] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:02:46.676095  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.554522  4000 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:46.616889  3989 raft_consensus.cc:697] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:02:46.676630  3951 consensus_meta.cc:287] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.989s	user 0.014s	sys 0.000s
W20260709 13:02:46.677984  3980 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.542s	user 0.000s	sys 0.002s
I20260709 13:02:46.718021  3965 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:46.718910  4003 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:46.677692  3720 consensus_queue.cc:237] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:46.720693  3935 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:46.722150  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1407) took 1879 ms. Trace:
I20260709 13:02:46.722082  3990 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:02:46.725258  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 149452us
W20260709 13:02:46.841598  1538 connection.cc:256] RPC call timeout handler was delayed by -1.0314974799999845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24480000000000002s for a response
W20260709 13:02:46.842430  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.707782  3817 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:02:46.698710  3977 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:46.731423  3952 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:46.725620  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1410) took 1882 ms. Trace:
I20260709 13:02:46.857251  1565 rpcz_store.cc:276] 0709 13:02:44.843584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.704342 (+860758us) service_pool.cc:224] Handling call
0709 13:02:45.726771 (+ 22429us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:45.727166 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.727192 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:45.779721 (+ 52529us) log.cc:844] Serialized 52 byte log entry
0709 13:02:46.176564 (+396843us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:46.176774 (+   210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.176938 (+   164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.725398 (+548460us) raft_consensus.cc:1685] finished
0709 13:02:46.725418 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.725556 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":229,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":31242,"thread_start_us":293155,"threads_started":1,"wal-append.queue_time_us":839943}
I20260709 13:02:46.735126  1717 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:46.736459  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:46.861268  1718 raft_consensus.cc:2368] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7.
W20260709 13:02:46.738384  3894 consensus_meta.cc:287] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.945s	user 0.015s	sys 0.003s
I20260709 13:02:46.718008  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1401) took 1879 ms. Trace:
I20260709 13:02:46.869932  1569 rpcz_store.cc:276] 0709 13:02:44.838965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.839095 (+   130us) service_pool.cc:224] Handling call
0709 13:02:44.874725 (+ 35630us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:45.097450 (+222725us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.097494 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:45.124468 (+ 26974us) log.cc:844] Serialized 52 byte log entry
0709 13:02:46.083187 (+958719us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:46.083230 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.083299 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.717738 (+634439us) raft_consensus.cc:1685] finished
0709 13:02:46.717759 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.717937 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9390,"raft.queue_time_us":51728,"raft.run_cpu_time_us":594,"raft.run_wall_time_us":17621,"thread_start_us":880010,"threads_started":2,"wal-append.queue_time_us":1366712}
I20260709 13:02:46.742806  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:46.745426  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:46.729516  3988 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
W20260709 13:02:46.781816  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.874714  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.876296  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:46.782228  2312 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.015s	sys 0.060s
W20260709 13:02:46.878008  2312 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.015s	sys 0.061s
I20260709 13:02:46.744946  3983 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:46.831965  1713 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:46.833950  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 58458.9us
W20260709 13:02:46.890386  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.837235  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.893496  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.846109  1567 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
W20260709 13:02:46.847285  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 72824.7us
I20260709 13:02:46.840240  1570 rpcz_store.cc:276] 0709 13:02:44.842878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.272454 (+429576us) service_pool.cc:224] Handling call
0709 13:02:45.321487 (+ 49033us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.896296 (+574809us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.896433 (+   137us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.934099 (+ 37666us) log.cc:844] Serialized 26 byte log entry
0709 13:02:46.181876 (+247777us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:46.181929 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.182032 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.721928 (+539896us) raft_consensus.cc:1685] finished
0709 13:02:46.721950 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.722087 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44396,"raft.queue_time_us":80762,"raft.run_cpu_time_us":433,"raft.run_wall_time_us":43737,"thread_start_us":667003,"threads_started":2,"wal-append.queue_time_us":294162}
I20260709 13:02:46.826372  3982 consensus_queue.cc:1059] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:02:46.827448  3941 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.871025  1564 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:46.872586  1720 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:46.739128  3985 consensus_queue.cc:260] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:46.835952  3963 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.444s	user 0.014s	sys 0.012s
I20260709 13:02:46.883121  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:46.897938  3791 consensus_meta.cc:287] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.036s	user 0.015s	sys 0.004s
I20260709 13:02:46.887682  1537 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:46.888332  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.697183  3688 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:46.807050  3877 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.847908  3680 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:46.864342  3977 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:46.993472  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:46.995608  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:46.996615  2344 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:02:47.003708  3989 consensus_queue.cc:237] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:46.899636  1611 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:46.854684  3965 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:47.023665  3965 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:46.885752  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 144847us
I20260709 13:02:47.028445  1719 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:46.828954  3951 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.040242  4001 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:47.040961  4001 raft_consensus.cc:697] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:47.041325  1720 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:02:46.979570  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:46.980840  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:46.900560  3825 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:46.920027  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:46.901607  3967 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.514s	user 0.020s	sys 0.051s
W20260709 13:02:47.014771  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.018828  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 119279us
I20260709 13:02:47.006072  3947 raft_consensus.cc:3055] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:47.031420  1640 raft_consensus.cc:2410] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:02:47.031505  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 153890us
I20260709 13:02:46.918612  3941 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.006559  3877 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:47.006593  3935 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.993738  4013 consensus_queue.cc:1059] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.123s
I20260709 13:02:47.048856  1569 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
W20260709 13:02:46.918892  3963 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.593s	user 0.015s	sys 0.013s
I20260709 13:02:47.044685  4001 consensus_queue.cc:237] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:47.066596  1538 leader_election.cc:336] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 3.229s (SENT)
W20260709 13:02:47.160656  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.264421  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:47.138494  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.262794  2313 leader_election.cc:336] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.964s (SENT)
W20260709 13:02:47.076785  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.269207  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:47.272454  2313 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:47.270254  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.275844  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 209323us
I20260709 13:02:47.274837  3993 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.278971  3993 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.275002  3894 consensus_queue.cc:1059] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:02:47.282107  4012 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:47.282670  3982 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:47.283267  3982 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:47.284623  3935 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.265427  3947 raft_consensus.cc:740] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:47.286619  3944 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:47.028025  3945 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.300s	user 0.009s	sys 0.012s
I20260709 13:02:47.298202  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:47.222275  4017 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:02:47.301851  1719 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:02:47.264935  3967 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.877s	user 0.024s	sys 0.078s
W20260709 13:02:47.300534  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.303787  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.054152  3936 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:47.108105  3952 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.093248  3951 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:47.205912  4000 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:47.032290  3680 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.265436  3985 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:47.325062  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.329272  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:47.321884  3965 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:47.329667  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:47.330006  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:47.327298  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.342463  1717 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:02:47.307158  3945 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:47.311473  3688 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:47.347853  3982 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:47.331449  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:47.321684  3977 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.353358  3977 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:47.325881  3970 consensus_meta.cc:287] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.140s	user 0.013s	sys 0.007s
I20260709 13:02:47.356979  3970 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.358240  3945 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:47.359875  3970 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:47.394753  3935 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:47.399670  3877 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:47.360383  3947 consensus_queue.cc:260] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.365336  3936 raft_consensus.cc:697] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:47.365984  3952 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:47.370271  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.377904  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.411931  3964 consensus_queue.cc:1059] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:47.381268  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:47.413074  1713 raft_consensus.cc:2376] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6. Re-sending same reply.
I20260709 13:02:47.379163  3953 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:47.293079  3825 raft_consensus.cc:697] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:47.408131  3817 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.413559  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:47.414162  3817 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.414013  2313 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:47.379160  3956 consensus_meta.cc:287] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.537s	user 0.013s	sys 0.000s
I20260709 13:02:47.413550  3964 raft_consensus.cc:3055] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:47.355257  3680 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:47.429852  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.430647  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.417073  2313 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:47.435592  3720 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.472085  1720 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
W20260709 13:02:47.472966  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.474624  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:47.475517  1713 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:47.475878  1722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:02:47.476414  1722 raft_consensus.cc:606] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:47.476728  1722 raft_consensus.cc:614] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:47.474493  1538 leader_election.cc:336] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.894s (SENT)
I20260709 13:02:47.483213  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:47.483963  1639 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:47.484792  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:47.430886  3960 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.428642  3964 raft_consensus.cc:740] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:47.442095  4005 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:47.442492  4016 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:47.490278  2313 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:47.415947  3877 consensus_queue.cc:1059] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:02:47.492343  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:47.437589  3688 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:47.494839  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:47.496253  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:47.580019  1646 raft_consensus.cc:2393] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 6: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:47.496659  4013 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:47.494912  1538 leader_election.cc:336] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.990s (SENT)
I20260709 13:02:47.498998  3952 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:47.506024  4026 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.584584  4026 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.587459  4026 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:47.512589  3825 consensus_queue.cc:237] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:47.513252  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1614) took 1322 ms. Trace:
I20260709 13:02:47.600471  1568 rpcz_store.cc:276] 0709 13:02:46.190681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.190836 (+   155us) service_pool.cc:224] Handling call
0709 13:02:47.513037 (+1322201us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400406ad4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:47.513190 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2773343616}
I20260709 13:02:47.514626  3944 raft_consensus.cc:3055] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:47.493183  3936 consensus_queue.cc:237] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.526460  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:47.528609  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:47.531123  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1949) took 1786 ms. Trace:
I20260709 13:02:47.531972  4010 raft_consensus.cc:3055] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:47.534727  3987 raft_consensus.cc:3055] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:47.540548  4003 consensus_queue.cc:1059] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:47.540966  4008 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:47.542234  1569 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:02:47.486263  3953 consensus_queue.cc:1059] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 13:02:47.544389  3879 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:47.545395  3817 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:47.547694  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:47.550304  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:47.655457  3953 raft_consensus.cc:3055] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:47.655969  3953 raft_consensus.cc:740] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:47.550676  2340 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:47.611133  4010 raft_consensus.cc:740] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:47.553051  3947 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:47.555721  3985 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.564388  3993 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:47.565054  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:47.565868  3975 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:47.567209  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:47.569335  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 267) took 2948 ms. Trace:
I20260709 13:02:47.436596  3720 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.579808  1686 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:47.497453  1644 raft_consensus.cc:1275] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 13:02:47.582273  1642 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:47.583513  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:47.512148  3992 log.cc:927] Time spent T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.819s	user 0.004s	sys 0.000s
I20260709 13:02:47.589156  3964 consensus_queue.cc:260] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:47.570922  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 296194us
I20260709 13:02:47.492189  4005 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.603355  1722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:02:47.430019  3956 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.606316  1640 rpcz_store.cc:276] 0709 13:02:45.744685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.748571 (+  3886us) service_pool.cc:224] Handling call
0709 13:02:47.530915 (+1782344us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54000a73d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:47.531061 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2928036736}
I20260709 13:02:47.443774  3941 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:47.635669  3987 raft_consensus.cc:740] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:47.666234  1722 raft_consensus.cc:606] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:47.666900  1722 raft_consensus.cc:614] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:47.666786  3987 consensus_queue.cc:260] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.668900  3987 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:47.591635  3970 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.669541   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:47.671197  3956 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:47.670233  3970 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:47.638430  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 143516us
W20260709 13:02:47.672626  1538 leader_election.cc:336] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.992s (SENT)
I20260709 13:02:47.492432  3960 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.645030  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:47.673957  4005 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:47.645015  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:47.675856  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 96145.2us
I20260709 13:02:47.673089  1538 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:47.613118  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:47.602622  3944 raft_consensus.cc:740] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:47.665340  3964 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:47.665716  1564 raft_consensus.cc:2410] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:47.651731  2341 raft_consensus.cc:1217] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Deduplicated request from leader. Original: 1.4->[3.5-5.6]   Dedup: 3.5->[5.6-5.6]
I20260709 13:02:47.698292  2341 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 5. Current term is 6. Ops: [3.5-5.6]
I20260709 13:02:47.698606  4003 raft_consensus.cc:3055] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:47.698777  1715 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:47.679606  2347 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:47.702214  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:47.703539  1718 raft_consensus.cc:1217] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Deduplicated request from leader. Original: 1.1->[3.2-3.2]   Dedup: 3.2->[]
I20260709 13:02:47.657289  3953 consensus_queue.cc:260] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.704409  1718 raft_consensus.cc:1240] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:02:47.680989  1639 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:47.682366  1644 pending_rounds.cc:85] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 2
I20260709 13:02:47.706980  1644 pending_rounds.cc:107] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:02:47.707296  1644 raft_consensus.cc:2889] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:47.683099  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.683431  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:47.681332  1566 rpcz_store.cc:276] 0709 13:02:44.621225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.621523 (+   298us) service_pool.cc:224] Handling call
0709 13:02:44.638529 (+ 17006us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.421290 (+1782761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.569079 (+1147789us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b54002251d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:47.569276 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3767954816}
I20260709 13:02:47.658656  4010 consensus_queue.cc:260] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:47.710806  3830 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:47.683838  1646 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:47.659809  3879 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.674641  1537 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:47.699039  4003 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:47.695585  1713 raft_consensus.cc:1217] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[3.2-3.2]   Dedup: 3.2->[]
I20260709 13:02:47.717867  1713 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:02:47.718214  4003 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:47.718750  3944 consensus_queue.cc:260] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.719883  4003 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:47.723183  1713 raft_consensus.cc:1217] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Deduplicated request from leader. Original: 2.2->[4.3-4.3]   Dedup: 4.3->[]
W20260709 13:02:47.724022  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.725116  1713 raft_consensus.cc:1240] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:47.720202  4010 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:47.720466   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:47.698911  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:47.730625  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 58027.9us
I20260709 13:02:47.720297  3944 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:47.732167  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 57600.7us
I20260709 13:02:47.733870  3985 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.735916  3680 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:47.736793  3680 raft_consensus.cc:697] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:47.738060  3680 consensus_queue.cc:237] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.698423  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:47.739426  1717 raft_consensus.cc:2393] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:47.740175  3975 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.597564  3688 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.743758  1717 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:47.744741  3688 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:47.732909  3877 consensus_queue.cc:1175] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "066ee0109aba4465ba17fa65e14ef504" responder_term: 4 status { last_received { term: 4 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 4 index: 6 } }
I20260709 13:02:47.747948  1717 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:02:47.750248  1718 raft_consensus.cc:1240] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:47.750131  1573 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:47.754055  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:47.750178  3894 consensus_queue.cc:1059] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:02:47.757484  1717 raft_consensus.cc:1240] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:47.699331  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:47.759367  1719 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:47.760565  3960 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:47.750957  3993 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.762382  3993 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:47.729725  3943 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:47.779276  1719 raft_consensus.cc:1240] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: [4.2-4.2]
W20260709 13:02:47.779692  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.712558  3830 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.781807  1722 raft_consensus.cc:1217] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Deduplicated request from leader. Original: 2.2->[5.3-5.3]   Dedup: 5.3->[]
I20260709 13:02:47.782270  1722 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 13:02:47.786088  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.786633  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.797435  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.803181  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:47.803930  1713 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:47.806048  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:47.668375  3720 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:47.808409  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:47.809489  1722 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:47.815822  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:47.758015  3894 raft_consensus.cc:3055] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:47.817080  1717 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:47.820928  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:47.821762  1721 raft_consensus.cc:2393] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 6: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:47.779856  3688 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:47.819205  3894 raft_consensus.cc:740] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:47.839819  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:47.788053  3830 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:47.844731  1719 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:47.823269  1471 catalog_manager.cc:5697] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 5 to 6, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 6 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:47.847604  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:47.848381  1644 raft_consensus.cc:2368] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4.
I20260709 13:02:47.849005  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:47.851524  1721 raft_consensus.cc:2393] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:47.853686  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:47.856628  1716 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:02:47.860265  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.864672  1720 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:02:47.871315  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:47.843402  3688 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.875909  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:47.876871  1646 raft_consensus.cc:2376] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5. Re-sending same reply.
W20260709 13:02:47.879061  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.882905  3977 raft_consensus.cc:1064] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
W20260709 13:02:47.883571  3977 raft_consensus.cc:1068] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: Unable to promote non-voter cf2f71592be8465fb18e36e15d0b2f89: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:02:47.882776  1471 catalog_manager.cc:5697] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:47.885799  3970 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:47.885288  1471 catalog_manager.cc:5697] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 1 to 5, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 5 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:47.888814  1471 catalog_manager.cc:5697] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:47.877131  3688 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:47.905579  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49150.2us
I20260709 13:02:47.908588  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:47.918171  1713 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:47.919143  1722 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:47.924093  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:47.924345  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:47.925035  1640 raft_consensus.cc:2393] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:47.925288  1643 raft_consensus.cc:2368] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 8.
I20260709 13:02:47.927601  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:47.929162  1642 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:02:47.930208  1642 pending_rounds.cc:85] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 5
I20260709 13:02:47.930516  1642 pending_rounds.cc:107] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:02:47.930773  1642 raft_consensus.cc:2889] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:47.933861  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.840101  4008 consensus_queue.cc:1059] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:02:47.937407  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:47.944296  1641 raft_consensus.cc:2368] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 8.
I20260709 13:02:47.944427  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:47.937969  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:47.945338  1648 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:02:47.941377  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:47.936022  1722 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:47.943001  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:47.947744  1642 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:02:47.941135  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:47.948895  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:47.949970  1646 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:47.952025  2341 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:47.938005  1471 catalog_manager.cc:5697] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:47.952612  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:47.957461  1644 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:02:47.957998  1644 pending_rounds.cc:85] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 4
I20260709 13:02:47.958252  1644 pending_rounds.cc:107] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:02:47.958508  1644 raft_consensus.cc:2889] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:47.958814  1644 pending_rounds.cc:107] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 5.6
W20260709 13:02:47.958827  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.959664  1640 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
W20260709 13:02:47.959816  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.959054  1644 raft_consensus.cc:2889] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:47.926576  4026 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.927745  3985 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:47.912954  3936 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:47.968647  1471 catalog_manager.cc:5697] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: leader changed from <none> to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:02:47.977998  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:47.989157  1648 raft_consensus.cc:2368] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4.
I20260709 13:02:47.991151  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:47.991557  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:47.992692  1647 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:47.994674  1719 raft_consensus.cc:2410] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:47.978255  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:47.980805  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:47.981904  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:47.995790  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:47.995930  2341 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:47.996673  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:47.997309  4033 log.cc:927] Time spent T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:02:47.998009  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.072001  1648 raft_consensus.cc:2393] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:47.946435  3975 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:47.994238  3879 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:48.002172  3960 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:48.003567  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:48.003577  1643 raft_consensus.cc:1275] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:48.004313  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:48.004524  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:48.079003  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.080001  1642 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:47.940759  3894 consensus_queue.cc:260] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:47.992210  4026 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:48.009954  3936 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:48.013140  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.085824  1646 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:47.977185  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:47.943624  3720 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:47.945116  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:48.022840  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:48.025611  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.031381  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:48.028653  1468 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38662 (request call id 207) took 10060 ms. Trace:
I20260709 13:02:48.010076  3953 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:48.052042  1566 raft_consensus.cc:2376] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5. Re-sending same reply.
I20260709 13:02:48.088802  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:48.067425  1565 raft_consensus.cc:2393] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 7: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:48.068413  1641 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:47.990975  4005 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:48.073868  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.090046  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.094978  1643 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:48.090127  1713 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 7: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:48.091293  1647 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:48.027926  3993 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:48.105048  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.105816  1722 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:02:48.103091  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:48.117312  1648 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:48.083160  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:48.050552  1564 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:48.101585  1646 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:48.081285  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:48.104286  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.047688  4029 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:02:48.027182  4001 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 13:02:48.123441  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:48.127175  1642 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:48.044207  3941 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:02:48.140105  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:48.142681  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:48.091429  3975 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.149336  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.157317  1641 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:48.167959  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:48.169872  1569 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:48.172858  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.175017  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:48.090914  1468 rpcz_store.cc:276] 0709 13:02:37.959465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.014532 (+ 55067us) service_pool.cc:224] Handling call
0709 13:02:48.019913 (+10005381us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:43.717590 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:43.718413 (+   823us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:43.718477 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:02:43.718509 (+    32us) tablet.cc:662] Decoding operations
0709 13:02:43.720394 (+  1885us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:43.720485 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:43.720525 (+    40us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:02:43.722853 (+  2328us) tablet.cc:701] Row locks acquired
0709 13:02:43.722921 (+    68us) write_op.cc:260] PREPARE: finished
0709 13:02:43.723363 (+   442us) write_op.cc:270] Start()
0709 13:02:43.723560 (+   197us) write_op.cc:276] Timestamp: P: 1783602163723042 usec, L: 0
0709 13:02:43.723578 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:02:43.725065 (+  1487us) log.cc:844] Serialized 6051 byte log entry
0709 13:02:46.413846 (+2688781us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b180005eb40. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:02:47.218857 (+805011us) op_driver.cc:464] REPLICATION: finished
0709 13:02:47.424738 (+205881us) write_op.cc:301] APPLY: starting
0709 13:02:47.425017 (+   279us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:47.425689 (+   672us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:47.425750 (+    61us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:47.509968 (+ 84218us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:47.510339 (+   371us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=15
0709 13:02:47.510382 (+    43us) write_op.cc:312] APPLY: finished
0709 13:02:47.510841 (+   459us) log.cc:844] Serialized 107 byte log entry
0709 13:02:47.602491 (+ 91650us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:47.635599 (+ 33108us) write_op.cc:454] Released schema lock
0709 13:02:47.648897 (+ 13298us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:47.649442 (+   545us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:47.649670 (+   228us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":185158,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":201403,"apply.run_cpu_time_us":7096,"apply.run_wall_time_us":224739,"num_ops":15,"prepare.queue_time_us":530001,"prepare.run_cpu_time_us":32354,"prepare.run_wall_time_us":823372,"raft.queue_time_us":182016,"raft.run_cpu_time_us":104981,"raft.run_wall_time_us":404839,"replication_time_us":3405368,"spinlock_wait_cycles":56038912,"thread_start_us":1306954,"threads_started":4,"wal-append.queue_time_us":869423}]]}
I20260709 13:02:48.072883  3961 consensus_queue.cc:1048] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.072s
I20260709 13:02:48.199026  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:48.203763  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:48.207579  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.212750  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.215059  1642 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:48.220876  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:48.222057  1566 raft_consensus.cc:2393] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:48.224406  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:48.226143  1641 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:48.233892  1569 raft_consensus.cc:1240] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:48.237788  1687 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:48.241412  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.242472  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.255827  1687 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:48.259825  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.126312  3943 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.089s
I20260709 13:02:48.261397  1720 raft_consensus.cc:2410] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:48.269122  1646 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:48.122753  3720 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:48.209594  2344 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:48.290863  3941 consensus_queue.cc:1059] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:48.292659  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:48.293476  1643 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:48.298689  3877 raft_consensus.cc:3055] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:48.299288  3877 raft_consensus.cc:740] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:48.273108  3977 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:48.300395  3877 consensus_queue.cc:260] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:48.304590  3877 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:48.306838  4005 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:48.314359  3987 consensus_meta.cc:287] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.645s	user 0.009s	sys 0.005s
I20260709 13:02:48.315716  1640 raft_consensus.cc:2410] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:02:48.317080  4032 log.cc:927] Time spent T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.397s	user 0.003s	sys 0.001s
I20260709 13:02:48.298604  1647 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
W20260709 13:02:48.318483  4035 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 13:02:48.320014  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.320858  1647 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:48.322371  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:48.276086  1565 raft_consensus.cc:1275] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:48.210119  3849 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.278234  1648 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:48.246634  3993 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.327039  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:48.330325  4003 consensus_meta.cc:287] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.607s	user 0.010s	sys 0.003s
I20260709 13:02:48.331681  1643 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:02:48.372440  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.358569  1647 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:48.280125  3817 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:48.364754  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.377827  1720 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:48.323208  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.274056  1687 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:48.274978  3985 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:48.279497  3947 consensus_meta.cc:287] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.615s	user 0.008s	sys 0.004s
I20260709 13:02:48.175537  3975 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:48.368654  3944 consensus_meta.cc:287] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.636s	user 0.015s	sys 0.001s
I20260709 13:02:48.383945  2349 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
W20260709 13:02:48.363754  3964 consensus_meta.cc:287] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.664s	user 0.013s	sys 0.002s
I20260709 13:02:48.373484  1643 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:48.424556  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.425235  1643 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:48.426879  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:48.427512  1643 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:48.378671  1640 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:48.427902  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.407338  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.366838  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:48.429023  1646 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:48.293227  3941 consensus_queue.cc:1243] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer cf2f71592be8465fb18e36e15d0b2f89 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:02:48.421404  1644 raft_consensus.cc:2410] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:48.429031  1648 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:48.430035  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:48.421365  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 148097us
I20260709 13:02:48.430173  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.430788  1719 raft_consensus.cc:2393] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:48.279698  3935 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.103s
I20260709 13:02:48.431349  1722 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:48.367614  1642 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:48.420408  1641 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:02:48.448158  2347 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:02:48.452936  1687 leader_election.cc:400] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4.
I20260709 13:02:48.453590  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.456691  1642 raft_consensus.cc:2410] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:48.458027  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.459270  1640 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:48.460795  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.461619  1640 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:48.476259  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:48.481007  1648 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:48.404753  3936 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:48.506269  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.508750  1642 raft_consensus.cc:2376] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5. Re-sending same reply.
W20260709 13:02:48.510231  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 57521.3us
I20260709 13:02:48.512758  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:48.516207  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.578609  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.526120  4026 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:48.375470  3993 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:48.611842  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.530567  1573 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.008s	sys 0.031s
W20260709 13:02:48.612625  1573 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.008s	sys 0.031s
I20260709 13:02:48.612782  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.427568  3989 raft_consensus.cc:2764] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:02:48.613760  1713 raft_consensus.cc:2393] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:02:48.543006  1568 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.034s	sys 0.067s
W20260709 13:02:48.480027  4010 consensus_meta.cc:287] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.753s	user 0.010s	sys 0.005s
I20260709 13:02:48.545434  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.369764  3849 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:48.474994  3947 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.420643  3985 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:48.556586  1567 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.015s	sys 0.042s
I20260709 13:02:48.571681  1722 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:48.572525  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.575553  1646 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:48.580917  1640 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:02:48.628490  1567 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.015s	sys 0.042s
I20260709 13:02:48.628646  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.583356  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:48.630087  1642 raft_consensus.cc:2376] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5. Re-sending same reply.
I20260709 13:02:48.541888  3983 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.080s
I20260709 13:02:48.598075  1647 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:48.601692  1648 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
W20260709 13:02:48.609037  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.609546  1571 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.000s	sys 0.054s
W20260709 13:02:48.638190  1571 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.000s	sys 0.054s
I20260709 13:02:48.629307  1640 raft_consensus.cc:2376] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6. Re-sending same reply.
I20260709 13:02:48.639425  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:48.614461  1639 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.000s	sys 0.034s
I20260709 13:02:48.619980  1720 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:02:48.561887  3825 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.012s	sys 0.018s
I20260709 13:02:48.643692  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:48.626017  1570 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.018s	sys 0.016s
I20260709 13:02:48.631222  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.645200  1642 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:48.646064  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.646396  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.646690  1719 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:48.592008  1643 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:48.647913  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:48.557824  3970 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.015s	sys 0.022s
W20260709 13:02:48.635047  1612 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:44424 dropped due to backpressure: service queue is full with 50 items
W20260709 13:02:48.615041  1568 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.035s	sys 0.067s
I20260709 13:02:48.649474  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:48.640053  1640 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:48.640105  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:48.641662  1639 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.000s	sys 0.035s
W20260709 13:02:48.652772  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 75616us
I20260709 13:02:48.654559  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.651561  1648 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:48.657856  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.636437  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:48.659233  1647 raft_consensus.cc:2376] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6. Re-sending same reply.
W20260709 13:02:48.663420  1570 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.018s	sys 0.016s
I20260709 13:02:48.665293  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 281) took 1079 ms. Trace:
I20260709 13:02:48.668380  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:48.669399  1647 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:48.664068  1648 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
W20260709 13:02:48.643653  3825 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.013s	sys 0.019s
I20260709 13:02:48.672238  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.676977  1646 raft_consensus.cc:2376] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6. Re-sending same reply.
W20260709 13:02:48.683384  1537 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.010s	sys 0.061s
W20260709 13:02:48.686374  1537 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.010s	sys 0.061s
W20260709 13:02:48.688462  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 535027us
I20260709 13:02:48.663305  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.625804  3947 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:48.665892  1570 rpcz_store.cc:276] 0709 13:02:47.586426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.589539 (+  3113us) service_pool.cc:224] Handling call
0709 13:02:47.589890 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.590100 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:47.818883 (+228783us) log.cc:844] Serialized 14 byte log entry
0709 13:02:48.664720 (+845837us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:48.664787 (+    67us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:48.664836 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.664905 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.665232 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":366064,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":71,"raft.queue_time_us":247069,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"thread_start_us":1071745,"threads_started":3,"wal-append.queue_time_us":483408}
W20260709 13:02:48.702209  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.650000  1612 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:44424 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1980)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1960)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1971)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1972)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1964)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1973)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1974)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1975)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1976)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1958)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1979)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 177)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1977)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1981)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 184)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 183)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1370)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 1978)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1366)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1368)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1369)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1371)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1374)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1375)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1378)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1379)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1380)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1381)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1382)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1383)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1384)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1389)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1390)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1391)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1392)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1395)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 185)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 186)
W20260709 13:02:48.768031  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.753912  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.768519  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.656123  3970 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.016s	sys 0.022s
W20260709 13:02:48.768810  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 54096.7us
W20260709 13:02:48.781515  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.761735  1536 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.018s	sys 0.022s
W20260709 13:02:48.785339  1536 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.019s	sys 0.022s
W20260709 13:02:48.786128  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.775350  1640 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
W20260709 13:02:48.733160  1538 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.007s	sys 0.024s
W20260709 13:02:48.787122  1538 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.013s	sys 0.030s
W20260709 13:02:48.787181  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.787456  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 759410us
W20260709 13:02:48.789199  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.769217  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.702921  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 49176.2us
I20260709 13:02:48.793028  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.795871  1687 leader_election.cc:400] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 3 pre-election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4.
I20260709 13:02:48.796718  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.798110  1722 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:48.734623  3720 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:48.799414  1643 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:02:48.817960  1687 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:48.746359  4041 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 13:02:48.804836  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.788380  3975 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:48.800450  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 217615us
W20260709 13:02:48.822103  1539 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.003s	sys 0.032s
W20260709 13:02:48.823268  1539 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.004s	sys 0.033s
W20260709 13:02:48.800184  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 362153us
I20260709 13:02:48.730232  4045 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:48.828395  1719 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:48.828680  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:02:48.830545  4042 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:02:48.831070  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:48.832139  4052 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:02:48.833720   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:48.834583  2342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 286) took 1156 ms. Trace:
I20260709 13:02:48.835402  2342 rpcz_store.cc:276] 0709 13:02:47.678097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.694894 (+ 16797us) service_pool.cc:224] Handling call
0709 13:02:47.706599 (+ 11705us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:47.707064 (+   465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:47.707102 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:47.816605 (+109503us) log.cc:844] Serialized 26 byte log entry
0709 13:02:48.358118 (+541513us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:48.369609 (+ 11491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.404751 (+ 35142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:48.834033 (+429282us) raft_consensus.cc:1685] finished
0709 13:02:48.834115 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.834469 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":383828,"threads_started":1,"wal-append.queue_time_us":759188}
W20260709 13:02:48.838240  4044 log.cc:927] Time spent T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:02:48.828749  4045 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:48.840960  4045 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:48.746981  4040 log.cc:927] Time spent T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:02:48.721940  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.795365  1640 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:48.824718  1722 raft_consensus.cc:1275] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:02:48.843962  4038 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.282s	user 0.000s	sys 0.008s
I20260709 13:02:48.845247  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1962) took 1093 ms. Trace:
I20260709 13:02:48.842792  1641 raft_consensus.cc:2410] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:48.845611  1718 rpcz_store.cc:276] 0709 13:02:47.752850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.768512 (+ 15662us) service_pool.cc:224] Handling call
0709 13:02:47.810250 (+ 41738us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:47.810490 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:47.810520 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:47.843510 (+ 32990us) log.cc:844] Serialized 26 byte log entry
0709 13:02:48.029926 (+186416us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:48.030039 (+   113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.030140 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:48.845033 (+814893us) raft_consensus.cc:1685] finished
0709 13:02:48.845055 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.845193 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2432,"thread_start_us":154829,"threads_started":1,"wal-append.queue_time_us":557822}
I20260709 13:02:48.847190  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:48.847924  1641 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:48.848664  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.849195  1641 raft_consensus.cc:2376] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7. Re-sending same reply.
I20260709 13:02:48.827330  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.850013  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:48.850224  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:48.852347  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.853379  1640 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
W20260709 13:02:48.855785  4039 log.cc:927] Time spent T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:02:48.850713  1641 raft_consensus.cc:2376] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8. Re-sending same reply.
I20260709 13:02:48.856114  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:48.857530  4046 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:02:48.857949  1640 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:48.858827  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.858880  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.859041  3985 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:48.859606  1640 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:48.859724  1643 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:48.860744  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.861490  1643 raft_consensus.cc:2393] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:48.862389  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.862991  1643 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:48.863620  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.863843  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.864099  1713 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:48.864355  1643 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:02:48.865578  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.866658  1646 raft_consensus.cc:2376] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8. Re-sending same reply.
I20260709 13:02:48.867404  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.867587  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.868331  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.868554  1647 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
W20260709 13:02:48.870447  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45914.7us
I20260709 13:02:48.866605  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.864287  1642 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:48.876125  1640 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:48.876248  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.877731  1647 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:02:48.865581  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.878657  1643 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:48.878805  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.883682  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.886776  1685 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:48.888018  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.884027  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:48.888852  1648 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:48.889071  1721 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:48.883749  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.885192  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:48.890972  1566 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.005s	sys 0.017s
W20260709 13:02:48.891363  1566 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.005s	sys 0.017s
I20260709 13:02:48.880872  4010 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.897686  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:48.899789  1646 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
W20260709 13:02:48.880972  4051 log.cc:927] Time spent T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:02:48.903339  1685 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:48.904721  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:48.906659  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.910835  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.911600  1719 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:48.907698  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.912992  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.914098  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.917196  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:48.918116  2347 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:48.915781  1718 raft_consensus.cc:2410] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:48.920226  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:48.920907  1646 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:02:48.922031  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.922996  1685 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:48.923589  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.935652  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:02:48.937909  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.816843  3879 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
W20260709 13:02:48.942639  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 125014us
I20260709 13:02:48.950047  3964 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:48.901149  4010 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:48.954114  4029 raft_consensus.cc:3055] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:48.898523  3993 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:48.916313  3977 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:48.959015  1721 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:02:48.964604  1714 raft_consensus.cc:1275] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:48.965054  1646 raft_consensus.cc:1240] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:48.958464  1687 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:48.908514  3970 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:48.959874  3935 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.937587  3956 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:48.954788  4029 raft_consensus.cc:740] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:48.961567  3989 consensus_queue.cc:1048] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:48.961920  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 165) took 1038 ms. Trace:
W20260709 13:02:48.966740  1612 connection.cc:256] RPC call timeout handler was delayed by -1.4836112220000359s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15480000000000002s for a response
I20260709 13:02:48.942085  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:48.938952  4045 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:48.971730  1687 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:48.974614  1685 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:48.960378  3877 consensus_meta.cc:287] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.655s	user 0.017s	sys 0.002s
W20260709 13:02:48.978669  1612 connection.cc:256] RPC call timeout handler was delayed by -1.47371122200002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1559s for a response
W20260709 13:02:49.001660  1612 connection.cc:256] RPC call timeout handler was delayed by -1.4566112219999923s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15780000000000002s for a response
W20260709 13:02:49.002032  1612 connection.cc:256] RPC call timeout handler was delayed by -1.3585112220000042s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16870000000000002s for a response
W20260709 13:02:49.002290  1612 connection.cc:256] RPC call timeout handler was delayed by -1.3477112219999867s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16990000000000002s for a response
W20260709 13:02:49.002532  1612 connection.cc:256] RPC call timeout handler was delayed by -1.3477112219999867s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16990000000000002s for a response
W20260709 13:02:49.002779  1612 connection.cc:256] RPC call timeout handler was delayed by -1.1065112219999946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1967s for a response
I20260709 13:02:49.002885  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:49.003719  1641 raft_consensus.cc:2376] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6. Re-sending same reply.
I20260709 13:02:49.005292  1641 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:49.008152  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:48.916797  4016 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
W20260709 13:02:49.010366  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.984283  3983 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:49.012380  3983 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:49.014750  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:49.015740  1642 raft_consensus.cc:2410] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:49.016197  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1370) took 1007 ms. Trace:
I20260709 13:02:49.016494  1642 rpcz_store.cc:276] 0709 13:02:48.008895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.891085 (+882190us) service_pool.cc:224] Handling call
0709 13:02:49.014554 (+123469us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b5800003098. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:02:49.016127 (+  1573us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2974080}
I20260709 13:02:49.021553  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:48.954670  4010 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:48.959761  3985 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.988245  3967 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:48.991564  3894 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:48.974500  3849 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:48.998186  4029 consensus_queue.cc:260] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:49.031822  3967 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:49.000092  4062 log.cc:927] Time spent T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:02:49.042397  3967 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:49.003053  1612 connection.cc:256] RPC call timeout handler was delayed by -1.1020112219999874s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19720000000000001s for a response
W20260709 13:02:49.042995  1612 connection.cc:256] RPC call timeout handler was delayed by -1.0138112220000153s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.207s for a response
I20260709 13:02:49.042450  4029 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:49.044828  1612 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:48.978498  1639 rpcz_store.cc:276] 0709 13:02:47.923157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.923303 (+   146us) service_pool.cc:224] Handling call
0709 13:02:47.923792 (+   489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:47.924116 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:47.924149 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:47.957084 (+ 32935us) log.cc:844] Serialized 26 byte log entry
0709 13:02:48.652169 (+695085us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:48.658687 (+  6518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.658810 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:48.961467 (+302657us) raft_consensus.cc:1685] finished
0709 13:02:48.961497 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.961796 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":464,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":15004,"thread_start_us":627500,"threads_started":1,"wal-append.queue_time_us":775190}
I20260709 13:02:48.985394  1687 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:48.969538  3936 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:49.055083  3993 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:48.997357  3947 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:49.059962  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 74654.5us
I20260709 13:02:49.061393  1687 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:49.052588  1612 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:49.053956  3936 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:49.034706  3985 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:49.071202  1687 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:49.072461  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:49.009311  2344 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:49.074589  1687 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:48.982920  3935 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:49.083962  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 84259.2us
I20260709 13:02:49.084106  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:49.072484  3953 consensus_meta.cc:287] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.957s	user 0.015s	sys 0.000s
W20260709 13:02:49.085143  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 292595us
I20260709 13:02:49.071954  4026 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:49.098097  3879 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:49.100941  1644 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:02:49.102052  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1383) took 1025 ms. Trace:
I20260709 13:02:49.102473  1644 rpcz_store.cc:276] 0709 13:02:48.076609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.100117 (+1023508us) service_pool.cc:224] Handling call
0709 13:02:49.100682 (+   565us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:49.101735 (+  1053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.101981 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:49.105424  3936 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:49.088887  4057 raft_consensus.cc:2764] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:02:49.107545  1645 raft_consensus.cc:2410] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:49.069267  3970 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:49.033342  3817 consensus_queue.cc:1059] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:02:49.068467  3956 raft_consensus.cc:697] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:49.112563  1686 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:02:49.118984  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.119916  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1421) took 1172 ms. Trace:
W20260709 13:02:49.121989  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 47835.4us
I20260709 13:02:49.123737  1687 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:49.120539  1573 rpcz_store.cc:276] 0709 13:02:47.947436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.947571 (+   135us) service_pool.cc:224] Handling call
0709 13:02:47.947827 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.948038 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:48.613296 (+665258us) log.cc:844] Serialized 14 byte log entry
0709 13:02:49.119434 (+506138us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:49.119517 (+    83us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:49.119586 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.119684 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.119840 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":382071,"prepare.queue_time_us":136111,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":5857,"thread_start_us":777683,"threads_started":2,"wal-append.queue_time_us":340468}
I20260709 13:02:49.082665  3952 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:49.076005  4075 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:49.196058  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:49.209179  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:49.196756  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:49.205111  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.212867  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1420) took 1266 ms. Trace:
I20260709 13:02:49.213459  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1428) took 1248 ms. Trace:
W20260709 13:02:49.215647  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 91888.9us
I20260709 13:02:49.216645  1687 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:49.115641  3970 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:49.213918  1568 rpcz_store.cc:276] 0709 13:02:47.965026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.965194 (+   168us) service_pool.cc:224] Handling call
0709 13:02:47.965547 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.965808 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:48.661295 (+695487us) log.cc:844] Serialized 14 byte log entry
0709 13:02:49.212887 (+551592us) log.cc:844] Serialized 14 byte log entry
0709 13:02:49.213123 (+   236us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:49.213169 (+    46us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:49.213219 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.213289 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.213404 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":314280,"prepare.queue_time_us":66876,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"thread_start_us":873293,"threads_started":2,"wal-append.queue_time_us":326852}
I20260709 13:02:49.224272  3989 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:49.224956  1687 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:49.227831  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:49.196949  4057 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.198520  4049 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:49.197449  3977 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:49.247356  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 152327us
I20260709 13:02:49.247879  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1368) took 1239 ms. Trace:
I20260709 13:02:49.249384  1687 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:49.251345  1612 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:49.174067  4056 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.109s
I20260709 13:02:49.223214  3936 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.278870  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:49.261835  1570 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:02:49.248319  1647 rpcz_store.cc:276] 0709 13:02:48.008285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.889114 (+880829us) service_pool.cc:224] Handling call
0709 13:02:48.890160 (+  1046us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:48.906520 (+ 16360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:48.906558 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:48.921853 (+ 15295us) log.cc:844] Serialized 52 byte log entry
0709 13:02:49.065894 (+144041us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:49.197387 (+131493us) log.cc:844] Serialized 14 byte log entry
0709 13:02:49.229982 (+ 32595us) log.cc:844] Serialized 14 byte log entry
0709 13:02:49.247542 (+ 17560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.247631 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:49.247670 (+    39us) raft_consensus.cc:1685] finished
0709 13:02:49.247681 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.247824 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43571,"prepare.queue_time_us":171945,"raft.queue_time_us":17746,"raft.run_cpu_time_us":486,"raft.run_wall_time_us":44155,"thread_start_us":220660,"threads_started":2,"wal-append.queue_time_us":102025}
I20260709 13:02:49.285279  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:49.226995  3935 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:49.291684  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1443) took 1177 ms. Trace:
I20260709 13:02:49.322971  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:49.323091  1566 rpcz_store.cc:276] 0709 13:02:48.114662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.268919 (+154257us) service_pool.cc:224] Handling call
0709 13:02:48.329977 (+ 61058us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:48.330432 (+   455us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:48.331187 (+   755us) log.cc:844] Serialized 14 byte log entry
0709 13:02:49.290913 (+959726us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:49.290979 (+    66us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:49.291039 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.291125 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.291584 (+   459us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":398370,"prepare.queue_time_us":324,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":402358,"thread_start_us":560896,"threads_started":1,"wal-append.queue_time_us":629679}
I20260709 13:02:49.284178  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:49.294884  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 45779.5us
I20260709 13:02:49.220920  3970 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:49.305272  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:49.260787  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:49.226703  4075 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:49.237617  3952 raft_consensus.cc:697] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:02:49.276963  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:49.285729  4073 log.cc:927] Time spent T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:02:49.246997  4043 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:49.231837  3983 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:49.344462  1647 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:02:49.361871  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.408170  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1389) took 1268 ms. Trace:
I20260709 13:02:49.240083  3977 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.348675  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1349) took 1736 ms. Trace:
W20260709 13:02:49.301432  4068 log.cc:927] Time spent T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:02:49.337981  3989 raft_consensus.cc:697] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:49.241819  4049 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:49.250093  3817 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:49.303476  3953 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:49.409123  1647 rpcz_store.cc:276] 0709 13:02:48.139668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.343465 (+1203797us) service_pool.cc:224] Handling call
0709 13:02:49.343957 (+   492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:49.407603 (+ 63646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.407963 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:49.254613  4026 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:49.427587  3989 consensus_queue.cc:237] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:49.429167  4071 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
W20260709 13:02:49.432498  4072 log.cc:927] Time spent T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:02:49.294842  3688 consensus_queue.cc:1059] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.120s
I20260709 13:02:49.416260  3977 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:49.438691  4059 raft_consensus.cc:3055] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:49.439106  4059 raft_consensus.cc:740] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:49.439893  4059 consensus_queue.cc:260] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:49.441502  4059 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:49.347097  4075 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:49.351917  4043 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:49.322644  3944 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.102s
I20260709 13:02:49.460418  3983 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:49.456422  3952 consensus_queue.cc:237] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:49.457286  4077 log.cc:927] Time spent T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 13:02:49.418828  3849 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.460987  3985 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:49.322706  3936 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:49.458362  4078 log.cc:927] Time spent T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:02:49.423254  4049 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:49.283452  3956 consensus_queue.cc:237] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:49.247748  4057 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:49.463038  4043 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:49.486277  4080 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.055s	user 0.002s	sys 0.000s
W20260709 13:02:49.482796  4081 log.cc:927] Time spent T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Append to log took a long time: real 0.057s	user 0.001s	sys 0.001s
I20260709 13:02:49.491987  3936 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:49.518698  3977 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:49.526185  3977 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:49.500082  4045 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:49.523749  4049 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:49.529287  4049 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:49.520468  4057 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:49.583420  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 299033us
I20260709 13:02:49.597801  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:49.598882  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:49.600597  1568 raft_consensus.cc:2410] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:49.597831  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:49.602470  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:49.602607  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:49.603565  1569 raft_consensus.cc:2410] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:49.603219  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:49.597854  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 271951us
I20260709 13:02:49.604943  1566 raft_consensus.cc:2410] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:49.602605  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:49.606480  1538 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:49.611455  1537 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:49.603288  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:49.613021  1537 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:49.376868  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:49.615502  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:49.616221  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 364868us
I20260709 13:02:49.410786  1645 rpcz_store.cc:276] 0709 13:02:47.612559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.612834 (+   275us) service_pool.cc:224] Handling call
0709 13:02:49.348430 (+1735596us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54000a69d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:49.348609 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3288167680}
I20260709 13:02:49.618896  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1384) took 1533 ms. Trace:
I20260709 13:02:49.619625  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1382) took 1546 ms. Trace:
I20260709 13:02:49.620615  1645 raft_consensus.cc:1275] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 2. (term mismatch)
I20260709 13:02:49.621136  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1380) took 1572 ms. Trace:
W20260709 13:02:49.616262  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.621366  1645 pending_rounds.cc:85] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 1
I20260709 13:02:49.622962  1645 pending_rounds.cc:107] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 13:02:49.620086  1639 rpcz_store.cc:276] 0709 13:02:48.073357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.046124 (+972767us) service_pool.cc:224] Handling call
0709 13:02:49.046459 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:49.053677 (+  7218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:49.053713 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:49.054324 (+   611us) log.cc:844] Serialized 26 byte log entry
0709 13:02:49.082626 (+ 28302us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:49.082673 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.082743 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:49.502573 (+419830us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b48002619d8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ddaeaa1 00007f824dda5b37 00007f824dda59ca 00007f824de9d26f 00007f824de9cf37 00007f824deb155c 00007f824deb1508 00007f824deb1488 00007f824deb1443 00007f824deb06ec 00007f824cddaa96
0709 13:02:49.618944 (+116371us) raft_consensus.cc:1685] finished
0709 13:02:49.618988 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.619405 (+   417us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9474,"raft.run_cpu_time_us":3999,"raft.run_wall_time_us":446458,"spinlock_wait_cycles":4679936,"thread_start_us":27790,"threads_started":1,"wal-append.queue_time_us":119129}
I20260709 13:02:49.623481  1645 raft_consensus.cc:2889] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:49.624867  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1391) took 1471 ms. Trace:
I20260709 13:02:49.625134  1639 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:49.621706  1642 rpcz_store.cc:276] 0709 13:02:48.048922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.017588 (+968666us) service_pool.cc:224] Handling call
0709 13:02:49.026624 (+  9036us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:49.027040 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:49.027066 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:49.027521 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 13:02:49.223585 (+196064us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:49.223636 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.223727 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:49.620605 (+396878us) raft_consensus.cc:1685] finished
0709 13:02:49.620642 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.620967 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2201,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":15299,"thread_start_us":142248,"threads_started":1,"wal-append.queue_time_us":284731}
I20260709 13:02:49.619477  1644 rpcz_store.cc:276] 0709 13:02:48.102161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.103366 (+1001205us) service_pool.cc:224] Handling call
0709 13:02:49.108850 (+  5484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:49.109748 (+   898us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:49.109784 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:49.115172 (+  5388us) log.cc:844] Serialized 26 byte log entry
0709 13:02:49.297730 (+182558us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:49.418131 (+120401us) log.cc:844] Serialized 14 byte log entry
0709 13:02:49.418364 (+   233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.418469 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:49.618286 (+199817us) raft_consensus.cc:1685] finished
0709 13:02:49.618326 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.618620 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42265,"prepare.queue_time_us":144302,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":6109,"raft.queue_time_us":1986,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":9434,"spinlock_wait_cycles":7160192,"thread_start_us":84849,"threads_started":1,"wal-append.queue_time_us":221571}
W20260709 13:02:49.495925  3830 consensus_meta.cc:287] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.597s	user 0.015s	sys 0.003s
I20260709 13:02:49.671747  3830 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:49.213405  1567 rpcz_store.cc:276] 0709 13:02:47.946412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.946535 (+   123us) service_pool.cc:224] Handling call
0709 13:02:47.946799 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.947024 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:48.629153 (+682129us) log.cc:844] Serialized 14 byte log entry
0709 13:02:49.212213 (+583060us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:49.212282 (+    69us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:49.212343 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.212431 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.212785 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":404899,"prepare.queue_time_us":202321,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":328531,"thread_start_us":859383,"threads_started":2,"wal-append.queue_time_us":357897}
I20260709 13:02:49.631333  1647 raft_consensus.cc:1275] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:49.674427  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1390) took 1521 ms. Trace:
I20260709 13:02:49.675154  1647 rpcz_store.cc:276] 0709 13:02:48.152817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.629882 (+1477065us) service_pool.cc:224] Handling call
0709 13:02:49.630555 (+   673us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:49.673994 (+ 43439us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.674308 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:49.675647  3879 raft_consensus.cc:3055] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:49.676329  3879 raft_consensus.cc:740] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:49.625272  1645 rpcz_store.cc:276] 0709 13:02:48.153093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.618287 (+1465194us) service_pool.cc:224] Handling call
0709 13:02:49.619383 (+  1096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:49.624429 (+  5046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.624766 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:49.680333  1645 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:02:49.652509  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 403105us
I20260709 13:02:49.525310  3967 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:49.655395  1642 raft_consensus.cc:1275] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:49.656559  3970 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:49.657487  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 185) took 1303 ms. Trace:
I20260709 13:02:49.697065  1644 rpcz_store.cc:276] 0709 13:02:48.353820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.656090 (+1302270us) service_pool.cc:224] Handling call
0709 13:02:49.656630 (+   540us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:49.657069 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:49.657099 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:49.657158 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:49.657195 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.657258 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.657398 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:49.702323  3970 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:49.648643  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1392) took 1469 ms. Trace:
I20260709 13:02:49.641465  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1369) took 1632 ms. Trace:
I20260709 13:02:49.666477  4029 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:49.666827  3688 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.667886  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1371) took 1623 ms. Trace:
I20260709 13:02:49.668744  3943 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:49.483460  3849 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:49.478482  3817 raft_consensus.cc:697] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:49.685459  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:49.687031  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:49.687618  1611 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:49.690984  1687 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:02:49.692940  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 269245us
I20260709 13:02:49.628999  1612 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:49.712126  4049 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:49.707578  1639 rpcz_store.cc:276] 0709 13:02:48.166246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.623946 (+1457700us) service_pool.cc:224] Handling call
0709 13:02:49.624543 (+   597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:49.633389 (+  8846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.635558 (+  2169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:49.705543  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1395) took 1463 ms. Trace:
W20260709 13:02:49.728386  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 115618us
I20260709 13:02:49.713996  3830 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:49.749521  1538 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:49.752826  1642 rpcz_store.cc:276] 0709 13:02:48.235651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.653921 (+1418270us) service_pool.cc:224] Handling call
0709 13:02:49.654708 (+   787us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:49.695910 (+ 41202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.698954 (+  3044us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:49.736387  3943 raft_consensus.cc:697] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:02:49.761013  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.761083  1572 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
W20260709 13:02:49.762742  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 75568.9us
I20260709 13:02:49.715492  3688 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:49.766134  1571 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:02:49.768819  1720 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:49.773777  4049 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:49.774125  3943 consensus_queue.cc:237] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:49.767203  1646 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.000s	sys 0.012s
W20260709 13:02:49.776543  1646 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.000s	sys 0.014s
I20260709 13:02:49.778632  3817 consensus_queue.cc:237] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:49.773830  3977 raft_consensus.cc:2764] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:02:49.707813  1640 rpcz_store.cc:276] 0709 13:02:48.008583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.890569 (+881986us) service_pool.cc:224] Handling call
0709 13:02:48.893959 (+  3390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:48.894205 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:49.064999 (+170794us) log.cc:844] Serialized 14 byte log entry
0709 13:02:49.639867 (+574868us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b18002ffc80. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a2523a
0709 13:02:49.640559 (+   692us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:49.640672 (+   113us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:49.640811 (+   139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.640947 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.641198 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23385,"prepare.queue_time_us":224620,"prepare.run_cpu_time_us":8789,"prepare.run_wall_time_us":68186,"raft.queue_time_us":38025,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":294,"spinlock_wait_cycles":3506688,"thread_start_us":223995,"threads_started":2,"wal-append.queue_time_us":179993}
I20260709 13:02:49.786376  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:49.787717  2344 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:49.787734  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:49.788897  2342 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:49.789114  3849 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:49.790700  3989 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:49.791775  3989 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:49.755025  3879 consensus_queue.cc:260] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:49.801121  1572 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1426) took 1850 ms (client timeout 2057 ms). Trace:
W20260709 13:02:49.802815  1572 rpcz_store.cc:269] 0709 13:02:47.950503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.950638 (+   135us) service_pool.cc:224] Handling call
0709 13:02:49.800765 (+1850127us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b5400405bd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:49.800994 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3570222336}
I20260709 13:02:49.759684  4029 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:49.765918  1643 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.013s	sys 0.016s
I20260709 13:02:49.715881  1648 rpcz_store.cc:276] 0709 13:02:48.009195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.890580 (+881385us) service_pool.cc:224] Handling call
0709 13:02:48.892735 (+  2155us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:48.893176 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:48.893207 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:48.893745 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 13:02:49.089162 (+195417us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:49.232847 (+143685us) log.cc:844] Serialized 14 byte log entry
0709 13:02:49.251712 (+ 18865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.251829 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:49.505411 (+253582us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b1800057ca0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d662a6 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ddaeaa1 00007f824dda5b37 00007f824dda59ca 00007f824de9d26f
0709 13:02:49.506146 (+   735us) spinlock_profiling.cc:243] Waited 38 ms on lock 0x7b48002619d8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ddaeaa1 00007f824dda5b37 00007f824dda59ca 00007f824de9d26f 00007f824de9cf37 00007f824deb155c 00007f824deb1508 00007f824deb1488 00007f824deb1443 00007f824deb06ec 00007f824cddaa96
0709 13:02:49.631963 (+125817us) raft_consensus.cc:1685] finished
0709 13:02:49.632009 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.632437 (+   428us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":123035,"prepare.queue_time_us":202069,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":7002,"raft.queue_time_us":18096,"raft.run_cpu_time_us":3501,"raft.run_wall_time_us":258038,"spinlock_wait_cycles":111989888,"thread_start_us":166084,"threads_started":2,"wal-append.queue_time_us":279431}
W20260709 13:02:49.819511  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 50457.2us
I20260709 13:02:49.820672  4056 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:49.822688  1643 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.014s	sys 0.016s
I20260709 13:02:49.766561  4060 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:02:49.827291  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 136581us
I20260709 13:02:49.827910  4054 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.828493  1687 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:49.829089  3967 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:49.828747  4054 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:49.829869  3830 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:49.830624  3830 raft_consensus.cc:697] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:49.831848  4054 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:49.804368  4057 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:49.832098  3830 consensus_queue.cc:237] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:49.827319  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 198587us
I20260709 13:02:49.835959  4029 raft_consensus.cc:697] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:49.837419  4029 consensus_queue.cc:237] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:49.818470  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1378) took 1791 ms. Trace:
I20260709 13:02:49.835994  1612 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:49.847338  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 112589us
W20260709 13:02:49.851431  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 130929us
I20260709 13:02:49.856405  4043 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:49.856601  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1366) took 1848 ms. Trace:
I20260709 13:02:49.859431  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1381) took 1791 ms. Trace:
I20260709 13:02:49.817497  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 1982) took 1010 ms. Trace:
I20260709 13:02:49.863605  3944 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:49.864217  1537 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:49.865149  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 186) took 1510 ms. Trace:
I20260709 13:02:49.866709  4049 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:49.868474  3688 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:49.860643  3935 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:49.840062  1646 rpcz_store.cc:276] 0709 13:02:48.027213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.966143 (+938930us) service_pool.cc:224] Handling call
0709 13:02:48.980741 (+ 14598us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:48.981193 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:48.981226 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:49.000904 (+ 19678us) log.cc:844] Serialized 52 byte log entry
0709 13:02:49.126253 (+125349us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:49.777138 (+650885us) log.cc:844] Serialized 14 byte log entry
0709 13:02:49.817380 (+ 40242us) log.cc:844] Serialized 14 byte log entry
0709 13:02:49.817814 (+   434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.817974 (+   160us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:49.818069 (+    95us) raft_consensus.cc:1685] finished
0709 13:02:49.818082 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.818380 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":134324,"prepare.queue_time_us":355625,"prepare.run_cpu_time_us":133,"prepare.run_wall_time_us":150,"raft.queue_time_us":22923,"raft.run_cpu_time_us":651,"raft.run_wall_time_us":4158,"thread_start_us":671414,"threads_started":2,"wal-append.queue_time_us":215205}
W20260709 13:02:49.879741  4107 log.cc:927] Time spent T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:02:49.871618  1641 rpcz_store.cc:276] 0709 13:02:48.067950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.028220 (+960270us) service_pool.cc:224] Handling call
0709 13:02:49.028562 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:49.029444 (+   882us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:49.029493 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:49.284019 (+254526us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b2c000120e8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824777ea11 00007f824778e372 00007f8248ec0a01 00007f8248e8543a 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2
0709 13:02:49.284769 (+   750us) log.cc:844] Serialized 26 byte log entry
0709 13:02:49.801930 (+517161us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:49.838178 (+ 36248us) log.cc:844] Serialized 14 byte log entry
0709 13:02:49.858564 (+ 20386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.858719 (+   155us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:49.858843 (+   124us) raft_consensus.cc:1685] finished
0709 13:02:49.859001 (+   158us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.859199 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56072,"prepare.queue_time_us":35997,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":56,"raft.queue_time_us":20835,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":6831,"spinlock_wait_cycles":37579136,"thread_start_us":474687,"threads_started":1,"wal-append.queue_time_us":488137}
I20260709 13:02:49.863626  1612 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:49.872234  1643 rpcz_store.cc:276] 0709 13:02:48.007673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.893698 (+886025us) service_pool.cc:224] Handling call
0709 13:02:48.895810 (+  2112us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:48.896602 (+   792us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:48.896654 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:48.903378 (+  6724us) log.cc:844] Serialized 26 byte log entry
0709 13:02:49.110538 (+207160us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:49.823419 (+712881us) log.cc:844] Serialized 14 byte log entry
0709 13:02:49.855397 (+ 31978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.855607 (+   210us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:49.856046 (+   439us) raft_consensus.cc:1685] finished
0709 13:02:49.856074 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.856478 (+   404us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":131707,"prepare.queue_time_us":228722,"prepare.run_cpu_time_us":11579,"prepare.run_wall_time_us":58686,"raft.queue_time_us":34624,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":29635,"thread_start_us":744181,"threads_started":2,"wal-append.queue_time_us":284777}
I20260709 13:02:49.902079  4075 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:49.903508  3935 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:49.872896  1720 rpcz_store.cc:276] 0709 13:02:48.806853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.819812 (+ 12959us) service_pool.cc:224] Handling call
0709 13:02:48.825508 (+  5696us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:49.772967 (+947459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.816947 (+ 43980us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b5400213254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:49.817336 (+   389us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1919543168}
I20260709 13:02:49.896382  1611 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:49.897359  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45424us
I20260709 13:02:49.906086  4075 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:49.909574  3989 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:49.894328  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 65965.8us
I20260709 13:02:49.902693  1612 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:49.767023  4089 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:49.916028  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:49.910149  3935 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:49.918936  1572 raft_consensus.cc:2410] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:02:49.923137  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 59124.3us
I20260709 13:02:49.926052  3680 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:49.926594  3680 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.926959  3680 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:49.927120  1612 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:49.928887  1612 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:49.873984  1647 rpcz_store.cc:276] 0709 13:02:48.354709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.676528 (+1321819us) service_pool.cc:224] Handling call
0709 13:02:49.677449 (+   921us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:49.679402 (+  1953us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:49.679454 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:49.680261 (+   807us) log.cc:844] Serialized 26 byte log entry
0709 13:02:49.745954 (+ 65693us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:49.840648 (+ 94694us) log.cc:844] Serialized 14 byte log entry
0709 13:02:49.864370 (+ 23722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.864551 (+   181us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:49.864634 (+    83us) raft_consensus.cc:1685] finished
0709 13:02:49.864678 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.865011 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22825,"prepare.queue_time_us":70022,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":79,"raft.queue_time_us":24026,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":328,"thread_start_us":65056,"threads_started":1,"wal-append.queue_time_us":104022}
I20260709 13:02:49.920786  3935 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.947811  3935 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:49.950799  4105 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.951545  4105 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:49.955400  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 102549us
I20260709 13:02:49.957018  1538 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:49.959455  4043 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:49.942653  3985 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:49.943401  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:49.946048  3947 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:49.946918  3993 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:49.967118  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:49.971108  3944 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:49.973551  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 44805.8us
I20260709 13:02:49.988338  1612 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:49.981422  1687 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:02:50.015765  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 75916.1us
I20260709 13:02:50.016875  1687 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:50.007058  1537 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:50.004905  4089 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:02:50.023326  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 59817.3us
W20260709 13:02:50.022727  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 65920.5us
I20260709 13:02:50.027815  3849 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.031149  3830 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:50.026840  3943 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:50.032141  3849 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:50.039003  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 51210.8us
I20260709 13:02:50.038578  3943 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:50.051477  3830 raft_consensus.cc:697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:50.061537  4105 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:50.062103  3830 consensus_queue.cc:237] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:50.065852  4090 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:50.066665  4090 raft_consensus.cc:697] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:50.067958  4090 consensus_queue.cc:237] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:50.060567  3943 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:50.086733  3849 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:50.088279  2340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 288) took 1409 ms. Trace:
I20260709 13:02:50.089679  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:50.090952  1572 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:02:50.090725  1612 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:50.095520  2312 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:50.110524  4029 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:50.101846  4057 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:50.097047  3849 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:50.123092  3849 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.124821  3849 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:50.126451  3680 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:50.136502  3680 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:50.089308  2340 rpcz_store.cc:276] 0709 13:02:48.674758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.697013 (+ 22255us) service_pool.cc:224] Handling call
0709 13:02:49.423541 (+726528us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b580003a298. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:02:49.433883 (+ 10342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:49.434349 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:49.692823 (+258474us) log.cc:844] Serialized 14 byte log entry
0709 13:02:50.023026 (+330203us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:50.030242 (+  7216us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:50.052143 (+ 21901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.076494 (+ 24351us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.082822 (+  6328us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15114,"prepare.queue_time_us":106783,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":38,"raft.queue_time_us":95638,"raft.run_cpu_time_us":415,"raft.run_wall_time_us":18944,"spinlock_wait_cycles":41565440,"thread_start_us":557977,"threads_started":3,"wal-append.queue_time_us":108845}
I20260709 13:02:50.143455  4088 raft_consensus.cc:1064] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:02:50.144696  4029 consensus_queue.cc:1048] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:02:50.142912  4113 log.cc:927] Time spent T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:02:50.148573  3688 consensus_queue.cc:1059] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:50.147598  4088 consensus_queue.cc:237] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:50.168068  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 77545.2us
W20260709 13:02:50.172186  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 101332us
I20260709 13:02:50.175488  4105 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.176546  4105 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:50.178187  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 133233us
W20260709 13:02:50.202209  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 55811.7us
I20260709 13:02:50.098088  3935 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:50.212865  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.216216  1687 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:50.217952  3688 consensus_queue.cc:1059] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:50.222721  1573 raft_consensus.cc:1275] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:50.223130  1567 raft_consensus.cc:1275] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:50.224521  1572 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:50.228231  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:50.229504  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:50.247923  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:50.249522  1612 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:50.236548  3830 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:50.255030  3879 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:50.258774  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:50.259068  1568 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:50.261070  1569 raft_consensus.cc:2376] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5. Re-sending same reply.
I20260709 13:02:50.269951  3956 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
W20260709 13:02:50.274456  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 58515.1us
W20260709 13:02:50.278071  2313 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.934s (SENT) [suppressed 35 similar messages]
I20260709 13:02:50.285933  1538 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:50.287647  1687 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:50.287870  3935 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:50.288254  4075 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:50.290249  3944 raft_consensus.cc:3055] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:50.277558  3936 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:50.297292  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:50.290956  3944 raft_consensus.cc:740] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:50.303453  3956 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:50.306046  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:50.307380  1569 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:50.307816  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:50.309432  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:50.311048  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:50.312462  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:50.315887  1568 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:50.323010  3944 consensus_queue.cc:260] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:50.324568  3944 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:50.293522  4049 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:02:50.277921  4105 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:50.337502  3688 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:50.340454  1570 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:50.340718  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 54946.2us
I20260709 13:02:50.342528  2312 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:50.343043  1538 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:50.343900  4104 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:50.344468  4104 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.344844  4104 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:50.334152  4029 raft_consensus.cc:3055] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:50.350576  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:02:50.356099  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 145372us
I20260709 13:02:50.356501  1567 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:50.359100  1569 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:02:50.361593  1611 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:50.364246  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 110599us
I20260709 13:02:50.340482  3688 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:50.348986  4029 raft_consensus.cc:740] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:02:50.380539  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 93083.6us
I20260709 13:02:50.371992  3936 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:50.388482  3936 raft_consensus.cc:697] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:50.390213  3936 consensus_queue.cc:237] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:50.383833  3849 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:50.396369  2340 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 LEARNER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:02:50.396957  3849 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:50.397997  4059 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:50.400441  4029 consensus_queue.cc:260] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:50.402456  4029 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:50.407027  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:02:50.408804  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 66044.5us
I20260709 13:02:50.376116  3688 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:50.413297  1538 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:50.426337  1612 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:50.430799  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.437176  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:50.438593  3935 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:50.436403  3817 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:02:50.442154  4060 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:50.442726  4060 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:50.443157  4060 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:50.454205  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:50.470955  1537 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:50.472777  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 92680.8us
W20260709 13:02:50.460889  2342 consensus_meta.cc:287] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.671s	user 0.010s	sys 0.011s
I20260709 13:02:50.478890  4059 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:50.481909  3935 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:50.477730  4054 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:02:50.482779  3935 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:50.486287  1612 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:50.487437  4057 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:50.501313  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.503418  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:50.504493  4127 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.478415  2342 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:50.504537  1569 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:02:50.513468  3975 consensus_queue.cc:1048] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:50.517035  1611 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:50.518365  4057 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:50.518870  4057 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:50.519275  4057 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:50.521072  1537 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:50.525449  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:50.532428  1564 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:50.526275  3935 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:50.537091  4054 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:50.545481  1686 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:50.549515  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:50.524952  4127 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:50.555971  1612 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:50.551403  4054 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:50.537814  4049 consensus_queue.cc:1048] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:02:50.560555  4059 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:50.563944  3817 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:02:50.570142  4054 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:50.586848  3688 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:50.572714  3936 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:50.595017  4060 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:50.596113  4088 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:50.598338  4088 raft_consensus.cc:697] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:50.599786  4088 consensus_queue.cc:237] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:50.600776  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 54919.7us
I20260709 13:02:50.569943  4059 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:50.549932  3975 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 13:02:50.602079  4059 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:50.604246  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:50.605223  1647 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:50.608335  1647 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:50.611666  3688 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:50.613952  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:50.614185  1566 raft_consensus.cc:1240] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:50.617815  1720 raft_consensus.cc:1275] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:50.624009  3849 consensus_queue.cc:1059] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:50.618240  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:50.619655  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:50.625694  4105 raft_consensus.cc:3055] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:50.626181  1646 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:50.626194  4105 raft_consensus.cc:740] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:50.629347  4105 consensus_queue.cc:260] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:50.625710  1641 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:50.630477  4105 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:50.630939  4127 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:50.632273  1471 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47518 (request call id 94) took 5867 ms. Trace:
I20260709 13:02:50.635106  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:50.637473  2314 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.671s (SENT)
I20260709 13:02:50.638085  2314 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:50.641865  3849 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:50.644568  4059 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:50.640743  1647 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
W20260709 13:02:50.651789  1538 connection.cc:256] RPC call timeout handler was delayed by -2.6149719910000613s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22210000000000002s for a response
I20260709 13:02:50.652470  1469 catalog_manager.cc:5697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 2 to 5, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 5 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:50.658654  1538 leader_election.cc:336] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.092s (SENT)
W20260709 13:02:50.659471  3943 consensus_meta.cc:287] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.554s	user 0.014s	sys 0.000s
I20260709 13:02:50.660329  1572 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:02:50.660192  3943 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:50.660705  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:50.661547  1571 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:50.661557  1647 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:50.662326  2312 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:50.664491  3935 consensus_queue.cc:1059] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:50.665020  3943 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:50.666345  1641 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
W20260709 13:02:50.666913  4128 log.cc:927] Time spent T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:02:50.667634  1646 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:50.669625  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:50.670549  3849 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:50.671420  1564 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:02:50.659349  1538 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:50.625212  1567 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
W20260709 13:02:50.687093  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 143157us
I20260709 13:02:50.619961  4060 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:50.689152  2312 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:50.700156  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:50.700981  1641 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:50.699116  1647 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:50.709604  1612 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:50.690518  3849 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.714818  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:50.715515  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:50.725524  1648 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:50.632696  1471 rpcz_store.cc:276] 0709 13:02:44.764794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.792408 (+ 27614us) service_pool.cc:224] Handling call
0709 13:02:48.307131 (+3514723us) spinlock_profiling.cc:243] Waited 9.06 ms on lock 0x7b5400002228. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ddaeaa1 00007f824ddd59b4 00007f824c8b93dc 00007f824c87ae0b 00007f824c87abe4 00007f824ed7b2d1 00007f824ed7bb2f 00007f824ec3ad4b 00007f824ed42c3e 00007f824d2032a4 00007f824d203232
0709 13:02:50.631967 (+2324836us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:48.942397 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:49.021005 (+ 78608us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:49.021049 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:02:49.021070 (+    21us) tablet.cc:662] Decoding operations
0709 13:02:49.022018 (+   948us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:49.022083 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:49.022108 (+    25us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:02:49.022640 (+   532us) tablet.cc:701] Row locks acquired
0709 13:02:49.022659 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:02:49.022842 (+   183us) write_op.cc:270] Start()
0709 13:02:49.022973 (+   131us) write_op.cc:276] Timestamp: P: 1783602169022777 usec, L: 0
0709 13:02:49.022999 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:02:49.023763 (+   764us) log.cc:844] Serialized 2515 byte log entry
0709 13:02:49.539793 (+516030us) op_driver.cc:464] REPLICATION: finished
0709 13:02:49.541249 (+  1456us) write_op.cc:301] APPLY: starting
0709 13:02:49.541373 (+   124us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:49.541686 (+   313us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:49.541867 (+   181us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:49.543797 (+  1930us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:49.544135 (+   338us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:02:49.544212 (+    77us) write_op.cc:312] APPLY: finished
0709 13:02:49.544625 (+   413us) log.cc:844] Serialized 53 byte log entry
0709 13:02:49.545782 (+  1157us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:49.545945 (+   163us) write_op.cc:454] Released schema lock
0709 13:02:49.546012 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:49.546533 (+   521us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:49.546576 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":141160,"reported_tablets":12,"spinlock_wait_cycles":20831104,"child_traces":[["op",{"apply.queue_time_us":1134,"apply.run_cpu_time_us":4173,"apply.run_wall_time_us":5221,"num_ops":6,"prepare.queue_time_us":593382,"prepare.run_cpu_time_us":41785,"prepare.run_wall_time_us":294575,"raft.queue_time_us":37600,"raft.run_cpu_time_us":1098,"raft.run_wall_time_us":1343,"replication_time_us":516637,"thread_start_us":673240,"threads_started":4,"wal-append.queue_time_us":270894,"wal-append.run_cpu_time_us":8603,"wal-append.run_wall_time_us":1233441}]]}
I20260709 13:02:50.702173  3943 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:50.714617  3849 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:50.730540  1538 connection.cc:256] RPC call timeout handler was delayed by -2.2234719910000535s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2656s for a response
W20260709 13:02:50.731040  1538 connection.cc:256] RPC call timeout handler was delayed by -1.7273510680000186s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2086s for a response
W20260709 13:02:50.731374  1538 connection.cc:256] RPC call timeout handler was delayed by -1.7129510680000521s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2102s for a response
I20260709 13:02:50.711709  4105 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:50.735773  1646 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
W20260709 13:02:50.737699  1538 connection.cc:256] RPC call timeout handler was delayed by -1.694951068000023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2122s for a response
W20260709 13:02:50.738193  1538 connection.cc:256] RPC call timeout handler was delayed by -1.2649495930000398s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2041s for a response
W20260709 13:02:50.738498  1538 connection.cc:256] RPC call timeout handler was delayed by -1.1290495930000475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21920000000000003s for a response
W20260709 13:02:50.738796  1538 connection.cc:256] RPC call timeout handler was delayed by -1.1209495930000344s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22010000000000002s for a response
W20260709 13:02:50.739086  1538 connection.cc:256] RPC call timeout handler was delayed by -1.0993495930000563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22250000000000003s for a response
I20260709 13:02:50.741132  1641 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:50.748586  1612 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:50.750417  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:50.753237  1648 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:50.753495  1469 catalog_manager.cc:5697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 4 to 5, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 5 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:50.753216  1644 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:50.730326  3935 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.773288  1639 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:50.729809  3967 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.757014  4105 raft_consensus.cc:697] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:02:50.789827  1538 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.885s (SENT)
I20260709 13:02:50.794077  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:50.795061  1720 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 8: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:50.777455  3935 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:50.798679  1644 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:50.800405  4057 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:50.803491  4104 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:50.806304  4133 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:50.807309  4133 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:50.807690  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:50.809350  4105 consensus_queue.cc:237] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:50.786899  1469 catalog_manager.cc:5697] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 3 to 4, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:50.829473  1647 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:50.776664  3975 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.830976  1538 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:50.839354  3975 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:50.840363  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 52608.9us
W20260709 13:02:50.841596  1538 leader_election.cc:336] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.988s (SENT)
I20260709 13:02:50.842953  3688 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:50.843811  3975 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:50.848075  2314 leader_election.cc:336] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 3.292s (SENT)
I20260709 13:02:50.849009  2314 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:50.853107  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:50.854738  2342 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:50.857584  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:50.859349  3975 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:50.777971  3936 raft_consensus.cc:3055] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:50.862653  4131 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:50.866191  1720 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:50.790558  4059 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:50.896438  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 65413.6us
I20260709 13:02:50.902511   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:50.902700  1469 catalog_manager.cc:5697] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:02:50.903527  3688 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.862694  3936 raft_consensus.cc:740] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:02:50.904711  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.904328  3688 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:50.905697  3936 consensus_queue.cc:260] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:50.907433  3936 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:50.908241  3688 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:50.908950  3688 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.863138  4104 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:50.905256  1469 catalog_manager.cc:5697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 3 to 4, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
W20260709 13:02:50.918617  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 168453us
I20260709 13:02:50.872798  3975 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.927066  1567 raft_consensus.cc:1240] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:50.930244  1569 raft_consensus.cc:1275] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:50.899147  4127 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:50.932379  1571 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:50.932924  1720 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:50.836546  3967 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:50.977901  1644 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:50.929831  1572 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:02:50.981875  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 137691us
I20260709 13:02:50.956058  1469 catalog_manager.cc:5697] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:50.928751  3975 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:50.991526  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:51.001526  1612 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:51.013959  1570 raft_consensus.cc:3055] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:51.014671  1570 raft_consensus.cc:740] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:02:51.019424  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 171421us
W20260709 13:02:51.021554  1538 leader_election.cc:336] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.086s (SENT)
I20260709 13:02:50.980741  4059 consensus_queue.cc:1048] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:02:51.025424  4057 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:51.025743  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:51.026139  1644 raft_consensus.cc:1275] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:50.979027  3688 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:51.044432  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.040861  1570 consensus_queue.cc:260] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:51.005179  3879 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:02:51.058543  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:51.041751  3817 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:02:51.040359  1469 catalog_manager.cc:5697] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 1 to 4, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:51.032933  4134 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:02:51.057912  4132 log.cc:927] Time spent T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.077s	user 0.002s	sys 0.000s
I20260709 13:02:51.021767  3956 consensus_queue.cc:1048] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.045s
W20260709 13:02:51.092931  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 74347.8us
I20260709 13:02:51.111066  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:51.115823  1716 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.043735  3935 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:51.095628  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 120585us
I20260709 13:02:51.099759  1566 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:51.116777  1570 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:51.116741  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:51.118341  4054 consensus_meta.cc:287] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.528s	user 0.012s	sys 0.007s
I20260709 13:02:51.094987  4141 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.135773  1469 catalog_manager.cc:5697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 3 to 4, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:51.157125  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:51.124619  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:51.175357  1718 raft_consensus.cc:1275] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:51.192003  1687 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:51.196854  1469 catalog_manager.cc:5697] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:51.207551  1469 catalog_manager.cc:5697] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:02:51.206707  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:51.207131  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:51.172919  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:51.219849  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 203619us
I20260709 13:02:51.234831  1469 catalog_manager.cc:5697] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 3 to 5, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 5 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:51.240547  1469 catalog_manager.cc:5697] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:51.156791  4054 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:51.246107  4135 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:51.267184  1640 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:51.267575  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:51.268007  1538 leader_election.cc:336] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.102s (SENT)
W20260709 13:02:51.282428  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.283814  1721 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:51.285197  1642 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:51.290570  1564 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.291419  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:51.311094  1716 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:02:51.264012  1469 catalog_manager.cc:5697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 2 to 4, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:51.287932  1715 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:02:51.265760  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:51.274783  4145 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:02:51.273487  4131 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:51.286173  3688 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:51.267684  4090 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:51.265249  2341 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:51.336858  4090 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.265307  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:51.281288  3967 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:51.338762  1719 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
W20260709 13:02:51.339736  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 71819us
I20260709 13:02:51.340699  1719 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:51.292119  4054 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:51.275871  4137 log.cc:927] Time spent T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
W20260709 13:02:51.341364  1538 leader_election.cc:336] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.170s (SENT)
I20260709 13:02:51.337359  1718 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:02:51.346966  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:51.263401  4141 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:51.371073  1721 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 8: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:51.370208  1564 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:02:51.371188  1719 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:51.376048  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:51.337404  4090 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:51.388732  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 253409us
I20260709 13:02:51.397863  4059 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.398518  4059 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:51.401751  4059 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:51.379171  4054 consensus_queue.cc:1059] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:02:51.403699  4054 raft_consensus.cc:3055] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:51.404218  4054 raft_consensus.cc:740] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:02:51.392406  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 229580us
I20260709 13:02:51.406230  1687 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:51.407591  1687 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:51.407655  1538 leader_election.cc:336] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.192s (SENT)
I20260709 13:02:51.410315  1538 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:51.413352  1570 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:51.392627  1645 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:51.328497  4135 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:51.395450  1469 catalog_manager.cc:5697] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 2 to 4, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 4 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:02:51.384690  3967 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.426507  1715 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:51.407049  3956 consensus_queue.cc:1059] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
W20260709 13:02:51.426759  4154 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:02:51.427448  4059 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:51.436020  3849 consensus_meta.cc:287] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.707s	user 0.016s	sys 0.000s
I20260709 13:02:51.409181  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:51.437665  1469 catalog_manager.cc:5697] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 3 to 4, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:51.425841  4054 consensus_queue.cc:260] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:51.438896  1687 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:51.442432  1721 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:51.457006  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:51.458535  1716 raft_consensus.cc:2393] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:51.441890  3967 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:51.447232  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:51.461148  1719 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.424791  4150 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:51.461696  1612 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:51.468429  1469 catalog_manager.cc:5697] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:02:51.470953  1469 catalog_manager.cc:5697] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 3 to 4, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
W20260709 13:02:51.474821  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 67377.5us
I20260709 13:02:51.468346  3956 consensus_queue.cc:1243] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:02:51.448469  4160 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:51.484681  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.465183  4141 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:51.467605  4150 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:51.483896  2341 raft_consensus.cc:2368] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6.
I20260709 13:02:51.500571  4150 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:51.503165  4156 log.cc:927] Time spent T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:02:51.489948  1687 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:51.504390  1537 leader_election.cc:400] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6.
I20260709 13:02:51.504901  1537 leader_election.cc:403] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:02:51.505147  3956 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:51.505683  4060 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6.
I20260709 13:02:51.475016  4049 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:51.507135  4049 raft_consensus.cc:697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:02:51.507604  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.508679  2312 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:02:51.499253  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.489583  4054 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.511001  1642 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7501558aa93542308c5fc758446387df"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:49580
I20260709 13:02:51.511530  4060 consensus_queue.cc:1059] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:51.512060  3688 consensus_queue.cc:1048] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:51.512575  4060 raft_consensus.cc:3055] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:51.514252  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:51.516041  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:51.513052  4060 raft_consensus.cc:740] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:51.518572  4060 consensus_queue.cc:260] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:51.519361  3967 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:51.505753  3956 raft_consensus.cc:697] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:51.519979  4060 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.520706  3956 consensus_queue.cc:237] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:51.500967  2340 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
W20260709 13:02:51.524013  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.523352  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:51.525889  1716 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:02:51.453567  3849 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:51.547452  1566 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:02:51.548074  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 107286us
I20260709 13:02:51.504846  1687 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:51.547060  1538 leader_election.cc:336] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.225s (SENT)
I20260709 13:02:51.591915  1721 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:51.593758  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:51.595292  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:51.596140  1564 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.601441  1570 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:51.603833  1642 raft_consensus.cc:606] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:51.604544  1642 raft_consensus.cc:614] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:51.508558  4049 consensus_queue.cc:237] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:51.542984  1469 catalog_manager.cc:5697] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 3 to 5, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:02:51.587011  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:51.487071  4160 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:51.545377  4135 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:51.620448  1571 raft_consensus.cc:2393] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:51.535732  4127 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:51.547391  4088 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.488770  4162 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:51.625427  2347 raft_consensus.cc:2368] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8.
W20260709 13:02:51.635327  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 124755us
W20260709 13:02:51.635483  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 88495.5us
I20260709 13:02:51.528661  4139 consensus_queue.cc:1059] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.099s
I20260709 13:02:51.627461  4088 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:51.620776  4160 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:51.723480  1719 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
W20260709 13:02:51.723549  1538 leader_election.cc:336] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.428s (SENT)
I20260709 13:02:51.732784  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:51.732889  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:51.736363  1564 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
W20260709 13:02:51.736717  1538 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.448s (SENT)
I20260709 13:02:51.737936  4059 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:51.737520  1538 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:51.638584  4127 raft_consensus.cc:697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:51.745656  4059 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:51.740401  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 125428us
W20260709 13:02:51.748476  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 243721us
I20260709 13:02:51.766927  1687 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:51.770598  1687 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:51.772848  1713 raft_consensus.cc:1240] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:51.774000  4143 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.716s	user 0.002s	sys 0.007s
I20260709 13:02:51.776631  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1400) took 1159 ms. Trace:
I20260709 13:02:51.777055  1643 rpcz_store.cc:276] 0709 13:02:50.616776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.617015 (+   239us) service_pool.cc:224] Handling call
0709 13:02:50.617533 (+   518us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:50.618161 (+   628us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:50.618191 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:50.687747 (+ 69556us) spinlock_profiling.cc:243] Waited 58.6 ms on lock 0x7b2c000120e8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824777ea11 00007f824778e372 00007f8248ec0a01 00007f8248e8543a 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2
0709 13:02:50.722524 (+ 34777us) log.cc:844] Serialized 52 byte log entry
0709 13:02:51.004944 (+282420us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:51.121792 (+116848us) log.cc:844] Serialized 14 byte log entry
0709 13:02:51.129266 (+  7474us) log.cc:844] Serialized 14 byte log entry
0709 13:02:51.272836 (+143570us) spinlock_profiling.cc:243] Waited 7.87 ms on lock 0x7b1800460120. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d98522 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:02:51.516431 (+243595us) spinlock_profiling.cc:243] Waited 5.76 ms on lock 0x7b1800460120. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287 00007f8248ea0208
0709 13:02:51.624364 (+107933us) spinlock_profiling.cc:243] Waited 7.34 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:02:51.640597 (+ 16233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:51.640688 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:51.776316 (+135628us) raft_consensus.cc:1685] finished
0709 13:02:51.776348 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:51.776510 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87376,"prepare.queue_time_us":301285,"prepare.run_cpu_time_us":157,"prepare.run_wall_time_us":10078,"raft.queue_time_us":93909,"raft.run_cpu_time_us":546,"raft.run_wall_time_us":54219,"spinlock_wait_cycles":183132288,"thread_start_us":403253,"threads_started":3,"wal-append.queue_time_us":303431}
I20260709 13:02:51.758236  4131 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:51.794206  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 172402us
I20260709 13:02:51.749428  3967 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:51.813925  1612 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:51.750186  4135 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:51.821704  4135 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:51.825779  4135 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:51.827960  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:51.744230  4127 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:51.746744  4059 consensus_queue.cc:1059] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 13:02:51.832600  4151 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:51.797614  4088 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:51.820478  3849 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:51.779954  3688 consensus_queue.cc:1059] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:02:51.748709  4134 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.851373  4165 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.847592  4131 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:51.806437  3817 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:51.884939  1538 leader_election.cc:336] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.838s (SENT)
I20260709 13:02:51.886173  1538 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:51.811336  3879 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:02:51.861567  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 117482us
W20260709 13:02:51.865605  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 117420us
I20260709 13:02:51.884383  4164 raft_consensus.cc:3055] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:51.898563  4164 raft_consensus.cc:740] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:51.898168  4171 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:02:51.860885  4151 consensus_queue.cc:1243] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 13:02:51.915885  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 102136us
I20260709 13:02:51.891170  4059 raft_consensus.cc:3055] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:51.940619  1612 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:51.885142  3817 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:51.900440  4164 consensus_queue.cc:260] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:51.907917  4134 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:51.947845  4090 consensus_meta.cc:287] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.532s	user 0.006s	sys 0.008s
I20260709 13:02:51.948319  4090 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:51.948889  4174 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:02:51.949226  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 76664.1us
I20260709 13:02:51.923996  3688 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:51.949581  4174 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:51.950487  1538 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 3.235s (SENT)
I20260709 13:02:51.941975  4059 raft_consensus.cc:740] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:02:51.952805  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.954005  1714 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:02:51.955735  4164 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.927175  4165 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:51.960392  3975 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.940s	user 0.011s	sys 0.007s
I20260709 13:02:51.942186  4139 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:02:51.938647  3879 raft_consensus.cc:697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:51.961833  4150 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:51.944818  3817 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:51.942330  4049 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:02:51.964782  4090 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:51.966054  4160 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:51.954107  4059 consensus_queue.cc:260] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:51.948788  4134 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:51.999660  4160 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:52.000200  3975 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:52.001603  1720 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:51.999713  4165 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:52.001869  4150 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:52.002890  3879 consensus_queue.cc:237] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:52.003968  3975 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:51.970782  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:51.983853  3935 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:52.000727  1717 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7. Ops: []
W20260709 13:02:51.970770  4142 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.911s	user 0.004s	sys 0.000s
I20260709 13:02:52.001494  4059 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:51.954155  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 788191us
I20260709 13:02:51.950403  3688 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:51.985360  3989 raft_consensus.cc:3055] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:02:52.016388  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 89745.8us
I20260709 13:02:52.025106  1611 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:52.027043  4134 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:52.028637  4134 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:52.029093  4134 raft_consensus.cc:697] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:52.024891  3989 raft_consensus.cc:740] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:52.030381  3989 consensus_queue.cc:260] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:52.031407  3989 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:52.023191  4165 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:51.984364  4174 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:52.038118  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 67483.5us
W20260709 13:02:52.039275  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.026372  1537 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:52.042135  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:52.043913  1564 raft_consensus.cc:2410] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:52.041308  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:52.044767  1571 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
W20260709 13:02:52.046576  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 200090us
I20260709 13:02:52.047465  1687 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:52.052891  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.056080  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:52.057066  1571 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:52.057189  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.059100  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:52.064785  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 191) took 1229 ms. Trace:
W20260709 13:02:52.065208  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 145897us
I20260709 13:02:52.066039  1570 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:02:52.069437  1612 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:52.064939  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 224281us
I20260709 13:02:52.065382  1717 rpcz_store.cc:276] 0709 13:02:50.832037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.904177 (+ 72140us) service_pool.cc:224] Handling call
0709 13:02:51.891751 (+987574us) spinlock_profiling.cc:243] Waited 9.85 ms on lock 0x7b580009db98. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:02:51.892017 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.019168 (+127151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:52.060840 (+ 41672us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b5400248e54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:52.061138 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271960704}
W20260709 13:02:52.072968  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 46683.3us
I20260709 13:02:52.061949  3688 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:52.077831  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.073669  3967 raft_consensus.cc:697] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:52.119196  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:52.121973  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:52.148291  4165 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:52.171613  4165 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:52.172194  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 107515us
I20260709 13:02:52.080992  3935 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:52.082516  3975 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:52.084829  4134 consensus_queue.cc:237] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:52.077860  3688 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.080956  4171 consensus_queue.cc:1059] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.055s
W20260709 13:02:52.133846  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 49006.7us
I20260709 13:02:52.086252  4179 consensus_queue.cc:1048] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:02:52.135249  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 193) took 1158 ms. Trace:
I20260709 13:02:52.138219  1537 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:52.087345  1612 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:52.086825  1611 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:52.145148  4172 raft_consensus.cc:3055] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:52.148434  2349 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:52.186993  1720 rpcz_store.cc:276] 0709 13:02:50.977161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.977498 (+   337us) service_pool.cc:224] Handling call
0709 13:02:52.135010 (+1157512us) spinlock_profiling.cc:243] Waited 992 ms on lock 0x7b540020a654. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:52.135155 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2282168576}
I20260709 13:02:52.190075  3967 consensus_queue.cc:237] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:52.195791  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 109058us
W20260709 13:02:52.197841  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 59827.5us
I20260709 13:02:52.198135  1611 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:52.180518  3688 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:52.210937  4131 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:52.219859  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 132593us
I20260709 13:02:52.194861  3975 raft_consensus.cc:697] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:52.138119  4184 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.121s
I20260709 13:02:52.226321  4175 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.188187  4135 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:52.204344  3935 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:52.246420  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 65865.5us
I20260709 13:02:52.187831  4172 raft_consensus.cc:740] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:02:52.242252  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.213884  4131 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:52.187040  3817 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:52.262426  1612 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:52.263263  1571 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:02:52.267207  2312 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:52.281526  1642 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
mode: GRACEFUL
new_leader_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:49580
I20260709 13:02:52.282408  1642 raft_consensus.cc:606] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Received request to transfer leadership to TS 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:52.282764  1642 raft_consensus.cc:614] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:52.291606  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 43571.5us
I20260709 13:02:52.248875  3935 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:52.310400  2313 leader_election.cc:336] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.934s (SENT)
I20260709 13:02:52.311059  2313 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:52.312083  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:52.262387  4060 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:52.275308  4131 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:52.317886  1570 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:02:52.281526  4174 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:52.330236  4174 raft_consensus.cc:697] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:52.331421  1611 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:52.331434  2349 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
W20260709 13:02:52.347919  2313 leader_election.cc:336] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.680s (SENT)
I20260709 13:02:52.348480  2313 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:52.314540  3935 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:52.278693  4179 consensus_queue.cc:1059] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:02:52.314328  4135 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:52.250638  4175 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:52.362743  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.369244  3849 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:52.348796  4181 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.099s
I20260709 13:02:52.374200  4135 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:52.374940  4135 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:52.369220  4174 consensus_queue.cc:237] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:52.386178  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.383535  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:52.388303  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 70706.5us
I20260709 13:02:52.329327  4170 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:52.390547  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:52.346807  3879 raft_consensus.cc:3055] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:52.411201  1648 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:52.427349  1571 raft_consensus.cc:1240] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:52.436522  1642 raft_consensus.cc:1275] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:02:52.439903  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49475.8us
I20260709 13:02:52.442291  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:52.441902  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:52.385911  4135 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:52.381533  4164 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.477594  1645 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:02:52.443272  1639 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:52.477624  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:52.443272  1643 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:52.442843  1641 pending_rounds.cc:85] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 1
I20260709 13:02:52.479619  1647 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:52.479934  1641 pending_rounds.cc:107] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 13:02:52.480343  1641 raft_consensus.cc:2889] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:52.412077  4165 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:52.482308  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:52.483953  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.488956  1640 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:52.452572  1646 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:52.318878  4192 consensus_queue.cc:1059] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.125s
I20260709 13:02:52.433411  4134 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:02:52.503852  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 293102us
I20260709 13:02:52.435390  4060 raft_consensus.cc:697] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 6 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:52.508992  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:52.511684  1648 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:52.442034  4179 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:52.429157  3879 raft_consensus.cc:740] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:52.513911  1646 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:49580
I20260709 13:02:52.516597  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:52.517699  1639 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:52.486596  4184 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:52.520548  1646 raft_consensus.cc:606] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:52.537156  1646 raft_consensus.cc:614] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:52.547005  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:52.513037  4165 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.496680  4164 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:52.436664  3975 consensus_queue.cc:237] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
W20260709 13:02:52.615559  4199 log.cc:927] Time spent T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 13:02:52.620936  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 155504us
I20260709 13:02:52.513998  4134 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:52.428210  4172 consensus_queue.cc:260] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:52.490417  4178 consensus_queue.cc:1059] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.139s
I20260709 13:02:52.597783  3817 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:52.641805  4191 log.cc:927] Time spent T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
W20260709 13:02:52.662532  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 105766us
W20260709 13:02:52.642155  4188 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.287s	user 0.000s	sys 0.001s
I20260709 13:02:52.559815  4192 consensus_queue.cc:1059] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.085s
I20260709 13:02:52.560950  4197 raft_consensus.cc:3055] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:52.559329  4165 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:52.625649  4194 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:52.642716  4207 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:52.709144  1648 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:52.690209  1647 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:02:52.739259  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:52.700800  4172 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:52.778286  1713 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:52.771296  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:52.608557  4184 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:52.705088  4197 raft_consensus.cc:740] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:02:52.819077  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.723775  4090 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:52.823546  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:52.824496  1719 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:52.826447  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:52.837154  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:02:52.839054  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.663964  4060 consensus_queue.cc:237] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:52.827080  1721 raft_consensus.cc:2376] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8. Re-sending same reply.
I20260709 13:02:52.647889  3879 consensus_queue.cc:260] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:52.837385  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:52.888751  1713 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:02:52.884272  1640 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:02:52.874010  1687 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:52.884272  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:52.897454  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.905192  4134 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:52.903174  1612 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.001s	sys 0.013s
W20260709 13:02:52.914853  1612 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.002s	sys 0.013s
I20260709 13:02:52.964074  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:52.965224  1640 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:52.985515  1719 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
W20260709 13:02:52.991775  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 743109us
I20260709 13:02:52.907372  4175 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:53.000793  3879 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:52.782213  4194 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:52.773180  4207 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:53.024052  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:53.035226  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.043200  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:53.021744  4200 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:53.047222  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 404380us
I20260709 13:02:53.031553  4054 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.028980  4175 raft_consensus.cc:3055] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:53.069960  4164 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:53.071136  4176 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.208s
I20260709 13:02:53.070619  4054 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:53.071879  4214 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:53.074832  1573 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
W20260709 13:02:53.075309  4209 log.cc:927] Time spent T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.291s	user 0.002s	sys 0.000s
W20260709 13:02:52.699635  4127 consensus_queue.cc:1175] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "066ee0109aba4465ba17fa65e14ef504" responder_term: 6 status { last_received { term: 5 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:53.071123  4213 consensus_queue.cc:1059] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.259s
I20260709 13:02:53.070132  4211 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.077812  4211 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:53.078912  4194 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:53.079957  4194 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.089429  1537 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:53.127354  1640 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:53.129935  4090 raft_consensus.cc:697] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:53.130228  1640 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:02:53.075393  4212 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
W20260709 13:02:53.131469  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 257551us
I20260709 13:02:53.080453  4211 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:53.082504  4175 raft_consensus.cc:740] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:53.106580  4164 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:53.206451  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 117137us
I20260709 13:02:53.209681  1640 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:53.217978  4224 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:53.246284  4224 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:53.247186  4224 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:52.999775  4197 consensus_queue.cc:260] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:53.218405  4090 consensus_queue.cc:237] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:53.105475  4131 consensus_meta.cc:287] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.769s	user 0.010s	sys 0.004s
I20260709 13:02:53.253305  4131 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:53.228135  1639 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.710s	user 0.012s	sys 0.000s
I20260709 13:02:53.072671  4214 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:53.257668  1639 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:02:53.292845  3956 raft_consensus.cc:3055] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:53.293960  4184 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:53.294263  3956 raft_consensus.cc:740] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:53.294963  4175 consensus_queue.cc:260] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:53.296468  4175 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:53.297623  4208 log.cc:927] Time spent T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.418s	user 0.000s	sys 0.001s
I20260709 13:02:53.299106  4218 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.299767  4218 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:53.066430  4200 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:53.303081  4233 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:53.320060  4226 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.322415  4200 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:53.284670  4164 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:53.317411  4184 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.324990  4184 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:53.320255  4233 raft_consensus.cc:697] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:53.302507  4054 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:53.328210  4139 raft_consensus.cc:3055] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:53.329119  4139 raft_consensus.cc:740] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:02:53.095707  4194 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:53.364833  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1653) took 1301 ms. Trace:
W20260709 13:02:53.300253  4196 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.611s	user 0.003s	sys 0.001s
I20260709 13:02:53.373364  1537 leader_election.cc:400] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8.
I20260709 13:02:53.374576  1537 leader_election.cc:403] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:02:53.373739  4059 raft_consensus.cc:2955] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Committing config change with OpId 5.5: config changed from index 2 to 5, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:02:53.383011  4230 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8.
I20260709 13:02:53.394305  4139 consensus_queue.cc:260] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:53.396566  4139 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:53.321256  4226 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:53.397603  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.400739  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.400779  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:53.425690  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.426095  1573 raft_consensus.cc:2410] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:53.414319  4194 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:53.426714  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 131763us
I20260709 13:02:53.365561  1564 rpcz_store.cc:276] 0709 13:02:52.049786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.063821 (+ 14035us) service_pool.cc:224] Handling call
0709 13:02:52.075698 (+ 11877us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:52.082520 (+  6822us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:52.175542 (+ 93022us) log.cc:844] Serialized 14 byte log entry
0709 13:02:52.877201 (+701659us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:52.909472 (+ 32271us) log.cc:844] Serialized 52 byte log entry
0709 13:02:52.914224 (+  4752us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:52.914362 (+   138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:52.914498 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:53.310033 (+395535us) raft_consensus.cc:1685] finished
0709 13:02:53.310062 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.351199 (+ 41137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26949,"prepare.queue_time_us":227702,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":4318,"raft.queue_time_us":318400,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":16254,"thread_start_us":720584,"threads_started":3,"wal-append.queue_time_us":1117255}
W20260709 13:02:53.436352  3688 consensus_meta.cc:287] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.214s	user 0.017s	sys 0.000s
W20260709 13:02:53.437261  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.438648  4172 consensus_meta.cc:287] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.648s	user 0.012s	sys 0.000s
W20260709 13:02:53.430519  1613 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.017s	sys 0.021s
W20260709 13:02:53.512871  1613 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.017s	sys 0.021s
W20260709 13:02:53.513687  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 68486.2us
W20260709 13:02:53.381458  3989 consensus_meta.cc:287] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.350s	user 0.011s	sys 0.005s
I20260709 13:02:53.371001  4218 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:53.528076  4224 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:53.552835  4224 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:53.553900  4225 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.554958  4225 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:53.560292  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 504236us
I20260709 13:02:53.562939  4165 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:53.517175  1717 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.898s	user 0.002s	sys 0.010s
W20260709 13:02:53.444625  3935 consensus_meta.cc:287] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.075s	user 0.010s	sys 0.006s
W20260709 13:02:53.567786  1717 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.003s	sys 0.011s
W20260709 13:02:53.579340  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 551547us
I20260709 13:02:53.515523  3688 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:53.530177  4184 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:53.590794  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.887s	user 0.014s	sys 0.032s
W20260709 13:02:53.591486  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.014s	sys 0.032s
I20260709 13:02:53.566886  4225 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:53.591069  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:53.594934  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:53.566404  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:53.595948  1564 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:53.597947  1612 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:53.610530  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.607946  4164 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:53.620990  4193 log.cc:927] Time spent T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.105s	user 0.007s	sys 0.002s
I20260709 13:02:53.623958  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:53.635069  2349 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:02:53.585048  3935 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:53.646358  4232 log.cc:927] Time spent T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:02:53.648972  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2014) took 1035 ms. Trace:
I20260709 13:02:53.649435  1720 rpcz_store.cc:276] 0709 13:02:52.613136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.613289 (+   153us) service_pool.cc:224] Handling call
0709 13:02:52.622564 (+  9275us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:52.687317 (+ 64753us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:52.687364 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:52.739799 (+ 52435us) log.cc:844] Serialized 52 byte log entry
0709 13:02:53.031285 (+291486us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:53.031412 (+   127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.031572 (+   160us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:53.648720 (+617148us) raft_consensus.cc:1685] finished
0709 13:02:53.648743 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.648899 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":497,"raft.queue_time_us":221528,"raft.run_cpu_time_us":663,"raft.run_wall_time_us":26338,"thread_start_us":186328,"threads_started":2,"wal-append.queue_time_us":687113}
I20260709 13:02:53.652581  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1444) took 1136 ms. Trace:
I20260709 13:02:53.653218  4131 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:53.653074  1643 rpcz_store.cc:276] 0709 13:02:52.516635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.535050 (+ 18415us) service_pool.cc:224] Handling call
0709 13:02:52.557413 (+ 22363us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:52.894218 (+336805us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:52.894466 (+   248us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:52.907580 (+ 13114us) log.cc:844] Serialized 52 byte log entry
0709 13:02:53.122260 (+214680us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:53.126800 (+  4540us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.130328 (+  3528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.131039 (+   711us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:53.652308 (+521269us) raft_consensus.cc:1685] finished
0709 13:02:53.652335 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.652512 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3783,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":356434,"raft.run_cpu_time_us":456,"raft.run_wall_time_us":63088,"thread_start_us":448412,"threads_started":2,"wal-append.queue_time_us":670167}
W20260709 13:02:53.659222  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.661664  4226 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:53.663437  4135 consensus_meta.cc:287] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.175s	user 0.019s	sys 0.000s
I20260709 13:02:53.663973  4135 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:53.602207  2344 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:02:53.686775  1564 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:02:53.690632  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:53.693579  1644 raft_consensus.cc:1240] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:02:53.699972  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:53.599176  4233 consensus_queue.cc:237] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:53.716130  4060 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:53.701601  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:53.704841  2347 raft_consensus.cc:2955] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 2 to 5, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:02:53.698990  4226 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:02:53.737166  4151 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.053s	user 0.029s	sys 0.030s
I20260709 13:02:53.721525  4060 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.745771  1573 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:53.731761  4175 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:02:53.749387  4249 log.cc:927] Time spent T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.057s	user 0.002s	sys 0.000s
W20260709 13:02:53.780227  4151 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.105s	user 0.029s	sys 0.030s
I20260709 13:02:53.747144  4127 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.787770  4175 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.788259  4060 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:53.738976  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:53.753232  4135 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:53.756471  4189 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:53.804065  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.768683  3935 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:53.758978  3879 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.741s	user 0.016s	sys 0.000s
W20260709 13:02:53.811873  2313 leader_election.cc:336] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.537s (SENT)
I20260709 13:02:53.748706  4226 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:53.808427  4175 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:53.820833  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.810483  4139 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
W20260709 13:02:53.826145  2314 leader_election.cc:336] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.271s (SENT)
I20260709 13:02:53.831221  2349 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:02:53.834972  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:53.835963  4233 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:53.836546  4233 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.836930  4233 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:53.828608  4229 raft_consensus.cc:3055] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:53.838443  4229 raft_consensus.cc:740] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:53.839574  4229 consensus_queue.cc:260] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:53.841094  4229 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:53.845045  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 670262us
I20260709 13:02:53.845552  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 201) took 1107 ms. Trace:
I20260709 13:02:53.845875  4195 consensus_queue.cc:1048] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:53.846235  1718 rpcz_store.cc:276] 0709 13:02:52.738747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.738909 (+   162us) service_pool.cc:224] Handling call
0709 13:02:53.845314 (+1106405us) spinlock_profiling.cc:243] Waited 938 ms on lock 0x7b540020a654. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:53.845477 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2156651264}
W20260709 13:02:53.849020  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 286441us
I20260709 13:02:53.849293  1687 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:53.851118  1687 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:53.851362  1538 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.041s (SENT)
I20260709 13:02:53.851841  1538 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:53.852905  4200 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:53.854492  4226 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:53.856289  1538 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.085s (SENT)
I20260709 13:02:53.856769  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1436) took 1444 ms. Trace:
I20260709 13:02:53.818367  4189 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:53.859288  2314 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.698s (SENT)
I20260709 13:02:53.879067  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1452) took 1040 ms. Trace:
I20260709 13:02:53.863214  4197 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:53.809747  4127 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:53.866024  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1443) took 1374 ms. Trace:
I20260709 13:02:53.867760  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1446) took 1048 ms. Trace:
I20260709 13:02:53.870388  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1445) took 1320 ms. Trace:
I20260709 13:02:53.871562  1538 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:53.871587  1644 rpcz_store.cc:276] 0709 13:02:52.412070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.412260 (+   190us) service_pool.cc:224] Handling call
0709 13:02:52.426550 (+ 14290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:53.695440 (+1268890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.856403 (+160963us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400183154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:53.856677 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2903448448}
W20260709 13:02:53.885597  2313 leader_election.cc:336] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 2.271s (SENT)
I20260709 13:02:53.899744  2313 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:53.889356  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.879773  1640 rpcz_store.cc:276] 0709 13:02:52.838412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.243920 (+405508us) service_pool.cc:224] Handling call
0709 13:02:53.244416 (+   496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:53.244754 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:53.245223 (+   469us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.878378 (+633155us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:53.878455 (+    77us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:53.878519 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.878616 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.878799 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":288351,"prepare.queue_time_us":5052,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":77,"thread_start_us":344265,"threads_started":1,"wal-append.queue_time_us":376369}
I20260709 13:02:53.896451  1648 rpcz_store.cc:276] 0709 13:02:52.819693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.819943 (+   250us) service_pool.cc:224] Handling call
0709 13:02:52.878499 (+ 58556us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.879262 (+   763us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:53.205393 (+326131us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.518004 (+312611us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.867027 (+349023us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:53.867108 (+    81us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:53.867362 (+   254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.867492 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.867677 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":354731,"prepare.queue_time_us":125453,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":90,"raft.queue_time_us":350596,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":331,"thread_start_us":585824,"threads_started":2,"wal-append.queue_time_us":436654}
W20260709 13:02:53.901908  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.904682  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.906952  1718 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:53.907763  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:53.908313  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 310480us
I20260709 13:02:53.909920  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:53.910475  1715 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:53.897843  1646 rpcz_store.cc:276] 0709 13:02:52.550457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.607336 (+ 56879us) service_pool.cc:224] Handling call
0709 13:02:52.644976 (+ 37640us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:53.120805 (+475829us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:53.120852 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:53.121669 (+   817us) log.cc:844] Serialized 52 byte log entry
0709 13:02:53.543811 (+422142us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:53.551766 (+  7955us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.869746 (+317980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.869844 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:53.869895 (+    51us) raft_consensus.cc:1685] finished
0709 13:02:53.869921 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.870299 (+   378us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":317151,"prepare.queue_time_us":7318,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":682380,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":9995,"thread_start_us":867000,"threads_started":2,"wal-append.queue_time_us":492434}
I20260709 13:02:53.914044  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:53.914645  1714 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:53.914857  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:53.915753  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:53.914765  1718 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
W20260709 13:02:53.916607  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.915747  1715 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:02:53.870282  3688 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:53.919754  4215 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:53.919906  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:53.920411  4215 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:53.921224  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:53.917215  4174 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:53.916824  4131 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:53.924006  4215 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:53.924563  4131 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:53.924896  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2015) took 1306 ms. Trace:
I20260709 13:02:53.925215  4134 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.915781  4213 consensus_queue.cc:1048] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:02:53.926003  4134 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:53.926769  4258 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 13:02:53.927237  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.927842  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.923493  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:53.896317  1642 rpcz_store.cc:276] 0709 13:02:52.491173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.512337 (+ 21164us) service_pool.cc:224] Handling call
0709 13:02:52.534099 (+ 21762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.534312 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:52.660601 (+126289us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.157824 (+497223us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.865207 (+707383us) spinlock_profiling.cc:243] Waited 8.73 ms on lock 0x7b18003d4200. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d12c0a 00007f824f452219 00007f8245d9aa40 00007f8245d954e4 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287 00007f8248ea0208
0709 13:02:53.865439 (+   232us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:53.865509 (+    70us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:53.865567 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.865697 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.865926 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":664159,"prepare.queue_time_us":341308,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":715961,"raft.run_cpu_time_us":409,"raft.run_wall_time_us":32445,"spinlock_wait_cycles":62430976,"thread_start_us":587494,"threads_started":2,"wal-append.queue_time_us":996580}
I20260709 13:02:53.923872  4135 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.925117  4131 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:53.852820  3956 consensus_queue.cc:260] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:53.938082  3956 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:53.938474  4228 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.939436  4131 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:53.939162  4228 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:53.929697  1611 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:53.918733  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:53.941526  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:53.941800  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:53.930219  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1447) took 1102 ms. Trace:
I20260709 13:02:53.943152  1647 rpcz_store.cc:276] 0709 13:02:52.828137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.828307 (+   170us) service_pool.cc:224] Handling call
0709 13:02:52.892307 (+ 64000us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.892550 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:53.129934 (+237384us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.929592 (+799658us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.929808 (+   216us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:53.929860 (+    52us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:53.929911 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.930009 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.930149 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":370593,"prepare.queue_time_us":138656,"prepare.run_cpu_time_us":134,"prepare.run_wall_time_us":3908,"raft.queue_time_us":361823,"spinlock_wait_cycles":11716480,"thread_start_us":650731,"threads_started":2,"wal-append.queue_time_us":561373}
W20260709 13:02:53.929059  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 55161.9us
I20260709 13:02:53.896311  3975 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:53.933872  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 77647.8us
W20260709 13:02:53.947965  1538 leader_election.cc:336] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.107s (SENT)
W20260709 13:02:53.948752  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.949198  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.949626  1538 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.122s (SENT)
I20260709 13:02:53.935348  4134 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:53.936277  4247 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.936128  4135 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:53.936697  4175 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:53.938607  1722 raft_consensus.cc:2955] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 2 to 5, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:02:53.946118  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 60594.4us
I20260709 13:02:53.915505  4226 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:53.925261  1717 rpcz_store.cc:276] 0709 13:02:52.618399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.618571 (+   172us) service_pool.cc:224] Handling call
0709 13:02:52.619068 (+   497us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:53.582621 (+963553us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:53.582662 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:53.607122 (+ 24460us) log.cc:844] Serialized 52 byte log entry
0709 13:02:53.720974 (+113852us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:53.793581 (+ 72607us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.924584 (+131003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.924656 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:53.924704 (+    48us) raft_consensus.cc:1685] finished
0709 13:02:53.924718 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.924842 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":166038,"prepare.queue_time_us":106097,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":278306,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":44116,"thread_start_us":1117189,"threads_started":3,"wal-append.queue_time_us":101477}
I20260709 13:02:53.956348  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:53.964849  1715 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:53.958020  1646 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:49580
I20260709 13:02:53.965682  1646 raft_consensus.cc:606] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:53.966372  1646 raft_consensus.cc:614] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:53.958137  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:53.902220  4233 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:53.918784  4254 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:53.962929  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.953142  4131 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:53.929021  4211 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.930991  3967 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.047s
W20260709 13:02:53.983417  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 132651us
W20260709 13:02:53.970172  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 54720.5us
I20260709 13:02:53.954078  3975 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:53.972946  2313 leader_election.cc:336] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.830s (SENT)
I20260709 13:02:53.987038  1721 raft_consensus.cc:1240] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:53.962713  1538 leader_election.cc:336] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.902s (SENT)
W20260709 13:02:53.990420  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 44855.3us
I20260709 13:02:53.994666  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:53.996393  3989 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:53.989718  4254 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:53.998083  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:53.997901  4211 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:02:54.000670  4211 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:53.982553  4131 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:54.001387  4131 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:54.001873  4233 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:54.010385  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 47687.1us
W20260709 13:02:54.011644  1538 leader_election.cc:336] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.201s (SENT)
I20260709 13:02:54.012234  1538 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:54.003801   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:02:54.014360  1565 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:02:54.015808  1569 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:54.016191  4230 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:54.017050  1568 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:54.016866  4230 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:54.017553  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:54.017776  4150 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:54.018251  2340 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:54.019809  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1441) took 1537 ms. Trace:
I20260709 13:02:54.022692  4211 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:54.022603  4206 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:54.023801  4211 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:54.025678  4214 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:53.990964  3975 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:54.028582  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 44219.3us
I20260709 13:02:53.961036  4247 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:54.029888  4260 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:02:54.034162  4247 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:53.996436  4231 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:54.035213  4135 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:53.997047  3989 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.035630  4231 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:54.036271  3989 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:54.036660  1687 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:54.029220  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:54.038491  1567 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:54.038591  4127 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:54.020253  1645 rpcz_store.cc:276] 0709 13:02:52.482264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.498643 (+ 16379us) service_pool.cc:224] Handling call
0709 13:02:52.518285 (+ 19642us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:52.604338 (+ 86053us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:52.667420 (+ 63082us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.866575 (+1199155us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:53.868433 (+  1858us) log.cc:844] Serialized 52 byte log entry
0709 13:02:53.868640 (+   207us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:53.868684 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.868958 (+   274us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:54.019456 (+150498us) raft_consensus.cc:1685] finished
0709 13:02:54.019479 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.019627 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":645818,"prepare.queue_time_us":384928,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":942839,"raft.run_cpu_time_us":568,"raft.run_wall_time_us":80577,"thread_start_us":629919,"threads_started":3,"wal-append.queue_time_us":938857}
I20260709 13:02:54.042273  1687 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:54.043411  1687 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:54.044260  4151 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:54.044646  1687 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:54.044963  4151 raft_consensus.cc:697] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:54.046257  4151 consensus_queue.cc:237] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:54.070098  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.069337  1715 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:02:53.987591  4228 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:54.186228  4230 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:54.193965  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 129205us
I20260709 13:02:54.073747  4233 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:54.198478  4220 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:54.196414  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 184818us
I20260709 13:02:54.020401  4254 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:54.079967  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 138736us
I20260709 13:02:54.199401  4220 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:54.200827  1538 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.221s (SENT)
I20260709 13:02:54.086167  4150 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:54.201498  1538 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:54.198992  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:54.087455  1717 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:54.202752  1716 raft_consensus.cc:2410] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:54.204102  4211 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:54.205289  4230 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:54.214051  4175 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:54.087121  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:54.216030  4189 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:54.196763  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 44961.2us
I20260709 13:02:54.209465  4206 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:02:54.114027  4247 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:54.220584  4247 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:54.225788  4247 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:54.209229  4207 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:54.194327  4231 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:54.226465  1538 leader_election.cc:336] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.692s (SENT)
I20260709 13:02:54.237203  2313 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:02:54.078490  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 58166.8us
I20260709 13:02:54.241284  1538 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:54.239624  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.239027   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:54.244902  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:54.200889  1611 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:54.095878  2314 leader_election.cc:336] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 3.307s (SENT)
I20260709 13:02:54.067075  4174 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:54.249254  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 48477us
I20260709 13:02:54.211076  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:54.248390  2314 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:54.259917  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.272405  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:54.073424  4088 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:54.275583  1573 raft_consensus.cc:1275] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:54.272763  4150 raft_consensus.cc:697] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:02:54.277052  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 181227us
I20260709 13:02:54.280484  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:54.277827  4172 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:54.281407  1566 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:54.302834  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:54.303838  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:54.305308  1566 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:54.282791  4150 consensus_queue.cc:237] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:54.277303  4254 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.316066  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:54.285111  4179 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.263914  4203 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:54.248358  4189 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:54.286515  4244 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:54.287140  4195 consensus_queue.cc:1059] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:54.292209  4206 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:54.292326  4060 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:54.293330  2343 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEARNER]: Advancing to term 6
I20260709 13:02:54.294106  4090 consensus_queue.cc:1059] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:54.329890  4090 consensus_queue.cc:1243] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer 066ee0109aba4465ba17fa65e14ef504 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:02:54.272244  3879 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.330417  3967 raft_consensus.cc:3055] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:54.263597  4197 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:54.331017  3967 raft_consensus.cc:740] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:54.298813  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:54.279472  1565 raft_consensus.cc:1275] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:54.332412  3967 consensus_queue.cc:260] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:54.300791  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.300694  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:54.354811  1718 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:54.314469  3975 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:54.360873  1720 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 6: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:54.279521  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:54.362315  3975 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:54.302496  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:54.306828  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.366477  3967 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:54.075574  4127 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:54.367789  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:54.087147  4253 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:54.273428  4228 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:54.316817  1717 raft_consensus.cc:2376] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8. Re-sending same reply.
I20260709 13:02:54.088339  4135 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.227s
I20260709 13:02:54.324446  4244 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:54.328683  4195 raft_consensus.cc:3055] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:54.398782  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:54.399286  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.399444  1647 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:54.331168  4060 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:54.277208  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:54.330760  3879 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:54.321702  4179 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:54.334470  4189 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:54.335824  2348 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:54.336349  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:54.319597  4254 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:54.321995  4203 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.341724  2340 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:02:54.300840  4174 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:54.247080  1567 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:54.397826  4253 raft_consensus.cc:697] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:54.409978  1642 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:54.416568  1565 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:54.325305  4088 raft_consensus.cc:697] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:54.391736  4131 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:54.396309  4134 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:54.445542  3688 consensus_queue.cc:1059] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:02:54.390985  4211 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:54.457056  1537 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:54.416568  1647 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:02:54.459216  1568 raft_consensus.cc:2410] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:02:54.475198  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.368217  4127 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:54.447072  3989 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:54.450999  4230 consensus_queue.cc:1059] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.071s
W20260709 13:02:54.491132  4278 log.cc:927] Time spent T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:02:54.492101  3879 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:54.495200  4179 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:54.395962  1718 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:02:54.421156  4195 raft_consensus.cc:740] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:54.420538  4228 raft_consensus.cc:697] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 5 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:54.446298  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:54.516949  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.501982  4203 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:54.473007  4197 consensus_queue.cc:1059] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 13:02:54.761346  4135 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:54.601107  1646 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:54.741412  4244 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:54.641898  4189 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:54.768008  4135 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:54.779582  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.786100  1720 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:02:54.775640  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.801522  1640 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:02:54.545739  1645 raft_consensus.cc:1240] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:54.580719  2341 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:54.775048  4135 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:54.786880  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.803421  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:54.817947  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:54.815040  2348 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:54.818727  1722 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
W20260709 13:02:54.657351  4281 log.cc:927] Time spent T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:02:54.657833  4060 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:54.641251  4175 raft_consensus.cc:3055] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:54.796972  4127 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:54.817060  4189 raft_consensus.cc:3055] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:54.642545  4174 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:54.603313  4274 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:02:54.827610  1641 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:54.774353  4228 consensus_queue.cc:237] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:54.546654  4211 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:54.775606  3989 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:54.831951  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 364471us
I20260709 13:02:54.841202  1640 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:02:54.853541  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.855336  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.856199  1538 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.652s (SENT)
I20260709 13:02:54.833484  4175 raft_consensus.cc:740] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:54.547158  4134 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:54.817190  4185 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
W20260709 13:02:54.843502  4284 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 13:02:54.821182  4172 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:02:54.833276  4131 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:54.834923  4189 raft_consensus.cc:740] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:54.894018  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:54.894966  1640 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:54.878841  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 421907us
I20260709 13:02:54.899988  1537 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:02:54.918139  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.888724  4277 log.cc:927] Time spent T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
W20260709 13:02:54.890177  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.820725  4088 consensus_queue.cc:237] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:54.889794  2343 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEARNER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:54.658172  4253 consensus_queue.cc:237] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:54.974494  4211 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:54.940750  4275 log.cc:927] Time spent T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.208s	user 0.000s	sys 0.002s
I20260709 13:02:54.847790  4195 consensus_queue.cc:260] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:54.979100  4242 log.cc:927] Time spent T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:02:54.976598  4220 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:54.998162  1648 raft_consensus.cc:1275] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:02:54.880375  1538 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.656s (SENT)
W20260709 13:02:55.011941  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.017799  1538 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:55.019717  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.024204  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.026399  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.002219  4195 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:55.035528  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:55.037240  1648 raft_consensus.cc:2410] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:54.985044  4175 consensus_queue.cc:260] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:55.046590  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1480) took 1040 ms. Trace:
I20260709 13:02:55.047226  1564 rpcz_store.cc:276] 0709 13:02:54.006303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.045880 (+ 39577us) service_pool.cc:224] Handling call
0709 13:02:54.046302 (+   422us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:54.046711 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:54.046755 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:54.047399 (+   644us) log.cc:844] Serialized 52 byte log entry
0709 13:02:54.398390 (+350991us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:54.398445 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.398525 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:55.046341 (+647816us) raft_consensus.cc:1685] finished
0709 13:02:55.046362 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:55.046524 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3879,"raft.queue_time_us":4206,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":13900,"thread_start_us":350366,"threads_started":1,"wal-append.queue_time_us":478002}
I20260709 13:02:55.050060  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.064558  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.068449  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:55.072737  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:55.075330  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:55.088799  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.095952  1648 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:55.040613  4224 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:55.107357  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.108441  1564 raft_consensus.cc:2410] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:55.110308  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.110616  4289 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:55.111238  1570 raft_consensus.cc:2410] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:55.111181  4289 raft_consensus.cc:697] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:55.015332  4189 consensus_queue.cc:260] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:55.113027  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.115410  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:55.062333  4220 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:55.134838  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.137595  1640 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:55.103349  4175 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:55.138587  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 237299us
I20260709 13:02:55.153880  3989 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:02:55.154175  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 273853us
I20260709 13:02:55.154718  3989 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:55.155925  1644 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 5 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:55.166806  4276 log.cc:927] Time spent T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.215s	user 0.000s	sys 0.002s
I20260709 13:02:55.167794  4289 consensus_queue.cc:237] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:55.174074  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.174304  4293 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:55.176235  4286 log.cc:927] Time spent T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
I20260709 13:02:55.179693  1645 raft_consensus.cc:1240] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260709 13:02:55.180224  4279 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.332s	user 0.002s	sys 0.003s
I20260709 13:02:55.182498  1640 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:55.184092  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:55.185977  4134 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:55.186933  4293 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:55.187649  4293 raft_consensus.cc:697] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:55.189431  4293 consensus_queue.cc:237] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:55.190275  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:55.191342  1722 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:55.192485  1685 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:55.147735  4295 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:02:55.197746  4229 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:55.121024  4185 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.200461  4134 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:55.200680  4185 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:55.201678  4134 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:55.202905  4239 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.203061  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:55.203584  4239 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:55.204706  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:55.205693  2343 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:55.148650  4189 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:55.208606  1642 raft_consensus.cc:1275] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:55.139045  4220 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:55.212713  4220 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:55.214284  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:55.204849  4134 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:55.119320  4224 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:55.112129  3849 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.168167  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:55.220031  1538 leader_election.cc:336] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 4.365s (SENT)
I20260709 13:02:55.161525  3989 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:55.222379  1568 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:55.257728  1538 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:55.258009  1716 raft_consensus.cc:2410] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:55.225495  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:55.227385  4254 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:55.236188  1715 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:55.241422  1648 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:55.243510  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:55.204463  4185 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:55.245884  1722 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
W20260709 13:02:55.253274  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.253394  4296 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.200390  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:55.280643  1646 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:55.280035  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.290205  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.309330  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 52248.7us
I20260709 13:02:55.240923  4295 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.246901  4160 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:55.328867  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.329360  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.330355  1570 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:02:55.339759  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.340540  1568 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:55.354282  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:55.356375  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:55.357689  1647 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:02:55.359190  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:55.360102  1640 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:55.364697  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:55.365581  3688 raft_consensus.cc:3055] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:55.365923  1643 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:55.366202  3688 raft_consensus.cc:740] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:55.371836  3688 consensus_queue.cc:260] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:55.373714  3688 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:55.379937  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:55.276005   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:55.390357  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 79228.8us
I20260709 13:02:55.392221  1685 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:55.331866  4293 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:55.398132  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 178131us
W20260709 13:02:55.343537  3967 consensus_meta.cc:287] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.974s	user 0.008s	sys 0.007s
W20260709 13:02:55.413225  1538 leader_election.cc:336] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.801s (SENT)
I20260709 13:02:55.328269  4239 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:55.265345  4205 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.151s
I20260709 13:02:55.264590  4220 consensus_queue.cc:1048] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.127s
W20260709 13:02:55.453192  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 61115.3us
I20260709 13:02:55.326612  4296 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:55.341686  4295 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:55.438499  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:55.438499  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.419646  4293 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.474880  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.432410  1646 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:55.462388  1713 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:55.264833  3849 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:55.415781  4300 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:02:55.474507  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 231137us
W20260709 13:02:55.475327  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 62176.6us
W20260709 13:02:55.489329  1538 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.843s (SENT)
I20260709 13:02:55.477306  1573 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:55.476734  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.439091  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:55.493866  1643 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:55.495030  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:55.496871  1567 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:55.475284  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.490150  1538 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:55.498071  1569 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:02:55.504081  1565 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 6: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:02:55.517706  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.519613  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:55.520397  1572 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:55.446321  4246 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.521982  4246 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:55.526103  4292 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.308s	user 0.001s	sys 0.001s
I20260709 13:02:55.529513  3817 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:55.530829  4224 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:55.532444  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 60873.8us
I20260709 13:02:55.534667  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.536218  4151 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:55.536907  4151 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:55.539027  1611 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:55.534695  4293 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:55.540758  4300 raft_consensus.cc:697] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:02:55.539047  4299 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.268s	user 0.000s	sys 0.002s
I20260709 13:02:55.542233  4300 consensus_queue.cc:237] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:55.543675  4228 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:55.544145  4228 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:55.544543  4228 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:55.526126  1687 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:02:55.550433  4203 consensus_meta.cc:287] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.731s	user 0.011s	sys 0.004s
I20260709 13:02:55.550211  4239 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:02:55.562570  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 73521.2us
I20260709 13:02:55.563609  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.523855  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:55.564695  1538 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.521s (SENT) [suppressed 34 similar messages]
I20260709 13:02:55.564749  1566 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:02:55.458091  4181 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:55.562945  1570 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:55.583534  1570 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:55.578271  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 208) took 1300 ms. Trace:
W20260709 13:02:55.580312  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45938.6us
I20260709 13:02:55.587193  1567 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:55.585883  1713 rpcz_store.cc:276] 0709 13:02:54.278233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.299439 (+ 21206us) service_pool.cc:224] Handling call
0709 13:02:54.299765 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.485629 (+1185864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.577497 (+ 91868us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400248e54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:55.578034 (+   537us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2651973504}
I20260709 13:02:55.598416  1566 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 13:02:55.599040  1566 pending_rounds.cc:85] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 2
I20260709 13:02:55.599574  1566 pending_rounds.cc:107] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:02:55.600075  1566 raft_consensus.cc:2889] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:55.600369  1566 pending_rounds.cc:107] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:02:55.600643  1566 raft_consensus.cc:2889] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:55.505350  4171 consensus_queue.cc:1048] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.098s
I20260709 13:02:55.605340  1565 raft_consensus.cc:1275] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:55.611593  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.551437  4239 raft_consensus.cc:697] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:55.550940  4203 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:55.636404  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.637442  1569 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:02:55.639078  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.642097  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:55.642469  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 51005.3us
I20260709 13:02:55.643450  1566 raft_consensus.cc:2410] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:55.659377  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.659677  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.660835  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.661180  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.662953  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.660537  1570 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:55.667014  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.600649  4134 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:02:55.608458  4220 consensus_queue.cc:1059] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:02:55.669885  1570 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:55.645741  3849 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:55.680991  1572 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
W20260709 13:02:55.685606  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.632786  3817 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:02:55.633558  4088 consensus_queue.cc:1059] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:02:55.691439  1570 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:55.656464  2343 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:02:55.671136  4296 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:55.705596  1565 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:55.645324  4229 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:02:55.659019  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:55.709074  4246 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:55.709836  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:55.711259  4181 raft_consensus.cc:3055] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:55.675925  4293 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:55.737942  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:55.662537  4205 consensus_queue.cc:1059] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:02:55.712116  2342 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
W20260709 13:02:55.748420  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 337476us
I20260709 13:02:55.700493  4233 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:55.754673  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:55.736218  4296 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.736374  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:55.712473  4151 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:55.779338  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.795259  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.796093  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.765679  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:02:55.778415  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.799674  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:55.800195  2347 raft_consensus.cc:2393] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:02:55.807878  2314 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.864s (SENT)
W20260709 13:02:55.783730  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.748665  4181 raft_consensus.cc:740] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:02:55.810494  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.799927  2312 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:55.816028  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:55.812834  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:55.827687  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:55.837067  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:55.840631  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:02:55.827180  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 185134us
I20260709 13:02:55.800361  4135 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.792596  4295 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:55.848050  1687 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:55.852658  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 288377us
W20260709 13:02:55.863757  1538 leader_election.cc:336] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.641s (SENT)
W20260709 13:02:55.841307  4189 consensus_meta.cc:287] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.633s	user 0.016s	sys 0.000s
W20260709 13:02:55.870949  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 58213.4us
I20260709 13:02:55.811234  4296 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:55.923631  1648 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:02:55.923234  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 75266.1us
I20260709 13:02:55.925611  1687 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:55.915357  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 51661.5us
I20260709 13:02:55.761678  4239 consensus_queue.cc:237] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:55.906302  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:02:55.917430  4195 consensus_meta.cc:287] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.879s	user 0.005s	sys 0.012s
W20260709 13:02:55.929979  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 286250us
I20260709 13:02:55.927210  1643 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 6. Ops: [4.3-4.3]
W20260709 13:02:55.933224  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 283705us
I20260709 13:02:55.921056  4312 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:55.928869  1538 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.648s (SENT)
I20260709 13:02:55.938920  1648 raft_consensus.cc:1217] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Deduplicated request from leader. Original: 3.2->[4.3-4.3]   Dedup: 4.3->[]
I20260709 13:02:55.947963  1648 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:02:55.948269  1538 leader_election.cc:336] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.705s (SENT)
I20260709 13:02:55.912508  4214 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:55.948889  1538 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:55.950181  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:55.951237  1640 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 6: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:55.926183  2344 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:55.928416  4171 raft_consensus.cc:3055] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:55.868041  4135 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:55.907239  3849 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
W20260709 13:02:55.964932  1538 leader_election.cc:336] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.816s (SENT)
I20260709 13:02:55.963356  1648 raft_consensus.cc:1275] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:55.986922  1538 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:55.976759  1640 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:55.947518  4312 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.973284  4197 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:55.949420  2343 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
W20260709 13:02:55.982656  4304 log.cc:927] Time spent T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.307s	user 0.002s	sys 0.000s
I20260709 13:02:56.008433  1646 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:02:56.003518  1568 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:55.974747  4214 raft_consensus.cc:697] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:55.976354  4171 raft_consensus.cc:740] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:56.000116  4315 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:56.038209  4328 raft_consensus.cc:1064] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:02:55.999845  4312 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:55.981581  4224 consensus_queue.cc:1048] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.043s
W20260709 13:02:56.031901  4316 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.281s	user 0.001s	sys 0.001s
I20260709 13:02:56.030687  4324 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:56.084352  4324 raft_consensus.cc:697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:56.085448  4324 consensus_queue.cc:237] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:56.086930  4171 consensus_queue.cc:260] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:56.090372  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 125496us
W20260709 13:02:56.091115  1538 leader_election.cc:336] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.952s (SENT)
I20260709 13:02:56.091521  1538 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:56.092128  1640 raft_consensus.cc:1240] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:02:56.093626  4246 consensus_queue.cc:1059] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:56.094364  1648 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 13:02:56.061137  4328 raft_consensus.cc:1068] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838: Unable to promote non-voter cf2f71592be8465fb18e36e15d0b2f89: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:02:56.062552  4135 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:56.062327  4214 consensus_queue.cc:237] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:56.064198  4313 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:02:56.067468  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:56.068742  4296 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:56.077293  4321 log.cc:927] Time spent T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:02:56.014971  4318 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:02:56.098232  1538 leader_election.cc:336] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.003s (SENT)
I20260709 13:02:56.114732  1640 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:02:56.125934  4318 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:56.033377  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 234610us
I20260709 13:02:56.127534  2312 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:56.128823  4335 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:56.129285  4335 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:56.129920  4335 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:56.133368  1538 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.056s (SENT)
W20260709 13:02:56.119892  4228 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.575s	user 0.011s	sys 0.008s
I20260709 13:02:56.137166  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:56.134048  1538 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:56.142149  1538 leader_election.cc:336] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.083s (SENT)
I20260709 13:02:56.142823  1538 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:56.143905  4296 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:56.144743  4296 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:56.141801  4228 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:56.102669  4315 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:56.028187  4331 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:56.146610  4331 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.147008  4331 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:56.148104  4228 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:56.156345  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:56.157069  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:56.157845  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.168555  4318 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:56.183868  2342 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55890 (request call id 253) took 1884 ms (client timeout 1597 ms). Trace:
I20260709 13:02:56.184774  2343 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:02:56.188191  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.190007  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:56.190737  1567 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:56.197182  4295 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:56.199200  4295 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:56.199911  4295 consensus_queue.cc:1243] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer cf2f71592be8465fb18e36e15d0b2f89 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:02:56.201392  1538 leader_election.cc:336] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.189s (SENT)
I20260709 13:02:56.202189  1538 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:56.202924  4320 log.cc:927] Time spent T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.251s	user 0.001s	sys 0.001s
I20260709 13:02:56.203128  4226 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:02:56.205168  1538 leader_election.cc:336] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.272s (SENT)
W20260709 13:02:56.200030  4327 log.cc:927] Time spent T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:02:56.226897  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.232484  1644 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:56.235832  1538 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:56.246889  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.253136  1640 raft_consensus.cc:1217] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Deduplicated request from leader. Original: 3.3->[5.4-5.4]   Dedup: 5.4->[]
I20260709 13:02:56.256209  1640 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 13:02:56.256743  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 328440us
W20260709 13:02:56.262751  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 57616.9us
I20260709 13:02:56.262786  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.221985  4171 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:56.267894  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1668) took 1149 ms. Trace:
I20260709 13:02:56.268702  1567 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:02:56.296579  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:56.282958  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.295207  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:56.297923  1648 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:56.296860  1564 rpcz_store.cc:276] 0709 13:02:55.118727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.149876 (+ 31149us) service_pool.cc:224] Handling call
0709 13:02:55.183401 (+ 33525us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.183657 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:55.332211 (+148554us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.266792 (+934581us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b180039b020. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d12c0a 00007f824f452219 00007f8245d9aa40 00007f8245d954e4 00007f8245d95455 00007f824c87d3c7 00007f8248e8f9a7 00007f8248e9f0bf 00007f8248e9f038
0709 13:02:56.267061 (+   269us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.267355 (+   294us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:56.267434 (+    79us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:56.267506 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.267607 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.267810 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":122977,"prepare.queue_time_us":727506,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":90,"raft.queue_time_us":8832,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":17579,"spinlock_wait_cycles":5585280,"thread_start_us":627133,"threads_started":2,"wal-append.queue_time_us":600240}
I20260709 13:02:56.225765  4203 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:56.290200  1647 raft_consensus.cc:1275] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:56.292068  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:56.303442  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.290529  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:56.291644  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:56.291318  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.304512  1646 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:56.291177  1642 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:56.293754  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.281843  4315 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:56.287995  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:56.184437  2342 rpcz_store.cc:269] 0709 13:02:54.299897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.306382 (+  6485us) service_pool.cc:224] Handling call
0709 13:02:56.183571 (+1877189us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400712854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:56.183754 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2729035648}
I20260709 13:02:56.305912  1611 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:56.292376  4319 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.344s	user 0.004s	sys 0.001s
I20260709 13:02:56.309742  4312 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:56.310565  4181 consensus_queue.cc:260] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:56.312597  1641 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:56.316221  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.311990  1468 catalog_manager.cc:5697] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 2 to 5, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:56.313936  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:56.314549  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.315871  1685 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:56.315183  1640 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:56.325482  1639 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:56.323257  4134 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:56.331768  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.265105  4228 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.309818  2343 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:02:56.319758  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 61275.8us
I20260709 13:02:56.346912  1646 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:02:56.354452  1644 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:02:56.400378  1640 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:56.407701  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.408874  1640 raft_consensus.cc:2410] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:56.409999  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:56.425155  1648 raft_consensus.cc:2410] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:02:56.435185  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 76765us
I20260709 13:02:56.321538  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:56.152971  4335 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:56.447628  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.440062  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.448920  1645 raft_consensus.cc:2393] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:56.309861  4315 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:56.466959  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 124118us
I20260709 13:02:56.465147  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:56.467285  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 151524us
I20260709 13:02:56.472733  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.490592  1644 raft_consensus.cc:2376] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6. Re-sending same reply.
I20260709 13:02:56.438366  4181 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:56.450227  4300 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.000s	sys 0.048s
I20260709 13:02:56.494963  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.498291  1644 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 6: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:02:56.500998  1686 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.000s	sys 0.036s
W20260709 13:02:56.503110  1686 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.000s	sys 0.037s
I20260709 13:02:56.436733  4189 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
W20260709 13:02:56.504109  2313 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.012s	sys 0.050s
I20260709 13:02:56.436777  4228 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:56.517385  2313 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.012s	sys 0.050s
W20260709 13:02:56.518235  1573 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.013s	sys 0.009s
W20260709 13:02:56.497411  4300 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.000s	sys 0.049s
W20260709 13:02:56.521148  1573 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.013s	sys 0.009s
W20260709 13:02:56.523123  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:56.524268  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:56.501523  4088 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.007s	sys 0.015s
I20260709 13:02:56.530011  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.532322  1645 raft_consensus.cc:2410] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:02:56.529971  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.013s	sys 0.035s
I20260709 13:02:56.533193  1644 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:02:56.525766  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 48284.2us
W20260709 13:02:56.533160  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.013s	sys 0.035s
I20260709 13:02:56.525665  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:56.502888  2344 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:02:56.539593  1648 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:02:56.540715  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 46784.8us
W20260709 13:02:56.528805  4088 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.007s	sys 0.016s
I20260709 13:02:56.603269  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:56.604761  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:56.604264  1648 raft_consensus.cc:2376] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3. Re-sending same reply.
I20260709 13:02:56.606632  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:56.607146  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 65453.9us
I20260709 13:02:56.607395  1718 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
W20260709 13:02:56.608251  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 72397.1us
I20260709 13:02:56.611011  1718 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:02:56.611649  1718 pending_rounds.cc:85] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 6
I20260709 13:02:56.612061  1718 pending_rounds.cc:107] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:02:56.612462  1718 raft_consensus.cc:2889] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:56.624344  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:56.624810  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:56.529132  4214 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:56.641338  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.643724  1687 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:02:56.645668  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.647318  1645 raft_consensus.cc:2410] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:56.650118  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:56.533600  4338 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:56.658373  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:56.659679  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 442580us
W20260709 13:02:56.660928  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.680071  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:56.684309  1687 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:02:56.632462  4246 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.022s	sys 0.034s
W20260709 13:02:56.693259  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.695587  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.696369  1644 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:02:56.702179  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:56.703538  1715 raft_consensus.cc:2410] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:02:56.704288  2312 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.000s	sys 0.045s
W20260709 13:02:56.706245  2312 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.000s	sys 0.045s
I20260709 13:02:56.706282  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.708087  1640 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:02:56.711819  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.737852  1640 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:56.746430  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.747731  1648 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 6: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:02:56.696305  4246 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.022s	sys 0.035s
I20260709 13:02:56.601334  4160 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.095s
W20260709 13:02:56.710526  4239 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.016s	sys 0.029s
I20260709 13:02:56.775069  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.683172  4220 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.077s
I20260709 13:02:56.761091  4246 consensus_queue.cc:1243] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer 066ee0109aba4465ba17fa65e14ef504 log is divergent from this leader: its last log entry 7.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:02:56.610262  4331 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:56.696931  4289 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:56.776998  1645 raft_consensus.cc:2410] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:02:56.762233  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.769153  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.784225  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:56.656814  4214 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:56.796633  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 112634us
I20260709 13:02:56.753495  4189 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:56.698699  4229 consensus_queue.cc:1048] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.075s
I20260709 13:02:56.751714  4139 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:56.807310  1646 raft_consensus.cc:2410] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:56.808681  1640 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:02:56.861127  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:56.810837  1644 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 6: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:56.812700  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:56.864176  1719 raft_consensus.cc:2410] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:02:56.778131  4239 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.016s	sys 0.030s
I20260709 13:02:56.815819  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:56.762027  3817 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:56.773149  4312 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:56.823186  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.837448  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.799988  4195 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:56.839993  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:56.843334  1565 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.014s	sys 0.005s
I20260709 13:02:56.895553  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:56.896258  1565 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.014s	sys 0.005s
I20260709 13:02:56.844894  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:56.769812  3989 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:02:56.807909  4246 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:56.823822  4134 raft_consensus.cc:697] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:02:56.901850  4195 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:56.849195  1687 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:56.913297  1719 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:56.862057  1564 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:56.884466  1647 raft_consensus.cc:2410] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:02:56.883507  2314 leader_election.cc:336] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.210s (SENT)
I20260709 13:02:56.866953  1645 raft_consensus.cc:2376] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6. Re-sending same reply.
I20260709 13:02:56.885737  1648 raft_consensus.cc:2410] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:56.894587  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:56.896715  1640 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:02:56.916232  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:56.897224  1721 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:02:56.897907  1642 raft_consensus.cc:2376] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6. Re-sending same reply.
I20260709 13:02:56.901852  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:56.918671  1568 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:56.829723  4289 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:56.924190  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 75230.8us
I20260709 13:02:56.911352  4336 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:56.873582  4315 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:56.936354  1687 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:56.929616  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:56.885898  3817 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:02:56.959795  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.968894  1687 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:56.967864  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:56.911406  4195 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:56.974664  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 335462us
I20260709 13:02:56.863261  4189 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:56.923775  4312 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:56.984452  1648 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:56.978873  1685 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:02:56.941286  4345 log.cc:927] Time spent T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:02:57.020716  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.020493  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:56.887063  4139 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:57.022737  1642 raft_consensus.cc:2410] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:56.939359  4343 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:56.983383  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.035235  1645 raft_consensus.cc:2393] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:57.044555  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.018276  3817 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:56.997538  1564 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:02:57.056705  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:57.053930  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 75157.3us
I20260709 13:02:57.057889  1642 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:02:57.062801  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.063755  1647 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:57.065485  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:57.072288  1648 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:02:57.069613  1687 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:56.982190  4220 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:57.083514  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.084174  1716 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:02:56.918900  4246 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:56.986351  4342 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.076s	user 0.000s	sys 0.003s
I20260709 13:02:57.095141  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:57.108865  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 50828.8us
I20260709 13:02:57.011466  4239 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:57.121680  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.124990  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 55491.9us
I20260709 13:02:57.097568  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.062299  4343 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:02:57.135562  4358 log.cc:927] Time spent T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.206s	user 0.002s	sys 0.001s
W20260709 13:02:57.167858  4356 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.234s	user 0.000s	sys 0.000s
I20260709 13:02:57.150520  1685 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:57.098026  4335 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:57.199589  2313 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:57.203023  1645 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:02:57.084275  4090 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:57.159734  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.159312  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.210867  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:57.211474  1721 raft_consensus.cc:2410] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:57.211791  1722 raft_consensus.cc:2368] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5.
I20260709 13:02:57.127776  4338 consensus_queue.cc:1048] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.085s
I20260709 13:02:57.117249  4239 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.214259  4239 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:57.207870  4343 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:57.217792  4335 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:57.096036  4350 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:57.226227  4350 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:57.246610  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.012893  4134 consensus_queue.cc:237] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:57.253893  4353 log.cc:927] Time spent T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.233s	user 0.000s	sys 0.004s
I20260709 13:02:57.272224  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.293974  1644 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:02:57.259699  4345 log.cc:927] Time spent T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
W20260709 13:02:57.295842  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 94482.5us
I20260709 13:02:57.302238  2313 leader_election.cc:400] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5.
I20260709 13:02:57.302688  2313 leader_election.cc:403] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:02:57.299147  1687 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:57.266803  4224 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:57.307116  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.321868  4330 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:57.322480  4330 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.322844  4330 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:57.322033  4233 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.350853  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2004) took 1257 ms. Trace:
I20260709 13:02:57.351385  1643 rpcz_store.cc:276] 0709 13:02:56.093271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.093418 (+   147us) service_pool.cc:224] Handling call
0709 13:02:56.093984 (+   566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:56.094657 (+   673us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:56.094689 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:56.095359 (+   670us) log.cc:844] Serialized 26 byte log entry
0709 13:02:56.473953 (+378594us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:56.778124 (+304171us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.887300 (+109176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.887429 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:57.350564 (+463135us) raft_consensus.cc:1685] finished
0709 13:02:57.350590 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.350773 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27501,"prepare.queue_time_us":821048,"raft.queue_time_us":76312,"raft.run_cpu_time_us":515,"raft.run_wall_time_us":36640,"thread_start_us":735959,"threads_started":3,"wal-append.queue_time_us":680873}
I20260709 13:02:57.354314  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.360216  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.361155  1644 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:02:57.311966  4335 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:57.364436  1643 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:02:57.301422  4214 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:57.365964  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1677) took 1419 ms. Trace:
I20260709 13:02:57.370352  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1538) took 1433 ms. Trace:
I20260709 13:02:57.379271  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.311811  4224 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:57.303840  4338 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:02:57.402964  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.404297  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:57.383054  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.391498  4337 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:57.392474  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.393133  4350 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:57.398072  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.399658  4139 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:57.402125  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.373039  1573 rpcz_store.cc:276] 0709 13:02:55.946131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.946637 (+   506us) service_pool.cc:224] Handling call
0709 13:02:55.946873 (+   236us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:55.950990 (+  4117us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:55.951020 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:55.951543 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 13:02:56.521829 (+570286us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:56.521937 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.365116 (+843179us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b54004056d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:02:57.365189 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:57.365251 (+    62us) raft_consensus.cc:1685] finished
0709 13:02:57.365263 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.365552 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":923,"raft.queue_time_us":11262,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":7101,"spinlock_wait_cycles":4650752,"thread_start_us":504124,"threads_started":1,"wal-append.queue_time_us":811484}
I20260709 13:02:57.378019  1565 rpcz_store.cc:276] 0709 13:02:55.936843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.937068 (+   225us) service_pool.cc:224] Handling call
0709 13:02:55.950977 (+ 13909us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.951174 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:56.296026 (+344852us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.996016 (+699990us) log.cc:844] Serialized 14 byte log entry
0709 13:02:57.369523 (+373507us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:57.369955 (+   432us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:57.370035 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.370134 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.370284 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":349784,"prepare.queue_time_us":345149,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":54,"raft.queue_time_us":528319,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":65006,"spinlock_wait_cycles":96953344,"thread_start_us":1011768,"threads_started":3,"wal-append.queue_time_us":853633}
I20260709 13:02:57.407804  1643 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:02:57.414011  1568 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:02:57.368825  4331 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:57.379359  4226 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:57.343918  4233 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:57.437307  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.389878  4224 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:57.440850  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.449879  1643 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:02:57.441565  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.387030  4214 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:57.444164  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:57.415455  4337 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:57.389801  4254 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 13:02:57.422725  4220 raft_consensus.cc:697] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:57.485857  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.487077  1643 raft_consensus.cc:2410] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:57.490861  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.490805  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.512689  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:57.513609  1722 raft_consensus.cc:2410] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:57.516701  2342 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:02:57.516850  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:57.518362  1643 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:57.518568  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.520167  1715 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:57.463805  4362 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:57.522193  4362 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:57.469159  4189 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:57.527639  4220 consensus_queue.cc:237] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:57.510711  4289 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:57.477286  4214 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:57.477742  4337 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:57.543901  4171 consensus_meta.cc:287] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.237s	user 0.011s	sys 0.007s
I20260709 13:02:57.544741  1721 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
W20260709 13:02:57.551095  1539 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.051s	sys 0.054s
W20260709 13:02:57.551931  1539 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.051s	sys 0.054s
I20260709 13:02:57.554077  2312 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:57.576393  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.495555  2340 raft_consensus.cc:1217] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 4.6->[6.7-11.9]   Dedup: 6.7->[9.8-11.9]
I20260709 13:02:57.577375  1644 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:57.562299  4139 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:57.555307  4362 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:57.605973  4324 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.454s	user 0.009s	sys 0.027s
W20260709 13:02:57.615149  4324 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.463s	user 0.010s	sys 0.028s
I20260709 13:02:57.610265  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.617508  1644 raft_consensus.cc:2410] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:02:57.584172  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:57.619601  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.620445  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.621713  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.602240  2340 pending_rounds.cc:85] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Aborting all ops after (but not including) 7
I20260709 13:02:57.648872  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:02:57.667479  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.604275  4246 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:57.682402  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.683868  1644 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 6: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:57.670293  2340 pending_rounds.cc:107] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:02:57.679481  4331 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:02:57.647146  4289 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:02:57.701431  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.694187  2340 raft_consensus.cc:2889] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:57.690840  4189 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:57.718662  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:57.787143  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.724599  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1624) took 1014 ms. Trace:
I20260709 13:02:57.728633  2348 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:57.736773  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.740895  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.741925  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:57.733405  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.761747  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.790589  1643 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:57.790778  1645 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:57.793412  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:57.707629  4373 log.cc:927] Time spent T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.103s	user 0.001s	sys 0.002s
I20260709 13:02:57.788405  1715 rpcz_store.cc:276] 0709 13:02:56.728673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.729024 (+   351us) service_pool.cc:224] Handling call
0709 13:02:57.519972 (+790948us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.520605 (+   633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.724359 (+203754us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b540002e9d0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:57.724527 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1740752256}
W20260709 13:02:57.796289  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.801512  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:02:57.733328  3817 consensus_meta.cc:287] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.626s	user 0.016s	sys 0.000s
I20260709 13:02:57.802852  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:57.794238  4239 consensus_meta.cc:287] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.545s	user 0.009s	sys 0.009s
W20260709 13:02:57.712231  4340 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 1.087s	user 0.001s	sys 0.001s
W20260709 13:02:57.793924  4195 consensus_meta.cc:287] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.768s	user 0.016s	sys 0.000s
W20260709 13:02:57.813561  4181 consensus_meta.cc:287] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.316s	user 0.015s	sys 0.007s
W20260709 13:02:57.782827  4343 consensus_meta.cc:287] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.562s	user 0.015s	sys 0.000s
I20260709 13:02:57.727063  4214 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:57.838672  1720 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:57.821167  4382 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.811704  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.822577  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.806284  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.801858  4371 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:57.812999  4335 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5.
I20260709 13:02:57.837713  1645 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:57.837327  4254 raft_consensus.cc:1064] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:02:57.782308  4289 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:57.861495  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:57.808115  4330 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:57.808986  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.823895  4293 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.801514  4189 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:57.807193  4381 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:57.833729  4233 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:57.839130  4229 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.854655  1645 raft_consensus.cc:3055] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:57.856305  1640 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:57.874431  1645 raft_consensus.cc:740] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:02:57.836026  4383 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:02:57.854205  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.866779  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.836670  4295 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:57.876040  1641 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
W20260709 13:02:57.854741  4377 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.174s	user 0.002s	sys 0.000s
I20260709 13:02:57.894081  1644 raft_consensus.cc:1275] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:57.858573  4139 raft_consensus.cc:697] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:02:57.899621  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 56696.2us
I20260709 13:02:57.853037  4374 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:57.830565  3817 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:57.906643  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.829756  4239 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:57.837150  4195 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:57.865267  4254 raft_consensus.cc:1068] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838: Unable to promote non-voter cf2f71592be8465fb18e36e15d0b2f89: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:02:57.915954  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.916589  1644 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:02:57.902719  1645 consensus_queue.cc:260] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:57.922272  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.923149  1721 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:02:57.924018  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.927042  2349 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:57.915196  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:57.928427  1645 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:57.841027  4343 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:57.931116  1713 raft_consensus.cc:2393] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:02:57.914004  4295 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:57.920317  4374 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:57.936864  4289 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:57.910888  4172 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.937503  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.941541  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.855160  4382 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:57.912163  4394 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:57.947311  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.947916  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1537) took 2020 ms. Trace:
I20260709 13:02:57.920920  4224 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:57.974514  4224 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:57.894450  4229 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:57.889206  4293 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:57.925700  4397 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.913146  4233 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:02:57.959914  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:57.861318  4371 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:57.966178  1572 rpcz_store.cc:276] 0709 13:02:55.927087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.927229 (+   142us) service_pool.cc:224] Handling call
0709 13:02:55.946847 (+ 19618us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:55.964286 (+ 17439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:55.964340 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:55.975175 (+ 10835us) log.cc:844] Serialized 26 byte log entry
0709 13:02:56.302119 (+326944us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:56.302165 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.302247 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:57.711498 (+1409251us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:02:57.940749 (+229251us) raft_consensus.cc:1685] finished
0709 13:02:57.940775 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.947829 (+  7054us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10944,"raft.queue_time_us":2851,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":20051,"spinlock_wait_cycles":37330176,"thread_start_us":287332,"threads_started":1,"wal-append.queue_time_us":626545}
I20260709 13:02:57.971838  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:57.888983  4381 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:57.974869  4382 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:58.057463  4195 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:58.010380  4239 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:58.118180  1716 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:58.010838  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:58.120529  1640 raft_consensus.cc:2410] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:58.056072  3817 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:58.008591  4343 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:58.059115  4151 raft_consensus.cc:3055] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:02:57.951543  4368 log.cc:927] Time spent T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
W20260709 13:02:58.053373  4395 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:02:58.023552  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.053776  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:58.128751  4229 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:58.005703  1644 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:02:58.131508  1721 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:02:58.007982  4389 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:02:58.137650  4151 raft_consensus.cc:740] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:02:58.146925  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.147864  4151 consensus_queue.cc:260] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:58.149053  1717 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:02:58.149725  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2057) took 1544 ms. Trace:
I20260709 13:02:58.150053  1717 rpcz_store.cc:276] 0709 13:02:56.605274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.605423 (+   149us) service_pool.cc:224] Handling call
0709 13:02:58.148519 (+1543096us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b580009db98. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60eee 0000000000375939 00007f824ddd41e0 00007f824de9d0c4 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:02:58.148714 (+   195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.149439 (+   725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.149666 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2553728}
I20260709 13:02:58.151947  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:58.153097  1643 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:02:58.141433  1641 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:02:58.124042  4330 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:02:58.146977  4315 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.162195  4315 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:57.951373  4139 consensus_queue.cc:237] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:57.976478  4394 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:02:58.060652  4337 consensus_meta.cc:287] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.526s	user 0.012s	sys 0.000s
I20260709 13:02:58.139019  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:58.173414  1640 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:02:58.174175  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1500) took 1104 ms. Trace:
I20260709 13:02:58.118561  4189 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:58.175520  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:58.177780  1640 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:02:58.142383  1645 raft_consensus.cc:1275] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 13:02:58.142706  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:58.178562  1645 pending_rounds.cc:85] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 2
I20260709 13:02:58.143929  1718 raft_consensus.cc:1275] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:02:58.178908  1644 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:58.179139  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:02:58.056205  4377 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:02:58.157992  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:58.160259  4381 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:58.180109  1641 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:58.180598  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1504) took 1108 ms. Trace:
I20260709 13:02:58.182020  1639 rpcz_store.cc:276] 0709 13:02:57.072051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.073847 (+  1796us) service_pool.cc:224] Handling call
0709 13:02:57.115950 (+ 42103us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:57.143347 (+ 27397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:57.143389 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:57.184921 (+ 41532us) log.cc:844] Serialized 52 byte log entry
0709 13:02:57.413798 (+228877us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:57.482138 (+ 68340us) log.cc:844] Serialized 14 byte log entry
0709 13:02:57.482635 (+   497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.482747 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:58.180385 (+697638us) raft_consensus.cc:1685] finished
0709 13:02:58.180405 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.180539 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36953,"prepare.queue_time_us":95578,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":14722,"raft.run_cpu_time_us":371,"raft.run_wall_time_us":34137,"thread_start_us":117816,"threads_started":1,"wal-append.queue_time_us":311919}
W20260709 13:02:58.183368  4383 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
I20260709 13:02:58.184306  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:58.184916  1639 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:02:58.185808  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:58.186364  1639 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
W20260709 13:02:58.185981  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.188174  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:58.189954  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:58.191094  1639 raft_consensus.cc:2368] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5.
I20260709 13:02:58.192080  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:58.192410  4293 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:58.193006  1639 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:02:58.179836  1640 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:02:58.194483  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:58.195261  1639 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:58.195596  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:58.195999  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:58.196698  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:58.197968  4331 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:58.198036  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:58.225847  4331 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:58.226516  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:58.227108  1643 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:58.228670  4331 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:58.199843  1717 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:02:58.201068  4374 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:58.201814  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.196978  1640 raft_consensus.cc:2368] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5.
I20260709 13:02:58.229892  4315 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:58.231626  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:58.231814  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:58.232441  1639 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:58.232863  1640 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:58.196775  1648 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:58.206728  1641 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:02:58.236068  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:58.237031  1641 raft_consensus.cc:2368] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5.
I20260709 13:02:58.237944  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:58.238471  1641 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:58.240522  1643 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:58.178890  1645 pending_rounds.cc:107] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:02:58.242534  1645 raft_consensus.cc:2889] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:58.243268  1645 pending_rounds.cc:107] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:02:58.223912  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:58.175081  1642 rpcz_store.cc:276] 0709 13:02:57.070080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.070361 (+   281us) service_pool.cc:224] Handling call
0709 13:02:57.071342 (+   981us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:57.071928 (+   586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:57.071971 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:57.083853 (+ 11882us) log.cc:844] Serialized 52 byte log entry
0709 13:02:57.937142 (+853289us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b180032f740. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a250d1 00007f8248e40a0f
0709 13:02:58.125771 (+188629us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:58.139059 (+ 13288us) log.cc:844] Serialized 14 byte log entry
0709 13:02:58.142377 (+  3318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.142446 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:58.142482 (+    36us) raft_consensus.cc:1685] finished
0709 13:02:58.167624 (+ 25142us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.174076 (+  6452us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24838,"prepare.queue_time_us":14119,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":67,"raft.queue_time_us":15382,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":35438,"spinlock_wait_cycles":3922688,"thread_start_us":188447,"threads_started":1,"wal-append.queue_time_us":1003071}
I20260709 13:02:58.054845  4295 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:58.245612  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.222807  1644 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:02:58.247327  1645 raft_consensus.cc:2889] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:58.257721  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:58.258889  2349 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:02:58.257715  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:58.265969  1641 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:58.253036  1644 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:02:58.274164  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 374266us
I20260709 13:02:58.275153  1642 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:02:58.281827  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 43681.7us
I20260709 13:02:58.283444  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:58.284502  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:58.284839  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:58.286595  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:58.287322  1643 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:02:58.288884  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:58.289718  1648 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:02:58.293953  1716 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:58.294409  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:58.296994  1721 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:02:58.292308  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:58.172497  4394 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:58.300854  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.301467  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:02:58.302505  4135 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:02:58.308374  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2056) took 1703 ms. Trace:
I20260709 13:02:58.309026  1720 rpcz_store.cc:276] 0709 13:02:56.604977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.623988 (+ 19011us) service_pool.cc:224] Handling call
0709 13:02:58.308164 (+1684176us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400172854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:02:58.308300 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2790979968}
I20260709 13:02:57.965265  4172 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:58.318045  1537 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:58.321457  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:58.324208  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:02:58.324599  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.325045  1641 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:58.325999  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:58.327107  4374 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:58.327385  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:58.302232  1717 raft_consensus.cc:2368] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7.
I20260709 13:02:58.328049  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:58.330188  4403 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:58.327517  4374 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:58.331533  4374 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:58.304315  4368 log.cc:927] Time spent T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:02:58.335907  1537 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:58.130995  4397 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:58.301694  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.344960  2345 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:02:58.346504  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:02:58.348048  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:58.350955  1538 connection.cc:256] RPC call timeout handler was delayed by -1.341792125999973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3336s for a response
W20260709 13:02:58.352279  1538 connection.cc:256] RPC call timeout handler was delayed by -1.090135250000003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.38480000000000003s for a response
I20260709 13:02:58.344779  1641 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:58.353047  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:02:58.353883  1643 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:02:58.354718  1718 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:58.357308  1640 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:02:58.355376  1687 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:58.360833  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:58.363400  1639 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:02:58.369166  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:58.363636  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:58.371554  1718 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:02:58.273430  4337 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:58.382586  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:58.384337  2347 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:02:58.389768  1721 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:58.392658  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 45775.9us
I20260709 13:02:58.397188  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:58.398698  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:58.399497  1718 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:02:58.377983  4381 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:58.403940  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:58.403870  1538 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.521s (SENT)
W20260709 13:02:58.406016  1538 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.748s (SENT)
I20260709 13:02:58.406558  1538 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:58.406848  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:58.408293  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:58.388618  4296 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.415536  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:58.378131  4233 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:58.352473  4371 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:58.378260  4362 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:58.421505  4413 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.342401  4214 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:02:58.403471  4189 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:58.345125  4315 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:02:58.424038  4365 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:58.411695  4336 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:58.473519  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 67270us
W20260709 13:02:58.474283  1685 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.196s	user 0.009s	sys 0.025s
W20260709 13:02:58.487928  1685 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.210s	user 0.010s	sys 0.025s
W20260709 13:02:58.488772  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 1.33847e+06us
W20260709 13:02:58.473886  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 50203.8us
I20260709 13:02:58.471427  4362 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:58.483208  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:58.411983  4381 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:58.435650  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:58.430059  4296 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:58.522835  1685 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:58.426908  4233 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:58.538195  1687 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:58.474015  4413 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:58.528774  4293 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:58.548205  4405 log.cc:927] Time spent T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:02:58.550889  4382 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:58.516621  4300 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:02:58.514257  4189 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:58.541772  4388 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:58.574251  1687 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:58.556663  4296 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:58.554937  4397 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:58.546718  4172 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:58.557185  2340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55890 (request call id 262) took 2304 ms. Trace:
I20260709 13:02:58.544749  4336 raft_consensus.cc:697] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:58.579799  2340 rpcz_store.cc:276] 0709 13:02:56.252596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.253303 (+   707us) service_pool.cc:224] Handling call
0709 13:02:57.495018 (+1241715us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b580003a298. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:02:57.495280 (+   262us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:02:57.830821 (+335541us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:58.138534 (+307713us) log.cc:844] Serialized 14 byte log entry
0709 13:02:58.318956 (+180422us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:58.427308 (+108352us) log.cc:844] Serialized 52 byte log entry
0709 13:02:58.474490 (+ 47182us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:58.482036 (+  7546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.490926 (+  8890us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:58.556688 (+ 65762us) raft_consensus.cc:1685] finished
0709 13:02:58.556850 (+   162us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.557054 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7282,"prepare.queue_time_us":113568,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":92216,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":19993,"spinlock_wait_cycles":6722304,"thread_start_us":359271,"threads_started":2,"wal-append.queue_time_us":109181}
I20260709 13:02:58.568584  4295 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:02:58.618534  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1513) took 1539 ms. Trace:
I20260709 13:02:58.544361  4254 consensus_queue.cc:1048] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 13:02:58.628741  4331 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:58.630432  4416 log.cc:927] Time spent T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.073s	user 0.000s	sys 0.003s
I20260709 13:02:58.542078  4220 consensus_queue.cc:1059] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:02:58.553591  4382 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.080s
W20260709 13:02:58.568125  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 162345us
I20260709 13:02:58.658355  1537 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:58.638386  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 66374.5us
I20260709 13:02:58.563150  4214 consensus_queue.cc:1048] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.030s
W20260709 13:02:58.649226  1612 connection.cc:256] RPC call timeout handler was delayed by -2.5407849669999791s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
W20260709 13:02:58.671186  1612 connection.cc:256] RPC call timeout handler was delayed by -2.4813849669999968s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
W20260709 13:02:58.672904  1612 connection.cc:256] RPC call timeout handler was delayed by -2.4417849669999896s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1613s for a response
W20260709 13:02:58.673341  1612 connection.cc:256] RPC call timeout handler was delayed by -2.3634849670000335s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17s for a response
W20260709 13:02:58.674898  1612 connection.cc:256] RPC call timeout handler was delayed by -2.307684967s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17620000000000002s for a response
W20260709 13:02:58.675446  1612 connection.cc:256] RPC call timeout handler was delayed by -2.3022849669999914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1768s for a response
W20260709 13:02:58.675944  1612 connection.cc:256] RPC call timeout handler was delayed by -2.2698849669999959s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1804s for a response
W20260709 13:02:58.650207  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:58.653045  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 83517.7us
I20260709 13:02:58.640061  1647 rpcz_store.cc:276] 0709 13:02:57.078835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.828897 (+750062us) service_pool.cc:224] Handling call
0709 13:02:57.833427 (+  4530us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:57.834830 (+  1403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:57.834875 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:57.837971 (+  3096us) log.cc:844] Serialized 52 byte log entry
0709 13:02:58.336019 (+498048us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:58.344254 (+  8235us) log.cc:844] Serialized 14 byte log entry
0709 13:02:58.344473 (+   219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.344571 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:58.594907 (+250336us) raft_consensus.cc:1685] finished
0709 13:02:58.594936 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.618441 (+ 23505us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6604,"prepare.queue_time_us":8085,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":5732,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":12867,"thread_start_us":444886,"threads_started":1,"wal-append.queue_time_us":546887}
W20260709 13:02:58.678622  1612 connection.cc:256] RPC call timeout handler was delayed by -2.2572849670000323s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
W20260709 13:02:58.679437  1612 connection.cc:256] RPC call timeout handler was delayed by -2.2572849670000323s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
W20260709 13:02:58.680124  1612 connection.cc:256] RPC call timeout handler was delayed by -2.2536849670000265s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18220000000000003s for a response
W20260709 13:02:58.680692  1612 connection.cc:256] RPC call timeout handler was delayed by -2.1726849670000092s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1912s for a response
W20260709 13:02:58.681483  1612 connection.cc:256] RPC call timeout handler was delayed by -2.1483849670000268s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19390000000000002s for a response
W20260709 13:02:58.682369  1612 connection.cc:256] RPC call timeout handler was delayed by -2.1366849670000079s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1952s for a response
I20260709 13:02:58.692487  1537 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:58.652781  4134 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:58.640951  4374 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:58.702158  4381 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:58.703634  4374 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:58.706900  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 213) took 1713 ms. Trace:
I20260709 13:02:58.706703  4134 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:58.709213  4134 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:58.711457  4362 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:58.712927  4413 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:58.713091  4394 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:58.672304  4371 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:58.714344  4189 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:58.682651  4088 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:58.714955  4226 consensus_queue.cc:1059] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:02:58.715941  4394 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:58.707542  1719 rpcz_store.cc:276] 0709 13:02:56.994265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.994643 (+   378us) service_pool.cc:224] Handling call
0709 13:02:56.995540 (+   897us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:57.027463 (+ 31923us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:57.027494 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:57.728442 (+700948us) spinlock_profiling.cc:243] Waited 35.1 ms on lock 0x7b2c00013bb8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824777ea11 00007f824778e372 00007f8248ec0a01 00007f8248e8543a 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2
0709 13:02:57.729060 (+   618us) log.cc:844] Serialized 52 byte log entry
0709 13:02:57.858314 (+129254us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:57.901247 (+ 42933us) log.cc:844] Serialized 14 byte log entry
0709 13:02:57.903262 (+  2015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.903348 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:58.156957 (+253609us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b4800190158. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ddaeaa1 00007f824dda5b37 00007f824dda59ca 00007f824de9d26f 00007f824de9cf37 00007f824deb155c 00007f824deb1508 00007f824deb1488 00007f824deb1443 00007f824deb06ec 00007f824cddaa96
0709 13:02:58.586010 (+429053us) spinlock_profiling.cc:243] Waited 3.02 ms on lock 0x7b1800436780. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:02:58.706666 (+120656us) raft_consensus.cc:1685] finished
0709 13:02:58.706687 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.706835 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17917,"prepare.queue_time_us":42900,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":3708,"raft.queue_time_us":33192,"raft.run_cpu_time_us":10650,"raft.run_wall_time_us":1113450,"spinlock_wait_cycles":136570880,"thread_start_us":128813,"threads_started":1,"wal-append.queue_time_us":182678}
I20260709 13:02:58.717517  4331 raft_consensus.cc:697] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:58.667088  4337 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:02:58.722411  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 44949us
I20260709 13:02:58.723199  4239 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:58.724689  1687 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:58.726094  1687 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:58.717154  4296 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:58.734640  4296 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:58.735198  4296 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:58.684589  1612 connection.cc:256] RPC call timeout handler was delayed by -2.1258849669999904s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19640000000000002s for a response
W20260709 13:02:58.736155  1612 connection.cc:256] RPC call timeout handler was delayed by -2.1123849670000254s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19790000000000002s for a response
W20260709 13:02:58.736498  1612 connection.cc:256] RPC call timeout handler was delayed by -2.0592849669999964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20379999999999998s for a response
W20260709 13:02:58.736796  1612 connection.cc:256] RPC call timeout handler was delayed by -2.0592849669999964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20379999999999998s for a response
W20260709 13:02:58.737056  1612 connection.cc:256] RPC call timeout handler was delayed by -2.03858496700002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2061s for a response
W20260709 13:02:58.737322  1612 connection.cc:256] RPC call timeout handler was delayed by -2.0241849669999965s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2077s for a response
W20260709 13:02:58.737593  1612 connection.cc:256] RPC call timeout handler was delayed by -1.9620849670000098s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2146s for a response
W20260709 13:02:58.737993  1612 connection.cc:256] RPC call timeout handler was delayed by -1.9359849670000244s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2175s for a response
W20260709 13:02:58.738392  1612 connection.cc:256] RPC call timeout handler was delayed by -1.913484966999988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22000000000000003s for a response
W20260709 13:02:58.738781  1612 connection.cc:256] RPC call timeout handler was delayed by -1.901784967000026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22130000000000002s for a response
W20260709 13:02:58.739185  1612 connection.cc:256] RPC call timeout handler was delayed by -1.901784967000026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22130000000000002s for a response
W20260709 13:02:58.739462  1612 connection.cc:256] RPC call timeout handler was delayed by -1.8909849670000085s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22250000000000003s for a response
I20260709 13:02:58.714401  4336 consensus_queue.cc:237] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:58.739777  1612 connection.cc:256] RPC call timeout handler was delayed by -1.8828849669999954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22340000000000002s for a response
W20260709 13:02:58.740089  1612 connection.cc:256] RPC call timeout handler was delayed by -1.7550849670000161s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2376s for a response
W20260709 13:02:58.740341  1612 connection.cc:256] RPC call timeout handler was delayed by -1.5993849669999918s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2549s for a response
W20260709 13:02:58.740615  1612 connection.cc:256] RPC call timeout handler was delayed by -1.2510849669999971s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2936s for a response
W20260709 13:02:58.740867  1612 connection.cc:256] RPC call timeout handler was delayed by -1.096284967000031s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3108s for a response
W20260709 13:02:58.741130  1612 connection.cc:256] RPC call timeout handler was delayed by -1.0620849670000325s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3146s for a response
W20260709 13:02:58.747097  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 197486us
I20260709 13:02:58.722144  4331 consensus_queue.cc:237] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:58.773116  1612 connection.cc:256] RPC call timeout handler was delayed by -1.0035849669999948s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3211s for a response
I20260709 13:02:56.534361   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:58.716105  4226 raft_consensus.cc:3055] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:58.782552  1468 catalog_manager.cc:5697] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 2 to 7, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 7 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
W20260709 13:02:58.794543  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 67383.9us
W20260709 13:02:58.798305  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 102091us
I20260709 13:02:58.800115  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:58.810767  1612 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:58.820155  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 201997us
W20260709 13:02:58.821393  2314 leader_election.cc:336] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.580s (SENT)
I20260709 13:02:58.827556  1612 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:58.809513  1538 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:58.838095  1538 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:58.780768  4413 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:58.842724  1612 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:58.785518  4134 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:58.790599  4226 raft_consensus.cc:740] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:58.827469  4224 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:02:58.854012  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 127996us
W20260709 13:02:58.854578  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 106192us
I20260709 13:02:58.830494  4289 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:58.846122  4422 log.cc:927] Time spent T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:02:58.833900  4088 raft_consensus.cc:697] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:58.840261  4295 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:58.880273  4088 consensus_queue.cc:237] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:58.883929  1468 catalog_manager.cc:5697] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:58.886607  1468 catalog_manager.cc:5697] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 1 to 4, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:58.830065  4394 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:02:58.889879  1468 catalog_manager.cc:5697] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 1 to 3, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } health_report { overall_health: HEALTHY } } }
I20260709 13:02:58.830356  4397 consensus_queue.cc:1048] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:02:58.830247  4381 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:02:58.833662  4371 consensus_queue.cc:1059] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:02:58.875995  1687 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:58.899363  1538 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:58.862391  4220 consensus_queue.cc:1243] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:02:58.906471  1538 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:58.862098  4160 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:58.866380  4374 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:58.914395  2314 leader_election.cc:336] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 4.402s (SENT)
I20260709 13:02:58.915230  2314 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:58.921273  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:58.913527  4246 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:58.928786  1714 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8. Ops: []
W20260709 13:02:58.929025  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.932904  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:02:58.933660  1639 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:58.934742  1644 raft_consensus.cc:1275] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:58.908102  4426 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:58.926589  4224 raft_consensus.cc:697] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:58.940099  4224 consensus_queue.cc:237] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:58.941134  1568 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:58.943295  3967 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:58.944548  1565 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:58.945108  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:58.946475  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:58.948016  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:58.918255  4160 raft_consensus.cc:697] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:58.949847  4430 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:58.950631  4160 consensus_queue.cc:237] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:58.952231  3967 raft_consensus.cc:697] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:02:58.913121  4394 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:58.948581  1715 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:58.955092  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:02:58.958518  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 52176us
I20260709 13:02:58.904551  4189 consensus_queue.cc:1059] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:02:58.918352  4172 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:59.003883  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:59.004750  1568 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:58.966141  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:58.966296  1538 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:58.875109  4134 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:59.011451  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:59.017611  1639 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
W20260709 13:02:58.995299  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 152713us
I20260709 13:02:58.873389  4413 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:59.030038  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:59.031481  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:58.963590  4394 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.022490  4239 consensus_queue.cc:1048] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:02:59.023262  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.002189  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:59.032253  1567 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:59.035132  1612 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:02:59.053421  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.054297  4433 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:59.055125  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:59.056430  4433 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:59.056632  1612 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:02:59.056943  4433 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:59.057391  4433 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:59.035732  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 69554.4us
I20260709 13:02:59.062777  1538 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:59.043609  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:59.065985  4088 consensus_queue.cc:1059] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:02:59.072957  1612 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:59.074697  1612 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:59.068516  4135 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:59.083635  1538 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:59.104526  1538 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:59.105707  1568 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:02:59.106313  1538 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:59.021507  4214 raft_consensus.cc:3055] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:59.096194  1567 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:02:59.120302  1715 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:02:59.045718  4394 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:59.020637  3967 consensus_queue.cc:237] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:02:59.134856  4431 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:02:59.109001  4135 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:59.079133  4419 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:59.121829  4433 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:59.126188  4214 raft_consensus.cc:740] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:59.123302  4430 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:59.151292  4312 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:59.141381  4424 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:59.164510  4134 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:59.152915  4135 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:59.170586  4402 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:02:59.177070  4189 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:59.177603  4189 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:02:59.178097  4189 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:59.153498  4419 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:59.186275  4419 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:59.188241  2314 leader_election.cc:336] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.176s (SENT)
I20260709 13:02:59.175482  4293 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:02:59.189090  4214 consensus_queue.cc:260] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:59.176098  4433 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:59.136514  4315 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:59.191234  4214 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:59.191334  4315 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:59.191830  1640 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:59.189952  4293 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:59.122684  4365 consensus_queue.cc:1059] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:02:59.189144  2314 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:59.194607  4315 raft_consensus.cc:3055] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:59.200780  4315 raft_consensus.cc:740] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:59.194770  4312 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:59.258073  4312 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:59.260675  4312 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:59.281575  4296 consensus_meta.cc:287] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.546s	user 0.018s	sys 0.000s
I20260709 13:02:59.283196  4228 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:59.297395  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:59.267521  4413 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:59.318535  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.247038  4315 consensus_queue.cc:260] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:59.328593  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:02:59.331069  1715 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:02:59.334228  2313 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:02:59.338213  1567 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:59.425412  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.336201  4296 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:59.497450  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:59.502938  1569 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:02:59.504922  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.509223  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:02:59.513154  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 406929us
I20260709 13:02:59.514225  1538 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:59.515841  1538 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:59.520170  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:59.521847  1567 raft_consensus.cc:2410] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:59.548128  1572 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:59.529980  4374 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:59.499022  4189 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:59.572749  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:59.575875  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 387673us
W20260709 13:02:59.576862  1612 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.014s	sys 0.021s
W20260709 13:02:59.577495  1612 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.014s	sys 0.021s
W20260709 13:02:59.565600  1537 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.020s	sys 0.040s
W20260709 13:02:59.579764  1537 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.022s	sys 0.040s
W20260709 13:02:59.580183  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 559170us
W20260709 13:02:59.592051  1570 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.025s	sys 0.033s
W20260709 13:02:59.593489  1570 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.026s	sys 0.033s
I20260709 13:02:59.589161  1537 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:02:59.629787  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:02:59.615399  4381 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:59.638584  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 563994us
W20260709 13:02:59.639750  1643 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.002s	sys 0.033s
W20260709 13:02:59.640328  1643 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.003s	sys 0.033s
I20260709 13:02:59.640166  1612 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:59.622056  4371 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:59.656209  1612 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:59.662418  1570 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:02:59.651180  4446 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 13:02:59.669116  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 335002us
I20260709 13:02:59.671496  1612 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:59.654194  4453 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:59.654995  4296 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:02:59.627216  4451 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:02:59.632143  4135 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:59.664161  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:59.642014  4445 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:02:59.666290  4228 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:59.693769  4434 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:59.694177  4420 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:59.768746  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 97344.6us
W20260709 13:02:59.742373  4419 consensus_meta.cc:287] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.555s	user 0.014s	sys 0.006s
W20260709 13:02:59.773787  2314 leader_election.cc:336] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.766s (SENT)
I20260709 13:02:59.730648  4453 raft_consensus.cc:697] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:02:59.777722  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 188616us
I20260709 13:02:59.791357  2314 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:02:59.798705  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:59.804584  1612 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:59.770401  4430 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:59.728971  4445 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:59.815797  1714 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:59.817608  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2078) took 1414 ms. Trace:
I20260709 13:02:59.820190  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:02:59.818212  1718 rpcz_store.cc:276] 0709 13:02:58.403537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.404587 (+  1050us) service_pool.cc:224] Handling call
0709 13:02:58.405008 (+   421us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:58.471927 (+ 66919us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:58.471976 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:59.060395 (+588419us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b2c00013bb8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824777ea11 00007f824778e372 00007f8248ec0a01 00007f8248e8543a 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2
0709 13:02:59.061316 (+   921us) log.cc:844] Serialized 52 byte log entry
0709 13:02:59.203414 (+142098us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:59.203478 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.203611 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:59.813374 (+609763us) raft_consensus.cc:1685] finished
0709 13:02:59.813402 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:59.817532 (+  4130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78715,"raft.queue_time_us":23619,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":40289,"spinlock_wait_cycles":5708032,"thread_start_us":132788,"threads_started":1,"wal-append.queue_time_us":429867}
I20260709 13:02:59.824163  1612 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:02:59.802531  4228 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:59.813441  4443 log.cc:927] Time spent T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
W20260709 13:02:59.833096  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.833385  1612 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:02:59.833346  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 104657us
I20260709 13:02:59.791829  4419 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:59.836625  1722 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:59.812783  4434 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:59.828531  4465 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:02:59.844762  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:02:59.831255  4295 raft_consensus.cc:3055] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:59.812968  4448 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:02:59.832324  4189 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:02:59.879488  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:59.879654  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:59.880761  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:02:59.881870  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:59.835657  4413 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:59.887889  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:02:59.856431  4430 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:59.900202  4419 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:02:59.901844  4441 log.cc:927] Time spent T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.243s	user 0.000s	sys 0.002s
W20260709 13:02:59.904129  4463 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 13:02:59.859520  4462 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:02:59.865161  4459 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.924110  4471 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:02:59.925055  4453 consensus_queue.cc:237] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:59.927296  4460 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:59.928411  4468 log.cc:927] Time spent T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
W20260709 13:02:59.930001  4450 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.281s	user 0.001s	sys 0.001s
W20260709 13:02:59.930660  4293 consensus_meta.cc:287] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.737s	user 0.019s	sys 0.001s
I20260709 13:02:59.857077  4295 raft_consensus.cc:740] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:02:59.883358  4296 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:02:59.930172  4458 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:02:59.895783  4448 consensus_queue.cc:1243] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:02:59.941530  4465 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:02:59.957779  2314 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.187s (SENT)
W20260709 13:02:59.963757  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 122972us
I20260709 13:02:59.970489  4464 consensus_queue.cc:1059] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:59.943118  4459 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:59.955358  4293 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:59.970309  4295 consensus_queue.cc:260] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:02:59.975346  4312 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:59.976070  4295 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:59.948343  4471 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:02:59.980420  4135 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:02:59.976449  4474 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
W20260709 13:02:59.986049  4394 consensus_meta.cc:287] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.846s	user 0.013s	sys 0.010s
I20260709 13:02:59.984781  4471 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:00.018543  4460 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:00.019375  4460 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:00.023648  4481 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:00.006541  4394 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:00.026846  4459 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:00.009253  4471 raft_consensus.cc:3055] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:00.030297  4471 raft_consensus.cc:740] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:00.031780  4471 consensus_queue.cc:260] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:02:59.951000  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 119799us
I20260709 13:02:59.951825  1714 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
W20260709 13:03:00.134801  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.967675  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:02:59.965268  4226 consensus_queue.cc:260] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:00.238683  4189 consensus_queue.cc:1059] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
W20260709 13:03:00.241233  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 309276us
I20260709 13:03:00.242056  4233 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:00.243420  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2081) took 1322 ms. Trace:
I20260709 13:03:00.246095  1567 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:00.239948  4293 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:00.250468  4189 consensus_queue.cc:1243] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:03:00.252282  2314 leader_election.cc:336] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.935s (SENT)
W20260709 13:03:00.253990  4486 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.216s	user 0.006s	sys 0.000s
I20260709 13:03:00.275368  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1552) took 1274 ms. Trace:
I20260709 13:03:00.276216  1571 rpcz_store.cc:276] 0709 13:02:59.000689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.001220 (+   531us) service_pool.cc:224] Handling call
0709 13:02:59.001602 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:59.509804 (+508202us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:59.509850 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:59.510790 (+   940us) log.cc:844] Serialized 26 byte log entry
0709 13:02:59.685984 (+175194us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:59.686051 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.686177 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:00.260941 (+574764us) raft_consensus.cc:1685] finished
0709 13:03:00.263566 (+  2625us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:00.275023 (+ 11457us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17067,"raft.queue_time_us":62149,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":19756,"thread_start_us":635997,"threads_started":2,"wal-append.queue_time_us":466559}
W20260709 13:03:00.286023  4214 consensus_meta.cc:287] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.094s	user 0.023s	sys 0.002s
W20260709 13:03:00.287117  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.253412  2314 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:00.282471  1565 raft_consensus.cc:1240] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:03:00.292237  4151 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:00.303393  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 90949.9us
I20260709 13:03:00.305166  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:00.282809  4394 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:00.332393  4471 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:00.247342  1716 rpcz_store.cc:276] 0709 13:02:58.920303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.920436 (+   133us) service_pool.cc:224] Handling call
0709 13:02:58.920788 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:58.949238 (+ 28450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:58.949269 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:58.954047 (+  4778us) log.cc:844] Serialized 26 byte log entry
0709 13:02:59.514820 (+560773us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:59.514940 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.518402 (+  3462us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:00.239531 (+721129us) raft_consensus.cc:1685] finished
0709 13:03:00.239918 (+   387us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:00.242017 (+  2099us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38868,"raft.queue_time_us":30823,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"thread_start_us":469908,"threads_started":1,"wal-append.queue_time_us":575715}
I20260709 13:03:00.347318  4480 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:00.364667  1567 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:00.307107  4460 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:00.325238  4312 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:00.374892  4312 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:00.386382  1538 leader_election.cc:336] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.678s (SENT)
I20260709 13:03:00.390671  4466 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:03:00.395259  4336 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.350s	user 0.026s	sys 0.071s
W20260709 13:03:00.396770  4336 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.376s	user 0.027s	sys 0.072s
I20260709 13:03:00.396456  4466 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:00.400658  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 95801.2us
I20260709 13:03:00.411760  1612 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:00.403527  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:00.444741  4480 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:03:00.404959  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 152977us
I20260709 13:03:00.412832  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:00.387706  4482 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:00.433449  4415 log.cc:927] Time spent T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.784s	user 0.003s	sys 0.003s
I20260709 13:03:00.434592  4330 raft_consensus.cc:1064] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:03:00.439523  1713 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
W20260709 13:03:00.440977  4421 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 1.792s	user 0.007s	sys 0.006s
I20260709 13:03:00.449369  1612 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:00.450133  1538 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:00.449083  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2080) took 1531 ms. Trace:
I20260709 13:03:00.455845  1719 rpcz_store.cc:276] 0709 13:02:58.919753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.919960 (+   207us) service_pool.cc:224] Handling call
0709 13:02:58.920250 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.920453 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:59.338842 (+418389us) log.cc:844] Serialized 14 byte log entry
0709 13:02:59.783354 (+444512us) log.cc:844] Serialized 14 byte log entry
0709 13:03:00.408219 (+624865us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b1800410320. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d12c0a 00007f824f452219 00007f8245d9aa40 00007f8245d954e4 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287 00007f8248ea0208
0709 13:03:00.447881 (+ 39662us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:00.448022 (+   141us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:00.448323 (+   301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.448541 (+   218us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:00.448831 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":615197,"prepare.queue_time_us":214677,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":4901,"raft.queue_time_us":650962,"raft.run_cpu_time_us":470,"raft.run_wall_time_us":8889,"spinlock_wait_cycles":5240576,"thread_start_us":791079,"threads_started":2,"wal-append.queue_time_us":490463}
I20260709 13:03:00.476017  1612 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:00.477855  4135 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:00.478580  4480 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:00.479996  4135 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:00.481563  4371 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:00.498353  4296 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:00.504016  4445 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:00.505220  1686 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:00.507517  4460 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:00.509713  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.514003  4371 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:00.514705  1573 raft_consensus.cc:3055] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:00.507468  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2075) took 2114 ms. Trace:
I20260709 13:03:00.518389  1573 raft_consensus.cc:740] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:00.525605  1612 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:00.512951  4330 consensus_queue.cc:237] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:00.542058  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 71012.3us
I20260709 13:03:00.556476  4480 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:00.539949  4452 log.cc:927] Time spent T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.819s	user 0.002s	sys 0.002s
W20260709 13:03:00.567555  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 206821us
I20260709 13:03:00.516784  4135 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:00.520114  1721 rpcz_store.cc:276] 0709 13:02:58.392488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.408243 (+ 15755us) service_pool.cc:224] Handling call
0709 13:02:58.411259 (+  3016us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:58.418682 (+  7423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:58.418728 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:58.419395 (+   667us) log.cc:844] Serialized 52 byte log entry
0709 13:02:58.497582 (+ 78187us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:58.497623 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.497727 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:00.439576 (+1941849us) spinlock_profiling.cc:243] Waited 8.77 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:00.505734 (+ 66158us) raft_consensus.cc:1685] finished
0709 13:03:00.505757 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:00.507223 (+  1466us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20431,"raft.queue_time_us":15267,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":8225,"spinlock_wait_cycles":20163328,"thread_start_us":77795,"threads_started":1,"wal-append.queue_time_us":171448}
I20260709 13:03:00.529273  4460 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:00.564999  4296 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.596443  4289 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:00.598173  4289 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:00.601251  4289 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:00.601557  4460 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:00.603751  4466 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:00.587886  1573 consensus_queue.cc:260] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:00.620436  1685 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:00.623260  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:00.580825  4336 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:00.631667  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1548) took 1625 ms. Trace:
W20260709 13:03:00.639842  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 150037us
I20260709 13:03:00.647351  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:00.648556  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.632920  4336 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:00.650988  4336 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:00.658363  4371 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:00.664880  1721 raft_consensus.cc:3055] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:00.665560  1721 raft_consensus.cc:740] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:00.648453  4296 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:00.667153  4341 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:00.670958  1573 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:00.632373  1564 rpcz_store.cc:276] 0709 13:02:58.961581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.972917 (+ 11336us) service_pool.cc:224] Handling call
0709 13:02:58.996760 (+ 23843us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:59.003825 (+  7065us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:59.003857 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:59.004384 (+   527us) log.cc:844] Serialized 26 byte log entry
0709 13:02:59.516132 (+511748us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:59.516192 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.516592 (+   400us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:00.448168 (+931576us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:00.569330 (+121162us) raft_consensus.cc:1685] finished
0709 13:03:00.569371 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:00.587246 (+ 17875us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8778,"raft.queue_time_us":16922,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":14715,"spinlock_wait_cycles":4766976,"thread_start_us":402754,"threads_started":1,"wal-append.queue_time_us":528389}
I20260709 13:03:00.646677  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:00.668254  4341 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:00.684967  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.684978  4328 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:00.694880  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 47911.2us
I20260709 13:03:00.701157  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1547) took 1739 ms. Trace:
W20260709 13:03:00.708938  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.714125  4135 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:00.715181  1571 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:03:00.705578  1565 rpcz_store.cc:276] 0709 13:02:58.961255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.961388 (+   133us) service_pool.cc:224] Handling call
0709 13:02:58.963715 (+  2327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:00.296989 (+1333274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.700681 (+403692us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54001b3d54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:00.701052 (+   371us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2682118400}
I20260709 13:03:00.733760  1571 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:03:00.737298  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:00.667160  1721 consensus_queue.cc:260] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:00.784792  4478 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.065s	user 0.000s	sys 0.002s
I20260709 13:03:00.784960  1721 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:00.794934  4453 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:00.795935  4453 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:00.798038  4296 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:00.802932  4341 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:00.806627  1566 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:03:00.784812  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 164808us
I20260709 13:03:00.815263  1564 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7. Ops: []
W20260709 13:03:00.827159  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:00.829504  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 68485.5us
I20260709 13:03:00.826661  1573 raft_consensus.cc:1275] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:03:00.823311  1713 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2079) took 2409 ms (client timeout 1501 ms). Trace:
W20260709 13:03:00.839774  4495 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.283s	user 0.005s	sys 0.000s
I20260709 13:03:00.842893  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2089) took 1030 ms. Trace:
W20260709 13:03:00.842972  1538 leader_election.cc:336] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.994s (SENT)
I20260709 13:03:00.848877  1538 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:00.773029  4312 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:00.853014  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:00.836078  1713 rpcz_store.cc:269] 0709 13:02:58.413926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.415339 (+  1413us) service_pool.cc:224] Handling call
0709 13:03:00.812486 (+2397147us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b5400247054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:00.823192 (+ 10706us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4643654016}
I20260709 13:03:00.855187  4135 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:00.864509  4226 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:00.864230  4135 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:00.868510  4312 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:00.870286  2347 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:00.883497  4444 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:00.876355  1612 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:00.887907  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:00.843674  1715 rpcz_store.cc:276] 0709 13:02:59.811956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.812162 (+   206us) service_pool.cc:224] Handling call
0709 13:02:59.827534 (+ 15372us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:00.409222 (+581688us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:00.409294 (+    72us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:00.415345 (+  6051us) log.cc:844] Serialized 52 byte log entry
0709 13:03:00.493114 (+ 77769us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:00.493231 (+   117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.513832 (+ 20601us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:00.828827 (+314995us) spinlock_profiling.cc:243] Waited 32.6 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:00.842135 (+ 13308us) raft_consensus.cc:1685] finished
0709 13:03:00.842172 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:00.842579 (+   407us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":580673,"raft.queue_time_us":581685,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"spinlock_wait_cycles":74940800,"thread_start_us":77144,"threads_started":1,"wal-append.queue_time_us":131139}
I20260709 13:03:00.877369  4480 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.906337  1721 raft_consensus.cc:1275] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:03:00.915604  3817 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
W20260709 13:03:00.915802  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 63413.6us
I20260709 13:03:00.916878  4315 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:00.951792  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:00.922405  1573 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:00.959053  1564 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:03:00.900563  4312 consensus_queue.cc:1059] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
W20260709 13:03:01.006651  2314 leader_election.cc:336] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.814s (SENT)
I20260709 13:03:00.985338  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:01.011154  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:01.011885  1573 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:01.031754  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.033736  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 190847us
W20260709 13:03:01.022162  4498 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.081s	user 0.002s	sys 0.002s
I20260709 13:03:01.042506  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 346) took 1598 ms. Trace:
I20260709 13:03:01.041946  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2083) took 2121 ms. Trace:
W20260709 13:03:01.033571  4503 log.cc:927] Time spent T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.265s	user 0.001s	sys 0.002s
I20260709 13:03:01.064635  1568 rpcz_store.cc:276] 0709 13:02:59.443705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.443876 (+   171us) service_pool.cc:224] Handling call
0709 13:02:59.478840 (+ 34964us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.423948 (+945108us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.424041 (+    93us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:00.424106 (+    65us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:00.424151 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.042021 (+617870us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b54002251d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:01.042073 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.042290 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2172549888}
I20260709 13:03:01.066709  1720 rpcz_store.cc:276] 0709 13:02:58.920872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.921022 (+   150us) service_pool.cc:224] Handling call
0709 13:03:00.987392 (+2066370us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b54003a39d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:01.041745 (+ 54353us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1496135424}
I20260709 13:03:01.070449  1573 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
W20260709 13:03:01.072129  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 46285.3us
I20260709 13:03:01.074530  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:01.082360  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.083130  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.981937  4480 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:01.045739  4453 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:01.095805  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 98721.5us
W20260709 13:03:01.085712  1538 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.995s (SENT) [suppressed 31 similar messages]
I20260709 13:03:01.100168  2345 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:03:01.042748  4444 consensus_queue.cc:1059] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.056s
W20260709 13:03:01.112825  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.117516  4172 raft_consensus.cc:3055] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:01.118168  4172 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:01.119602  4135 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:01.124569  2314 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.943s (SENT)
I20260709 13:03:01.130245  4508 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.131485  4508 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:01.133122  4135 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:01.138890  1570 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:01.141813  4444 raft_consensus.cc:3055] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:03:01.145599  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.153415  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 276872us
W20260709 13:03:01.157052  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.146360  1564 raft_consensus.cc:1275] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:01.153018  4480 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:01.136251  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.137451  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 52667.9us
I20260709 13:03:01.166563  4300 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:01.171160  4507 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.097s	user 0.003s	sys 0.000s
W20260709 13:03:01.171922  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.165712  4336 consensus_meta.cc:287] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.512s	user 0.012s	sys 0.006s
I20260709 13:03:01.181829  4300 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:01.187886  4160 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:01.189148  4160 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:01.190213  1537 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:01.160128  4444 raft_consensus.cc:740] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:01.202800  1612 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:01.191639  4233 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:01.203461  4508 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:01.224871  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.225495  4508 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:01.225950  2348 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 13:03:01.227082  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 163532us
I20260709 13:03:01.228423  4135 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:01.187932  4336 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:01.229858  4239 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.230504  4135 consensus_queue.cc:1059] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:01.230690  4239 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:01.231623  4135 raft_consensus.cc:3055] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:01.232053  4135 raft_consensus.cc:740] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:01.232342  4336 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:01.233325  4135 consensus_queue.cc:260] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:01.235988  1611 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:01.156597  1468 catalog_manager.cc:5697] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 1 to 5, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:01.243705  4185 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.244292  4135 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:01.244455  4185 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:01.242381  4328 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:03:01.247387  4185 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:01.249991  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.240783  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:01.251974  2348 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
W20260709 13:03:01.244805  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 44250.7us
W20260709 13:03:01.255231  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.255268  2349 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEARNER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:03:01.260402  1569 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:01.234297  4233 raft_consensus.cc:697] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:01.277086  1611 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:01.261252  4394 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:01.278522  4510 log.cc:927] Time spent T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.122s	user 0.002s	sys 0.000s
W20260709 13:03:01.298620  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 114140us
I20260709 13:03:01.287164  1468 catalog_manager.cc:5697] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 1 to 5, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:01.304514  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:01.281505  4508 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:01.286091  4328 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:01.289808  4239 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:01.329316  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:01.330401  1715 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:01.319873  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:01.336434  1721 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:01.308665  4508 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:01.350075  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:01.320418  4160 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:01.301573  4394 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:01.351969  4480 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:01.357161  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:01.368299  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 105118us
I20260709 13:03:01.344463  4508 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:01.373899  4233 consensus_queue.cc:237] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:01.411160  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:01.297300  4444 consensus_queue.cc:260] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:01.426862  1687 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:01.366509  4480 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:01.440343  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 55693.7us
W20260709 13:03:01.440380  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 70348.1us
I20260709 13:03:01.437840  1468 catalog_manager.cc:5697] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 1 to 5, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:01.451879  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 63102.2us
I20260709 13:03:01.457609  4185 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:01.458824  4185 raft_consensus.cc:697] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 8 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:01.364364  4328 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:01.472492  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:01.465739  1469 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38650 (request call id 196) took 16424 ms (client timeout 15000 ms). Trace:
I20260709 13:03:01.426321  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:01.446990  4480 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:01.476230  1720 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
W20260709 13:03:01.477615  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 191738us
W20260709 13:03:01.490002  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 412243us
W20260709 13:03:01.495997  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.461557  4300 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:01.472299  4394 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:01.518246  4300 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:01.521063  4512 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:01.521589  4300 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:01.522430  1648 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:03:01.535825  1715 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:01.515915  1468 catalog_manager.cc:5697] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:01.530309  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:01.533766  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:01.534849  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:01.526450  2314 leader_election.cc:336] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.582s (SENT)
I20260709 13:03:01.543462  2314 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:01.494999  2343 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:01.544025  1685 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:01.558475  4135 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:01.589223  1720 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5. Ops: []
W20260709 13:03:01.591547  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.593407  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:01.610983  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 49681.8us
I20260709 13:03:01.619626  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:01.612960  2313 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:01.612950  4467 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:01.644275  1644 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:03:01.646287  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:01.660152  1644 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
W20260709 13:03:01.662287  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 146535us
I20260709 13:03:01.647773  1468 catalog_manager.cc:5697] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 4 to 5, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:01.656447  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:01.682242  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 47786.8us
I20260709 13:03:01.652868  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:01.683568  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.684525  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.686070  1648 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:01.538856  4185 consensus_queue.cc:237] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:01.716643  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.478193  1469 rpcz_store.cc:269] 0709 13:02:44.867383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.887607 (+ 20224us) service_pool.cc:224] Handling call
0709 13:02:46.596600 (+1708993us) spinlock_profiling.cc:243] Waited 44.3 ms on lock 0x7b540030ffb8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec44173 00007f824ed4237e 00007f824d2032a4 00007f824d203232 00007f824d203161 00007f824d2030dc 00007f824d202392 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:01.278284 (+14681684us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:52.145440 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:52.171253 (+ 25813us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:52.171317 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:02:52.171338 (+    21us) tablet.cc:662] Decoding operations
0709 13:02:52.189957 (+ 18619us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:52.190082 (+   125us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:52.190107 (+    25us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:02:52.192573 (+  2466us) tablet.cc:701] Row locks acquired
0709 13:02:52.192661 (+    88us) write_op.cc:260] PREPARE: finished
0709 13:02:52.192852 (+   191us) write_op.cc:270] Start()
0709 13:02:52.193333 (+   481us) write_op.cc:276] Timestamp: P: 1783602172192777 usec, L: 0
0709 13:02:52.193363 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:02:52.328400 (+135037us) log.cc:844] Serialized 7138 byte log entry
0709 13:02:53.331073 (+1002673us) op_driver.cc:464] REPLICATION: finished
0709 13:02:53.660094 (+329021us) write_op.cc:301] APPLY: starting
0709 13:02:53.660213 (+   119us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:53.660686 (+   473us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:53.660839 (+   153us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:53.714645 (+ 53806us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:53.715692 (+  1047us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=18
0709 13:02:53.715779 (+    87us) write_op.cc:312] APPLY: finished
0709 13:02:53.716689 (+   910us) log.cc:844] Serialized 125 byte log entry
0709 13:02:53.721560 (+  4871us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:53.736614 (+ 15054us) write_op.cc:454] Released schema lock
0709 13:02:53.739120 (+  2506us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:53.739632 (+   512us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:53.739662 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":128439,"reported_tablets":45,"spinlock_wait_cycles":101882624,"child_traces":[["op",{"apply.queue_time_us":328639,"apply.run_cpu_time_us":8596,"apply.run_wall_time_us":79514,"num_ops":18,"prepare.queue_time_us":162084,"prepare.run_cpu_time_us":19852,"prepare.run_wall_time_us":289889,"raft.queue_time_us":713441,"raft.run_cpu_time_us":48276,"raft.run_wall_time_us":276955,"replication_time_us":1137481,"thread_start_us":711577,"threads_started":4,"wal-append.queue_time_us":141441,"wal-append.run_cpu_time_us":32792,"wal-append.run_wall_time_us":2258486}]]}
W20260709 13:03:01.719772  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 204816us
I20260709 13:03:01.726475  2313 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:01.719915  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.711669  4312 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:01.620851  4135 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:01.766867  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 44266.6us
I20260709 13:03:01.776237  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:01.742237  4467 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:01.786854  4341 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:01.787253  4315 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:01.798247  4315 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:03:01.800208  1641 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:03:01.801519  4315 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:01.805019  4135 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:01.805789  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:01.810647  4195 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:01.813419  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:01.795534  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 79659us
I20260709 13:03:01.802397  4341 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:01.802531  1641 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:03:01.841985  2313 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:01.809252  1722 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:01.867677  1644 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:03:01.857429  1648 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: []
W20260709 13:03:01.910928  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 69086.1us
W20260709 13:03:01.912771  1537 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.501s (SENT)
I20260709 13:03:01.915395  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:01.843165  1538 leader_election.cc:336] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.791s (SENT)
W20260709 13:03:01.881595  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 558865us
I20260709 13:03:01.925405  1686 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:01.815824  4233 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.817756  4328 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:01.926393  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 69248.8us
I20260709 13:03:01.927349  1538 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:01.570111  4512 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:01.846990  1468 catalog_manager.cc:5697] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:01.963126  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.868115  4523 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:01.972770  1645 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:03:01.923767  4483 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:01.980849  1641 raft_consensus.cc:1240] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:02.014242  1566 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7. Ops: []
W20260709 13:03:02.015806  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 53242.7us
W20260709 13:03:02.027885  4508 consensus_meta.cc:287] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.601s	user 0.010s	sys 0.002s
I20260709 13:03:02.028396  4508 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:02.029881  1643 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:02.032197  4341 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:02.034013  1644 raft_consensus.cc:1275] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:03:02.039077  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.941248  4233 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:02.051847  1718 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:03:02.056196  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.051017  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:02.088786  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.052572  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:02.112434  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 317482us
W20260709 13:03:02.118475  1570 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1561) took 2167 ms (client timeout 1976 ms). Trace:
I20260709 13:03:02.019069  4453 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:02.119105  1570 rpcz_store.cc:269] 0709 13:02:59.878364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.881572 (+  3208us) service_pool.cc:224] Handling call
0709 13:03:02.038416 (+2156844us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400306954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:02.045396 (+  6980us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2662462208}
I20260709 13:03:02.114025  1468 catalog_manager.cc:5697] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:02.041970  4524 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:02.149030  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 170493us
I20260709 13:03:02.151654  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:02.051403  4300 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:02.044250  4521 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:02.153017  1645 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
W20260709 13:03:02.154422  4515 log.cc:927] Time spent T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.369s	user 0.000s	sys 0.002s
I20260709 13:03:02.041262  4523 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:02.196909  1612 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:02.193764  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 177125us
I20260709 13:03:02.199851  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:02.173822  1468 catalog_manager.cc:5697] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 1 to 3, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:02.183574  4520 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.045187  4172 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:02.213447  1566 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:03:02.215004  1572 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:03:02.215679  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:02.222534  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:02.215858  1568 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:03:02.224922  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:02.225520  1568 raft_consensus.cc:606] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:02.226007  1568 raft_consensus.cc:614] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:02.212867  4533 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.226783  4521 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.187541  4453 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:02.257354  4528 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
W20260709 13:03:02.258152  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.262800  4537 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:02.267395  4543 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:02.270321  1569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1567) took 2389 ms (client timeout 1766 ms). Trace:
W20260709 13:03:02.270812  1569 rpcz_store.cc:269] 0709 13:02:59.880297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.887563 (+  7266us) service_pool.cc:224] Handling call
0709 13:03:02.269892 (+2382329us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400014854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:02.270054 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2693699456}
I20260709 13:03:02.229277  4542 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:02.273164  4531 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:03:02.296828  4172 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:02.301098  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 122149us
I20260709 13:03:02.302804  1612 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:02.321535  2314 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.020s	sys 0.027s
W20260709 13:03:02.322412  2314 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.021s	sys 0.027s
W20260709 13:03:02.322971  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:02.324249  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 45356.8us
I20260709 13:03:02.233817  4538 consensus_queue.cc:1059] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.088s
I20260709 13:03:02.348814  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 347) took 1586 ms. Trace:
I20260709 13:03:02.349320  2349 rpcz_store.cc:276] 0709 13:03:00.756286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.759312 (+  3026us) service_pool.cc:224] Handling call
0709 13:03:01.237953 (+478641us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.258108 (+ 20155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.335363 (+1077255us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b5400467650. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:02.342882 (+  7519us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1062423296}
W20260709 13:03:02.368778  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.319994  4512 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:02.378005  1570 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:02.408336  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:02.409185  1569 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
W20260709 13:03:02.363147  4480 consensus_meta.cc:287] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.840s	user 0.007s	sys 0.007s
W20260709 13:03:02.420791  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 52070.7us
I20260709 13:03:02.421912  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:02.424253  1566 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:02.286139  4532 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.117s
W20260709 13:03:02.431911  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:02.408890  4531 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
W20260709 13:03:02.426815  4526 log.cc:927] Time spent T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.402s	user 0.002s	sys 0.000s
I20260709 13:03:02.463536  1569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:03:02.465889  1569 raft_consensus.cc:606] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Received request to transfer leadership to TS 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:02.466365  1569 raft_consensus.cc:614] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:02.481915  1611 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:02.435846  4532 consensus_queue.cc:1243] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:03:02.495898  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.446159  4527 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:02.519731  1538 leader_election.cc:336] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.908s (SENT)
I20260709 13:03:02.480947  4467 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:02.437546  4480 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:02.522221  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 219618us
I20260709 13:03:02.539997  4233 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:02.540228  1612 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:02.541623  4511 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:03:02.281864  4312 consensus_queue.cc:1175] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "066ee0109aba4465ba17fa65e14ef504" responder_term: 7 status { last_received { term: 6 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:03:02.543452  4540 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
W20260709 13:03:02.543843  4536 log.cc:927] Time spent T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
W20260709 13:03:02.545715  4547 log.cc:927] Time spent T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:03:02.560318  4523 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:02.223238  4520 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:02.563678  1639 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:02.227233  4533 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:02.227679  4521 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:02.263839  4537 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:02.268102  4543 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:02.184801  4524 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:02.567214  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1555) took 1075 ms. Trace:
W20260709 13:03:02.568070  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.517009  4527 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:02.570546  4556 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.571233  4512 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:02.578934  1570 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:03:02.666172  1570 raft_consensus.cc:606] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Received request to transfer leadership to TS 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:02.547581  4467 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:02.675871  4527 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:02.583007  4549 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.677047  4467 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:02.677215  4549 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:02.679725  4549 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:02.366195  4542 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:02.680842  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2098) took 1236 ms. Trace:
I20260709 13:03:02.682278  4542 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:02.567919  2314 leader_election.cc:336] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.902s (SENT)
I20260709 13:03:02.683023  2314 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:02.683977  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1576) took 2780 ms. Trace:
W20260709 13:03:02.684440  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 137444us
I20260709 13:03:02.590972  4559 raft_consensus.cc:3055] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:02.686684  4524 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:02.687299  4559 raft_consensus.cc:740] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:02.592674  4511 consensus_queue.cc:1059] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:02.593331  4562 consensus_queue.cc:1059] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:03:02.689929  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 718493us
I20260709 13:03:02.690294  4562 raft_consensus.cc:3055] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:02.690927  4562 raft_consensus.cc:740] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:02.594636  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 54545.8us
W20260709 13:03:02.596396  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 114583us
I20260709 13:03:02.696125  1612 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:02.696722  4551 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:02.697587  4511 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.698215  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 225) took 1177 ms. Trace:
I20260709 13:03:02.699581  1612 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:02.699375  1713 rpcz_store.cc:276] 0709 13:03:01.521122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.521768 (+   646us) service_pool.cc:224] Handling call
0709 13:03:01.522187 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.522536 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:01.972944 (+450408us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.697830 (+724886us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.697885 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:02.697929 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.698016 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.698140 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":552063,"prepare.queue_time_us":132515,"prepare.run_cpu_time_us":140,"prepare.run_wall_time_us":126,"thread_start_us":606678,"threads_started":2,"wal-append.queue_time_us":243169}
I20260709 13:03:02.701742  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:02.702138  4512 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:02.702773  1569 raft_consensus.cc:2410] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:03:02.703048  4512 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:02.595832  4555 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:02.703791  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:02.704185  4555 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:02.704617  1568 raft_consensus.cc:2393] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 7: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:02.704727  4555 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:02.705693  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:02.706326  1571 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:02.707275  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:02.708127  1573 raft_consensus.cc:2410] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:03:02.708837  4328 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:03:02.611974  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:02.592805  4539 log.cc:927] Time spent T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 13:03:02.705730  4229 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.613547  1566 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:02.577353  4480 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:02.711432  4229 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:02.712258  1570 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:51464
W20260709 13:03:02.662029  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.582425  4453 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:02.557330  4457 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:02.663218  4544 log.cc:927] Time spent T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.190s	user 0.004s	sys 0.000s
I20260709 13:03:02.712903  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:02.715440  1564 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:02.686383  4558 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:02.684484  1572 rpcz_store.cc:276] 0709 13:02:59.903199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.542670 (+639471us) service_pool.cc:224] Handling call
0709 13:03:02.045953 (+1503283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.219109 (+173156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.683713 (+464604us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b5400306950. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:02.683871 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3415159040}
I20260709 13:03:02.712853  1570 raft_consensus.cc:606] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:02.701226  1612 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:03:02.684670  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 761653us
I20260709 13:03:02.719122  1570 raft_consensus.cc:614] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:02.681339  1721 rpcz_store.cc:276] 0709 13:03:01.444970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.450323 (+  5353us) service_pool.cc:224] Handling call
0709 13:03:01.465507 (+ 15184us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.465937 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:01.634384 (+168447us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.886935 (+252551us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.680289 (+793354us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b18003e28a0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d12c0a 00007f824f452219 00007f8245d9aa40 00007f8245d954e4 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287 00007f8248ea0208
0709 13:03:02.680446 (+   157us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.680499 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:02.680545 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.680614 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.680769 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":735364,"prepare.queue_time_us":206465,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":814926,"raft.run_cpu_time_us":569,"raft.run_wall_time_us":43533,"spinlock_wait_cycles":75542144,"thread_start_us":335110,"threads_started":2,"wal-append.queue_time_us":322436}
I20260709 13:03:02.622263  1646 rpcz_store.cc:276] 0709 13:03:01.492732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.552580 (+ 59848us) service_pool.cc:224] Handling call
0709 13:03:01.753845 (+201265us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:01.926548 (+172703us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:03:02.189689 (+263141us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.408439 (+218750us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:02.475315 (+ 66876us) log.cc:844] Serialized 52 byte log entry
0709 13:03:02.475594 (+   279us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:02.475975 (+   381us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.481721 (+  5746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.481809 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.566839 (+ 85030us) raft_consensus.cc:1685] finished
0709 13:03:02.566868 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.567004 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5436,"prepare.queue_time_us":121829,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":5041,"raft.queue_time_us":550183,"raft.run_cpu_time_us":764,"raft.run_wall_time_us":105895,"thread_start_us":391434,"threads_started":3,"wal-append.queue_time_us":295453}
I20260709 13:03:02.710884  4522 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:02.746035  1611 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:02.718448  4135 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:02.693349  4562 consensus_queue.cc:260] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:02.795828  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 94750us
W20260709 13:03:02.786741  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:02.798087  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:02.805085  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 59134.7us
I20260709 13:03:02.766585  4524 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:02.779943  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:02.811973  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 100080us
I20260709 13:03:02.813138  1611 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:02.698284  4511 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:02.824776  4336 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.825520  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:02.826781  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:02.834124  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.799549  4543 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:02.743898  4520 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:02.796622  4229 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:02.784282  4341 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:03:02.800490  4537 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:02.814778  4444 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:02.839564  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 146141us
W20260709 13:03:02.830780  1538 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.501s (SENT)
I20260709 13:03:02.851155  1538 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:02.828711  4172 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:02.829020  4565 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.833154  4552 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:02.743480  4457 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:02.722386  4556 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:02.861964  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 45370.5us
I20260709 13:03:02.863724  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1570) took 1073 ms. Trace:
W20260709 13:03:02.863786  1538 leader_election.cc:336] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.520s (SENT)
I20260709 13:03:02.881188  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:02.864554  4537 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:02.866070  4508 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:02.843551  4562 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:02.844832  4524 raft_consensus.cc:697] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:02.879140  1612 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:02.902789  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2104) took 1179 ms. Trace:
I20260709 13:03:02.807840  4522 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:02.904726  1717 rpcz_store.cc:276] 0709 13:03:01.723173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.723525 (+   352us) service_pool.cc:224] Handling call
0709 13:03:01.760023 (+ 36498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.913376 (+153353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.913415 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:01.913467 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:01.913514 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.902404 (+988890us) spinlock_profiling.cc:243] Waited 64.8 ms on lock 0x7b5400212854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:02.902472 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.902700 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149090048}
W20260709 13:03:02.915793  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 88214.4us
W20260709 13:03:02.915772  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.840899  4336 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:02.885155  4328 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:02.871523  4552 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:02.926517  4564 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:02.875451  4565 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:02.932157  1570 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "130bc5ff39b847b1b3204626f284f838"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:03:02.934371  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:02.937120  4556 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:02.938198  4420 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:02.921867  4555 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:02.880885  1647 rpcz_store.cc:276] 0709 13:03:01.644405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.754538 (+110133us) service_pool.cc:224] Handling call
0709 13:03:01.781972 (+ 27434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.810335 (+ 28363us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:01.975097 (+164762us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.716981 (+741884us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.717057 (+    76us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:02.717130 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.717312 (+   182us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.717488 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9307,"prepare.queue_time_us":237533,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":201240,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":22846,"spinlock_wait_cycles":16124160,"thread_start_us":753608,"threads_started":3,"wal-append.queue_time_us":322333}
I20260709 13:03:02.874516  4172 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:02.929557  4328 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:02.957357  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1575) took 1109 ms. Trace:
I20260709 13:03:02.931566  4552 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:02.945930  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 66934.5us
I20260709 13:03:02.949409  1570 raft_consensus.cc:606] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Received request to transfer leadership to TS 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:02.976902  1570 raft_consensus.cc:614] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:02.956418  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 92693.2us
W20260709 13:03:02.979279  1538 leader_election.cc:336] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.639s (SENT)
I20260709 13:03:02.980261  1538 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:02.945240  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:02.965459  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:02.975204  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 48915.3us
W20260709 13:03:02.971074  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 105650us
I20260709 13:03:02.985520  1572 raft_consensus.cc:2410] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:03:02.987649  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:02.962793  4554 log.cc:927] Time spent T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.389s	user 0.006s	sys 0.000s
I20260709 13:03:02.972885  1648 rpcz_store.cc:276] 0709 13:03:01.848203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.928744 (+ 80541us) service_pool.cc:224] Handling call
0709 13:03:01.974071 (+ 45327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.983183 (+  9112us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:03:02.151616 (+168433us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.423858 (+272242us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.423920 (+    62us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:02.424043 (+   123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.957044 (+533001us) spinlock_profiling.cc:243] Waited 8.87 ms on lock 0x7b54000a5ad4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:02.957100 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.957268 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":65196,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":246997,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":18735,"spinlock_wait_cycles":20389760,"thread_start_us":379176,"threads_started":3,"wal-append.queue_time_us":160118}
I20260709 13:03:02.954602  4522 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:02.968729  4556 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:02.962193  4457 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:03.006222  1612 leader_election.cc:400] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7.
I20260709 13:03:03.006641  1612 leader_election.cc:403] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:03.014040  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:03.016463  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:02.991393  4341 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:03:03.018522  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:03.021154  1713 pending_rounds.cc:85] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 3
I20260709 13:03:03.021142  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:03.021790  1538 leader_election.cc:336] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.650s (SENT)
I20260709 13:03:03.022341  1538 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:03.023648   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:03.026017   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:03.026306  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:03.027570  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:03.027719  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:03.028308  1642 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:03.028811  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:03.029130  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.998545  4537 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:03.030336  4226 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:03.031108  4336 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:03.031904  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:03.032538  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:03.021798  1713 pending_rounds.cc:107] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:03:03.033517  1713 raft_consensus.cc:2889] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:03.035519  4565 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:03.049507  4577 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:03.050740  1687 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:03.051733  4511 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:03.051872  4090 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:03.052456  4090 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:03.053076  4090 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:03.052179  4577 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:03.054643  1720 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:03:03.016943  4135 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7.
W20260709 13:03:03.060475  4546 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.567s	user 0.004s	sys 0.000s
I20260709 13:03:03.061581  4172 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:03.016608  4566 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:03.066715  4555 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:03.067358  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:03.067754  4312 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.068490  4312 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:03.072849  4312 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:02.992751  4524 consensus_queue.cc:237] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:03.076977  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:03.079412  4566 raft_consensus.cc:697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:03.076275  4090 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:03.080674  4566 consensus_queue.cc:237] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:03.070488  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 79889.5us
I20260709 13:03:03.088728  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:03.083106  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:03.010377  4556 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:02.941622  4574 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 13:03:03.090888  4533 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:03.090909  4555 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
W20260709 13:03:03.111327  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 89590.1us
I20260709 13:03:03.105118  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:03.112869  1572 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:03.105352  4090 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:03.108558  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 91891.5us
I20260709 13:03:03.104323  4556 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:03.117065  4185 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:03.111936  1564 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:03.113144  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:03.118934  1566 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
W20260709 13:03:03.102581  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 48962.4us
W20260709 13:03:03.127622  1538 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.713s (SENT)
I20260709 13:03:03.132791  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1550) took 1715 ms. Trace:
I20260709 13:03:03.133499  1639 rpcz_store.cc:276] 0709 13:03:01.413676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.438450 (+ 24774us) service_pool.cc:224] Handling call
0709 13:03:01.457179 (+ 18729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.613410 (+1156231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.126602 (+513192us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540007ec54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:03.128770 (+  2168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2516992384}
I20260709 13:03:03.086372  4328 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:03.128881  4574 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:03:03.144917  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:03.140503  4578 consensus_queue.cc:1059] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:03:03.149719  4341 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:03.150698  4573 consensus_queue.cc:1059] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:03.150620  4581 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:03:03.154289  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:03.154886  1566 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:03.147953  1685 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:03.153582  1719 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
W20260709 13:03:03.160285  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.136564  4090 consensus_queue.cc:1048] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:03:03.160799  4577 raft_consensus.cc:3055] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:03.161756  4577 raft_consensus.cc:740] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:03.163156  4581 raft_consensus.cc:3055] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:03.163471  4527 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.163630  4581 raft_consensus.cc:740] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:03.164180  4527 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:03.165284  4521 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:03.162880  4559 consensus_queue.cc:260] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:03.140815  4336 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:03:03.166807  1687 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:03.131175  4580 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:03:03.172427  2312 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:03.177587  1611 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:03.182099  1611 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:03.190421  4549 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:03:03.199083  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 85521.7us
I20260709 13:03:03.191813  4559 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:03.203410  1538 leader_election.cc:336] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.727s (SENT)
I20260709 13:03:03.188722  4524 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:03.176590  4578 consensus_queue.cc:1048] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:03:03.201006  4160 consensus_queue.cc:1059] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:03:03.210924  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:03.210583  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:03.212347  4538 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:03.193882  4312 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:03.215380  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 72586.5us
I20260709 13:03:03.215728  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:03.194474  4574 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:03.162962  4577 consensus_queue.cc:260] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:03.222136  1720 raft_consensus.cc:2376] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6. Re-sending same reply.
I20260709 13:03:03.199674  4573 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:03.196813  4549 raft_consensus.cc:697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:03.222491  2349 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:03.201572  4543 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.166144  4581 consensus_queue.cc:260] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:03.221421  4524 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:03.224495  4574 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.246755  4574 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:03.230072  4312 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.234200  3817 consensus_queue.cc:1048] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
W20260709 13:03:03.249809  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 49588.6us
I20260709 13:03:03.242029  4336 raft_consensus.cc:3055] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:03.250505  4336 raft_consensus.cc:740] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:03.251682  4336 consensus_queue.cc:260] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:03.253047  4336 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:03.253304  1538 leader_election.cc:336] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.813s (SENT)
I20260709 13:03:03.236299  4538 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:03.261045  4581 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:03.262980  4538 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:03.243892  4573 raft_consensus.cc:697] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:03.264565  4573 consensus_queue.cc:237] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:03.245839  4521 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:03.249001  4537 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:03.258742  1538 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:03.257303  4527 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:03.263265  4341 raft_consensus.cc:3055] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:03.285946  4341 raft_consensus.cc:740] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:03.287778  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:03.280404  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:03.290866  1566 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:03:03.292302  2312 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:03.287107  4341 consensus_queue.cc:260] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:03.295104  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 71059.5us
I20260709 13:03:03.295526  4341 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:03.298983  4229 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:03.302110  1687 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:03.283591  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:03.307380  1570 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:03:03.294454  4574 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:03.308921  1611 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:03.310178  4538 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:03.310638  4538 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:03.311424  4538 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:03.310438  4549 consensus_queue.cc:237] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:03.310624  4574 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:03.318868  4529 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:03.307893  2349 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:03.319773  4529 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:03.320914  4172 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:03.320963  4529 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:03.322431  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.324008  4172 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.324535  4172 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:03.329864  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.330606  4574 consensus_queue.cc:1059] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:03.331996  4578 raft_consensus.cc:3055] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:03.332396  4578 raft_consensus.cc:740] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:03.333267  4578 consensus_queue.cc:260] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:03.334571  4578 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:03.334968  1686 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:03.336391  4090 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:03.337085  4090 raft_consensus.cc:697] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:03.336992  4172 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:03.338541  4090 consensus_queue.cc:237] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:03.340183  4538 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:03.342911  4538 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:03.343765  4577 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:03.270956  4328 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:03.344069  4172 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:03.248622  4524 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:03.244451  4543 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:03.258518  4312 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:03.369379  1572 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:03.376814  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:03.378031  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:03.378832  1566 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:03.383839  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:03.384567  1570 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:03.387503  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:03.389233  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:03.390309  1642 raft_consensus.cc:2410] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:03.393311  1570 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:03.396936  1640 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:03.407598  1537 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:03.374819  4444 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:03:03.426033  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:03.427034  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.425017  1611 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:03.430197  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:03.376852  4229 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:03.432673  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 179413us
I20260709 13:03:03.444216  1564 raft_consensus.cc:1240] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:03.445232  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:03.449970  1566 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:03:03.440001  1611 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:03.452505  2312 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:03.461037  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:03.461790  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:03.438426  4226 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:03.454077  4229 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:03.454237  4522 raft_consensus.cc:2764] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:03:03.446600  4529 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:03.446300  4508 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:03.477591  4229 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:03.479794  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.480048  4538 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:03.499884  4538 raft_consensus.cc:697] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:03.500366  1570 raft_consensus.cc:1275] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:03:03.488121  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:03.502635  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.488596  1718 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:03.489843  1720 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:03.514216  2313 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:03.532078  1566 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:03.532546  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:03.488226  4558 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:03.484980  4543 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:03.496531  4508 raft_consensus.cc:697] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:03.524371  4226 raft_consensus.cc:697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 6 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:03.500895  4538 consensus_queue.cc:237] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:03.529991  4467 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:03.488353  4529 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:03.581426  1719 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 5. Current term is 6. Ops: []
W20260709 13:03:03.582717  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.589843  4521 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:03.615793  1640 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:03.617847  1687 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:03.619107  1570 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 13:03:03.606586  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 140868us
W20260709 13:03:03.603874  4552 consensus_meta.cc:287] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.608s	user 0.014s	sys 0.002s
I20260709 13:03:03.595252  4558 raft_consensus.cc:697] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:03.639137  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:03.648823  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:03.650725  1640 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:03.656198  4524 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:03.656564  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:03.657696  1640 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:03.658962  4524 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:03.661994  4312 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:03.664727  4312 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:03.666272  4508 consensus_queue.cc:237] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:03.673372  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.671986  1687 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:03.639449  4529 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:03.685985  1570 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:03.638082  4552 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:03.710860  1566 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:03:03.717420  1640 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
W20260709 13:03:03.724556  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 123161us
I20260709 13:03:03.724788  1687 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:03.726218  1639 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:03.731634  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:03.670535  4226 consensus_queue.cc:237] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:03.724015  4336 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:03.765760  4529 raft_consensus.cc:697] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:03:03.772104  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.781283  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:03.785888  1570 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:03.756919  4336 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.762252  4328 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:03.813889  1566 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:03.817584  1570 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:03:03.735983  4558 consensus_queue.cc:237] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:03.824167  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:03.824484  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:03.828101  2349 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:03.828662  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.804400  4336 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:03.826725  4328 raft_consensus.cc:697] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 8 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:03.902810  2349 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:03.904191  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:03.905035  1566 raft_consensus.cc:2368] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10.
W20260709 13:03:03.907990  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.915426  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:03.916217  1566 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:03.918779  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:03.919754  1564 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:03.929790  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:03.931638  1571 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:03.934168  1573 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:03:03.943943  4597 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:03:03.949474  1568 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:03.951088  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:03.874941  4529 consensus_queue.cc:237] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:03.957903  4577 consensus_meta.cc:287] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.596s	user 0.003s	sys 0.011s
I20260709 13:03:03.986948  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:03.996722  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:03.997051  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 89992.2us
I20260709 13:03:03.967119  4552 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:04.012334  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:04.012334  1564 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:04.019249  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:03.998935  2314 leader_election.cc:336] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.623s (SENT)
I20260709 13:03:03.961877  4549 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:03:04.022696  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:04.024190  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:04.031845  4602 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:04.037531  2345 raft_consensus.cc:2368] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 7. Current term is 8.
I20260709 13:03:04.048036  1566 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:04.050283  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:03.965204  4328 consensus_queue.cc:237] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:04.051138  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 52246.9us
W20260709 13:03:04.053284  4603 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.146s	user 0.003s	sys 0.000s
W20260709 13:03:04.054414  4610 log.cc:927] Time spent T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:03:04.060098  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.045599  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:04.084369  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:04.085192  1568 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:04.085747  1567 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:04.087435  1570 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:04.086488  1569 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:04.099730  1566 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
W20260709 13:03:04.103091  4600 log.cc:927] Time spent T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 13:03:04.102787  2343 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:04.112561  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:04.125703  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:04.126322  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:04.132401  4605 log.cc:927] Time spent T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.188s	user 0.000s	sys 0.004s
I20260709 13:03:04.133282  4609 consensus_queue.cc:1059] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:04.134430  4609 raft_consensus.cc:3055] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:04.134781  4609 raft_consensus.cc:740] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:04.133620  4577 raft_consensus.cc:2955] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Committing config change with OpId 3.4: config changed from index 2 to 4, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:04.135774  4609 consensus_queue.cc:260] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:04.137063  4609 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:04.138840  4576 raft_consensus.cc:2955] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Committing config change with OpId 6.6: config changed from index 2 to 6, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:03:04.109325  4559 consensus_meta.cc:287] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.888s	user 0.013s	sys 0.000s
W20260709 13:03:04.144227  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:04.145552  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.129809  1571 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
W20260709 13:03:04.152735  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.149799  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:04.162573  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:04.173759  1565 raft_consensus.cc:2368] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10.
I20260709 13:03:04.190136  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:04.196656  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:04.198259  2312 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:04.187657  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:04.201370  1718 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:04.160003  3817 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:04.225235  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:04.226217  1715 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:04.190377  4172 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.224464  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:04.248974  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.245573  3817 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:04.249665  1612 connection.cc:256] RPC call timeout handler was delayed by -3.4681453840000245s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1663s for a response
W20260709 13:03:04.250377  1612 connection.cc:256] RPC call timeout handler was delayed by -3.3097453839999957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1839s for a response
I20260709 13:03:04.251796  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:04.252513  1647 raft_consensus.cc:2376] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7. Re-sending same reply.
I20260709 13:03:04.220320  4602 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:04.254747  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:04.256227  1715 raft_consensus.cc:1217] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 4.3->[6.4-8.5]   Dedup: 8.5->[]
I20260709 13:03:04.256958  1715 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 13:03:04.258966  1641 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:04.260196  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:04.261382  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:04.262072  1641 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:03:04.263036  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:04.265339  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:04.267457  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:04.268930  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:04.270376  4613 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:04.271323  4613 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:04.272650  4538 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:04.273952  4538 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:04.280684  1538 leader_election.cc:336] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.537s (SENT)
I20260709 13:03:04.286171  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:04.297427  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:04.308972  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.311715  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:04.313145  1572 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:03:04.316035  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:04.320780  1713 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:04.203553  4336 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:04.329080  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:04.338578  1538 leader_election.cc:336] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.995s (SENT)
I20260709 13:03:04.339249  1538 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:04.355500  1714 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:03:04.358094  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:04.359179  1640 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:04.373793  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.379475  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:04.383888  1646 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
W20260709 13:03:04.359234  1538 leader_election.cc:336] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.106s (SENT)
I20260709 13:03:04.396251  1538 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:04.397269  1568 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:03:04.402022  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:04.405345  1648 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:03:04.407531  1648 pending_rounds.cc:85] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 3
I20260709 13:03:04.277031  4172 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:04.406440  1715 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:04.408254  1648 pending_rounds.cc:107] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:03:04.409731  1648 raft_consensus.cc:2889] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:04.415851  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.348810  4522 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:04.433208  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 78157.3us
I20260709 13:03:04.440860  1719 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:04.446158  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:04.450685  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:04.453944  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 89745.9us
I20260709 13:03:04.468216  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:04.456414  2347 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:04.484395  1572 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:04.359578  4573 consensus_queue.cc:1059] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.086s
I20260709 13:03:04.521680  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:04.387111  4543 consensus_queue.cc:1059] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 13:03:04.559713  1639 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:04.425575  3817 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:04.468665  1642 raft_consensus.cc:2955] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:04.578588  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:04.577494  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:04.571596  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:04.576334  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:04.579031  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:04.580003  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:04.585156  1639 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:03:04.586824  1647 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:04.493366  1646 raft_consensus.cc:2955] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 2 to 6, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:04.416719  4623 consensus_queue.cc:1048] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.186s
I20260709 13:03:04.605140  1640 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:04.604602  1648 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:04.560281  4602 raft_consensus.cc:697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:04.570562  4420 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:04.613519  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:04.579993  3879 raft_consensus.cc:3055] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:04.438596  4522 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:03:04.579074  4601 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:04.578681  4336 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:04.635697  1572 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:04.594384  4226 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:04.632354  4538 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:04.659987  1572 raft_consensus.cc:1275] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:04.661758  1572 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:03:04.662084  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:04.662763  1641 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:03:04.663666  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:04.665973  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:04.687595  1639 raft_consensus.cc:1275] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:03:04.691171  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:04.702643  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 343436us
I20260709 13:03:04.709187  4592 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:04.710150  4592 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:04.632980  3879 raft_consensus.cc:740] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:03:04.744065  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 380272us
I20260709 13:03:04.752925  1572 raft_consensus.cc:1275] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:03:04.622478  4606 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.118s
I20260709 13:03:04.664798  4601 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:04.823330  1713 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
W20260709 13:03:04.842016  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 260953us
W20260709 13:03:04.952611  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.340543  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:05.028731  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:05.341590  1717 raft_consensus.cc:2410] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:03:05.238252  4624 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.590s	user 0.000s	sys 0.001s
W20260709 13:03:05.163606  4637 log.cc:927] Time spent T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:03:05.164069  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:05.065603  4600 log.cc:927] Time spent T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.314s	user 0.000s	sys 0.000s
W20260709 13:03:05.079221  4627 log.cc:927] Time spent T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.441s	user 0.000s	sys 0.000s
I20260709 13:03:04.816175  1714 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:03:05.283025  1539 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.853s	user 0.009s	sys 0.007s
W20260709 13:03:05.345981  1539 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.009s	sys 0.007s
I20260709 13:03:05.346413  1645 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:04.695724  4420 raft_consensus.cc:3055] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:05.347919  4420 raft_consensus.cc:740] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:05.349905  4420 consensus_queue.cc:260] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:05.111716  1643 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.007s	sys 0.010s
I20260709 13:03:05.332989   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:03:05.383443  1643 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.013s	sys 0.016s
W20260709 13:03:05.342644  4634 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.248s	user 0.000s	sys 0.000s
W20260709 13:03:05.351222  4630 log.cc:927] Time spent T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 13:03:05.068928  4602 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:05.354291  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1715) took 1016 ms. Trace:
I20260709 13:03:05.355352  3879 consensus_queue.cc:260] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:05.376750  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1612) took 1036 ms. Trace:
W20260709 13:03:05.404434  1716 consensus_meta.cc:287] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.024s	user 0.016s	sys 0.000s
W20260709 13:03:05.387660  2314 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.012s	sys 0.010s
I20260709 13:03:05.421602  1716 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
W20260709 13:03:05.422475  2314 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.012s	sys 0.010s
W20260709 13:03:05.427680  2314 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.032s (SENT)
I20260709 13:03:05.430296  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:05.423354  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.027988  4524 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:05.392894  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 52883.2us
W20260709 13:03:05.160066  4576 consensus_meta.cc:287] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.019s	user 0.010s	sys 0.004s
W20260709 13:03:05.162427  4625 log.cc:927] Time spent T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.562s	user 0.001s	sys 0.000s
W20260709 13:03:05.331722  4229 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.004s	sys 0.038s
I20260709 13:03:05.411334  1572 rpcz_store.cc:276] 0709 13:03:04.337920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.662851 (+324931us) service_pool.cc:224] Handling call
0709 13:03:04.750330 (+ 87479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.282271 (+531941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.354201 (+ 71930us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.340862  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1618) took 1143 ms. Trace:
I20260709 13:03:05.468008  1571 rpcz_store.cc:276] 0709 13:03:04.197260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.198328 (+  1068us) service_pool.cc:224] Handling call
0709 13:03:04.198678 (+   350us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:04.246493 (+ 47815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:04.246539 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:04.310857 (+ 64318us) log.cc:844] Serialized 52 byte log entry
0709 13:03:04.600226 (+289369us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:04.600267 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.600333 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:05.340609 (+740276us) raft_consensus.cc:1685] finished
0709 13:03:05.340634 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.340800 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37932,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":104,"thread_start_us":186744,"threads_started":1,"wal-append.queue_time_us":796390}
I20260709 13:03:05.476814  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:04.919634  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.421665  1645 rpcz_store.cc:276] 0709 13:03:04.340637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.342780 (+  2143us) service_pool.cc:224] Handling call
0709 13:03:04.395796 (+ 53016us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.346096 (+950300us) spinlock_profiling.cc:243] Waited 5.27 ms on lock 0x7b54002772a0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248dfcdc0 00007f8248e871c0 00007f8248e88231 00007f8248e84d8f 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c
0709 13:03:05.347248 (+  1152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.376650 (+ 29402us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12121984}
I20260709 13:03:05.440666  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1674) took 1112 ms. Trace:
I20260709 13:03:05.455955  1564 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:03:05.478925  1716 rpcz_store.cc:276] 0709 13:03:04.312526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.328763 (+ 16237us) service_pool.cc:224] Handling call
0709 13:03:05.424630 (+1095867us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.447046  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:05.377578  4312 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:05.409832  4538 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:05.334609  4592 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:05.459033  4420 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:05.473688  4229 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.005s	sys 0.039s
I20260709 13:03:05.513406  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:05.530864  1573 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:03:05.536741  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:05.509967  4609 consensus_meta.cc:287] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.351s	user 0.019s	sys 0.001s
W20260709 13:03:05.540697  1641 consensus_meta.cc:287] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.854s	user 0.014s	sys 0.001s
I20260709 13:03:05.463181  4577 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 13:03:05.617527  1641 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:05.618350  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 234) took 1003 ms. Trace:
I20260709 13:03:05.618676  1641 rpcz_store.cc:276] 0709 13:03:04.614900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.661730 (+ 46830us) service_pool.cc:224] Handling call
0709 13:03:05.618224 (+956494us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:05.581171  4633 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.923s	user 0.009s	sys 0.009s
I20260709 13:03:05.620314  4522 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:05.596688  4649 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:03:05.530889  4538 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:05.620419  1719 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:05.628811  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1713) took 1436 ms. Trace:
I20260709 13:03:05.463150  4524 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:05.637764  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1620) took 1367 ms. Trace:
I20260709 13:03:05.578264  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:05.629359  1566 rpcz_store.cc:276] 0709 13:03:04.192549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.203927 (+ 11378us) service_pool.cc:224] Handling call
0709 13:03:04.218857 (+ 14930us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:04.245727 (+ 26870us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:03:04.245771 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:04.280743 (+ 34972us) log.cc:844] Serialized 26 byte log entry
0709 13:03:04.598438 (+317695us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:04.598495 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.598593 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:05.628567 (+1029974us) raft_consensus.cc:1685] finished
0709 13:03:05.628588 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.628742 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6473,"raft.queue_time_us":29289,"raft.run_cpu_time_us":606,"raft.run_wall_time_us":604,"thread_start_us":266593,"threads_started":1,"wal-append.queue_time_us":1335700}
W20260709 13:03:05.640280  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 156932us
W20260709 13:03:05.637738  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 934136us
I20260709 13:03:05.641263  1685 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:05.629550  4654 log.cc:927] Time spent T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:03:05.643914  2347 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55890 (request call id 295) took 1336 ms (client timeout 1655 ms). Trace:
I20260709 13:03:05.621351  4522 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.638350  1565 rpcz_store.cc:276] 0709 13:03:04.270347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.273766 (+  3419us) service_pool.cc:224] Handling call
0709 13:03:04.274401 (+   635us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:04.274922 (+   521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:04.274951 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:04.275693 (+   742us) log.cc:844] Serialized 26 byte log entry
0709 13:03:04.467651 (+191958us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:04.467713 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.467814 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:05.637318 (+1169504us) raft_consensus.cc:1685] finished
0709 13:03:05.637352 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.637662 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2160,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":10021,"thread_start_us":191567,"threads_started":1,"wal-append.queue_time_us":475146}
I20260709 13:03:05.503823  4575 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:03:05.581048  4641 log.cc:927] Time spent T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
I20260709 13:03:05.673290  4609 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:05.682729  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1616) took 1566 ms. Trace:
I20260709 13:03:05.608872  4613 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:05.654605  2347 rpcz_store.cc:269] 0709 13:03:04.296660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.296879 (+   219us) service_pool.cc:224] Handling call
0709 13:03:05.632761 (+1335882us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b5400600254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:05.632937 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":363431808}
I20260709 13:03:05.665612  1685 leader_election.cc:400] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10.
I20260709 13:03:05.683295  1570 rpcz_store.cc:276] 0709 13:03:04.116407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.123193 (+  6786us) service_pool.cc:224] Handling call
0709 13:03:04.123602 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:04.124078 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:04.124117 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:04.124602 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 13:03:04.280146 (+155544us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:04.280216 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.280301 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:05.682465 (+1402164us) raft_consensus.cc:1685] finished
0709 13:03:05.682491 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.682652 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1454,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":145,"thread_start_us":154762,"threads_started":1,"wal-append.queue_time_us":326662}
I20260709 13:03:05.625612  4538 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:05.530608  4592 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:03:05.747180  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 81851.1us
I20260709 13:03:05.747697  4592 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:05.748404  1685 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:05.750053  4592 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:05.698268  4552 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.751045  4226 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:05.751533  4226 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:05.703934  4172 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:05.709257  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:05.712085  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:05.712934  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:05.757490  1641 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 6: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:05.713744  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:05.758512  1645 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:05.717968  4652 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:05.718930  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:05.763072  1686 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.0.107.254:38281, user_credentials={real_user=slave}} blocked reactor thread for 76627.4us
I20260709 13:03:05.763507  1647 raft_consensus.cc:2410] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:05.722510  4575 raft_consensus.cc:697] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:05.727289  4650 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:05.728600  1538 connection.cc:256] RPC call timeout handler was delayed by -1.164735038999936s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19610000000000002s for a response
I20260709 13:03:05.732156  4601 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:05.682945  4522 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:05.750083  4577 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:05.751489  4552 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:05.703249  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.761183  1644 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:05.761653  1639 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.046s	sys 0.091s
I20260709 13:03:05.594937  4543 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:05.675326  4524 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:05.655941  4578 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.118s
W20260709 13:03:05.765431  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.792876  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:05.794272  1640 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:05.773708  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:05.768040  1538 leader_election.cc:336] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.771s (SENT)
I20260709 13:03:05.795939  1538 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:05.777051  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:05.777316  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 236) took 1024 ms. Trace:
I20260709 13:03:05.797469  1644 rpcz_store.cc:276] 0709 13:03:04.753169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.711807 (+958638us) service_pool.cc:224] Handling call
0709 13:03:05.777104 (+ 65297us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:05.778457  1639 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.962s	user 0.046s	sys 0.091s
I20260709 13:03:05.783224  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:05.800246  1645 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:05.785476  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1624) took 1310 ms. Trace:
I20260709 13:03:05.787853  1685 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:05.805320  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:05.807188  1641 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:05.514086  4312 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:05.795214  1647 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
W20260709 13:03:05.835978  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 48022.6us
I20260709 13:03:05.802491  1643 rpcz_store.cc:276] 0709 13:03:04.475729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.476020 (+   291us) service_pool.cc:224] Handling call
0709 13:03:04.560326 (+ 84306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.560588 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:05.386040 (+825452us) log.cc:844] Serialized 14 byte log entry
0709 13:03:05.785121 (+399081us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:05.785175 (+    54us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:05.785221 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.785288 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.785407 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8630,"prepare.queue_time_us":918398,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":7726,"raft.queue_time_us":235672,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":126,"thread_start_us":1186556,"threads_started":3}
W20260709 13:03:05.843437  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 66550.5us
W20260709 13:03:05.846804  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 242871us
I20260709 13:03:05.773670  4652 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:05.861929  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:05.862991  2347 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:03:05.865742  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:05.869170  1647 raft_consensus.cc:2410] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:03:05.887609  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 119602us
I20260709 13:03:05.896744  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:05.906958  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1622) took 1480 ms. Trace:
W20260709 13:03:05.909297  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 244752us
I20260709 13:03:05.911821  1642 rpcz_store.cc:276] 0709 13:03:04.426918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.427069 (+   151us) service_pool.cc:224] Handling call
0709 13:03:04.468290 (+ 41221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.468541 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:04.704343 (+235802us) log.cc:844] Serialized 14 byte log entry
0709 13:03:05.341862 (+637519us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b180034b1c0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d662a6 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9
0709 13:03:05.630598 (+288736us) log.cc:844] Serialized 14 byte log entry
0709 13:03:05.905518 (+274920us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:05.905586 (+    68us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:05.905668 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.906612 (+   944us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.906852 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":892399,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":67,"raft.queue_time_us":66116,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":143,"spinlock_wait_cycles":1424833408,"thread_start_us":561513,"threads_started":2}
I20260709 13:03:05.937745  2346 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:05.842893  4657 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:05.777720  4650 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:05.939981  2312 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:05.937115  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:05.940666  1686 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:03:05.950718  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 51554.7us
I20260709 13:03:05.825825  4543 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:05.957134  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:05.953676  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 115815us
I20260709 13:03:05.956787  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:05.963183  1647 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:05.965920  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:05.967298  1565 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:05.967561  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:05.895660  4646 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:05.985267  1641 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:05.987138  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1628) took 1308 ms. Trace:
I20260709 13:03:05.985327  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:05.991009  4552 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:05.959107  4602 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:05.992259  4602 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.992944  4602 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:05.993042  4226 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:05.992435  4522 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:06.003288  1685 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:05.959760  4623 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:06.015414  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.019076  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.022272  1642 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:05.983898  4646 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.026891  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.037586  1644 raft_consensus.cc:2410] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:03:06.036494  4229 consensus_queue.cc:1059] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:03:05.942679  4575 consensus_queue.cc:237] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:06.052713  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.055068  4543 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:06.056337  4671 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:06.041747  4341 consensus_queue.cc:1059] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:03:06.058290  4341 raft_consensus.cc:3055] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:06.058766  4341 raft_consensus.cc:740] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:06.060170  4341 consensus_queue.cc:260] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:06.060580  4420 consensus_meta.cc:287] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.000s
I20260709 13:03:06.061610  4341 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:06.063952  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.065323  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.066020  1646 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:06.066121  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.066990  1641 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:03:06.067289  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:06.067878  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.068989  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.071539  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.072820  1642 raft_consensus.cc:2410] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:06.044152  1611 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:06.074457  1611 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:06.075165  1641 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:03:06.015184  4623 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:06.101681  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:06.077569  1646 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
W20260709 13:03:06.082268  4659 log.cc:927] Time spent T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:03:06.082623  4552 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:03:06.120635  2314 leader_election.cc:336] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.642s (SENT)
I20260709 13:03:06.121939  2314 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:06.122444  1641 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8. Ops: [5.3-7.4]
I20260709 13:03:06.083277  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:06.099094  1647 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:06.123757  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:06.096905  4172 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:06.113981  4665 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:03:06.114665  4668 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:03:06.118149  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:06.129750  1571 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:03:06.130179  1643 raft_consensus.cc:2368] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8.
W20260709 13:03:06.119699  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:06.131512  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:06.132652  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:06.133000  4576 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.010s	sys 0.017s
W20260709 13:03:06.104701  4645 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.318s	user 0.002s	sys 0.002s
I20260709 13:03:06.061689  4664 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:06.092216  4623 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:06.136694  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:06.160159  1568 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
I20260709 13:03:06.128847  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.078516  4652 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:06.096926  4592 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:06.036942  4646 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:06.043955  4609 consensus_queue.cc:1059] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
W20260709 13:03:06.145260  1538 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.598s (SENT)
I20260709 13:03:06.138690  4172 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:06.153246  4616 consensus_queue.cc:1059] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:03:06.154253  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:06.155967  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.157963  1646 raft_consensus.cc:1217] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Deduplicated request from leader. Original: 4.2->[5.3-5.3]   Dedup: 5.3->[]
I20260709 13:03:06.204954  1646 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:03:06.205477  4665 raft_consensus.cc:697] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:06.158378  1647 raft_consensus.cc:1217] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Deduplicated request from leader. Original: 3.2->[4.3-4.3]   Dedup: 4.3->[]
I20260709 13:03:06.217303  1647 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:03:06.162163  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:06.219470  1640 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:06.169960  4650 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:06.140179  4576 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.011s	sys 0.018s
I20260709 13:03:06.163529  1644 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
W20260709 13:03:06.178215  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 175011us
I20260709 13:03:06.116200  4538 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:03:06.181725  1641 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:03:06.120414  4524 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:03:06.188902  1538 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:06.277314  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.140625  4420 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:06.143079  4666 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:06.206540  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:06.281801  1646 raft_consensus.cc:2368] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8.
I20260709 13:03:06.158185  1643 raft_consensus.cc:1217] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Deduplicated request from leader. Original: 3.2->[5.3-5.3]   Dedup: 5.3->[]
I20260709 13:03:06.224196  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:06.282627  1643 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:03:06.282794  1568 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
W20260709 13:03:06.208716  4643 log.cc:927] Time spent T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.351s	user 0.001s	sys 0.001s
W20260709 13:03:06.277935  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 132725us
W20260709 13:03:06.287626  1538 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.719s (SENT)
W20260709 13:03:06.291605  1538 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.809s (SENT)
I20260709 13:03:06.293478  1538 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:06.229202  4552 raft_consensus.cc:697] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:06.298614  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 72604.7us
I20260709 13:03:06.308166  1685 leader_election.cc:400] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10.
I20260709 13:03:06.177961  4664 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:06.310835  1685 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:06.312247  4522 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:06.189843  4592 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:06.345455  1564 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:03:06.351837  4558 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:06.358564  4602 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:06.360693  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:06.364490  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:06.400589  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 89866.2us
I20260709 13:03:06.367691  4172 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:06.299376  4616 consensus_queue.cc:1243] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer cf2f71592be8465fb18e36e15d0b2f89 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:06.370683  4552 consensus_queue.cc:237] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:06.279832  4613 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:06.415231  4613 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:06.284085  4650 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:03:06.358579  4680 log.cc:927] Time spent T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:03:06.382747  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.423962  1644 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:06.425030  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.425812  1644 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:06.382887  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.398785  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.382067  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.405776  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.406200  4341 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:06.408248  4602 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:06.412037  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:06.405660  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:06.379647  2345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55890 (request call id 297) took 2051 ms. Trace:
I20260709 13:03:06.446806  1643 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:06.428661  1640 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:06.450946  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.427680  1646 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
W20260709 13:03:06.425573  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 134048us
W20260709 13:03:06.454344  1538 leader_election.cc:336] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.822s (SENT)
I20260709 13:03:06.455152  1538 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:06.383955  4592 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:06.423097  4636 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:03:06.456341  1645 raft_consensus.cc:2368] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 6. Current term is 7.
I20260709 13:03:06.459014  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:06.446826  2345 rpcz_store.cc:276] 0709 13:03:04.328143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.328303 (+   160us) service_pool.cc:224] Handling call
0709 13:03:06.378245 (+2049942us) spinlock_profiling.cc:243] Waited 32 ms on lock 0x7b580003a298. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:03:06.378693 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.379190 (+   497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.379244 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:06.379290 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:06.379317 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.379378 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.379577 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73689984}
W20260709 13:03:06.461591  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:06.466390  1538 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.839s (SENT)
I20260709 13:03:06.291716  4420 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:06.498464  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:06.503297  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.420397  4578 consensus_queue.cc:1048] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:03:06.503734  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.508075  4647 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:06.509910  1647 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:03:06.510972  1646 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:06.479430  4650 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:06.519135  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.519672  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2130) took 2240 ms. Trace:
I20260709 13:03:06.520112  1715 rpcz_store.cc:276] 0709 13:03:04.279419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.291596 (+ 12177us) service_pool.cc:224] Handling call
0709 13:03:04.313163 (+ 21567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.409183 (+ 96020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.519256 (+2110073us) spinlock_profiling.cc:243] Waited 93 ms on lock 0x7b540002e9d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:06.519506 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":213999360}
I20260709 13:03:06.478197  4341 raft_consensus.cc:697] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 8 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:06.508394  4613 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:06.423604  4558 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:06.501963  4538 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.536202  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:06.536962  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 70663.5us
I20260709 13:03:06.537084  1645 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
W20260709 13:03:06.537823  1538 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.844s (SENT)
I20260709 13:03:06.538300  1538 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:06.539742  1538 leader_election.cc:336] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.883s (SENT)
I20260709 13:03:06.540278  1538 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:06.524093  4679 log.cc:927] Time spent T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.089s	user 0.000s	sys 0.002s
W20260709 13:03:06.546732  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 143456us
I20260709 13:03:06.584239  4341 consensus_queue.cc:237] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:06.542622  4226 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:06.436474  4665 consensus_queue.cc:237] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:06.628304  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:06.577505  4636 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:03:06.645831  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.649595  1647 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:06.653134  4226 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:06.669706  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1685) took 1105 ms. Trace:
I20260709 13:03:06.671463  1713 rpcz_store.cc:276] 0709 13:03:05.564742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.576807 (+ 12065us) service_pool.cc:224] Handling call
0709 13:03:05.577286 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:05.654669 (+ 77383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:05.654712 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:05.676612 (+ 21900us) log.cc:844] Serialized 26 byte log entry
0709 13:03:06.376898 (+700286us) spinlock_profiling.cc:243] Waited 67.3 ms on lock 0x7b18002a7de0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a250d1 00007f8248e40a0f
0709 13:03:06.505059 (+128161us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:06.505123 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.505231 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:06.669285 (+164054us) raft_consensus.cc:1685] finished
0709 13:03:06.669327 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.669606 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75630,"raft.queue_time_us":78965,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":22691,"spinlock_wait_cycles":154857728,"thread_start_us":127892,"threads_started":1,"wal-append.queue_time_us":973391}
I20260709 13:03:06.682716  1645 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:06.706868  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.708565  1647 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:06.713833  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.720710  1644 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:06.721431  4609 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.722406  4609 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:06.727808  1715 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:03:06.730950  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.732110  1644 raft_consensus.cc:2410] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:03:06.732360  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:06.733109  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.735595  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:06.737679  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:06.739733  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:06.594478  4538 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:06.742268  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:06.743000  1721 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
W20260709 13:03:06.739205  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 199504us
I20260709 13:03:06.744040  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.744952  1645 raft_consensus.cc:2410] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:03:06.750654  1537 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:06.751667  4543 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:06.751330  4226 consensus_queue.cc:1059] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:06.752189  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 398) took 1111 ms. Trace:
I20260709 13:03:06.753834  4543 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:06.754204  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.754894  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:06.755599  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.754976  1640 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:06.757197  1644 raft_consensus.cc:2410] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:03:06.754237  4543 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.758312  4543 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:06.753378  1566 rpcz_store.cc:276] 0709 13:03:05.641304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.641456 (+   152us) service_pool.cc:224] Handling call
0709 13:03:06.751484 (+1110028us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b5800011198. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:03:06.751699 (+   215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.751876 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.751902 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:06.751953 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:06.751980 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.752032 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.752135 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2511616}
W20260709 13:03:06.761127  1538 leader_election.cc:336] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.911s (SENT)
I20260709 13:03:06.762387  1538 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:06.700482  4676 raft_consensus.cc:3055] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:06.677742  4420 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:06.766534  1538 leader_election.cc:336] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.917s (SENT)
W20260709 13:03:06.764806  2313 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.000s	sys 0.055s
W20260709 13:03:06.762979  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 57767.9us
W20260709 13:03:06.757757  4687 log.cc:927] Time spent T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.105s	user 0.001s	sys 0.001s
W20260709 13:03:06.772972  2313 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.000s	sys 0.056s
I20260709 13:03:06.773084  1647 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:06.782312  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:06.783082  1564 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:03:06.784116  1646 raft_consensus.cc:2410] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:03:06.773280  1538 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:06.776716  1566 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:06.763362  4664 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:06.764273  4676 raft_consensus.cc:740] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:06.802023  4420 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:06.801508  4676 consensus_queue.cc:260] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:06.804169  4676 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:06.805603  4229 raft_consensus.cc:3055] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:03:06.809365  1456 connection.cc:573] client connection to 127.0.107.195:34553 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:03:06.810492  1456 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 43 similar messages]
W20260709 13:03:06.815761  4646 consensus_meta.cc:287] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.612s	user 0.014s	sys 0.000s
I20260709 13:03:06.816371  4646 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:06.817842  1572 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:06.819376  4690 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.820053  4690 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:06.786624  1538 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.932s (SENT)
I20260709 13:03:06.795275  4558 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:06.776409  4691 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:03:06.797452  1573 raft_consensus.cc:1275] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:03:06.821472  1538 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:06.806699  4229 raft_consensus.cc:740] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:06.775029  4657 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:06.836570  1568 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:06.838815  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:06.844498  1646 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:06.857347  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 70754.8us
W20260709 13:03:06.861353  1456 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 50932.9us
I20260709 13:03:06.824334  4611 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:06.842953  4623 consensus_meta.cc:287] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.664s	user 0.017s	sys 0.000s
I20260709 13:03:06.882526  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.824339  4341 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:06.885179  1646 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:06.843832  4226 consensus_queue.cc:1243] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:03:06.909869  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:06.913384  1564 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:06.913422  1646 raft_consensus.cc:2410] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:06.889184  4657 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:06.974833  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.951968  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:06.955991  4629 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 2.299s	user 0.003s	sys 0.020s
I20260709 13:03:06.960276  4693 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:06.963356  4538 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:06.895654  4623 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:06.896494  4341 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:06.970237  4690 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:06.892207  4611 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:07.034159  1643 consensus_meta.cc:287] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.586s	user 0.013s	sys 0.002s
I20260709 13:03:07.034991  1643 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
W20260709 13:03:07.023370  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 182622us
I20260709 13:03:07.068300  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:07.069330  1643 raft_consensus.cc:2376] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6. Re-sending same reply.
W20260709 13:03:07.071113  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.072165  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 213316us
W20260709 13:03:07.074059  1538 leader_election.cc:336] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.009s (SENT)
I20260709 13:03:07.074777  1538 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:07.076748  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 105477us
I20260709 13:03:07.078485  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.079589  1646 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:03:07.087778  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:07.076097  1564 raft_consensus.cc:2955] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:03:07.097524  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 53660us
W20260709 13:03:07.105613  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.122529  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:07.123734  1643 raft_consensus.cc:2368] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8.
I20260709 13:03:07.125772  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:07.126793  1646 raft_consensus.cc:2376] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9. Re-sending same reply.
I20260709 13:03:07.128595  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:07.130820  4698 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:07.131014  1643 raft_consensus.cc:2376] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6. Re-sending same reply.
I20260709 13:03:07.131526  4698 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:07.135643  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1629) took 2403 ms. Trace:
I20260709 13:03:07.136147  1648 rpcz_store.cc:276] 0709 13:03:04.732204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.134669 (+2402465us) service_pool.cc:224] Handling call
0709 13:03:07.135064 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.135261 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:07.135288 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:07.135345 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:07.135381 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.135452 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.135577 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:06.930145  4229 consensus_queue.cc:260] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:07.138702  1685 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:03:07.139117  4696 log.cc:927] Time spent T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.118s	user 0.002s	sys 0.000s
I20260709 13:03:07.140020  4691 consensus_queue.cc:1059] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:07.141356  4691 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:07.141856  4691 raft_consensus.cc:697] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:07.142683  4623 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:07.110944  4699 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:03:07.157390  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:07.168979  4672 raft_consensus.cc:3055] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:07.172013  4646 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:07.167307  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.175447  1570 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:03:07.144985  4657 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:07.221810  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 147820us
I20260709 13:03:07.110667  4690 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.082s
W20260709 13:03:07.224925  1538 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.063s (SENT)
I20260709 13:03:07.173555  4226 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:07.234532  1538 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:07.234920  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.177336  4672 raft_consensus.cc:740] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:07.247594  1646 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:07.232997  4657 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:07.256579  4341 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:07.258116  4657 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:07.259495  4611 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:07.142760  4691 consensus_queue.cc:237] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:07.264056  4312 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:07.272635  2314 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.596s (SENT)
W20260709 13:03:07.274782  4543 consensus_meta.cc:287] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.005s
I20260709 13:03:07.283411  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:07.285974  1646 raft_consensus.cc:2410] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:03:07.240468  1642 raft_consensus.cc:2955] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 2 to 5, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:03:07.291519  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 66563.8us
W20260709 13:03:07.292949  1538 leader_election.cc:336] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.147s (SENT)
I20260709 13:03:07.299366  1565 raft_consensus.cc:3055] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:07.299889  1565 raft_consensus.cc:740] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:03:07.301502  2314 leader_election.cc:336] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.843s (SENT)
I20260709 13:03:07.302083  2314 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:07.314363  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.336024  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:07.339267  1538 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:07.276801  4543 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:07.300761  1565 consensus_queue.cc:260] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:07.345485  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 206906us
I20260709 13:03:07.355695  1565 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:07.337013  4701 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:03:07.358165  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.359150  1646 raft_consensus.cc:2410] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:03:07.359555  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 215462us
W20260709 13:03:07.382222  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 46572.7us
I20260709 13:03:07.408299  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.435089  1646 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
W20260709 13:03:07.405385  4686 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:03:07.435597  4341 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:07.416716  4703 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:07.340194  4172 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:07.435863  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:07.394451  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2095) took 1329 ms. Trace:
W20260709 13:03:07.442466  2314 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 3.292s (SENT)
I20260709 13:03:07.443148  2314 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:07.390623  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:07.409468  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 116589us
W20260709 13:03:07.449592  1538 leader_election.cc:336] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.175s (SENT)
I20260709 13:03:07.451452  1538 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:07.355875  4538 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:03:07.454119  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.459957  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.461124  1643 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
W20260709 13:03:07.456032  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.423401  4675 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:03:07.441462  1642 rpcz_store.cc:276] 0709 13:03:06.064986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.080896 (+ 15910us) service_pool.cc:224] Handling call
0709 13:03:06.081825 (+   929us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:03:06.127269 (+ 45444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.127304 (+    35us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:03:06.144920 (+ 17616us) log.cc:844] Serialized 642 byte log entry
0709 13:03:06.381532 (+236612us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:06.425413 (+ 43881us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.239828 (+814415us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b1800456280. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d12c0a 00007f824f452219 00007f8245d9aa40 00007f8245d954e4 00007f8245d95455 00007f824c87d3c7 00007f8248e8f9a7 00007f8248e9f0bf 00007f8248e9f038
0709 13:03:07.240094 (+   266us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.358267 (+118173us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.359829 (+  1562us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.393837 (+ 34008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.393992 (+   155us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:07.394061 (+    69us) raft_consensus.cc:1685] finished
0709 13:03:07.394079 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.394351 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":834821,"prepare.queue_time_us":915554,"prepare.run_cpu_time_us":208,"prepare.run_wall_time_us":8976,"raft.queue_time_us":65438,"raft.run_cpu_time_us":663,"raft.run_wall_time_us":110578,"spinlock_wait_cycles":148627072,"thread_start_us":173665,"threads_started":1,"wal-append.queue_time_us":350926}
I20260709 13:03:07.503443  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.503773  4609 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:07.462949  4172 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:07.471148  4664 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:07.471617  1538 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.212s (SENT)
I20260709 13:03:07.471911  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:07.472203  4703 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:07.439630  4671 consensus_queue.cc:1048] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:03:07.474466  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.458523  4538 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:07.478556  1569 raft_consensus.cc:2955] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 2 to 6, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:07.480692  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.424292  4720 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:07.419157  4672 consensus_queue.cc:260] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:07.499258  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:05.988229  1639 rpcz_store.cc:276] 0709 13:03:04.679133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.736020 (+ 56887us) service_pool.cc:224] Handling call
0709 13:03:04.736751 (+   731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.737030 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:05.798859 (+1061829us) log.cc:844] Serialized 14 byte log entry
0709 13:03:05.986483 (+187624us) log.cc:844] Serialized 14 byte log entry
0709 13:03:05.986873 (+   390us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:05.986910 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:05.986942 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.986996 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.987091 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9503,"prepare.queue_time_us":864799,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":34,"raft.queue_time_us":13943,"raft.run_cpu_time_us":992,"raft.run_wall_time_us":992,"spinlock_wait_cycles":12280576,"thread_start_us":1160478,"threads_started":2}
I20260709 13:03:07.513234  1643 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:07.514858  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.535905  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.536662  1642 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:07.537729  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.538760  1645 raft_consensus.cc:2410] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:03:07.539587  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.540052  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.540663  1643 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 2.
I20260709 13:03:07.518062  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.536494  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:07.523130  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.530403  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 58825.2us
I20260709 13:03:07.537915  2347 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:07.531802  1719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1676) took 3064 ms (client timeout 3123 ms). Trace:
I20260709 13:03:07.537765  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.515169  1612 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:07.540630  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.516009  4524 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:07.541818  1640 raft_consensus.cc:2376] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9. Re-sending same reply.
W20260709 13:03:07.543062  1538 leader_election.cc:336] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.297s (SENT)
I20260709 13:03:07.542006  1639 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:07.550077  1538 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:07.550350  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.543519  1646 raft_consensus.cc:2410] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:03:07.543437  1719 rpcz_store.cc:269] 0709 13:03:04.467687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.467951 (+   264us) service_pool.cc:224] Handling call
0709 13:03:07.531571 (+3063620us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54002990d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:07.531718 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2647860608}
I20260709 13:03:07.549629  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:07.536127  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.553566  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 47515.2us
I20260709 13:03:07.543666  1642 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:07.557785  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.558723  1642 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:07.559875  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.561290  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.545434  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.562789  1645 raft_consensus.cc:2410] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:03:07.564227  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.565276  1645 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:07.552312  1643 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:07.543346  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2106) took 1263 ms. Trace:
I20260709 13:03:07.382543  4698 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:07.555042  1611 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:07.544586  1612 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:07.567252  1611 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:07.566468  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.568740  1642 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:07.568630  1643 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:03:07.566102  1641 rpcz_store.cc:276] 0709 13:03:06.280027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.281672 (+  1645us) service_pool.cc:224] Handling call
0709 13:03:06.301719 (+ 20047us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:06.302262 (+   543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.302310 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:06.303935 (+  1625us) log.cc:844] Serialized 26 byte log entry
0709 13:03:06.726635 (+422700us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:07.542572 (+815937us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.543057 (+   485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.543127 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:07.543170 (+    43us) raft_consensus.cc:1685] finished
0709 13:03:07.543181 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.543296 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":603512,"prepare.queue_time_us":607806,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":14706,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":24596,"thread_start_us":437107,"threads_started":2,"wal-append.queue_time_us":299906}
I20260709 13:03:07.569698  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.571314  1640 raft_consensus.cc:2410] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:03:07.570580  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.572036  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.572337  1641 raft_consensus.cc:2410] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:07.572762  1639 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
W20260709 13:03:07.534612  4686 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:03:07.571873  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:07.576943  2345 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:07.577488  1565 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:07.574612  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.579178  1639 raft_consensus.cc:2410] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:07.536588  4538 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:07.541160  3879 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:07.589031  1640 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:03:07.588699  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:07.603416  1719 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:07.590782  1538 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.593s (SENT)
I20260709 13:03:07.592674  1643 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:03:07.593268  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.572789  4682 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:07.609941  1538 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:07.605726  1641 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:03:07.611106  1648 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8. Ops: [7.3-7.3]
W20260709 13:03:07.609208  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 64657.7us
I20260709 13:03:07.567585  4665 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:07.613265  4665 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:07.613837  4665 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:07.615607  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:07.612808  1612 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:07.616518  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:07.616513  2348 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 3.
I20260709 13:03:07.619096  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:07.591578  4664 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:07.620127  1715 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 12: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:07.620865  4664 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:07.586961  4703 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:07.609444  4737 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:07.623435  4737 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:07.623883  4737 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:07.625965  4698 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:07.625849  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:07.549872  4676 consensus_queue.cc:1059] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
W20260709 13:03:07.633229  1538 connection.cc:256] RPC call timeout handler was delayed by -1.4467816559999847s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2979s for a response
W20260709 13:03:07.634867  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.638300  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.638293  1538 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.015s (SENT)
I20260709 13:03:07.638916  1538 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:07.640048  1538 connection.cc:256] RPC call timeout handler was delayed by -1.2586816560000216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31880000000000003s for a response
W20260709 13:03:07.640836  1538 leader_election.cc:336] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.667s (SENT)
I20260709 13:03:07.641332  1538 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:07.596771  4524 raft_consensus.cc:697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:07.646323  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:07.647002  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:07.649165  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:07.651861  4665 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:07.656224  1612 leader_election.cc:400] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 7. Current term is 8.
I20260709 13:03:07.656752  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:07.657397  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 90238.3us
I20260709 13:03:07.657047  1612 leader_election.cc:403] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:07.659582  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:07.642549  4698 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:07.664364  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.664491  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:07.647133  4703 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:07.689416  4737 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:07.662814  4684 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 7. Current term is 8.
I20260709 13:03:07.707311  4684 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:07.708350  4684 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:07.666071  4664 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:03:07.663134  4543 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:07.711768  4524 consensus_queue.cc:237] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:07.676571  4652 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
W20260709 13:03:07.705390  1722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 240) took 1996 ms (client timeout 1865 ms). Trace:
W20260709 13:03:07.707526  4521 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:03:07.719578  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:07.758469  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:07.790390  1648 raft_consensus.cc:1275] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:03:07.791507  2347 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:07.721932  1643 raft_consensus.cc:1240] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:07.794175  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:07.797422  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:07.797863  1538 leader_election.cc:336] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.894s (SENT)
I20260709 13:03:07.798327  1642 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:07.801051  4665 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:07.797081  1722 rpcz_store.cc:269] 0709 13:03:05.708637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.708949 (+   312us) service_pool.cc:224] Handling call
0709 13:03:07.705084 (+1996135us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b5400213254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:07.705278 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1534365568}
I20260709 13:03:07.804698  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.805441  1641 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
W20260709 13:03:07.810312  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.811380  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.709445  4684 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:07.815476  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:07.815843  2313 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:07.816425  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.814836  1686 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:07.824302  1639 raft_consensus.cc:1240] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:07.826566  1644 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:07.828943  2313 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:07.830911  2313 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:07.824474  4226 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.832001  4226 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:07.835662  4226 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:07.838541  4172 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:07.845024  4738 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:07.845837  4738 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:07.846378  4529 consensus_queue.cc:1059] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:07.847394  4704 raft_consensus.cc:3055] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:07.847793  4704 raft_consensus.cc:740] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:07.850195  4571 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:07.850708  4571 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.851068  4571 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:07.849262  4738 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:07.848642  4704 consensus_queue.cc:260] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:07.855647  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:07.857122  1722 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:07.828092  2348 raft_consensus.cc:2955] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:07.859489  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:07.831707  4644 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:07.863157  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:07.865051  4226 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:07.866005  4226 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:07.867336  4172 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:07.873867  1722 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:07.854954  4738 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:07.884344  1714 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:03:07.884511  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:07.882604  1538 leader_election.cc:336] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.642s (SENT)
I20260709 13:03:07.886246  1538 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:07.891505  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:07.891903  1719 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 8 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:07.892048  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:07.895529  1716 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 13:03:07.897611  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.899106  1538 leader_election.cc:336] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.019s (SENT)
I20260709 13:03:07.899649  1538 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:07.901522  1685 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:07.858311  4704 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:07.912029  2314 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.769s (SENT)
I20260709 13:03:07.915448  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:07.916333  1713 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:03:07.860877  4522 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.860875  4521 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.857152  4665 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:07.929176  4616 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:07.882342  2346 raft_consensus.cc:2955] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 2 to 6, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:07.961076  4720 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:03:07.961990  4521 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:07.962805  3879 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:07.964390  4538 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:07.869372  4664 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:07.931154  4226 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:07.966017  4664 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:07.966928  4538 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:07.967937  4538 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:07.968498  4538 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:07.874292  4312 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:03:07.945143  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.970973  4522 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:07.972640  3879 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.951866  1538 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:07.868712  4682 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:07.973285  3879 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:07.973812  4682 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:07.975910  3879 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:07.960379  4616 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.977149  4616 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:07.938737  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:07.986114  1717 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:07.989509  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:07.991402  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:07.884088  4740 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:07.993829  4740 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:07.996542  4740 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:07.991681  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:07.917307  1722 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:03:07.964821  4521 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:08.004469  1538 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:08.012585  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2131) took 1258 ms. Trace:
I20260709 13:03:08.013924  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:08.014006  1538 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:08.013147  1640 rpcz_store.cc:276] 0709 13:03:06.754305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.576961 (+822656us) service_pool.cc:224] Handling call
0709 13:03:07.588600 (+ 11639us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.603019 (+ 14419us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.012333 (+409314us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b5400183150. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:08.012512 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25934592}
I20260709 13:03:08.020521  1715 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:07.998402  1713 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:08.021044  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:08.022473  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:08.023756  1722 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 12: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:07.921109  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:07.921564  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:07.919065  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.917160  4543 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:03:08.036741  1612 connection.cc:256] RPC call timeout handler was delayed by -3.1629739329999893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15080000000000002s for a response
W20260709 13:03:08.043774  1612 connection.cc:256] RPC call timeout handler was delayed by -3.158473932999982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1513s for a response
W20260709 13:03:08.044234  1612 connection.cc:256] RPC call timeout handler was delayed by -3.091873932999988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1587s for a response
W20260709 13:03:08.044513  1612 connection.cc:256] RPC call timeout handler was delayed by -2.9613739329999476s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17320000000000002s for a response
W20260709 13:03:08.044816  1612 connection.cc:256] RPC call timeout handler was delayed by -2.9136739329999841s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1785s for a response
W20260709 13:03:08.045094  1612 connection.cc:256] RPC call timeout handler was delayed by -2.8965739329999565s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1804s for a response
W20260709 13:03:08.045750  1612 connection.cc:256] RPC call timeout handler was delayed by -2.8686739329999682s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1835s for a response
I20260709 13:03:08.047168  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:08.046315  1612 connection.cc:256] RPC call timeout handler was delayed by -2.852473932999942s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18530000000000002s for a response
W20260709 13:03:08.047996  1612 connection.cc:256] RPC call timeout handler was delayed by -2.84707393299999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1859s for a response
W20260709 13:03:08.048424  1612 connection.cc:256] RPC call timeout handler was delayed by -2.7930739329999597s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19190000000000002s for a response
W20260709 13:03:08.048792  1612 connection.cc:256] RPC call timeout handler was delayed by -2.61127393299995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2121s for a response
W20260709 13:03:08.049216  1612 connection.cc:256] RPC call timeout handler was delayed by -2.5986739329999864s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2135s for a response
W20260709 13:03:08.050097  1612 connection.cc:256] RPC call timeout handler was delayed by -2.5275739329999851s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2214s for a response
I20260709 13:03:07.942410  4576 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:08.050532  1612 connection.cc:256] RPC call timeout handler was delayed by -1.837273932999949s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2981s for a response
W20260709 13:03:08.052016  1612 connection.cc:256] RPC call timeout handler was delayed by -1.7094739329999697s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3123s for a response
W20260709 13:03:08.052500  1612 connection.cc:256] RPC call timeout handler was delayed by -1.5636739329999614s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3285s for a response
W20260709 13:03:08.053015  1612 connection.cc:256] RPC call timeout handler was delayed by -1.52767393299996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3325s for a response
W20260709 13:03:08.053452  1612 connection.cc:256] RPC call timeout handler was delayed by -1.1964739329999929s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3693s for a response
I20260709 13:03:08.039706  1538 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:08.040908  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:08.042243  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:08.060194  1565 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:08.062647  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:08.013116  4577 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.046284  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:08.110541  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:08.025875  4578 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.025323  4698 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:07.905257  4738 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:07.915495  4703 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:08.065884  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:08.068964  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.016418  4720 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:08.119234  4682 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:08.041988  4740 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:08.153618  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:08.064244  4533 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:08.156651  4533 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:08.157171  4533 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:08.113229  4577 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:08.104681  4576 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:08.136360  1717 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:08.140748  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2126) took 1410 ms. Trace:
I20260709 13:03:08.162247  4647 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:08.163100  4647 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:08.133834  4571 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:08.168398  4571 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:08.162592  1647 rpcz_store.cc:276] 0709 13:03:06.730474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.732826 (+  2352us) service_pool.cc:224] Handling call
0709 13:03:08.140510 (+1407684us) spinlock_profiling.cc:243] Waited 39.2 ms on lock 0x7b54002aeed4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:08.140653 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90184960}
I20260709 13:03:08.062629  4664 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:08.135120  4522 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:08.042303  4665 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:08.184170  4665 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:08.126935  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 71999.6us
I20260709 13:03:08.191059  4229 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:08.191759  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.192507  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:08.191517  1612 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:08.204455  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:08.134845  4672 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:08.129612  4707 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:08.210947  2313 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:07.730588   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:08.227133  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:08.137524  4737 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:08.209249  1537 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:08.138778  4738 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:08.233986  1572 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:08.237154  1639 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:08.243058  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:08.243744  2345 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:08.244601  2314 leader_election.cc:336] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.071s (SENT)
I20260709 13:03:08.245110  2314 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:08.245694  1687 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:08.246253  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:08.192344  4522 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:08.248247  1537 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:08.151121  4720 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:08.249776  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:08.129808  4578 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:08.256479  1642 raft_consensus.cc:1275] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:03:08.256356  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:08.250392  1565 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:08.265071  1565 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
W20260709 13:03:08.269804  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 78425.4us
I20260709 13:03:08.231786  4707 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:08.240363  4312 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:08.279807  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:08.256227  4676 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:03:08.282680  1648 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:08.329303  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:08.305828  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:08.290400  4576 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:08.298246  2345 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:08.298698  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:08.303471  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:08.195437  4740 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:08.249799  4693 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:08.304850  1612 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:08.332468  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 53715.8us
I20260709 13:03:08.267652  4737 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:08.284766  4577 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:08.349004  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:08.252771  4703 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:08.260682  4720 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:08.256161  4692 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:08.364141  1686 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:08.304685  4226 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:08.307940  1470 catalog_manager.cc:5697] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:08.337256  1565 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:08.397267  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:08.395593  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 90866.8us
I20260709 13:03:08.398100  1714 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:08.383383  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:08.329519  4707 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:08.364650  4639 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.400833  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:08.401532  1570 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:08.404806  1611 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:08.380982  4538 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:08.372066  4705 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:08.390797  4692 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.417055  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:08.417920  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:08.385147  4665 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:08.422443  1612 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:08.422309  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 382796us
I20260709 13:03:08.426903  1612 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:08.427760  1538 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:08.430202  1537 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:08.425447  4705 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:08.416703  4724 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:08.407620  4740 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:08.437827  1612 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:08.438342  4682 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:08.439067  1612 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:08.418216  4226 raft_consensus.cc:697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:08.440261  4740 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:08.441373  4740 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:08.426204  4538 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:08.443426  4705 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:08.441421  4226 consensus_queue.cc:237] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:08.414832  4524 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:08.439013  4682 raft_consensus.cc:697] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:08.475945  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 48082.3us
I20260709 13:03:08.459223  1611 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:08.425729  4692 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:08.407708  4738 consensus_queue.cc:1059] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 13:03:08.505200  1685 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:08.512863  1538 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:08.530347  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.477556  4650 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:08.467672  4577 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:08.408650  4639 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:08.472601  4724 raft_consensus.cc:697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:03:08.555797  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 50715.3us
I20260709 13:03:08.480095  4543 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:08.504925  4664 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:03:08.572072  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 171331us
I20260709 13:03:08.505342  4524 raft_consensus.cc:697] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:08.606861  1685 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:08.607488  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 94760us
I20260709 13:03:08.609283  1538 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:08.560654  4665 raft_consensus.cc:3055] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:08.556001  4650 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:08.563486  4708 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:08.561699  4577 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:08.549618  4705 consensus_queue.cc:1059] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:03:08.614528  4665 raft_consensus.cc:740] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:08.593972  4664 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:08.575795  4543 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:08.620851  4650 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:08.624166  1538 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:08.616680  4743 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.137s	user 0.000s	sys 0.002s
I20260709 13:03:08.631172  1685 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:08.633095  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 269118us
W20260709 13:03:08.635017  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 196033us
I20260709 13:03:08.644207  1612 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:08.646044  1612 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:08.622180  4713 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:08.630882  4704 consensus_meta.cc:287] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.685s	user 0.004s	sys 0.008s
I20260709 13:03:08.629669  4703 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:08.630067  4725 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:08.632773  4664 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:08.637606  4543 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:08.626920  4665 consensus_queue.cc:260] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:08.684882  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 160877us
I20260709 13:03:08.688684  4749 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:08.658202  4738 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:08.679425  4581 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:08.680631  4676 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.040s
W20260709 13:03:08.719411  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 73487.1us
W20260709 13:03:08.726436  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 106921us
I20260709 13:03:08.728063  1612 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:08.731055  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 44883.2us
W20260709 13:03:08.709697  4741 log.cc:927] Time spent T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.294s	user 0.000s	sys 0.003s
I20260709 13:03:08.699159  4749 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:08.701457  4737 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:08.676402  4748 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 13:03:08.750025  1612 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:08.660609  4724 consensus_queue.cc:237] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:08.749820  4734 log.cc:927] Time spent T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:03:08.726115  4665 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:08.735601  4738 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:08.765108  4657 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:08.690161  4524 consensus_queue.cc:237] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:08.687242  4682 consensus_queue.cc:237] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:08.768108  4749 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:08.895489  4657 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:08.940203  4657 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:08.843091  4639 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:08.901122  4533 consensus_meta.cc:287] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.737s	user 0.008s	sys 0.004s
I20260709 13:03:08.941387  4533 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:08.943964  4533 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:08.944258  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 250254us
I20260709 13:03:08.779367  4574 consensus_queue.cc:1048] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.084s
W20260709 13:03:08.903266  4616 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.925s	user 0.014s	sys 0.006s
W20260709 13:03:08.904567  4647 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.741s	user 0.013s	sys 0.001s
W20260709 13:03:08.911167  4312 consensus_meta.cc:287] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.562s	user 0.012s	sys 0.002s
I20260709 13:03:08.924075  4578 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:08.925248  1470 catalog_manager.cc:5697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 5 to 7, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 7 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:03:08.930862  4747 log.cc:927] Time spent T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.303s	user 0.004s	sys 0.000s
I20260709 13:03:08.739425  4678 consensus_queue.cc:1059] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.105s
I20260709 13:03:08.945402  1570 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:08.945379  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:08.946952  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:08.843014  4577 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:08.961407  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 228399us
I20260709 13:03:08.964630  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1648) took 1143 ms. Trace:
I20260709 13:03:08.973039  1568 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:08.837162  4742 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:08.968021  4312 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:08.947533  4746 log.cc:927] Time spent T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.287s	user 0.003s	sys 0.012s
W20260709 13:03:08.966435  4684 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.134s	user 0.011s	sys 0.005s
I20260709 13:03:08.990059  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:09.006278  1718 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
W20260709 13:03:08.974651  4754 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.263s	user 0.000s	sys 0.002s
W20260709 13:03:09.004669  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 305275us
I20260709 13:03:09.012107  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:08.977530  1645 rpcz_store.cc:276] 0709 13:03:07.821229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.821383 (+   154us) service_pool.cc:224] Handling call
0709 13:03:07.821844 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:07.822251 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:07.822281 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:07.822792 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 13:03:08.014053 (+191261us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:08.014093 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.014399 (+   306us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:08.964257 (+949858us) raft_consensus.cc:1685] finished
0709 13:03:08.964304 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:08.964535 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1313,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":6780,"thread_start_us":176915,"threads_started":1,"wal-append.queue_time_us":544600}
I20260709 13:03:08.965226  4647 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:08.998224  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:09.019445  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:09.022285  1714 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:09.001935  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:08.957118  4738 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:03:09.027000  4647 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:09.021468  1568 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0df07602233941598bee7b19607b2e3b"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:03:09.032096  1568 raft_consensus.cc:606] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:09.035989  1568 raft_consensus.cc:614] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:09.040657  1568 raft_consensus.cc:1275] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:03:09.058980  4692 consensus_meta.cc:287] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.543s	user 0.011s	sys 0.001s
I20260709 13:03:09.059649  4692 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:09.063519  4664 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:09.065078  1568 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 13:03:09.066155  4672 consensus_meta.cc:287] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.834s	user 0.012s	sys 0.003s
W20260709 13:03:09.077067  4752 log.cc:927] Time spent T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:03:08.965368  4616 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:09.081571  4763 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
W20260709 13:03:09.250665  1686 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.025s	sys 0.025s
W20260709 13:03:09.253908  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 200288us
W20260709 13:03:09.328890  1686 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.026s	sys 0.025s
W20260709 13:03:09.232591  4707 consensus_meta.cc:287] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.816s	user 0.011s	sys 0.005s
W20260709 13:03:09.231025  4747 log.cc:927] Time spent T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.220s	user 0.000s	sys 0.000s
I20260709 13:03:09.011876  4684 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:09.378513  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 49171.5us
W20260709 13:03:09.332394  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 638590us
W20260709 13:03:09.387524  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 373358us
I20260709 13:03:09.393044  1641 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:09.324898  4690 consensus_queue.cc:1059] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
W20260709 13:03:09.382028  4752 log.cc:927] Time spent T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:03:09.375592  4522 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:09.407269  1646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.000s	sys 0.015s
W20260709 13:03:09.420465  1646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.000s	sys 0.016s
W20260709 13:03:09.426486  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 46660us
W20260709 13:03:09.428694  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 401001us
W20260709 13:03:09.431896  1612 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.026s	sys 0.054s
W20260709 13:03:09.432257  1612 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.026s	sys 0.054s
W20260709 13:03:09.432753  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 645139us
I20260709 13:03:09.434060  1612 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:09.395211  4577 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.447815  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:09.372316  4707 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:09.417057  4664 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:09.440622  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:09.441149  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 57901.4us
I20260709 13:03:09.448804  1644 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:03:09.491124  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:09.492923  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:09.438643  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:09.489523  1642 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:09.466169  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:09.496483  1647 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:09.496483  1648 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:09.504380  4765 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:09.505466  4765 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:09.515865  1720 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:09.516819  4571 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.517530  4524 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:03:09.516709  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 82870.1us
I20260709 13:03:09.482668  4692 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:09.520395  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:09.522014  4664 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:09.486341  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1661) took 1266 ms. Trace:
I20260709 13:03:09.524200  1641 rpcz_store.cc:276] 0709 13:03:08.219469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.219607 (+   138us) service_pool.cc:224] Handling call
0709 13:03:08.243497 (+ 23890us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.393858 (+1150361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.485765 (+ 91907us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54001f3954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:09.486227 (+   462us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2643168128}
I20260709 13:03:09.493549  4684 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:09.525357  4524 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:09.518440  4571 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:09.528810  4665 consensus_meta.cc:287] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.686s	user 0.013s	sys 0.001s
I20260709 13:03:09.487450  4616 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:09.470556  4758 raft_consensus.cc:3055] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:09.422258  4533 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.530076  1612 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:09.523689  4664 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:09.461275  4769 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.228s	user 0.000s	sys 0.000s
I20260709 13:03:09.431483  4703 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.558135  1640 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:03:09.558271  1612 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:09.556100  4682 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:09.576203  1640 pending_rounds.cc:85] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 3
I20260709 13:03:09.481365  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1676) took 1134 ms. Trace:
I20260709 13:03:09.510989  4765 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:09.600163  1640 pending_rounds.cc:107] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:03:09.578948  1612 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:09.602784  1570 rpcz_store.cc:276] 0709 13:03:08.346694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.412247 (+ 65553us) service_pool.cc:224] Handling call
0709 13:03:08.417572 (+  5325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.972446 (+554874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.480898 (+508452us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b5400014d54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:09.481245 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1213532544}
I20260709 13:03:09.606231  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:09.553292  4758 raft_consensus.cc:740] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:09.608834  4684 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:03:09.609011  4650 consensus_meta.cc:287] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.952s	user 0.011s	sys 0.004s
I20260709 13:03:09.585150  4774 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:09.613971  4757 raft_consensus.cc:3055] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:09.617241  4742 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:09.626268  4650 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:09.627923  1573 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:09.628216  4684 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:09.628338  4758 consensus_queue.cc:260] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:09.524008  4229 consensus_meta.cc:287] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.332s	user 0.017s	sys 0.000s
W20260709 13:03:09.577888  4543 consensus_meta.cc:287] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.891s	user 0.016s	sys 0.000s
I20260709 13:03:09.602001  4664 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:09.622216  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:09.624847  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:09.632759  4650 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:09.630415  4742 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:09.632462  4757 raft_consensus.cc:740] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:09.604161  1640 raft_consensus.cc:2889] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:09.633692  2314 leader_election.cc:336] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.021s (SENT)
I20260709 13:03:09.635162  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:09.650568  2314 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:09.430518  4522 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:09.619591  1570 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:09.644698  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 317759us
I20260709 13:03:09.626735  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:09.647179  1721 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:03:09.658375  1721 pending_rounds.cc:85] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 8
I20260709 13:03:09.658980  1721 pending_rounds.cc:107] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:03:09.659336  1721 raft_consensus.cc:2889] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:09.644817  1644 raft_consensus.cc:2376] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8. Re-sending same reply.
I20260709 13:03:09.663014  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:09.621348  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1681) took 1204 ms. Trace:
I20260709 13:03:09.669577  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:09.457341  4577 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:09.671087  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:09.673957  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 412) took 1592 ms. Trace:
I20260709 13:03:09.585193  4703 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:09.572892  4533 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:09.629261  4684 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:09.683496  1571 rpcz_store.cc:276] 0709 13:03:08.081537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.082933 (+  1396us) service_pool.cc:224] Handling call
0709 13:03:08.083488 (+   555us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:08.517832 (+434344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.517858 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:08.524705 (+  6847us) log.cc:844] Serialized 26 byte log entry
0709 13:03:08.764867 (+240162us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:08.764929 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.765271 (+   342us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:09.673228 (+907957us) raft_consensus.cc:1685] finished
0709 13:03:09.673261 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.673827 (+   566us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3214,"raft.queue_time_us":37568,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":21138,"thread_start_us":567708,"threads_started":2,"wal-append.queue_time_us":432362}
I20260709 13:03:09.722491  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:09.704416  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:09.705696  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:09.728703  1641 raft_consensus.cc:2376] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8. Re-sending same reply.
W20260709 13:03:09.675318  1538 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.025s	sys 0.037s
W20260709 13:03:09.710170  1569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.042s	sys 0.024s
W20260709 13:03:09.730872  1538 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.026s	sys 0.037s
W20260709 13:03:09.731033  1569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.042s	sys 0.024s
I20260709 13:03:09.735643  4521 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:09.736476  4767 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:09.736593  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:09.737226  4771 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.681s	user 0.000s	sys 0.002s
I20260709 13:03:09.737144  4767 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:09.737609  1647 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:09.740355  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1663) took 1407 ms. Trace:
I20260709 13:03:09.741659  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:09.742542  4703 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:09.740975  1572 rpcz_store.cc:276] 0709 13:03:08.333307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.338496 (+  5189us) service_pool.cc:224] Handling call
0709 13:03:08.338833 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:08.367287 (+ 28454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.367317 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:08.367786 (+   469us) log.cc:844] Serialized 26 byte log entry
0709 13:03:08.567863 (+200077us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:08.567928 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.568022 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:09.738954 (+1170932us) spinlock_profiling.cc:243] Waited 27.4 ms on lock 0x7b2400166988. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245c1e9c4 00007f8245c1cacb 00007f8245dd7dc9 00007f8245dd7a55 00007f8245dd6fb0 00007f8248a4d488 00007f8248a4dbe8 00007f8248a25398 00007f8248a2076a 00007f8248a1fbb6
0709 13:03:09.739936 (+   982us) raft_consensus.cc:1685] finished
0709 13:03:09.739956 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.740285 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28063,"raft.queue_time_us":30243,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":10727,"spinlock_wait_cycles":63063424,"thread_start_us":199711,"threads_started":1,"wal-append.queue_time_us":293127}
I20260709 13:03:09.741015  4767 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:09.743559  4790 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:09.744179  4790 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:09.744976  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1684) took 1243 ms. Trace:
I20260709 13:03:09.743120  1644 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:09.745530  1572 rpcz_store.cc:276] 0709 13:03:08.502585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.743660 (+1241075us) service_pool.cc:224] Handling call
0709 13:03:09.744016 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.744418 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:09.744462 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:09.744534 (+    72us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:09.744584 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.744730 (+   146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.744913 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:09.747880  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:09.741546  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1729) took 1696 ms. Trace:
I20260709 13:03:09.749595  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 419) took 1234 ms. Trace:
I20260709 13:03:09.740451  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:09.752003  4533 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:09.665167  1568 rpcz_store.cc:276] 0709 13:03:08.416594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.296769 (+880175us) service_pool.cc:224] Handling call
0709 13:03:09.297222 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.297501 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:09.337352 (+ 39851us) log.cc:844] Serialized 14 byte log entry
0709 13:03:09.620983 (+283631us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:09.621035 (+    52us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:09.621078 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.621151 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.621283 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8499,"prepare.queue_time_us":39314,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":2555,"raft.queue_time_us":2026,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":1992,"thread_start_us":283197,"threads_started":1}
I20260709 13:03:09.713975  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:09.671651  4678 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.753026  4757 consensus_queue.cc:260] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:09.755015  1647 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:09.755546  4757 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:09.753845  4678 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:09.657008  4742 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:09.760644  4742 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:09.761598  1645 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:03:09.638852  4774 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:09.762106  4707 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:09.763645  1568 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:09.729423  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 418) took 1301 ms. Trace:
I20260709 13:03:09.764536  1571 rpcz_store.cc:276] 0709 13:03:08.427832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.728283 (+1300451us) service_pool.cc:224] Handling call
0709 13:03:09.728679 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.729041 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:09.729071 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:09.729162 (+    91us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:09.729201 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.729263 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.729363 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:09.717721  4772 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.388s	user 0.000s	sys 0.003s
I20260709 13:03:09.649814  4543 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:09.776506  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:09.726222  4720 consensus_meta.cc:287] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.326s	user 0.012s	sys 0.007s
I20260709 13:03:09.768913  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 421) took 1054 ms. Trace:
I20260709 13:03:09.775311  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 420) took 1083 ms. Trace:
I20260709 13:03:09.780525  1571 rpcz_store.cc:276] 0709 13:03:08.714216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.766735 (+1052519us) service_pool.cc:224] Handling call
0709 13:03:09.768123 (+  1388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.768394 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:09.768432 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:09.768485 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:09.768525 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.768579 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.768738 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:09.783090  1568 rpcz_store.cc:276] 0709 13:03:08.691569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.756687 (+1065118us) service_pool.cc:224] Handling call
0709 13:03:09.757305 (+   618us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.774837 (+ 17532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.775212 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:09.764627  4749 consensus_meta.cc:287] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.827s	user 0.010s	sys 0.006s
I20260709 13:03:09.788617  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:09.799176  1570 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
W20260709 13:03:09.786268  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 369508us
I20260709 13:03:09.749543  1644 raft_consensus.cc:2393] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 6: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:09.805419  1537 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:09.816991  1716 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:09.818300  1568 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:09.795692  1572 rpcz_store.cc:276] 0709 13:03:08.514936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.748168 (+1233232us) service_pool.cc:224] Handling call
0709 13:03:09.748594 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.749125 (+   531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:09.749168 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:09.749230 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:09.749278 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.749360 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.749518 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:09.806527  4783 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:09.662892  4682 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:09.776064  4767 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.750634  1569 rpcz_store.cc:276] 0709 13:03:08.045001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.055146 (+ 10145us) service_pool.cc:224] Handling call
0709 13:03:08.114657 (+ 59511us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:08.130614 (+ 15957us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:03:08.478522 (+347908us) log.cc:844] Serialized 14 byte log entry
0709 13:03:08.934555 (+456033us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:09.731802 (+797247us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:09.732754 (+   952us) log.cc:844] Serialized 26 byte log entry
0709 13:03:09.735368 (+  2614us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:09.735431 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.741186 (+  5755us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:09.741250 (+    64us) raft_consensus.cc:1685] finished
0709 13:03:09.741350 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.741492 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":216568,"prepare.queue_time_us":501840,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":257972,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":2461,"spinlock_wait_cycles":66459520,"thread_start_us":1315820,"threads_started":3,"wal-append.queue_time_us":399958}
I20260709 13:03:09.825599  2349 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:09.832294  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:09.836274  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:09.835629  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:09.845435  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:09.827759  4783 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.850394  1611 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:09.860452  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:09.861977  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:09.837968  4578 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.793134  4720 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:09.869102  2345 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:09.869601  1685 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:09.873147  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:09.881598  4800 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:09.876909  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2153) took 1461 ms. Trace:
I20260709 13:03:09.882622  4800 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:09.883159  1714 rpcz_store.cc:276] 0709 13:03:08.415891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.416779 (+   888us) service_pool.cc:224] Handling call
0709 13:03:09.876678 (+1459899us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b54001afed4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:09.876837 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1380985088}
I20260709 13:03:09.877377  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:09.885597  1571 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:09.887297  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:09.882490  1716 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:09.887491  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:09.873643  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 240032us
I20260709 13:03:09.815740  4749 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:09.890831  4794 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.891018  1717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:35480
I20260709 13:03:09.891868  4794 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:09.892632  1717 raft_consensus.cc:606] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Received request to transfer leadership
I20260709 13:03:09.893803  1719 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:03:09.894407  1719 raft_consensus.cc:606] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:09.894906  1719 raft_consensus.cc:614] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:09.899840  4682 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:09.900713  4800 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:09.901552  4800 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:09.901111  4787 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:09.902415  4787 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:09.911616  4781 log.cc:927] Time spent T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:03:09.914731  4571 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:09.864122  4783 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:09.920112  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:09.932396  2345 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:09.861291  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:09.913194  4787 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:09.870222  4664 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:09.936504  1564 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
W20260709 13:03:09.954798  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 99218.4us
I20260709 13:03:09.959290  4713 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:09.959692  4748 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:09.959920  4698 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:09.960197  4765 consensus_queue.cc:1059] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:09.918455  4703 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:09.995527  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:09.929598  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:09.872077  4788 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:09.903591  4772 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:03:09.975260  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:09.838871  4767 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:09.893460  4749 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:10.021730  2345 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:09.895350  4577 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:09.985160  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:09.886657  4578 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:09.987646  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 80182.8us
I20260709 13:03:09.927910  4774 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:09.982800  2346 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:10.020757  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.030339  1538 leader_election.cc:336] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.932s (SENT)
I20260709 13:03:09.853965  4229 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:03:10.039254  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:10.016446  1571 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:10.024694  1570 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:10.024771  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 155319us
I20260709 13:03:10.030964  1538 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:10.026984  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:10.023927  4676 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:10.014989  4664 raft_consensus.cc:697] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:10.041445  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:10.047524  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:10.048012  2345 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
W20260709 13:03:10.043166  1538 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.961s (SENT)
I20260709 13:03:10.045485  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:10.049216  1538 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:10.044920  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:09.990909  4787 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:10.055765  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.050984  1639 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:10.058250  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:10.060105  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:10.060840  1639 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:10.014813  4698 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:10.061712  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:10.023557  4703 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:10.037227  4522 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:10.044580  4676 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.081472  4676 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:10.082317  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:10.083256  2349 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:10.085866  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:10.088253  4522 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:10.047040  4684 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:10.070475  4543 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:10.091552  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:10.063550  4787 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:10.093005  4703 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:10.093209  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:10.080487  4698 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:10.043676  4665 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:10.095371  4787 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:10.098448  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1725) took 2078 ms. Trace:
W20260709 13:03:10.097272  4750 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 1.233s	user 0.000s	sys 0.012s
I20260709 13:03:09.994750  4748 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:09.991293  4713 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:10.133984  4801 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:03:10.057021  4749 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:10.047086  4765 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:10.137966  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:10.235312  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 364739us
I20260709 13:03:10.242318  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:10.139484  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:10.131283  4664 consensus_queue.cc:237] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:10.150120  1571 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:03:10.247133  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:10.151394  1538 leader_election.cc:336] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.243s (SENT)
I20260709 13:03:10.248348  1538 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:10.249454  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 98128.7us
I20260709 13:03:10.202690  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:10.206327  1686 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:03:10.075773  4797 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.166s	user 0.002s	sys 0.000s
W20260709 13:03:10.253906  1538 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.516s (SENT)
I20260709 13:03:10.214462  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:10.254915  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:10.216881  2314 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.099s (SENT)
I20260709 13:03:10.260111  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:10.223352  4802 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.265255  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:10.046356  4800 consensus_queue.cc:1059] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 13:03:10.233263  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:10.223538  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 153904us
I20260709 13:03:10.224479  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:10.254371  1538 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:10.214365  2340 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:03:10.282549  4793 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.216009  1567 rpcz_store.cc:276] 0709 13:03:08.019924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.020079 (+   155us) service_pool.cc:224] Handling call
0709 13:03:08.046063 (+ 25984us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:08.046602 (+   539us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:03:08.304124 (+257522us) spinlock_profiling.cc:243] Waited 7.26 ms on lock 0x7b1800431140. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f824c87d3c7 00007f8248e8f9a7 00007f8248e9f0bf 00007f8248e9f038
0709 13:03:08.556165 (+252041us) log.cc:844] Serialized 14 byte log entry
0709 13:03:08.661290 (+105125us) spinlock_profiling.cc:243] Waited 1.73 ms on lock 0x7b1800431140. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a2523a 00007f8248e8fa2a
0709 13:03:09.426419 (+765129us) log.cc:844] Serialized 14 byte log entry
0709 13:03:09.432276 (+  5857us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:09.432783 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 13:03:09.432943 (+   160us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:09.432989 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.433081 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:10.098176 (+665095us) raft_consensus.cc:1685] finished
0709 13:03:10.098201 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.098377 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":293949,"prepare.queue_time_us":572045,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":86,"raft.queue_time_us":184808,"raft.run_cpu_time_us":442,"raft.run_wall_time_us":18932,"spinlock_wait_cycles":20661504,"thread_start_us":888968,"threads_started":3,"wal-append.queue_time_us":403434}
I20260709 13:03:10.023609  4788 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:10.261885  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 45245.8us
I20260709 13:03:10.255828  4787 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:10.240324  4671 consensus_queue.cc:1048] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:03:10.283262  4793 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:10.284092  4720 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:10.287912  4765 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:03:10.300248  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 57115.4us
W20260709 13:03:10.306319  4809 log.cc:927] Time spent T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.171s	user 0.000s	sys 0.003s
I20260709 13:03:10.307003  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1723) took 2288 ms. Trace:
I20260709 13:03:10.307680  1715 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:03:10.307735  1566 rpcz_store.cc:276] 0709 13:03:08.019145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.021622 (+  2477us) service_pool.cc:224] Handling call
0709 13:03:08.037041 (+ 15419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:08.037524 (+   483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.037569 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:08.070303 (+ 32734us) log.cc:844] Serialized 26 byte log entry
0709 13:03:08.504478 (+434175us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:08.504527 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.504610 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:10.096777 (+1592167us) spinlock_profiling.cc:243] Waited 8.2 ms on lock 0x7b2400166868. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245c1e9c4 00007f8245c1cacb 00007f8245dd7dc9 00007f8245dd7a55 00007f8245dd709d 00007f8248a0910e 00007f8248a2088c 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221
0709 13:03:10.302212 (+205435us) raft_consensus.cc:1685] finished
0709 13:03:10.302236 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.306901 (+  4665us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":544,"raft.queue_time_us":5428,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":58216,"spinlock_wait_cycles":18858368,"thread_start_us":365303,"threads_started":1,"wal-append.queue_time_us":650028}
W20260709 13:03:10.284719  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 259448us
W20260709 13:03:10.310467  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 104307us
I20260709 13:03:10.314811  1687 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:10.323071  1570 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
W20260709 13:03:10.347891  1612 connection.cc:256] RPC call timeout handler was delayed by -2.2862975479999932s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:03:10.387799  1639 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:10.385012  4783 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:10.397876  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 116771us
I20260709 13:03:10.299402  4767 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:10.287632  4802 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:10.400784  4799 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:10.406541  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 71717.8us
I20260709 13:03:10.406787  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:10.407107  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:10.321434  4757 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:10.409655  1639 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:10.386480  4684 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.424309  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:10.430274  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:10.433246  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:10.434152  1639 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:10.435140  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:10.435957  1639 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:10.445500  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:10.450791  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:10.454198  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:10.455541  1567 raft_consensus.cc:2393] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 3: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:10.455958  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:10.458813  1714 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:10.461194  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:10.495590  1715 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:03:10.466019  1714 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:03:10.425009  4757 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.500871  1639 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
W20260709 13:03:10.463101  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 148964us
I20260709 13:03:10.442144  4529 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:10.509034  4529 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:10.509469  4529 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:10.504868  4757 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:10.519043  4704 raft_consensus.cc:993] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: : Instructing follower 066ee0109aba4465ba17fa65e14ef504 to start an election
I20260709 13:03:10.508028  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:10.521557  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:10.513819  1719 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:10.432174  4578 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:10.534933  4676 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:10.522300  1639 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:10.403692  4799 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:10.497900  4797 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:03:10.423095  4577 consensus_queue.cc:1059] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.099s
I20260709 13:03:10.549337  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:10.496140  4742 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:10.552652  1639 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:10.555382  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:10.532950  4798 raft_consensus.cc:1081] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Signalling peer 066ee0109aba4465ba17fa65e14ef504 to start an election
I20260709 13:03:10.570494  1639 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:03:10.571244  4682 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:10.566936  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 170125us
I20260709 13:03:10.571942  4682 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:10.572072  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:10.572727  1612 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:10.574657  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:10.569480  2314 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:10.575929  4682 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:10.576962  4682 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:10.576975  4812 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:10.576795  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:10.565433  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1682) took 1124 ms. Trace:
I20260709 13:03:10.573863  1612 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:10.580418  1643 rpcz_store.cc:276] 0709 13:03:09.440584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.442455 (+  1871us) service_pool.cc:224] Handling call
0709 13:03:09.444334 (+  1879us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:09.516313 (+ 71979us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:03:09.516699 (+   386us) log.cc:844] Serialized 14 byte log entry
0709 13:03:09.785740 (+269041us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:09.828155 (+ 42415us) log.cc:844] Serialized 52 byte log entry
0709 13:03:09.828357 (+   202us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:09.828706 (+   349us) log.cc:844] Serialized 14 byte log entry
0709 13:03:09.838254 (+  9548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.838338 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:10.565088 (+726750us) raft_consensus.cc:1685] finished
0709 13:03:10.565125 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.565350 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101857,"prepare.queue_time_us":2170,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":82,"raft.queue_time_us":97641,"raft.run_cpu_time_us":605,"raft.run_wall_time_us":25089,"spinlock_wait_cycles":30976,"thread_start_us":267765,"threads_started":1,"wal-append.queue_time_us":349528}
I20260709 13:03:10.443682  4684 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:10.545359  4708 raft_consensus.cc:3055] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:10.578898  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:10.601999  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:10.496740  4814 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.156s
I20260709 13:03:10.606741  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:10.611385  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 209385us
I20260709 13:03:10.630376  1685 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:10.611874  1612 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:10.615885  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:10.638092  4800 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:10.638857  1567 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:03:10.641871  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 133447us
I20260709 13:03:10.640751  1643 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:10.613905  2346 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.573s	user 0.009s	sys 0.007s
I20260709 13:03:10.645020  1687 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:10.613803  4708 raft_consensus.cc:740] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:10.640772  4774 raft_consensus.cc:3055] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:10.647272  4774 raft_consensus.cc:740] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:03:10.648623  4816 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:03:10.648288  4708 consensus_queue.cc:260] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:10.648816  4774 consensus_queue.cc:260] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:10.651235  2346 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
W20260709 13:03:10.682436  4762 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 1.755s	user 0.004s	sys 0.000s
I20260709 13:03:10.682875  4529 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:10.651473  4827 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:10.652734  4682 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:10.614289  4802 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:10.653144  4793 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:10.557596  4742 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:10.679986  1685 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:10.630559  4813 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:10.689172  4690 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:10.646576  4812 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:03:10.698648  4799 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:10.698516  4713 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:10.709527  4690 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.743727  4690 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:10.743129  4676 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:10.763178  4742 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:10.767524  4790 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:10.768328  4616 consensus_queue.cc:1175] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cf2f71592be8465fb18e36e15d0b2f89" responder_term: 5 status { last_received { term: 5 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 5 index: 6 } }
I20260709 13:03:10.777268  4757 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:10.780222  4678 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:10.784170  4757 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:10.711091  4813 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:10.786070  4791 raft_consensus.cc:3055] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:10.792325  4788 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:10.804381  4783 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:10.807413  4748 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:10.786594  4791 raft_consensus.cc:740] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:10.819943  4813 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:10.825872  4791 consensus_queue.cc:260] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:10.830106  4791 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:10.832901  4794 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:10.882103  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1670) took 2539 ms. Trace:
I20260709 13:03:10.883199  1565 rpcz_store.cc:276] 0709 13:03:08.342416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.403534 (+ 61118us) service_pool.cc:224] Handling call
0709 13:03:08.404090 (+   556us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:08.415438 (+ 11348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.415471 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:08.416038 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 13:03:08.643768 (+227730us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:08.643816 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.643899 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:10.652319 (+2008420us) spinlock_profiling.cc:243] Waited 47.3 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:10.880784 (+228465us) raft_consensus.cc:1685] finished
0709 13:03:10.880871 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.881658 (+   787us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31652,"raft.queue_time_us":14087,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":48933,"spinlock_wait_cycles":108892416,"thread_start_us":201026,"threads_started":1,"wal-append.queue_time_us":358923}
I20260709 13:03:10.892285  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1694) took 1016 ms. Trace:
I20260709 13:03:10.897491  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1695) took 1019 ms. Trace:
I20260709 13:03:10.897092  1565 rpcz_store.cc:276] 0709 13:03:09.877135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.887961 (+1010826us) service_pool.cc:224] Handling call
0709 13:03:10.889159 (+  1198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.891483 (+  2324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:10.891565 (+    82us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.891649 (+    84us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:10.891750 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.891986 (+   236us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.892176 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.901988  1567 rpcz_store.cc:276] 0709 13:03:09.877460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.888596 (+1011136us) service_pool.cc:224] Handling call
0709 13:03:10.895076 (+  6480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.896530 (+  1454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:10.896555 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.896652 (+    97us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:10.896718 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.896832 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.897192 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.903522  1566 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:10.904822  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1696) took 1026 ms. Trace:
I20260709 13:03:10.668171  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1677) took 1229 ms. Trace:
I20260709 13:03:10.915092  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1669) took 2572 ms. Trace:
I20260709 13:03:10.916062  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1740) took 2054 ms. Trace:
I20260709 13:03:10.923079  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:10.923660  1568 rpcz_store.cc:276] 0709 13:03:08.861978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.811448 (+949470us) service_pool.cc:224] Handling call
0709 13:03:09.814063 (+  2615us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.818709 (+  4646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.915548 (+1096839us) spinlock_profiling.cc:243] Waited 4.02 ms on lock 0x7b54002b9154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:10.915866 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9250560}
I20260709 13:03:10.925308  1641 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:10.925977  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:10.931342  1568 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:10.697293  1643 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:10.921404  1639 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
W20260709 13:03:10.933252  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 253197us
I20260709 13:03:10.917536  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1690) took 1046 ms. Trace:
I20260709 13:03:10.937062  1569 rpcz_store.cc:276] 0709 13:03:09.871156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.871661 (+   505us) service_pool.cc:224] Handling call
0709 13:03:09.920223 (+ 48562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.299941 (+379718us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:10.299979 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.300035 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:10.300064 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.916942 (+616878us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b54001b6554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:10.917044 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.917330 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":872735488}
I20260709 13:03:10.942754  1570 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:10.944373  1570 raft_consensus.cc:1275] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:03:10.919575  1640 rpcz_store.cc:276] 0709 13:03:09.438502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.438650 (+   148us) service_pool.cc:224] Handling call
0709 13:03:09.438964 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.649971 (+211007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.667754 (+1017783us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b540007f154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:10.668063 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":272995712}
I20260709 13:03:10.946099  1571 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:03:10.927388  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 315502us
I20260709 13:03:10.949867  1612 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:10.928550  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1679) took 2549 ms. Trace:
I20260709 13:03:10.954418  1573 rpcz_store.cc:276] 0709 13:03:08.396042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.222417 (+826375us) service_pool.cc:224] Handling call
0709 13:03:09.222839 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.628349 (+405510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.927817 (+1299468us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b5400013454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:10.928294 (+   477us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":931226496}
I20260709 13:03:10.954706  1470 catalog_manager.cc:5697] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:10.961968  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1689) took 3099 ms (client timeout 1776 ms). Trace:
W20260709 13:03:10.963054  1720 rpcz_store.cc:269] 0709 13:03:07.862697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.862848 (+   151us) service_pool.cc:224] Handling call
0709 13:03:10.961169 (+3098321us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54003903d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:10.961535 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3141019520}
W20260709 13:03:10.964196  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 319162us
W20260709 13:03:10.968245  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.971657  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.973564  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 242) took 3379 ms. Trace:
I20260709 13:03:10.974295  1721 rpcz_store.cc:276] 0709 13:03:07.593679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.593891 (+   212us) service_pool.cc:224] Handling call
0709 13:03:07.600780 (+  6889us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.661095 (+2060315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.972749 (+1311654us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b54000230d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:10.973262 (+   513us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4699240192}
I20260709 13:03:10.979874  4529 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:10.982530  4665 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.957513  1564 rpcz_store.cc:276] 0709 13:03:08.342180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.355578 (+ 13398us) service_pool.cc:224] Handling call
0709 13:03:08.355813 (+   235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.978927 (+1623114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.914477 (+935550us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b5400306454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:10.914891 (+   414us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3634920704}
I20260709 13:03:10.983830  4665 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:10.982421  4703 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:10.905231  1566 rpcz_store.cc:276] 0709 13:03:09.877961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.901979 (+1024018us) service_pool.cc:224] Handling call
0709 13:03:10.902827 (+   848us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.904434 (+  1607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.904726 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.971908  1713 raft_consensus.cc:1275] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:03:10.911108  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1699) took 1032 ms. Trace:
I20260709 13:03:11.000123  1565 rpcz_store.cc:276] 0709 13:03:09.878807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.909193 (+1030386us) service_pool.cc:224] Handling call
0709 13:03:10.909956 (+   763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.910522 (+   566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:10.910556 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.910628 (+    72us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:10.910675 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.910740 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.910869 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.999966  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:11.007587  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 250) took 1351 ms. Trace:
I20260709 13:03:11.008299  1648 rpcz_store.cc:276] 0709 13:03:09.638002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.755670 (+117668us) service_pool.cc:224] Handling call
0709 13:03:09.756154 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.865043 (+108889us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:09.865086 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:09.865163 (+    77us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:09.865201 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.969123 (+1103922us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54002db754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:10.969227 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.989521 (+ 20294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249664768}
I20260709 13:03:11.016132  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:11.013290  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:11.017596  1641 raft_consensus.cc:2393] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:11.054893  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:11.055505  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2183) took 1084 ms. Trace:
I20260709 13:03:11.055871  2345 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:11.055959  1641 rpcz_store.cc:276] 0709 13:03:09.971107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.015506 (+1044399us) service_pool.cc:224] Handling call
0709 13:03:11.055397 (+ 39891us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:11.034021  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:11.031888  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:11.033325  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:11.025362  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1650) took 3203 ms. Trace:
I20260709 13:03:11.047555  2348 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:11.026350  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 239) took 1406 ms. Trace:
W20260709 13:03:11.038767  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.024456  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:11.139111  1612 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:11.057621  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:11.141326  1645 rpcz_store.cc:276] 0709 13:03:09.619737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.624473 (+  4736us) service_pool.cc:224] Handling call
0709 13:03:11.025885 (+1401412us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54001a9854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:11.026098 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":266889984}
I20260709 13:03:11.054380  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:11.020493  1687 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:03:11.156876  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 137024us
I20260709 13:03:11.176469  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:11.179489  4804 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.180709  4804 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:11.048246  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:11.026849  4664 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:11.035596  4665 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:11.193499  2347 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:11.161008  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:11.022061  4742 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:11.032821  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:11.197423  4707 raft_consensus.cc:3055] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:11.198262  1644 raft_consensus.cc:2410] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:03:11.197420  4664 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:11.198769  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1703) took 3181 ms. Trace:
I20260709 13:03:11.200004  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:11.194257  1614 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.010s	sys 0.010s
I20260709 13:03:11.037914  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:11.194264  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:11.022977  4798 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:11.202222  4707 raft_consensus.cc:740] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:11.165953  4807 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:11.208725  1614 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.011s	sys 0.011s
I20260709 13:03:11.202135  2349 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:03:11.022300  4703 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:11.209715  1566 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:11.211386  1648 raft_consensus.cc:2410] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:11.213430  4665 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:11.216113  1640 raft_consensus.cc:2393] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 6: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:11.022712  4684 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:11.203972  1713 rpcz_store.cc:276] 0709 13:03:08.017299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.045675 (+ 28376us) service_pool.cc:224] Handling call
0709 13:03:10.970304 (+2924629us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.992626 (+ 22322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.198447 (+205821us) spinlock_profiling.cc:243] Waited 2.85 s on lock 0x7b54003903d0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:11.198665 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6553935744}
W20260709 13:03:11.208686  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2186) took 1172 ms (client timeout 1516 ms). Trace:
W20260709 13:03:11.221139  1644 rpcz_store.cc:269] 0709 13:03:10.031209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.676661 (+645452us) service_pool.cc:224] Handling call
0709 13:03:11.160472 (+483811us) spinlock_profiling.cc:243] Waited 3.54 ms on lock 0x7b5800003098. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:03:11.203663 (+ 43191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8142720}
I20260709 13:03:11.222942  1568 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:03:11.225957  1640 raft_consensus.cc:1240] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:11.230863  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1692) took 1369 ms. Trace:
I20260709 13:03:11.228660  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2184) took 1231 ms. Trace:
I20260709 13:03:11.230088  4524 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:11.233215  1647 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:11.226056  4814 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:11.230130  1642 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:11.234794  4814 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:11.235349  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2197) took 1196 ms. Trace:
I20260709 13:03:11.234990  4832 consensus_queue.cc:1059] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:11.235800  1647 rpcz_store.cc:276] 0709 13:03:10.038737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.023708 (+984971us) service_pool.cc:224] Handling call
0709 13:03:11.176131 (+152423us) spinlock_profiling.cc:243] Waited 5.83 ms on lock 0x7b5800003098. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60eee 0000000000375939 00007f824ddd41e0 00007f824de9d0c4 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:03:11.235180 (+ 59049us) spinlock_profiling.cc:243] Waited 52.5 ms on lock 0x7b5400183654. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:11.235279 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":134153344}
I20260709 13:03:11.236817  4832 raft_consensus.cc:3055] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:11.237156  4774 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:11.237392  4832 raft_consensus.cc:740] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:11.235031  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2191) took 1201 ms. Trace:
I20260709 13:03:11.232436  1648 rpcz_store.cc:276] 0709 13:03:09.995540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.023961 (+1028421us) service_pool.cc:224] Handling call
0709 13:03:11.226918 (+202957us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:11.231971  1640 rpcz_store.cc:276] 0709 13:03:09.858751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.217304 (+1358553us) service_pool.cc:224] Handling call
0709 13:03:11.217962 (+   658us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.227761 (+  9799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.228070 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:11.238415  1642 rpcz_store.cc:276] 0709 13:03:10.033412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.963725 (+930313us) service_pool.cc:224] Handling call
0709 13:03:11.193868 (+230143us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b5800003098. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:03:11.234958 (+ 41090us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8085760}
I20260709 13:03:11.234371  4831 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:11.248067  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 253) took 1608 ms. Trace:
I20260709 13:03:11.030102  4767 consensus_queue.cc:1059] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.210s
W20260709 13:03:11.248561  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 244392us
I20260709 13:03:11.247675  4831 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:11.248605  1645 rpcz_store.cc:276] 0709 13:03:09.639152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.196265 (+1557113us) service_pool.cc:224] Handling call
0709 13:03:11.247192 (+ 50927us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.247617 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.247644 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:11.247710 (+    66us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:11.247767 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.247836 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.247970 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:11.140810  1646 rpcz_store.cc:276] 0709 13:03:07.822200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.822346 (+   146us) service_pool.cc:224] Handling call
0709 13:03:08.141516 (+319170us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:08.142059 (+   543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.142102 (+    43us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:08.142981 (+   879us) log.cc:844] Serialized 78 byte log entry
0709 13:03:08.498637 (+355656us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:08.779421 (+280784us) log.cc:844] Serialized 14 byte log entry
0709 13:03:08.786734 (+  7313us) log.cc:844] Serialized 14 byte log entry
0709 13:03:09.421368 (+634634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.421488 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:09.421559 (+    71us) raft_consensus.cc:1685] finished
0709 13:03:09.421579 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.084359 (+662780us) spinlock_profiling.cc:243] Waited 3.06 ms on lock 0x7b18003e3680. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d98522 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:03:11.024870 (+940511us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54002aeed0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:11.025095 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20458,"prepare.queue_time_us":567021,"prepare.run_cpu_time_us":342,"prepare.run_wall_time_us":345,"raft.queue_time_us":164724,"raft.run_cpu_time_us":15222,"raft.run_wall_time_us":230417,"spinlock_wait_cycles":709012480,"thread_start_us":1129074,"threads_started":3,"wal-append.queue_time_us":425168,"wal-append.run_cpu_time_us":14311,"wal-append.run_wall_time_us":1820446}
I20260709 13:03:11.228834  1537 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:11.235349  4814 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:11.282616  1645 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:11.283336  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:11.284006  4758 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:11.284052  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1700) took 1421 ms. Trace:
I20260709 13:03:11.284868  1645 rpcz_store.cc:276] 0709 13:03:09.862549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.256597 (+1394048us) service_pool.cc:224] Handling call
0709 13:03:11.282093 (+ 25496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.283565 (+  1472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.283933 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:11.287757  4831 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:11.288905  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 66128.9us
I20260709 13:03:11.289207  4832 consensus_queue.cc:260] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:11.314503  4832 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:11.315671  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:11.290527  4682 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:11.317421  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:11.290992  4748 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:11.318059  4682 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:11.234162  4524 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:11.278548  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:11.320348  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:11.321297  1643 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:11.321884  1538 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:11.323784  1538 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:11.324054  1718 raft_consensus.cc:1240] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:11.325935  4767 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:11.328501  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:11.294332  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:11.329533  1720 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:11.320387  4524 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:11.328507  1721 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:11.327113  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:11.331709  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:11.295177  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:11.337180  1612 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:03:11.337792  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 75708.3us
I20260709 13:03:11.339067  1686 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:11.341051  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 320816us
I20260709 13:03:11.296710  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1699) took 1434 ms. Trace:
I20260709 13:03:11.306564  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1697) took 1445 ms. Trace:
I20260709 13:03:11.342510  1642 rpcz_store.cc:276] 0709 13:03:09.862232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.240805 (+1378573us) service_pool.cc:224] Handling call
0709 13:03:11.280665 (+ 39860us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.296235 (+ 15570us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.296270 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:11.296346 (+    76us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:11.296426 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.296494 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.296635 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52096}
I20260709 13:03:11.323889  4748 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:11.342566  1648 rpcz_store.cc:276] 0709 13:03:09.861498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.243788 (+1382290us) service_pool.cc:224] Handling call
0709 13:03:11.276788 (+ 33000us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.306143 (+ 29355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.306175 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:11.306240 (+    65us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:11.306280 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.306349 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.306465 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36992}
I20260709 13:03:11.343912  4748 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:11.344394  4748 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:11.312687  4690 consensus_meta.cc:287] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.553s	user 0.010s	sys 0.007s
I20260709 13:03:11.345932  4690 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:11.347412  4664 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:11.348301  1713 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 8: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:11.349279  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2195) took 1307 ms. Trace:
I20260709 13:03:11.349929  1648 rpcz_store.cc:276] 0709 13:03:10.038386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.344242 (+1305856us) service_pool.cc:224] Handling call
0709 13:03:11.344906 (+   664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.345256 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.345300 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:11.345347 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:11.345390 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.345445 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.345565 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:11.353040  4804 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:11.354255  4804 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:11.354779  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:11.355271  4804 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:11.356693  4690 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:11.357157  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2196) took 1313 ms. Trace:
I20260709 13:03:11.357980  1648 rpcz_store.cc:276] 0709 13:03:10.038552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.351003 (+1312451us) service_pool.cc:224] Handling call
0709 13:03:11.351479 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.351928 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.351953 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:11.352016 (+    63us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:11.352129 (+   113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.352190 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.352303 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:11.358649  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:11.359407  4671 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:11.360927  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:11.361703  4671 consensus_queue.cc:1059] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:11.364159  4798 raft_consensus.cc:3055] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:11.364008  4684 consensus_queue.cc:1059] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:11.364691  4798 raft_consensus.cc:740] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:11.364811  4577 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:11.365120  4684 consensus_queue.cc:1243] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer 066ee0109aba4465ba17fa65e14ef504 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:03:11.365924  4684 raft_consensus.cc:3055] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:11.366425  4684 raft_consensus.cc:740] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:11.365973  4577 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:11.367275  4798 consensus_queue.cc:260] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:11.367587  4684 consensus_queue.cc:260] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:11.369338  1717 raft_consensus.cc:1240] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:11.303346  2345 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:11.362016  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2189) took 1330 ms. Trace:
I20260709 13:03:11.376832  1639 rpcz_store.cc:276] 0709 13:03:10.032371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.936501 (+904130us) service_pool.cc:224] Handling call
0709 13:03:11.360700 (+424199us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b5800003098. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:03:11.361940 (+  1240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7739136}
I20260709 13:03:11.374620  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1701) took 1510 ms. Trace:
I20260709 13:03:11.382187  1646 rpcz_store.cc:276] 0709 13:03:09.862865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.278043 (+1415178us) service_pool.cc:224] Handling call
0709 13:03:11.372203 (+ 94160us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.372818 (+   615us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.372856 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:11.372949 (+    93us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:11.373326 (+   377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.373415 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.373564 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:11.369517  4798 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:11.369989  4684 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:11.320431  4682 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:11.405174  2347 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:03:11.386304  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:11.407676  4524 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:11.411010  4708 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:11.370780  4577 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:11.413995  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 96495.1us
W20260709 13:03:11.416013  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 118409us
W20260709 13:03:11.418967  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.385217  4831 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:11.425395  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:11.421558  1470 catalog_manager.cc:5697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:11.439014  1687 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:11.441426  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.441587  4226 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:11.442535  4226 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:11.444989  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1698) took 1583 ms. Trace:
I20260709 13:03:11.445433  1640 rpcz_store.cc:276] 0709 13:03:09.861868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.240545 (+1378677us) service_pool.cc:224] Handling call
0709 13:03:11.305629 (+ 65084us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.322221 (+ 16592us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.322248 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:11.322309 (+    61us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:11.322347 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.444730 (+122383us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b5400184554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:11.444777 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.444906 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37400960}
I20260709 13:03:11.446450  4707 consensus_queue.cc:260] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:11.448089  4707 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:11.450037  1722 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 12: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:11.437296  4831 raft_consensus.cc:697] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 5 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:11.464443  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 43726us
I20260709 13:03:11.473593  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:11.474157  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:11.479493  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:11.486128  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 147317us
I20260709 13:03:11.496531  1566 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:11.495625  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:11.461691  4804 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:11.457755  4748 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:11.460801  4831 consensus_queue.cc:237] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:11.490900  4690 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:11.519704  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:11.522333  1569 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:03:11.522955  1611 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:11.543342  1565 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:11.490464  1470 catalog_manager.cc:5697] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: config changed from index 2 to 5, term changed from 5 to 8, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 8 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:11.561214  4705 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:11.494504  4524 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:03:11.521329  4226 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:11.534960  4690 raft_consensus.cc:697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 8 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:11.604208  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:11.607978  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 122667us
W20260709 13:03:11.611586  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 99143.7us
I20260709 13:03:11.625248  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1744) took 1706 ms. Trace:
I20260709 13:03:11.624073  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:11.648078  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 192157us
I20260709 13:03:11.570784  1566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:40670
I20260709 13:03:11.632278  4524 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:11.695957  1569 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 4. Current term is 5. Ops: []
W20260709 13:03:11.683754  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 298965us
I20260709 13:03:11.691210  1611 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:11.700519  1571 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 7: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:11.697940  1566 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:11.627843  1567 rpcz_store.cc:276] 0709 13:03:09.918602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.903802 (+985200us) service_pool.cc:224] Handling call
0709 13:03:10.906316 (+  2514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.907564 (+  1248us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:11.358705 (+451141us) spinlock_profiling.cc:243] Waited 9.54 ms on lock 0x7b18002d0a60. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f824c87d3c7 00007f8248e8f9a7 00007f8248e9f0bf
0709 13:03:11.358859 (+   154us) log.cc:844] Serialized 14 byte log entry
0709 13:03:11.624796 (+265937us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:11.624860 (+    64us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:11.624934 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.625020 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.625182 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3731,"prepare.queue_time_us":97630,"prepare.run_cpu_time_us":122,"prepare.run_wall_time_us":51177,"raft.queue_time_us":8466,"spinlock_wait_cycles":21949312,"thread_start_us":473695,"threads_started":2}
I20260709 13:03:11.721271  1566 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:11.732591  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 110825us
I20260709 13:03:11.710427  4524 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:11.729113  1713 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:11.725540  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:11.744417  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 114171us
I20260709 13:03:11.725010  4521 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:03:11.771946  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 47056.9us
I20260709 13:03:11.775117  1611 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:11.775197  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1696) took 1903 ms. Trace:
I20260709 13:03:11.746075  4524 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:11.777221  1686 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:11.762022  4521 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:11.754797  4742 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:11.778071  1647 rpcz_store.cc:276] 0709 13:03:09.861145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.236947 (+1375802us) service_pool.cc:224] Handling call
0709 13:03:11.237428 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.258867 (+ 21439us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:11.458265 (+199398us) log.cc:844] Serialized 14 byte log entry
0709 13:03:11.763978 (+305713us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:11.764057 (+    79us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:11.764140 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.764224 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.764385 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":419790,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":3500,"raft.queue_time_us":13298,"thread_start_us":393471,"threads_started":2}
W20260709 13:03:11.830015  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 63108.1us
W20260709 13:03:11.833227  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 76007.1us
I20260709 13:03:11.813565  4742 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:11.842114  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 117724us
W20260709 13:03:11.849521  2314 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.663s (SENT) [suppressed 23 similar messages]
W20260709 13:03:11.850812  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 178448us
I20260709 13:03:11.857741  4665 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:11.858388  4665 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.858799  4665 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:11.839653  4577 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:11.867660  4577 raft_consensus.cc:697] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:11.870357  4748 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:11.846601  1538 leader_election.cc:336] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 3.157s (SENT)
I20260709 13:03:11.796541  4690 consensus_queue.cc:237] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:11.884233  1538 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:11.844758  4742 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:11.879199  4832 consensus_meta.cc:287] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.564s	user 0.014s	sys 0.004s
W20260709 13:03:11.903059  1468 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38666 (request call id 213) took 23184 ms (client timeout 15000 ms). Trace:
I20260709 13:03:11.888090  1573 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:03:11.916079  1573 raft_consensus.cc:606] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:11.918648  1573 raft_consensus.cc:614] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:11.928090  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 93276.1us
W20260709 13:03:11.932097  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.913002  4846 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.097s	user 0.000s	sys 0.002s
I20260709 13:03:11.943439  1612 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:11.914175  4577 consensus_queue.cc:237] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:11.942343  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.938088  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.903884  1468 rpcz_store.cc:269] 0709 13:02:48.718322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.727460 (+  9138us) service_pool.cc:224] Handling call
0709 13:02:49.654637 (+927177us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54002b0054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248def090 00007f8248e8df3a 00007f824ec441cc 00007f824ed4237e 00007f824d2032a4 00007f824d203232 00007f824d203161 00007f824d2030dc 00007f824d202392 00007f82471c048c 00007f82471bf8c6
0709 13:03:11.902508 (+22247871us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:02.880123 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:02.962373 (+ 82250us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:02.962426 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:03:02.962451 (+    25us) tablet.cc:662] Decoding operations
0709 13:03:02.963928 (+  1477us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:02.964073 (+   145us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:02.964108 (+    35us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:03:03.072059 (+107951us) tablet.cc:701] Row locks acquired
0709 13:03:03.072189 (+   130us) write_op.cc:260] PREPARE: finished
0709 13:03:03.072912 (+   723us) write_op.cc:270] Start()
0709 13:03:03.073163 (+   251us) write_op.cc:276] Timestamp: P: 1783602183072722 usec, L: 0
0709 13:03:03.073191 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:03:03.178953 (+105762us) log.cc:844] Serialized 5242 byte log entry
0709 13:03:04.011190 (+832237us) op_driver.cc:464] REPLICATION: finished
0709 13:03:04.249349 (+238159us) write_op.cc:301] APPLY: starting
0709 13:03:04.249458 (+   109us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:04.352802 (+103344us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:04.415103 (+ 62301us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:04.840911 (+425808us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:04.841323 (+   412us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 13:03:04.883253 (+ 41930us) write_op.cc:312] APPLY: finished
0709 13:03:05.078812 (+195559us) log.cc:844] Serialized 95 byte log entry
0709 13:03:05.427567 (+348755us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:05.459740 (+ 32173us) write_op.cc:454] Released schema lock
0709 13:03:05.480749 (+ 21009us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:05.621024 (+140275us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:05.621085 (+    61us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5311624,"reported_tablets":49,"spinlock_wait_cycles":416465280,"child_traces":[["op",{"apply.queue_time_us":228957,"apply.run_cpu_time_us":27412,"apply.run_wall_time_us":1371548,"num_ops":13,"prepare.queue_time_us":267404,"prepare.run_cpu_time_us":104940,"prepare.run_wall_time_us":546577,"raft.queue_time_us":382074,"raft.run_cpu_time_us":16562,"raft.run_wall_time_us":222713,"replication_time_us":898270,"spinlock_wait_cycles":281344,"thread_start_us":745710,"threads_started":5,"wal-append.queue_time_us":647167,"wal-append.run_cpu_time_us":30841,"wal-append.run_wall_time_us":2481888}]]}
W20260709 13:03:11.949391  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 116550us
I20260709 13:03:11.910379  4748 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:11.952111  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.927392  4814 consensus_meta.cc:287] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.625s	user 0.005s	sys 0.010s
I20260709 13:03:11.961220  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 251) took 2323 ms. Trace:
W20260709 13:03:11.961735  1644 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.018s	sys 0.049s
W20260709 13:03:11.962291  1644 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.018s	sys 0.049s
W20260709 13:03:11.963579  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 167295us
W20260709 13:03:11.971593  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.920651  4664 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:11.957175  4682 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:03:11.996944  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:11.961709  1641 rpcz_store.cc:276] 0709 13:03:09.638340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.182424 (+1544084us) service_pool.cc:224] Handling call
0709 13:03:11.225198 (+ 42774us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:11.362580 (+137382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.362625 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:11.363270 (+   645us) log.cc:844] Serialized 26 byte log entry
0709 13:03:11.614532 (+251262us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:11.686856 (+ 72324us) log.cc:844] Serialized 14 byte log entry
0709 13:03:11.687081 (+   225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.687140 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:11.960973 (+273833us) raft_consensus.cc:1685] finished
0709 13:03:11.960996 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.961153 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":136028,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":74,"raft.queue_time_us":68077,"raft.run_cpu_time_us":895,"raft.run_wall_time_us":18027,"thread_start_us":305217,"threads_started":2,"wal-append.queue_time_us":427867}
I20260709 13:03:12.003593  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:11.933295  4804 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:03:12.012046  2345 raft_consensus.cc:2376] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5. Re-sending same reply.
W20260709 13:03:12.003367  4798 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.599s	user 0.011s	sys 0.005s
I20260709 13:03:12.013091  1566 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:12.012105  1538 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 3.302s (SENT)
I20260709 13:03:12.018174  1538 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:11.964112  4814 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:12.042243  1566 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:12.042205  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 163239us
W20260709 13:03:12.034231  4774 consensus_meta.cc:287] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.796s	user 0.014s	sys 0.003s
I20260709 13:03:12.042802  1686 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:12.032437  4831 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:12.077867  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:12.078855  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 135618us
I20260709 13:03:12.066319  4767 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:12.106202  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:12.042534  4664 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:12.086346  4738 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:12.123435  1715 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:12.133152  1566 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
W20260709 13:03:12.135334  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 143584us
W20260709 13:03:12.137154  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:12.138250  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 97383.7us
W20260709 13:03:12.152894  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.113790  4671 consensus_queue.cc:1059] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:03:12.158624  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:12.120955  4831 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:03:12.161150  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:12.158339  4577 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:12.165194  4609 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.165849  4577 raft_consensus.cc:697] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:12.166415  4609 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:12.168934  4609 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:12.170603  4855 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:03:12.170747  4671 raft_consensus.cc:3055] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:12.171629  4671 raft_consensus.cc:740] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:12.172567  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1694) took 2313 ms. Trace:
I20260709 13:03:12.174048  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:12.175154  1566 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:03:12.133040  4738 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:12.153178  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:12.173204  1644 rpcz_store.cc:276] 0709 13:03:09.859526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.238742 (+1379216us) service_pool.cc:224] Handling call
0709 13:03:11.277779 (+ 39037us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:11.370790 (+ 93011us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.370820 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:11.372328 (+  1508us) log.cc:844] Serialized 26 byte log entry
0709 13:03:11.962811 (+590483us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:11.962855 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.962943 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:12.172057 (+209114us) raft_consensus.cc:1685] finished
0709 13:03:12.172223 (+   166us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:12.172491 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40388,"raft.queue_time_us":42897,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":216,"spinlock_wait_cycles":80256,"thread_start_us":589370,"threads_started":1,"wal-append.queue_time_us":675971}
I20260709 13:03:12.180501  4767 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:12.168138  4577 consensus_queue.cc:237] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:12.137183  4852 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:03:12.155982  4708 consensus_meta.cc:287] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.703s	user 0.020s	sys 0.001s
W20260709 13:03:12.192888  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 69192.4us
I20260709 13:03:12.157246  4682 consensus_queue.cc:1059] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
W20260709 13:03:12.201406  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.156430  4703 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:12.186462  4850 log.cc:927] Time spent T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.118s	user 0.002s	sys 0.000s
I20260709 13:03:12.221215  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:12.219308  1569 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:03:12.225378  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:12.231123  1573 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 7: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:03:12.232342  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 197443us
I20260709 13:03:12.236083  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:12.236397  4682 raft_consensus.cc:3055] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:12.235908  4867 consensus_queue.cc:1059] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:12.236985  4682 raft_consensus.cc:740] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:12.237360  4664 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:12.237819  4867 raft_consensus.cc:3055] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:12.238195  4867 raft_consensus.cc:740] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:12.236439  4814 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:12.239001  4865 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:12.239850  4865 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:12.241001  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2188) took 2209 ms. Trace:
I20260709 13:03:12.239245  4738 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:12.251309  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 82647.5us
I20260709 13:03:12.278402  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:12.281090  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:12.282377  2347 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:03:12.173156  4671 consensus_queue.cc:260] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:12.283757  1686 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:03:12.280030  4758 consensus_meta.cc:287] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.995s	user 0.016s	sys 0.004s
I20260709 13:03:12.317296  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:12.323107  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2178) took 1007 ms. Trace:
I20260709 13:03:12.287511  4703 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.224468  4852 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:12.238890  4521 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:12.335960  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:12.336961  1570 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:12.323635  1715 rpcz_store.cc:276] 0709 13:03:11.315321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.319117 (+  3796us) service_pool.cc:224] Handling call
0709 13:03:11.319575 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.124733 (+805158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.322835 (+198102us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b5400172854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:12.323025 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1716853120}
I20260709 13:03:12.241484  1645 rpcz_store.cc:276] 0709 13:03:10.031810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.357797 (+1325987us) service_pool.cc:224] Handling call
0709 13:03:11.358477 (+   680us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:11.387274 (+ 28797us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.387315 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:11.454018 (+ 66703us) log.cc:844] Serialized 26 byte log entry
0709 13:03:11.790833 (+336815us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:11.829943 (+ 39110us) log.cc:844] Serialized 14 byte log entry
0709 13:03:11.836567 (+  6624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.836766 (+   199us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:12.240769 (+404003us) raft_consensus.cc:1685] finished
0709 13:03:12.240792 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:12.240940 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31468,"prepare.queue_time_us":103233,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":32540,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":21801,"thread_start_us":304792,"threads_started":1,"wal-append.queue_time_us":534504}
I20260709 13:03:12.319531  4708 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:12.341234  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:12.352481  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.354223  4703 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:12.355623  1722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:35480
I20260709 13:03:12.356424  1722 raft_consensus.cc:606] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Received request to transfer leadership
I20260709 13:03:12.357105  4703 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:12.357942  4872 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:12.358599  4872 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:12.359330  1538 leader_election.cc:336] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.751s (SENT)
I20260709 13:03:12.360713  1538 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:12.354311  4682 consensus_queue.cc:260] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:12.342365  4708 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.362399  4708 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:12.364879  4872 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:12.365622  4872 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.366374  4872 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:12.387476  1570 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:03:12.384300  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:12.398782  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 57893.9us
I20260709 13:03:12.390479  4774 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:12.400025  4682 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:12.418761  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.424628  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:12.424830  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:12.401531  4524 consensus_meta.cc:287] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.561s	user 0.005s	sys 0.009s
I20260709 13:03:12.325515  4767 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:12.392000  4748 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.432885  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:12.426951  4703 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:12.450727  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:12.451073  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:12.457525  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:12.462953  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:12.463054  1646 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:12.342295  4798 consensus_queue.cc:1059] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 13:03:12.440182  4671 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:12.467778  4874 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:12.469362  4852 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:12.467860  4647 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:12.470726  2314 leader_election.cc:336] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.132s (SENT)
I20260709 13:03:12.471637  4872 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:12.472347  4876 consensus_queue.cc:1059] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:03:12.460990  1641 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:12.475620  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:12.451951  4707 consensus_meta.cc:287] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.003s	user 0.019s	sys 0.003s
I20260709 13:03:12.406108  4774 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:12.495747  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 136335us
I20260709 13:03:12.487906  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:12.496416  2314 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:12.443998  4524 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:12.486595  4684 consensus_meta.cc:287] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.067s	user 0.010s	sys 0.008s
I20260709 13:03:12.508957  1644 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:12.510542  1564 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 12.
I20260709 13:03:12.519577  4861 raft_consensus.cc:3055] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:12.521931  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:12.522403  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:12.527559  2347 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:12.521919  2313 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:12.531930  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:12.505191  4665 consensus_meta.cc:287] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.646s	user 0.013s	sys 0.005s
I20260709 13:03:12.464193  4748 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:12.538141  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:12.540259  1640 raft_consensus.cc:2410] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:12.540723  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:12.545985  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:12.502293  4852 consensus_queue.cc:1059] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:03:12.547292  2342 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:12.549533  4876 consensus_queue.cc:1243] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:03:12.541397  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:12.538761  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 94217.4us
I20260709 13:03:12.525303  4690 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.567134  1642 raft_consensus.cc:2393] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:12.567818  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:12.570043  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 131576us
I20260709 13:03:12.572160  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:12.449550  4871 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.210s
I20260709 13:03:12.578379  1713 raft_consensus.cc:3055] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:12.527184  4861 raft_consensus.cc:740] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:12.572878  2313 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:12.579139  1713 raft_consensus.cc:740] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:12.597097  2347 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:12.500914  4647 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:12.606638  4647 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:12.610131  1716 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:12.612325  4672 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.613749  4672 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:12.610127  4878 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:12.615484  1639 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:12.617322  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:12.583770  4690 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:12.620493  4524 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:12.620652  4690 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:12.622360  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:12.622598  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2194) took 2584 ms. Trace:
I20260709 13:03:12.623533  1643 rpcz_store.cc:276] 0709 13:03:10.038143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.322802 (+1284659us) service_pool.cc:224] Handling call
0709 13:03:11.323316 (+   514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:11.408720 (+ 85404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.408887 (+   167us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:11.409773 (+   886us) log.cc:844] Serialized 26 byte log entry
0709 13:03:11.624946 (+215173us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:12.219072 (+594126us) spinlock_profiling.cc:243] Waited 7.9 ms on lock 0x7b18003e4580. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f824c87d3c7 00007f8248e8f9a7 00007f8248e9f0bf
0709 13:03:12.219216 (+   144us) log.cc:844] Serialized 14 byte log entry
0709 13:03:12.219433 (+   217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.219506 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:12.620219 (+400713us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b18003e4580. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:03:12.622377 (+  2158us) raft_consensus.cc:1685] finished
0709 13:03:12.622401 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:12.622535 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89812,"prepare.queue_time_us":216847,"prepare.run_cpu_time_us":136,"prepare.run_wall_time_us":2518,"raft.queue_time_us":104041,"raft.run_cpu_time_us":426,"raft.run_wall_time_us":43895,"spinlock_wait_cycles":69883520,"thread_start_us":433839,"threads_started":2,"wal-append.queue_time_us":482366}
I20260709 13:03:12.625748  1568 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:12.626212  4758 raft_consensus.cc:993] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: : Instructing follower cf2f71592be8465fb18e36e15d0b2f89 to start an election
I20260709 13:03:12.627424  1568 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:12.627545  4708 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:12.628144  4867 consensus_queue.cc:260] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:12.629372  1717 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 13: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:12.502007  4874 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:12.613560  4794 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:12.630419  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:12.634222  4774 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:12.670094  1568 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:12.673197  4865 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:12.676631  4708 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:12.630785  4705 raft_consensus.cc:3055] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:03:12.680311  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 107540us
I20260709 13:03:12.682168  4767 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:12.682302  4863 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:03:12.632308  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 161612us
I20260709 13:03:12.553736  4665 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:12.643381  1719 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:03:12.625238  4521 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:12.759404  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 43602.2us
I20260709 13:03:12.629583  4867 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:12.738790  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:12.775648  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:12.778537  1719 raft_consensus.cc:606] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 8 LEADER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:12.696713  1646 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:12.698228  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:12.690284  1644 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:12.665944  4524 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.783910  2313 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:12.690424  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 188891us
I20260709 13:03:12.779254  1570 raft_consensus.cc:2393] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:12.791766  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:12.794854  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:12.796298  1642 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 9: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:12.800951  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:12.801733  2347 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:12.807613  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1711) took 1064 ms. Trace:
I20260709 13:03:12.808140  1567 rpcz_store.cc:276] 0709 13:03:11.743493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.197617 (+454124us) service_pool.cc:224] Handling call
0709 13:03:12.218334 (+ 20717us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.323018 (+104684us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:12.460128 (+137110us) log.cc:844] Serialized 14 byte log entry
0709 13:03:12.807235 (+347107us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:12.807291 (+    56us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:12.807371 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.807437 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:12.807553 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":235347,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"thread_start_us":387264,"threads_started":2}
I20260709 13:03:12.779588  4705 raft_consensus.cc:740] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:03:12.800171  4742 consensus_meta.cc:287] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.853s	user 0.010s	sys 0.006s
I20260709 13:03:12.825462  1685 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:12.836197  2347 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:03:12.837596  1686 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:12.839155  4647 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:12.838122  4878 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:12.840021  4647 raft_consensus.cc:697] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:12.840225  4878 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:12.794495  4524 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:12.841025  4647 consensus_queue.cc:237] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:12.842926  4524 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:12.842695  4705 consensus_queue.cc:260] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:12.795286  4861 consensus_queue.cc:260] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:12.845698  4703 consensus_queue.cc:1059] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:12.846741  4767 raft_consensus.cc:3055] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:12.846876  4861 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:12.847235  4767 raft_consensus.cc:740] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:12.803792  4804 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:12.860378  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:12.830921  4742 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:12.874326  1644 raft_consensus.cc:2410] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:12.859860  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:12.860252  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:12.881973  1647 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:12.859498  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:12.883890  1646 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
W20260709 13:03:12.884713  1538 leader_election.cc:336] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.777s (SENT)
I20260709 13:03:12.877177  1640 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:12.862084  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:12.862445  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:12.902972  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:12.905139  1643 raft_consensus.cc:3055] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:12.905737  1643 raft_consensus.cc:740] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:12.908430  1641 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:12.904155  1644 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:12.866004  4748 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:12.872117  4665 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:12.848392  4767 consensus_queue.cc:260] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:12.876922  4742 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:12.839051  4671 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:12.848389  4703 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.161s
I20260709 13:03:13.007462  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:13.073421  1644 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
W20260709 13:03:13.074566  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 248552us
W20260709 13:03:13.020407  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.032274  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:13.081038  1640 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:13.040486  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:13.078374  1644 pending_rounds.cc:85] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 4
I20260709 13:03:13.094363  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:13.096177  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:13.097002  1645 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:13.053874  4881 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.098081  4881 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:13.098256  1643 consensus_queue.cc:260] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:13.099776  4742 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:13.100108  1643 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:13.100551  4742 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:13.110935  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:13.112166  1646 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:13.116103  2347 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
W20260709 13:03:13.117348  1538 leader_election.cc:336] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.859s (SENT)
I20260709 13:03:13.092267  1644 pending_rounds.cc:107] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:03:13.132205  1644 raft_consensus.cc:2889] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:13.133430  1639 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:13.134833  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:13.027808  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 179046us
I20260709 13:03:13.137032  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:13.101327  4881 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:13.077216  4671 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.139430  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:13.149911  1640 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:13.146955  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:13.144883  1641 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:13.126272  4865 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.031090  4878 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:13.154259  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:13.163344  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:12.914173  4521 consensus_queue.cc:1059] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.241s
I20260709 13:03:13.172284  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:13.143132  4774 consensus_queue.cc:1059] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 13:03:13.156780  4671 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:13.175635  1573 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:13.176815  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:13.174441  1646 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:13.159828  4682 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.725s	user 0.016s	sys 0.000s
I20260709 13:03:13.176573  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:13.237938  1645 raft_consensus.cc:2410] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
I20260709 13:03:13.202688  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:13.175201  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 432) took 1820 ms. Trace:
W20260709 13:03:13.204707  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1706) took 3067 ms (client timeout 2063 ms). Trace:
I20260709 13:03:13.205960  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:13.207839  4742 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:13.176692  4865 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:13.160670  4577 consensus_queue.cc:1059] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:03:13.211043  1639 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:13.217813  4888 log.cc:927] Time spent T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.072s	user 0.001s	sys 0.001s
I20260709 13:03:13.143831  4703 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:03:13.226985  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 73159.4us
W20260709 13:03:13.227941  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 148531us
W20260709 13:03:13.242329  2314 leader_election.cc:336] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.231s (SENT)
W20260709 13:03:13.240017  1716 rpcz_store.cc:269] 0709 13:03:10.137101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.137225 (+   124us) service_pool.cc:224] Handling call
0709 13:03:13.175154 (+3037929us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b5400211454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:13.204551 (+ 29397us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4772253568}
I20260709 13:03:13.242235  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:13.239923  1564 rpcz_store.cc:276] 0709 13:03:11.354205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.354458 (+   253us) service_pool.cc:224] Handling call
0709 13:03:13.174527 (+1820069us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54001b6054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:13.174904 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2656399232}
I20260709 13:03:13.246501  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:13.195953  4748 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:13.266363  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:13.267077  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:13.267560  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:13.268646  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.254431  1611 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:13.255213  2314 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:13.250384  1641 raft_consensus.cc:2410] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:03:13.259649  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:13.260684  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:13.261729  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:13.273360  1565 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:13.273594  1569 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:13.262578  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:13.263250  4865 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:13.263425  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:13.276568  1611 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:13.278024  1642 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:13.282289  1644 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:13.283303  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:13.284125  1644 raft_consensus.cc:2410] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:03:13.281502  1611 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:13.287218  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:13.290503  1645 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:13.291571  4609 raft_consensus.cc:3055] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:13.291996  4609 raft_consensus.cc:740] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:13.292091  4881 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:13.292994  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:13.280189  4742 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:13.293805  1568 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:13.294035  4742 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:13.294461  4742 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:13.294960  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:13.295743  1568 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:13.296568  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:13.297309  1568 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:13.299674  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:13.300625  1642 raft_consensus.cc:2393] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:13.302268  1566 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:13.286178  4682 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:13.303277  4682 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:13.303584  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:13.301565  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:13.308192  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.312023  1639 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:13.320719  1643 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:03:13.307184  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:13.323458  1538 leader_election.cc:336] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.324s (SENT)
I20260709 13:03:13.323985  4609 consensus_queue.cc:260] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:13.326366  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 84128.7us
I20260709 13:03:13.326956  1648 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:13.328379  1648 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:13.303651  4682 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:13.331071  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:13.333542  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:13.307497  2312 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:13.336326  1640 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:13.340656  1569 raft_consensus.cc:2410] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 13:03:13.345264  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 53313.3us
I20260709 13:03:13.321306  1643 pending_rounds.cc:85] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 7
I20260709 13:03:13.346638  1643 pending_rounds.cc:107] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:03:13.347018  1643 raft_consensus.cc:2889] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:13.355451  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 48067.4us
I20260709 13:03:13.367259  1640 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:03:13.334267  4665 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:13.378207  1642 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:13.335754  4609 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:13.364953  4881 raft_consensus.cc:2764] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:03:13.385842  4665 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:13.374722  4879 raft_consensus.cc:3055] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:13.402707  4879 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:13.401736  4665 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:13.411510  1646 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:13.422619  1571 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
W20260709 13:03:13.401595  4867 consensus_meta.cc:287] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.612s	user 0.010s	sys 0.005s
I20260709 13:03:13.431406  1568 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
W20260709 13:03:13.453293  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 103309us
I20260709 13:03:13.457731  1611 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:13.471203  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:13.475821  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 261223us
I20260709 13:03:13.422498  4871 consensus_queue.cc:1059] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:03:13.452745  4672 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:13.403823  4879 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:13.516785  1565 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
W20260709 13:03:13.517205  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 110877us
W20260709 13:03:13.521340  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 155916us
I20260709 13:03:13.627823  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:13.629210  2345 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:13.552722  1685 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:13.628665  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 107613us
W20260709 13:03:13.647599  1538 leader_election.cc:336] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.602s (SENT)
I20260709 13:03:13.648196  1538 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:13.529488  4879 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:13.642473  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:13.656816  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:13.657752  4894 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:13.653384  4748 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.671108  4665 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:13.672549  4871 raft_consensus.cc:3055] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:03:13.673583  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 44425.8us
W20260709 13:03:13.674163  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 121526us
W20260709 13:03:13.642845  4872 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
W20260709 13:03:13.886827  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.799917  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:13.720016  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1711) took 1238 ms. Trace:
I20260709 13:03:13.822358  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:13.875195  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:13.866333  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.685849  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:13.800347  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:13.684154  1645 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:13.919359  1647 rpcz_store.cc:276] 0709 13:03:12.481404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.495086 (+ 13682us) service_pool.cc:224] Handling call
0709 13:03:12.528525 (+ 33439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.883431 (+354906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.719641 (+836210us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b5400098b54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:13.719927 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":810598656}
I20260709 13:03:13.935096  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:13.805583  4880 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 1.117s	user 0.008s	sys 0.013s
W20260709 13:03:13.770183  4861 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.923s	user 0.009s	sys 0.010s
I20260709 13:03:13.895372  4884 raft_consensus.cc:3055] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:13.647231  4900 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:13.695762  4865 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:13.941494  1644 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.044s	sys 0.082s
I20260709 13:03:13.941812  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:13.898026  4705 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:13.897173  1713 consensus_queue.cc:260] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:13.822664  4521 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:13.956885  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:13.871212  4742 consensus_meta.cc:287] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.576s	user 0.012s	sys 0.003s
I20260709 13:03:13.786185  4895 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:13.822065  4874 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:13.785622  4707 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:13.958998  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:13.867604  4767 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:13.954753  4748 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:13.960044  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:13.830128  4703 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:13.960691  1644 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.045s	sys 0.082s
I20260709 13:03:13.915966  4871 raft_consensus.cc:740] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:13.993477  2342 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:03:14.008292  4707 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.008509  4874 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:14.009512  4874 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:14.010226  1720 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:03:14.010665  4707 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:13.798873  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:13.963274  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:14.011194  4908 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.011888  1719 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:13.992357  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:14.012681  1715 raft_consensus.cc:1275] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:03:14.013298  4703 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:14.014125  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.800630  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:14.015239  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1725) took 1142 ms. Trace:
I20260709 13:03:14.010457  4665 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:14.015966  1714 raft_consensus.cc:2410] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:14.006215  1717 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:03:14.017680  4703 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:13.960853  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 503223us
I20260709 13:03:13.944142  4884 raft_consensus.cc:740] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:14.021618  1611 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:14.021974  1722 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:14.022679  4665 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:14.022800  1611 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:14.023252  4665 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.023736  4665 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:14.015678  1644 rpcz_store.cc:276] 0709 13:03:12.873182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.332648 (+459466us) service_pool.cc:224] Handling call
0709 13:03:13.333080 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.333565 (+   485us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:13.366413 (+ 32848us) log.cc:844] Serialized 14 byte log entry
0709 13:03:14.014864 (+648451us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:14.014924 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:14.014988 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.015054 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.015179 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35491,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":77,"raft.queue_time_us":1900,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":50,"thread_start_us":608746,"threads_started":1}
I20260709 13:03:14.026786  4871 consensus_queue.cc:260] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.028409  4871 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:14.021268  4884 consensus_queue.cc:260] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.032352  4884 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:14.034538  4861 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:14.037353  4861 raft_consensus.cc:697] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:13.701606  4671 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.043601  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1728) took 1025 ms. Trace:
I20260709 13:03:14.035064  4521 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:14.011770  4908 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.039947  1713 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:13.960831  4900 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:14.041002  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 401354us
I20260709 13:03:14.034713  4874 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:14.044121  1639 rpcz_store.cc:276] 0709 13:03:13.018607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.365330 (+346723us) service_pool.cc:224] Handling call
0709 13:03:13.396265 (+ 30935us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.401145 (+  4880us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:13.499669 (+ 98524us) log.cc:844] Serialized 14 byte log entry
0709 13:03:14.043069 (+543400us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:14.043151 (+    82us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:14.043229 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.043325 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.043527 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8968,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":60,"thread_start_us":503107,"threads_started":2}
W20260709 13:03:14.055827  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:14.058995  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.059827  1716 raft_consensus.cc:2368] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7.
I20260709 13:03:14.065022  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:13.935426  4894 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:14.077683  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.082608  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1760) took 1891 ms. Trace:
W20260709 13:03:14.049484  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.094022  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:14.093286  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 452169us
I20260709 13:03:14.038870  4742 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.062335  4577 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.107477  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:14.083201  1572 rpcz_store.cc:276] 0709 13:03:12.191341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.209371 (+ 18030us) service_pool.cc:224] Handling call
0709 13:03:12.226416 (+ 17045us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:12.313319 (+ 86903us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:12.313357 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:12.338213 (+ 24856us) log.cc:844] Serialized 26 byte log entry
0709 13:03:12.490350 (+152137us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:12.490413 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.490512 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.804672 (+1314160us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:14.076759 (+272087us) raft_consensus.cc:1685] finished
0709 13:03:14.076795 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.082507 (+  5712us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7231,"raft.queue_time_us":47844,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":10538,"spinlock_wait_cycles":59474688,"thread_start_us":82386,"threads_started":1,"wal-append.queue_time_us":267062}
I20260709 13:03:14.063933  4900 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:14.103998  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:14.118000  1715 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:03:14.140425  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1732) took 1116 ms. Trace:
W20260709 13:03:14.145507  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.149865  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:14.151364  1716 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
W20260709 13:03:14.137665  4682 consensus_meta.cc:287] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.803s	user 0.008s	sys 0.007s
I20260709 13:03:14.155055  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:14.141242  1640 rpcz_store.cc:276] 0709 13:03:13.023753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.369391 (+345638us) service_pool.cc:224] Handling call
0709 13:03:13.399908 (+ 30517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.457337 (+ 57429us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:13.480088 (+ 22751us) log.cc:844] Serialized 14 byte log entry
0709 13:03:14.138765 (+658677us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:14.138835 (+    70us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:14.138922 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.139085 (+   163us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.140358 (+  1273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6213,"prepare.queue_time_us":22998,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":67,"raft.queue_time_us":90456,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":643459,"threads_started":2,"wal-append.queue_time_us":570483}
I20260709 13:03:14.160391  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:14.163555  1715 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:14.167914  1687 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:14.170229  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:14.178099  4748 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:14.181152  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:14.182060  4742 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:14.182399  1718 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:14.183763  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:14.184700  1718 raft_consensus.cc:2393] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:14.124984  4692 consensus_queue.cc:1175] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "066ee0109aba4465ba17fa65e14ef504" responder_term: 6 status { last_received { term: 5 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:14.186108  4908 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:14.186826  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:14.164619  4682 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.188350  4894 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:14.189544  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.174912  4524 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:14.191058  1538 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:14.194105  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:14.194810  2345 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 3.
I20260709 13:03:14.190174  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:14.149525  4906 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:14.196902  1537 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:14.189363  4682 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:14.081784  4861 consensus_queue.cc:237] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.204138  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:14.174971  4609 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:14.212028  4609 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.212523  4609 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:14.173520  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:14.222499  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:14.225302  1564 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:03:14.225950  1640 raft_consensus.cc:1275] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:14.224547  1565 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
W20260709 13:03:14.225092  1470 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47518 (request call id 95) took 17600 ms (client timeout 15000 ms). Trace:
W20260709 13:03:14.231549  1470 rpcz_store.cc:269] 0709 13:02:56.625238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.709556 (+ 84318us) service_pool.cc:224] Handling call
0709 13:02:58.052013 (+1342457us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b5400310010. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d610b2 00007f824ecdaa89 00007f824ec72da6 00007f824ec44088 00007f824ed4237e 00007f824d2032a4 00007f824d203232 00007f824d203161 00007f824d2030dc
0709 13:03:14.225002 (+16172989us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:12.169986 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:12.170895 (+   909us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:12.170955 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:03:12.170974 (+    19us) tablet.cc:662] Decoding operations
0709 13:03:12.171884 (+   910us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:12.171985 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:12.172022 (+    37us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:03:12.172876 (+   854us) tablet.cc:701] Row locks acquired
0709 13:03:12.172960 (+    84us) write_op.cc:260] PREPARE: finished
0709 13:03:12.173213 (+   253us) write_op.cc:270] Start()
0709 13:03:12.173379 (+   166us) write_op.cc:276] Timestamp: P: 1783602192173135 usec, L: 0
0709 13:03:12.173402 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:03:12.320946 (+147544us) log.cc:844] Serialized 2132 byte log entry
0709 13:03:13.424594 (+1103648us) op_driver.cc:464] REPLICATION: finished
0709 13:03:13.726199 (+301605us) write_op.cc:301] APPLY: starting
0709 13:03:13.726340 (+   141us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:13.726659 (+   319us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:13.726861 (+   202us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:13.858339 (+131478us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:13.858895 (+   556us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:03:13.858976 (+    81us) write_op.cc:312] APPLY: finished
0709 13:03:13.859453 (+   477us) log.cc:844] Serialized 47 byte log entry
0709 13:03:13.859737 (+   284us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:13.859899 (+   162us) write_op.cc:454] Released schema lock
0709 13:03:13.859996 (+    97us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:13.862410 (+  2414us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:13.862452 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4355753,"reported_tablets":12,"spinlock_wait_cycles":8663936,"child_traces":[["op",{"apply.queue_time_us":301171,"apply.run_cpu_time_us":4900,"apply.run_wall_time_us":136148,"num_ops":5,"prepare.queue_time_us":217045,"prepare.run_cpu_time_us":105480,"prepare.run_wall_time_us":656739,"raft.queue_time_us":370735,"raft.run_cpu_time_us":36448,"raft.run_wall_time_us":153615,"replication_time_us":1251000,"thread_start_us":709571,"threads_started":4,"wal-append.queue_time_us":481507}]]}
W20260709 13:03:14.238988  2314 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.663s (SENT)
W20260709 13:03:14.206530  4907 log.cc:927] Time spent T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.135s	user 0.002s	sys 0.000s
I20260709 13:03:14.154354  4577 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.246191  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:14.246750  1565 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:14.186975  4671 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:14.242091  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:14.243111  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:14.243880  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:14.262661  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 113080us
I20260709 13:03:14.269956  1639 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:14.273798  4911 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:14.273556  1471 catalog_manager.cc:5697] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 7 to 8, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 8 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:14.275695  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:14.276376  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:14.277189  4767 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:14.277866  4767 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.278334  4767 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:14.278527  4903 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:03:14.266106  4872 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:14.282352  4872 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.285524  1570 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:14.286418  4577 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:14.287438  4521 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:14.258224  4524 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:14.290894  4912 log.cc:927] Time spent T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:03:14.227146  4906 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.249047  4865 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:14.308885  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:14.300331  1471 catalog_manager.cc:5697] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:14.223780  4913 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:14.249322  4682 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:14.317166  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 145692us
I20260709 13:03:14.320880  1538 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:14.322039  1713 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:14.322077  1538 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:14.327117  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2194) took 1799 ms. Trace:
I20260709 13:03:14.327800  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:14.328658  1572 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:14.327595  1713 rpcz_store.cc:276] 0709 13:03:12.523225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.531376 (+  8151us) service_pool.cc:224] Handling call
0709 13:03:13.215493 (+684117us) spinlock_profiling.cc:243] Waited 58.5 ms on lock 0x7b5400531720. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e0306f 00007f8248e7bf60 00007f8248e7d7de 00007f8248e8927c 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2
0709 13:03:13.896079 (+680586us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b5400531720. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e02520 00007f8248e7bf70 00007f8248e7d7de 00007f8248e8927c 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2
0709 13:03:14.322585 (+426506us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":800787584}
I20260709 13:03:14.334213  1537 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:14.349505  1565 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
W20260709 13:03:14.351604  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 112642us
I20260709 13:03:14.350332  4913 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.331676  4758 raft_consensus.cc:1081] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Signalling peer cf2f71592be8465fb18e36e15d0b2f89 to start an election
I20260709 13:03:14.383750  4917 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:03:14.398125  4758 raft_consensus.cc:1084] unable to start election on peer cf2f71592be8465fb18e36e15d0b2f89: Not found: unknown peer
I20260709 13:03:14.332732  4894 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:14.398985  4894 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:14.398689  4917 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:14.399642  4894 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:14.337802  4682 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:14.412415  4682 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:14.394064  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 63072.8us
I20260709 13:03:14.419232  1538 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:14.326350  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 166063us
I20260709 13:03:14.422550  1686 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:03:14.419730  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 83542.5us
I20260709 13:03:14.424118  1687 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:03:14.343458  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 239218us
I20260709 13:03:14.417295  4908 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:14.388845  4913 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:14.342828  4919 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:14.420493  4915 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:14.342231  4521 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:14.421551  4874 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:14.350927  4871 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:14.455300  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 70151.6us
I20260709 13:03:14.456156  1611 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:14.458269  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 76872.7us
I20260709 13:03:14.420341  4748 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:03:14.459192  1611 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:14.417966  4665 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.442337  4917 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:14.467638  1685 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:03:14.481423  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 57371.9us
W20260709 13:03:14.484261  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 61812.6us
I20260709 13:03:14.486922  1685 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:14.490444  4927 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:14.491820  4927 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:14.455437  4521 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:14.455183  4915 raft_consensus.cc:697] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 6 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:14.510893  4930 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:14.510804  1687 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:14.511360  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.461395  4861 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:14.516243  2345 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:14.516243  2342 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:14.452546  4908 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.533849  2345 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:03:14.506428  4692 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:14.498445  4917 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:14.539484  1687 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:14.540730  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.457983  4871 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.509297  4867 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:14.562269  4867 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.563501  4867 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:14.544051  4917 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.564409  4917 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:14.565277  4705 consensus_meta.cc:287] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.566s	user 0.008s	sys 0.001s
I20260709 13:03:14.566885  4609 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:14.498733  4894 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.572103  4894 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:14.506929  4865 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:14.510071  4521 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:14.577733  1687 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:14.542412  1612 connection.cc:256] RPC call timeout handler was delayed by -1.6870025860000055s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1668s for a response
W20260709 13:03:14.580897  1612 connection.cc:256] RPC call timeout handler was delayed by -1.3729025860000093s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2017s for a response
I20260709 13:03:14.544215  1572 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
W20260709 13:03:14.585213  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.458132  4919 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.552171  4920 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.107s
I20260709 13:03:14.541702  4692 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.553296  4921 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:14.610954  1687 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:14.612990  4924 consensus_queue.cc:1059] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:14.613579  4758 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:14.614759  4758 raft_consensus.cc:697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:14.616281  4904 raft_consensus.cc:3055] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:14.616891  4904 raft_consensus.cc:740] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:03:14.557317  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.559399  4671 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:14.618433  1538 leader_election.cc:336] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.721s (SENT)
I20260709 13:03:14.618299  4904 consensus_queue.cc:260] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.619005  1538 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:14.560858  2342 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:14.619845  4904 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:14.623302  4647 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:14.625029  4647 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:03:14.581338  1612 connection.cc:256] RPC call timeout handler was delayed by -1.3315025859999992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20630000000000004s for a response
W20260709 13:03:14.626905  1612 connection.cc:256] RPC call timeout handler was delayed by -1.3315025859999992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20630000000000004s for a response
W20260709 13:03:14.627279  1612 connection.cc:256] RPC call timeout handler was delayed by -1.2442025859999717s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21600000000000003s for a response
W20260709 13:03:14.627821  1612 connection.cc:256] RPC call timeout handler was delayed by -1.1749025859999733s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2237s for a response
I20260709 13:03:14.628820  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:14.539162  4874 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.598770  1611 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:14.600598  1721 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:03:14.605612  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.631474  4874 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:14.634959  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:14.636665  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:14.638453  1573 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:14.638829  4874 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:14.639447  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:14.640087  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.640638  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:14.644834  1564 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:14.645102  1570 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:14.646695  1565 raft_consensus.cc:1275] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:14.646955  1570 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:14.647512  4774 consensus_queue.cc:1059] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:14.649307  4672 raft_consensus.cc:3055] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:14.524189  4861 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.649799  4672 raft_consensus.cc:740] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:14.651664  4861 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:14.579455  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet eca30b43c26f420c965ee25f475581a0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:03:14.652969  1456 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 127020us
I20260709 13:03:14.653443  4665 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:14.653833  1713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:35480
I20260709 13:03:14.654639  1713 raft_consensus.cc:606] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Received request to transfer leadership
I20260709 13:03:14.640725  4924 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.642129  4814 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:03:14.668896  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 50493.2us
W20260709 13:03:14.669420  4926 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:03:14.669654  1538 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.722s (SENT)
I20260709 13:03:14.670295  1538 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:14.670579  4862 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.580229  4865 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.651047  4672 consensus_queue.cc:260] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.655071  1713 raft_consensus.cc:614] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:14.657050  1572 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:14.655228  1570 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:03:14.658710  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:14.678740  2345 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:14.660524  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:14.680104  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:14.641587  1571 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:14.655048  4609 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:14.663753  4922 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:03:14.664937  4923 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 13:03:14.667240  4928 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.667333  4872 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:14.602391  4705 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.647104  4690 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:03:14.661316  4577 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:14.671798  4862 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:14.681282  1538 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.758s (SENT)
I20260709 13:03:14.690770  4665 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:14.616498  4758 consensus_queue.cc:237] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.702726  4931 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.703084  4705 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.703444  4931 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:14.690007  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 91358.9us
I20260709 13:03:14.685375  1685 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:14.656239  4692 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:14.707541  4900 consensus_meta.cc:287] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.570s	user 0.013s	sys 0.000s
I20260709 13:03:14.640071  1573 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:14.702590  1538 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:14.708153  4900 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.675889  4865 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:14.710177  4933 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:14.710861  4933 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:14.711573  4908 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:14.712273  4524 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:14.713126  4933 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:14.713867  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 56338.9us
I20260709 13:03:14.691203  4742 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:14.714924  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:14.680013  4911 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:14.717592  4871 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:14.719485  4906 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:14.719622  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:14.710394  4894 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:14.720979  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:14.675477  4915 consensus_queue.cc:237] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:14.690676  4672 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:14.750983  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:14.751716  1565 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:14.705416  4705 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:14.754894  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 126334us
I20260709 13:03:14.662024  4930 raft_consensus.cc:697] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:14.724062  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:14.724174  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 49806.1us
I20260709 13:03:14.683784  4921 raft_consensus.cc:697] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:14.724958  4901 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:14.746178  1685 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:14.746681  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:14.746822  4814 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:14.698067  4928 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:14.764904  4665 raft_consensus.cc:697] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:14.767797  2347 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:14.755961  4867 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.778419  4931 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:14.784793  2345 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:14.789980  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:14.791330  1570 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
I20260709 13:03:14.675896  4924 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.795297  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:14.793660  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:14.800413  1567 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:03:14.773950  4703 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:14.805275  1685 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:03:14.804172  4767 consensus_meta.cc:287] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.525s	user 0.015s	sys 0.001s
I20260709 13:03:14.810206  4705 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:14.810043  4922 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:14.810961  4705 raft_consensus.cc:697] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:14.811944  4705 consensus_queue.cc:237] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:14.812291  4928 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:14.813701  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 48207.4us
I20260709 13:03:14.815654  1612 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:14.817126  1612 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:14.817996  4921 consensus_queue.cc:237] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.818022  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:14.765190  4927 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:14.823354  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:14.822567  1570 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:14.786361  4708 consensus_queue.cc:1059] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:03:14.789387  4671 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:14.848008  2314 leader_election.cc:336] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.057s (SENT)
I20260709 13:03:14.848791  2314 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:14.775595  4917 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.792950  4609 consensus_queue.cc:1059] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:03:14.789464  4577 consensus_queue.cc:1048] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 13:03:14.847152  4690 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:14.859633  4690 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.860049  4690 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:14.822160  4933 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.859958  4665 consensus_queue.cc:237] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.860800  4933 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:14.856671  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 54107.9us
I20260709 13:03:14.826828  4928 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:14.863154  4867 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:14.871342  1569 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:14.873104  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:14.875113  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 58147.3us
I20260709 13:03:14.863391  4933 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:14.881095  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:14.881319  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:14.850101  4904 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:14.884539  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:14.891073  4917 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:14.892506  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:14.893193  1642 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:14.892974  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:14.894578  1641 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:14.894830  4939 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:14.895538  4939 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.826416  4798 consensus_queue.cc:1059] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:03:14.898402  1643 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:14.899442  1685 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:14.857889  4930 consensus_queue.cc:237] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.826586  4767 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:14.876184  1538 leader_election.cc:336] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.574s (SENT)
I20260709 13:03:14.903141  1568 raft_consensus.cc:2393] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 4: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:14.857306  4927 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:14.903474  1538 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:14.898288  4807 raft_consensus.cc:3055] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:14.917750  1570 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:14.918601  1640 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 13:03:14.918756  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:14.906805  1687 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:14.909138  2313 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:14.868309  4931 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:14.919337  1570 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:14.919562  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:14.905062  1612 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:14.918985  1640 pending_rounds.cc:85] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 4
I20260709 13:03:14.921316  1640 pending_rounds.cc:107] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 13:03:14.921706  1640 raft_consensus.cc:2889] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:14.923736  1642 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:14.902581  4671 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:14.853770  4897 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:03:14.947118  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:14.868836  4922 consensus_queue.cc:1048] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:03:14.933079  4671 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.935005  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:14.935755  1641 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:14.857767  4692 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.934729  1639 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
W20260709 13:03:14.966572  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 90437us
I20260709 13:03:14.917217  4807 raft_consensus.cc:740] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:14.983018  4807 consensus_queue.cc:260] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:14.984484  4692 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:14.928130  4874 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:14.996815  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.917819  4928 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:14.928170  4931 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:15.017511  4936 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.017973  4935 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:15.019155  4942 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:03:15.020097  4690 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:15.020572  4945 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.152s	user 0.000s	sys 0.002s
I20260709 13:03:15.021389  1570 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
W20260709 13:03:15.023476  1538 leader_election.cc:336] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.655s (SENT)
I20260709 13:03:15.026167  4924 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:15.031561  4647 raft_consensus.cc:3055] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:15.031957  4939 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:14.950157  4900 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:14.965392  4930 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:14.964035  4671 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:15.063922  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 187963us
W20260709 13:03:15.069178  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 162461us
I20260709 13:03:15.069973  4672 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:03:15.072367  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.139858  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:15.077867  1713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:57572
W20260709 13:03:15.078696  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.085292  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:15.096750  4862 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:15.097935  4690 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:15.079156  4807 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:15.097994  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:15.122116  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.121162  1611 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:15.047636  4931 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:15.138370  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:15.166967  1713 raft_consensus.cc:606] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:15.168363  1572 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:15.094861  4647 raft_consensus.cc:740] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:15.045128  4928 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:15.083247  4936 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:15.175853  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 54823.2us
W20260709 13:03:16.015939  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 876223us
I20260709 13:03:16.016269  4913 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.016858  4913 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:16.019706  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.019464  4900 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:16.019511  4913 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:15.232405  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.176510  1685 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:15.940264  4933 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:16.023221  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:16.023452  4647 consensus_queue.cc:260] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.024822  4647 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:16.025156  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:16.023059  4933 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:15.175674  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:03:16.014183  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 990747us
I20260709 13:03:16.028013  2349 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:15.596959  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:16.029074  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:15.953341  1565 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.000s	sys 0.016s
W20260709 13:03:16.029908  1565 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.002s	sys 0.016s
W20260709 13:03:15.758028  4865 consensus_meta.cc:287] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.048s	user 0.013s	sys 0.000s
I20260709 13:03:15.171489  1566 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:03:15.610893  4917 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.162876  4930 raft_consensus.cc:2764] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 13:03:16.031009  1571 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:16.030761  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:15.238528  4682 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:16.056874  1573 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:16.053005  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57572 (request call id 17) took 1156 ms. Trace:
I20260709 13:03:16.058018  1713 rpcz_store.cc:276] 0709 13:03:14.888194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.888359 (+   165us) service_pool.cc:224] Handling call
0709 13:03:16.044844 (+1156485us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:16.058522  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:16.025545  4928 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:15.175357  4908 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.032508  1611 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:16.033660  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:16.045905  4921 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.060s	user 0.011s	sys 0.044s
I20260709 13:03:15.134091  4767 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:15.653277  4521 consensus_meta.cc:287] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.051s	user 0.014s	sys 0.000s
I20260709 13:03:16.043792  1572 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:16.031724  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:16.059717  1569 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.919s	user 0.033s	sys 0.033s
I20260709 13:03:16.083985  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1766) took 1001 ms. Trace:
W20260709 13:03:16.084384  1569 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.944s	user 0.034s	sys 0.034s
I20260709 13:03:16.084417  1572 rpcz_store.cc:276] 0709 13:03:15.082988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.084987 (+  1999us) service_pool.cc:224] Handling call
0709 13:03:16.083920 (+998933us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:16.063774  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:16.017158  1612 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:16.086328  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:16.087898  1572 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:03:16.089319  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:16.090111  2312 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:16.091728  4521 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.092701  1566 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
W20260709 13:03:16.096553  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 79457.8us
I20260709 13:03:16.092161  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:16.090272  1572 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:03:16.058990  4682 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:03:16.065271  4921 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.080s	user 0.011s	sys 0.050s
I20260709 13:03:16.118192  1572 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:16.095692  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1767) took 1012 ms. Trace:
I20260709 13:03:16.050302  4917 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.122139  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1768) took 1038 ms. Trace:
I20260709 13:03:16.124032  1565 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:03:16.123834  1572 rpcz_store.cc:276] 0709 13:03:15.083537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.103537 (+1020000us) service_pool.cc:224] Handling call
0709 13:03:16.117993 (+ 14456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.121846 (+  3853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.122077 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:16.124799  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1789) took 1039 ms. Trace:
I20260709 13:03:16.125195  1565 rpcz_store.cc:276] 0709 13:03:15.084995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.123470 (+1038475us) service_pool.cc:224] Handling call
0709 13:03:16.123811 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:16.124570 (+   759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.124746 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:16.125792  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1769) took 1039 ms. Trace:
I20260709 13:03:16.126235  1572 rpcz_store.cc:276] 0709 13:03:15.086620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.124958 (+1038338us) service_pool.cc:224] Handling call
0709 13:03:16.125249 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.125416 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.125440 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:16.125482 (+    42us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:16.125519 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.125605 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.125742 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:16.127168  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1770) took 1040 ms. Trace:
I20260709 13:03:16.127575  1572 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:16.127569  1565 rpcz_store.cc:276] 0709 13:03:15.087070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.126370 (+1039300us) service_pool.cc:224] Handling call
0709 13:03:16.126614 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.126767 (+   153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.126789 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:16.126827 (+    38us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:16.126867 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.127026 (+   159us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.127123 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:16.122121  1566 rpcz_store.cc:276] 0709 13:03:15.083256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.093598 (+1010342us) service_pool.cc:224] Handling call
0709 13:03:16.094447 (+   849us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.094827 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.094875 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:16.094951 (+    76us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:16.095008 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.095244 (+   236us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.095589 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:16.129813  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1772) took 1042 ms. Trace:
I20260709 13:03:16.130458  1565 rpcz_store.cc:276] 0709 13:03:15.087732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.129034 (+1041302us) service_pool.cc:224] Handling call
0709 13:03:16.129286 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.129511 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.129528 (+    17us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:16.129554 (+    26us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:16.129573 (+    19us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.129605 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.129762 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:16.051855  4865 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:16.086994  1611 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:16.133725  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1771) took 1046 ms. Trace:
I20260709 13:03:16.132061  1565 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:16.138352  1572 rpcz_store.cc:276] 0709 13:03:15.087396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.127126 (+1039730us) service_pool.cc:224] Handling call
0709 13:03:16.127373 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.128108 (+   735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.133671 (+  5563us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:16.139006  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1795) took 1034 ms. Trace:
I20260709 13:03:16.140175  1565 rpcz_store.cc:276] 0709 13:03:15.105001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.131446 (+1026445us) service_pool.cc:224] Handling call
0709 13:03:16.131707 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.138564 (+  6857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.138916 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:16.121577  4939 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:16.140534  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 460) took 1020 ms. Trace:
I20260709 13:03:16.174746  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1773) took 1086 ms. Trace:
I20260709 13:03:16.174536  1572 rpcz_store.cc:276] 0709 13:03:15.120578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.139494 (+1018916us) service_pool.cc:224] Handling call
0709 13:03:16.139851 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.140092 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.140129 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:16.140186 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:16.140229 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.140314 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.140474 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:16.175550  1566 rpcz_store.cc:276] 0709 13:03:15.088052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.129700 (+1041648us) service_pool.cc:224] Handling call
0709 13:03:16.143405 (+ 13705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.143590 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.143611 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:16.174411 (+ 30800us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:16.174462 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.174574 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.174689 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:16.176369  2349 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
W20260709 13:03:16.140976  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 54123.7us
I20260709 13:03:16.117129  4904 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:16.081374  4928 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:16.122313  4682 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:16.160480  1647 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: [4.6-8.8]
I20260709 13:03:16.130089  4935 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:16.170822  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.171579  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:16.173804  2314 leader_election.cc:336] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.515s (SENT)
I20260709 13:03:16.177016  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55890 (request call id 339) took 1002 ms. Trace:
I20260709 13:03:16.177855  1611 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:16.151199  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 121827us
I20260709 13:03:16.183399  2314 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:16.083109  4908 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.183641  2349 rpcz_store.cc:276] 0709 13:03:15.175038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.175256 (+   218us) service_pool.cc:224] Handling call
0709 13:03:16.176958 (+1001702us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:16.143765  4665 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:16.185750  1641 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 6: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:16.188424  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.196163  1571 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:16.199383  4917 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:16.199611  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1764) took 1289 ms. Trace:
I20260709 13:03:16.196879  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1792) took 1093 ms. Trace:
I20260709 13:03:16.201727  1646 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:03:16.201998  1571 rpcz_store.cc:276] 0709 13:03:15.103818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.024876 (+921058us) service_pool.cc:224] Handling call
0709 13:03:16.196796 (+171920us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:16.205662  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.199990  1564 rpcz_store.cc:276] 0709 13:03:14.910154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.022936 (+112782us) service_pool.cc:224] Handling call
0709 13:03:15.023330 (+   394us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:15.038407 (+ 15077us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.038457 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:15.054632 (+ 16175us) log.cc:844] Serialized 52 byte log entry
0709 13:03:15.282456 (+227824us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:15.282520 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.282621 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:16.199333 (+916712us) raft_consensus.cc:1685] finished
0709 13:03:16.199360 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.199538 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15633,"raft.queue_time_us":20625,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":3982,"thread_start_us":224835,"threads_started":1,"wal-append.queue_time_us":1029367}
W20260709 13:03:16.206804  1538 leader_election.cc:336] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.644s (SENT)
I20260709 13:03:16.205340  4904 raft_consensus.cc:697] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:16.180828  4935 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.209131  1538 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:16.209813  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.187335  4928 raft_consensus.cc:697] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:16.192864  4665 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.217886  1645 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:16.204510  4939 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:16.213312  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.219537  4767 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:16.201210  4748 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:16.221979  4665 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:16.227479  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.229087  1646 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:03:16.214442  4935 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:16.231043  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.235677  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.236652  4708 raft_consensus.cc:3055] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:16.237162  4708 raft_consensus.cc:740] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:16.238173  4933 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:16.238174  4708 consensus_queue.cc:260] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.239106  4933 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:16.240058  4708 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:16.229676  4671 consensus_meta.cc:287] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.090s	user 0.012s	sys 0.005s
I20260709 13:03:16.223068  4939 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:16.245829  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.247845  4895 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:16.248085  4919 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:16.249828  1646 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:03:16.251528  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.252321  1647 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:03:16.253858  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.254874  1646 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:03:16.244961  4908 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:16.231197  4748 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.253561  4904 consensus_queue.cc:237] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.247138  4939 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:16.232791  4767 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:16.260946  4521 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:16.273026  4928 consensus_queue.cc:237] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.246762  4671 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:16.270660  4807 consensus_meta.cc:287] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.046s	user 0.010s	sys 0.005s
W20260709 13:03:16.316519  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 109746us
W20260709 13:03:16.321619  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 287212us
I20260709 13:03:16.310230  4865 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:16.330260  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 152123us
I20260709 13:03:16.343719  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1776) took 1170 ms. Trace:
I20260709 13:03:16.348965  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:16.358477  1537 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:16.359277  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:16.355509  4895 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:16.344420  1565 rpcz_store.cc:276] 0709 13:03:15.172941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.144044 (+971103us) service_pool.cc:224] Handling call
0709 13:03:16.144368 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:16.160630 (+ 16262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.169624 (+  8994us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:16.180901 (+ 11277us) log.cc:844] Serialized 26 byte log entry
0709 13:03:16.334937 (+154036us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:16.334996 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.335117 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:16.343371 (+  8254us) raft_consensus.cc:1685] finished
0709 13:03:16.343402 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.343628 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1824,"raft.queue_time_us":8649,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":8590,"thread_start_us":122668,"threads_started":1,"wal-append.queue_time_us":134480}
I20260709 13:03:16.368383  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:16.367650  4970 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:16.355985  4955 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:03:16.377853  4970 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:16.396281  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 218373us
I20260709 13:03:16.374106  4895 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.392074  4973 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:16.341320  4807 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.384488  2345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55890 (request call id 338) took 1326 ms. Trace:
W20260709 13:03:16.397279  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 1.22094e+06us
I20260709 13:03:16.361660  4690 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.342947  4962 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:16.398227  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 354977us
I20260709 13:03:16.357913  4521 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:16.396735  1538 leader_election.cc:336] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.872s (SENT)
I20260709 13:03:16.417805  1538 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:16.410029  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 236275us
W20260709 13:03:16.410374  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet eca30b43c26f420c965ee25f475581a0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:03:16.407840  2345 rpcz_store.cc:276] 0709 13:03:15.058650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.058820 (+   170us) service_pool.cc:224] Handling call
0709 13:03:15.059284 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.115740 (+ 56456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.115800 (+    60us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:15.115874 (+    74us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:15.115931 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.384123 (+1268192us) spinlock_profiling.cc:243] Waited 56.1 ms on lock 0x7b5400549054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:16.384191 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.384401 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":128950144}
W20260709 13:03:16.411846  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 89398.8us
I20260709 13:03:16.420915  4919 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:16.421846  4919 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.424501  4919 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:16.425045  4919 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.424721  2314 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:16.426005  4972 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:16.426859  1612 connection.cc:256] RPC call timeout handler was delayed by -1.5654996979999964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1589s for a response
W20260709 13:03:16.427562  1612 connection.cc:256] RPC call timeout handler was delayed by -1.5537996979999775s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1602s for a response
W20260709 13:03:16.427942  1612 connection.cc:256] RPC call timeout handler was delayed by -1.5483996979999688s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16080000000000003s for a response
W20260709 13:03:16.428278  1612 connection.cc:256] RPC call timeout handler was delayed by -1.5411996979999572s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16160000000000002s for a response
W20260709 13:03:16.428574  1612 connection.cc:256] RPC call timeout handler was delayed by -1.5051996979999558s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1656s for a response
W20260709 13:03:16.428910  1612 connection.cc:256] RPC call timeout handler was delayed by -1.4097996979999721s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17620000000000002s for a response
W20260709 13:03:16.429301  1612 connection.cc:256] RPC call timeout handler was delayed by -1.3971996979999517s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1776s for a response
W20260709 13:03:16.429738  1612 connection.cc:256] RPC call timeout handler was delayed by -1.3161996979999913s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18660000000000002s for a response
W20260709 13:03:16.430112  1612 connection.cc:256] RPC call timeout handler was delayed by -1.2936996979999549s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18910000000000002s for a response
W20260709 13:03:16.430495  1612 connection.cc:256] RPC call timeout handler was delayed by -1.2927996979999534s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1892s for a response
W20260709 13:03:16.430840  1612 connection.cc:256] RPC call timeout handler was delayed by -1.1496996980000063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20510000000000003s for a response
I20260709 13:03:16.407512  1611 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:16.360687  4904 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:16.431392  1612 connection.cc:256] RPC call timeout handler was delayed by -1.0848996979999583s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21230000000000004s for a response
W20260709 13:03:16.432582  1612 connection.cc:256] RPC call timeout handler was delayed by -1.0641996979999817s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2146s for a response
I20260709 13:03:16.433174  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:16.435212  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.443480  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.444558  1639 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:16.435938  4930 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:16.445936  4879 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:16.434314  4919 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.426661  4977 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:16.447634  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:16.448684  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 52029.7us
W20260709 13:03:16.454473  1538 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.931s (SENT)
I20260709 13:03:16.455153  1538 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:16.450819  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:16.451822  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1761) took 1544 ms. Trace:
W20260709 13:03:16.456246  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 97864.6us
I20260709 13:03:16.426690  4972 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:16.456437  1567 rpcz_store.cc:276] 0709 13:03:14.906981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.917386 (+ 10405us) service_pool.cc:224] Handling call
0709 13:03:14.920237 (+  2851us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:14.931333 (+ 11096us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:14.931383 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:14.932037 (+   654us) log.cc:844] Serialized 26 byte log entry
0709 13:03:15.136483 (+204446us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:15.136540 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.136641 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:16.451589 (+1314948us) raft_consensus.cc:1685] finished
0709 13:03:16.451612 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.451762 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":186,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":24626,"thread_start_us":80227,"threads_started":1,"wal-append.queue_time_us":1147706}
W20260709 13:03:16.458765  1568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 449) took 2176 ms (client timeout 1748 ms). Trace:
I20260709 13:03:16.446486  4879 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:16.460564  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:16.475279  1568 rpcz_store.cc:269] 0709 13:03:14.275144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.275290 (+   146us) service_pool.cc:224] Handling call
0709 13:03:16.451792 (+2176502us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b5400012054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:16.451920 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1440381824}
I20260709 13:03:16.431413  4973 raft_consensus.cc:697] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:16.431887  4904 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.415637  4900 consensus_queue.cc:1059] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:03:16.477015  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:16.477779  1642 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:16.442380  4917 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:16.479313  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:16.480064  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.447981  4964 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:16.490674  1645 raft_consensus.cc:2393] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 6: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:16.434139  4521 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:16.480929  1538 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.943s (SENT)
I20260709 13:03:16.429040  4908 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:03:16.497910  4917 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.499305  4908 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:16.499941  4908 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.500669  4917 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:16.416272  4807 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.502784  4970 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:16.503360  4807 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:16.429418  4690 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.503980  4936 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:16.504631  4895 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:16.505831  4690 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:16.505852  4935 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:16.508503  4671 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:16.432545  4962 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.478075  4748 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:16.511116  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.511865  1645 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:16.513151  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.481930  4879 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:16.515767  4874 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.478s	user 0.017s	sys 0.000s
I20260709 13:03:16.515928  4521 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:16.516577  4874 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:16.516737  4962 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:16.517393  1573 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:16.455971  4965 consensus_queue.cc:1059] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 13:03:16.518458  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.519181  1645 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:16.519310  4965 raft_consensus.cc:3055] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:16.519606  4935 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:16.520141  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.519665  4965 raft_consensus.cc:740] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:16.526508  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.526242  1639 raft_consensus.cc:2376] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6. Re-sending same reply.
I20260709 13:03:16.528026  1538 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:16.530717  1573 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:16.534111  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.534919  1645 raft_consensus.cc:2410] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:03:16.535862  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.536805  1642 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:16.539700  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.541366  1642 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:16.540468  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.542488  1645 raft_consensus.cc:2393] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 6: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:16.545374  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.540189  4908 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:16.554984  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.557364  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:16.558086  1643 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:16.548146  4930 raft_consensus.cc:697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:16.562132  4930 consensus_queue.cc:237] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.546536  4874 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:03:16.564085  4957 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.473s	user 0.005s	sys 0.000s
I20260709 13:03:16.568454  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.562891  4682 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.570613  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1751) took 1670 ms. Trace:
I20260709 13:03:16.571136  1648 rpcz_store.cc:276] 0709 13:03:14.899902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.917318 (+ 17416us) service_pool.cc:224] Handling call
0709 13:03:14.957079 (+ 39761us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:15.056116 (+ 99037us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.056161 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:15.058367 (+  2206us) log.cc:844] Serialized 26 byte log entry
0709 13:03:15.131781 (+ 73414us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:15.132903 (+  1122us) log.cc:844] Serialized 14 byte log entry
0709 13:03:15.133285 (+   382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.133374 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:16.563432 (+1430058us) spinlock_profiling.cc:243] Waited 3.9 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:16.570296 (+  6864us) raft_consensus.cc:1685] finished
0709 13:03:16.570398 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.570547 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4880,"prepare.queue_time_us":343,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":119269,"raft.run_cpu_time_us":87506,"raft.run_wall_time_us":888644,"spinlock_wait_cycles":8965504,"thread_start_us":30368,"threads_started":1,"wal-append.queue_time_us":964128}
I20260709 13:03:16.575537  4682 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:16.576253  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:16.576989  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 96097us
W20260709 13:03:16.590382  1538 leader_election.cc:336] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.949s (SENT)
I20260709 13:03:16.590847  1538 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:16.591823  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:16.592729  1645 raft_consensus.cc:2376] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6. Re-sending same reply.
I20260709 13:03:16.577934  4973 consensus_queue.cc:237] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:16.582656  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.601068  1639 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
W20260709 13:03:16.584077  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.550307  4708 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:16.603359  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:16.604663  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:16.605931  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:16.607856  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.587990  1643 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
W20260709 13:03:16.611101  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 181533us
I20260709 13:03:16.604219  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:16.613534  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.612390  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:16.608618  1648 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
W20260709 13:03:16.562157  4931 consensus_meta.cc:287] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.517s	user 0.009s	sys 0.004s
I20260709 13:03:16.527408  4965 consensus_queue.cc:260] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.618362  1720 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 8: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:16.694471  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:16.765902  4964 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 6 LEADER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:16.691777  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:16.705466  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 243905us
W20260709 13:03:16.714957  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 124614us
I20260709 13:03:16.770381  4965 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:16.770920  1538 leader_election.cc:336] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.966s (SENT)
I20260709 13:03:16.771621  1538 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:16.690171  4665 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.774382  4915 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:16.770388  1719 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:03:16.775435  1647 raft_consensus.cc:2368] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 5. Current term is 6.
I20260709 13:03:16.776582  4879 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:16.725524  2313 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:16.777493  1538 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.547s (SENT)
I20260709 13:03:16.643220  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:16.619225  4904 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:16.691849  4748 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.692086  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:16.645134  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:16.802943  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.803223  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:16.823766  1640 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:16.786134  1573 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1793) took 1682 ms (client timeout 2158 ms). Trace:
W20260709 13:03:16.824395  1573 rpcz_store.cc:269] 0709 13:03:15.104143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.058210 (+954067us) service_pool.cc:224] Handling call
0709 13:03:16.785973 (+727763us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b5400305a54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:16.786074 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1052762880}
I20260709 13:03:16.600914  4927 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:16.803023  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.689986   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:16.808779  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:16.803678  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.810315  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:16.809374  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:16.828065  1568 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:16.828421  1567 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:16.811088  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:16.815681  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:16.828981  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:16.823311  1718 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 6: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:16.823829  1717 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:16.799352  1538 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:16.825805  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:16.645802  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:16.821026  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 95630.4us
I20260709 13:03:16.837230  1573 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:16.839320  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:16.839874  1645 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 9: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:16.840519  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:16.841113  1567 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:16.829166  1572 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
W20260709 13:03:16.786947  4959 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.590s	user 0.000s	sys 0.003s
I20260709 13:03:16.845429  1566 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:16.788379  4961 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.666s	user 0.002s	sys 0.000s
I20260709 13:03:16.847509  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:16.860919  1568 raft_consensus.cc:2393] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 6: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:16.863117  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:16.606868  4966 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:16.852944  1611 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:16.866580  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.618853  4874 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:16.931289  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:16.789532  4879 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:16.780017  4665 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:16.693122  4521 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:16.646232  4935 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:16.999815  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:16.948040  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:16.826632  4936 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:16.780179  4767 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:16.933727  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 72177.2us
I20260709 13:03:16.997786  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:16.932369  1611 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:16.785405  4972 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:16.931350  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:16.989585  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:17.028095  2340 raft_consensus.cc:2410] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:03:16.967046  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:17.001478  1572 raft_consensus.cc:1275] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:03:17.002722  2349 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:03:17.029167  1645 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:16.618212  4708 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.826994  4915 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.781162  4931 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:17.019068  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:16.838330  4748 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.955808  4900 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:16.845924  4862 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:17.049109  1612 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:16.795123  4939 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:16.856175  4927 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:17.008890  4911 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:17.061702  4708 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:17.064831  4939 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:17.068768  4862 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:17.072638  4939 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:17.073894  1643 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:17.018442  4874 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:17.048667  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:17.032769  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 100478us
I20260709 13:03:17.055821  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:17.087234  1715 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:17.046525  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:17.093962  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 44879.5us
I20260709 13:03:17.116237  1611 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:17.119354  1612 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:17.137804  1644 raft_consensus.cc:2393] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 6: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:17.139760  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:17.140740  1645 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:17.165585  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:17.167371  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:17.168605  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:17.169339  1715 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:17.169871  4748 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:17.170329  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:17.170840  4972 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:17.171087  1715 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:03:17.172266  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:17.172672  1571 raft_consensus.cc:1275] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:17.174047  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:17.156070  2312 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.001s	sys 0.022s
I20260709 13:03:17.177567  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:17.178052  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:17.179533  1715 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 8: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:03:17.011276  4894 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.140s	user 0.035s	sys 0.092s
I20260709 13:03:17.176158  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:17.176209  2312 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.002s	sys 0.022s
W20260709 13:03:17.218116  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 855255us
I20260709 13:03:17.173367  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:17.046772  4665 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:17.233546  4665 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:17.234193  4665 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:17.163556  4928 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.004s	sys 0.020s
W20260709 13:03:17.237350  4928 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.004s	sys 0.020s
I20260709 13:03:17.180344  4874 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:17.241925  4874 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:17.241971  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:17.173308  4862 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:17.180898  1641 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:03:17.051057  4915 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:17.248842  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:17.249594  1645 raft_consensus.cc:2376] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9. Re-sending same reply.
I20260709 13:03:16.966471  4966 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:17.250945  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:17.251771  1645 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:17.010215  4521 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:17.252915  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:17.254606  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:17.259045  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1762) took 2349 ms. Trace:
I20260709 13:03:17.262493  1569 rpcz_store.cc:276] 0709 13:03:14.909455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.977596 (+ 68141us) service_pool.cc:224] Handling call
0709 13:03:15.059425 (+ 81829us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:15.083772 (+ 24347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.083819 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:15.135659 (+ 51840us) log.cc:844] Serialized 52 byte log entry
0709 13:03:16.085197 (+949538us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:16.355431 (+270234us) log.cc:844] Serialized 14 byte log entry
0709 13:03:16.355636 (+   205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.355721 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:16.786138 (+430417us) spinlock_profiling.cc:243] Waited 3.28 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:17.194987 (+408849us) raft_consensus.cc:1685] finished
0709 13:03:17.258559 (+ 63572us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:17.258931 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":270108,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":34113,"raft.run_cpu_time_us":510,"raft.run_wall_time_us":45526,"spinlock_wait_cycles":7543680,"thread_start_us":1176572,"threads_started":2,"wal-append.queue_time_us":1003659}
I20260709 13:03:17.263008  1645 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
W20260709 13:03:17.266824  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 150641us
I20260709 13:03:17.274896  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:17.262467  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 485001us
W20260709 13:03:17.279436  1538 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.998s (SENT) [suppressed 41 similar messages]
I20260709 13:03:17.019663  4935 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:17.231581  1714 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
W20260709 13:03:17.200824  4894 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.333s	user 0.035s	sys 0.092s
I20260709 13:03:17.177781  2342 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:17.289018  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:17.288308  4874 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:17.288662  4975 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.195919  4900 raft_consensus.cc:697] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:17.299621  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 177252us
W20260709 13:03:17.302099  1719 consensus_meta.cc:287] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.518s	user 0.007s	sys 0.003s
I20260709 13:03:17.311434  1719 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:17.312634  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:17.313431  1722 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:17.313491  1719 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:17.315371  1566 raft_consensus.cc:1275] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:03:17.315814  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 359804us
I20260709 13:03:17.268043  4966 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:17.285274  4939 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:17.321902  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:17.317214  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:17.280743  1538 leader_election.cc:336] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.001s (SENT)
I20260709 13:03:17.332325  1538 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:17.309650  1640 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:17.335975  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:17.336872  1640 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:17.345484  1714 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:17.346457  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:17.347429  1639 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:17.348848  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:17.300294  4928 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:17.365136  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:17.303423  4874 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:17.466049  1639 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:17.467163  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:17.444662  4900 consensus_queue.cc:237] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:17.421150  2349 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
W20260709 13:03:17.479362  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 198687us
W20260709 13:03:17.480898  1538 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.138s (SENT)
W20260709 13:03:17.482133  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 171006us
I20260709 13:03:17.483317  1612 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:17.386324  4939 raft_consensus.cc:697] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:17.484683  1612 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:17.481526  1538 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:17.491374  1612 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:17.326074  4927 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:17.497658  4962 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.498418  4904 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.499617  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:17.398315  4862 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.317173  4975 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:17.482362  1639 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:17.497804  4967 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.501911  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:17.506042  1640 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
W20260709 13:03:17.500833  1538 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.197s (SENT)
I20260709 13:03:17.508776  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:17.518440  1714 raft_consensus.cc:2368] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7.
I20260709 13:03:17.397006  4915 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:17.515061  1685 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:17.517300  1538 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:17.529631  4966 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:17.534893  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:17.535745  1640 raft_consensus.cc:2376] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6. Re-sending same reply.
I20260709 13:03:17.537460  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:17.507071  4908 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:17.499419  4972 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:17.498602  4962 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:17.514766  4928 raft_consensus.cc:2764] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:03:17.523286  4862 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:17.505759  4913 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:17.550840  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:17.546381  4998 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:03:17.557525  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:17.562142  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 61367.1us
I20260709 13:03:17.547547  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:17.547098  4577 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:03:17.563450  1538 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.220s (SENT)
W20260709 13:03:17.562374  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 71464.2us
I20260709 13:03:17.498968  4904 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:17.485450  4939 consensus_queue.cc:237] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:17.571956  1538 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:17.512148  4983 consensus_queue.cc:1059] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
W20260709 13:03:17.558408  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 48547.5us
W20260709 13:03:17.572978  1567 consensus_meta.cc:287] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.713s	user 0.012s	sys 0.004s
I20260709 13:03:17.589309  1567 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:17.553045  4966 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:17.594934  4966 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:17.596509  4862 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:17.600450  4807 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:03:17.603143  4996 log.cc:927] Time spent T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:03:17.595279  4935 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:17.602864  4807 consensus_queue.cc:1059] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:17.604363  4874 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:17.543037  4967 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:17.556888  4915 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:17.604444  4807 consensus_queue.cc:1243] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:17.588591  4577 raft_consensus.cc:697] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:17.607282  4967 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:17.607411  4904 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:17.608474  4966 raft_consensus.cc:3055] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:17.609441  4966 raft_consensus.cc:740] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:17.589766  1612 leader_election.cc:400] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7.
I20260709 13:03:17.610491  4966 consensus_queue.cc:260] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:17.612769  1640 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:17.616633  1612 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:17.625865  1612 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:17.627009  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 63597.8us
W20260709 13:03:17.628469  1538 leader_election.cc:336] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.233s (SENT)
I20260709 13:03:17.629292  1538 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:17.571873  4931 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:17.631302  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45215.6us
W20260709 13:03:17.651830  1538 leader_election.cc:336] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.243s (SENT)
I20260709 13:03:17.652872  1538 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:17.602126  4989 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:17.634709  5004 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:17.657125  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:17.656853  1717 consensus_meta.cc:287] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.819s	user 0.009s	sys 0.005s
W20260709 13:03:17.658535  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 58632.3us
I20260709 13:03:17.658947  1717 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:17.626116  4913 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:17.658138  2340 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:03:17.608341  4577 consensus_queue.cc:237] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:17.660806  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1758) took 1055 ms. Trace:
I20260709 13:03:17.569329  4982 consensus_queue.cc:1059] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 13:03:17.663604  1717 rpcz_store.cc:276] 0709 13:03:16.605190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.605364 (+   174us) service_pool.cc:224] Handling call
0709 13:03:17.660708 (+1055344us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:17.676627  4975 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:17.678697  4962 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:17.632195  4972 consensus_queue.cc:1059] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
W20260709 13:03:17.738917  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 46272.4us
W20260709 13:03:17.679498  1538 leader_election.cc:336] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.914s (SENT)
I20260709 13:03:17.680042  4874 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:03:17.681340  4965 consensus_meta.cc:287] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.910s	user 0.015s	sys 0.006s
I20260709 13:03:17.587078  4928 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260709 13:03:17.648696  4915 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:17.745509  4915 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:17.632035  4983 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:17.657531  4904 consensus_queue.cc:1059] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:03:17.683816  1640 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:03:17.684142  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:17.654487  4991 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:17.740432  1538 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:17.740700  1687 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:17.750488  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 66439.6us
W20260709 13:03:17.754494  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 75037.2us
I20260709 13:03:17.761068  4865 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:17.744743  4939 raft_consensus.cc:3055] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:17.763954  2313 leader_election.cc:400] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7.
I20260709 13:03:17.806463  2313 leader_election.cc:403] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:17.768195  4965 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:17.769516  4962 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.754839  4874 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:17.784364  4975 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:17.790843  4917 raft_consensus.cc:3055] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:17.791064  4908 raft_consensus.cc:3055] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:03:17.755515  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 62306.4us
W20260709 13:03:17.792728  4665 consensus_meta.cc:287] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.558s	user 0.012s	sys 0.003s
W20260709 13:03:17.796653  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 148812us
W20260709 13:03:17.801364  1538 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.972s (SENT)
W20260709 13:03:17.810887  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 47179.2us
I20260709 13:03:17.818692  1538 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:17.810992  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:17.819347  2313 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:17.821673  1612 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:17.812783  4917 raft_consensus.cc:740] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:17.824326  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:17.811517  4874 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:17.812635  4965 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:17.814659  4962 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:17.814736  4908 raft_consensus.cc:740] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:17.806659  4939 raft_consensus.cc:740] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:17.813057  4933 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7.
I20260709 13:03:17.825390  1567 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
W20260709 13:03:17.832433  1538 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.434s (SENT)
I20260709 13:03:17.840994  1538 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:17.842517  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:17.843528  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:17.842547  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:17.844362  1567 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:17.846041  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:17.847745  4939 consensus_queue.cc:260] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:17.849269  4939 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:17.811923  4895 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.850039  4895 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:17.852495  4895 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:17.853295  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:17.855151  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:17.855530  4988 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:17.856379  1564 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:17.858069  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:17.858379  1566 raft_consensus.cc:1275] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:03:17.858717  1571 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:17.831660  4884 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:17.831347  4665 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:17.864415  1568 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:17.864610  4665 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:17.817024  4913 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:17.865736  4913 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:17.866334  4915 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:17.867933  4913 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:17.826772  4682 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:03:17.871768  1644 raft_consensus.cc:2368] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10.
I20260709 13:03:17.831504  4874 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:17.875537  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1772) took 1024 ms. Trace:
I20260709 13:03:17.876245  1645 rpcz_store.cc:276] 0709 13:03:16.851362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.466678 (+615316us) service_pool.cc:224] Handling call
0709 13:03:17.875264 (+408586us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b5400183154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:17.875458 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":544042880}
I20260709 13:03:17.876936  1721 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:17.880015  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:17.880888  1570 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:17.886408  1567 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:17.887081  1568 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:03:17.891649  2312 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:17.893426  1571 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:17.895586  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:17.893707  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 61252.2us
I20260709 13:03:17.838847  4983 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:17.902981  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:17.904098  1572 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:17.907014  1564 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:17.909160  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:17.914932  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:17.915921  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:17.917109  1566 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:17.833961  4965 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:17.922621  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:17.928277  1571 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:17.926734  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:17.890105  4975 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:17.933755  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:17.934864  2347 raft_consensus.cc:2368] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4.
I20260709 13:03:17.929602  4962 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:17.937781  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:17.938427  1566 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:17.910737  4933 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:17.825567  4917 consensus_queue.cc:260] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:17.947885  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet eca30b43c26f420c965ee25f475581a0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:03:17.920449  4913 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:17.950035  4983 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.986822  1537 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:17.834645  4908 consensus_queue.cc:260] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:17.970239  1646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2242) took 1521 ms (client timeout 1949 ms). Trace:
I20260709 13:03:17.972110  5010 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:17.992172  5010 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:17.993916  5007 log.cc:927] Time spent T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:03:17.994841  5010 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:17.972934  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.996992  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:17.997316  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.998041  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:17.975917  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2264) took 1426 ms. Trace:
I20260709 13:03:17.999351  1642 rpcz_store.cc:276] 0709 13:03:16.553772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.602141 (+ 48369us) service_pool.cc:224] Handling call
0709 13:03:16.609295 (+  7154us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:16.609876 (+   581us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.609908 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:16.610418 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 13:03:17.767916 (+1157498us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b18004030c0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a250d1 00007f8248e40a0f
0709 13:03:17.816191 (+ 48275us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:17.816241 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.816342 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:17.975419 (+159077us) raft_consensus.cc:1685] finished
0709 13:03:17.975457 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:17.975823 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11968,"raft.queue_time_us":172,"raft.run_cpu_time_us":512,"raft.run_wall_time_us":82737,"spinlock_wait_cycles":3908864,"thread_start_us":47926,"threads_started":1,"wal-append.queue_time_us":1340551}
I20260709 13:03:17.983651  4865 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:17.984516  4915 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:17.985468  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:17.986212  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:17.946816  4933 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:18.009466  4933 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:17.990129  4983 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:18.008723  4884 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:18.014299  4884 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.014556  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1759) took 1194 ms. Trace:
I20260709 13:03:18.014755  4884 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:18.015021  1639 rpcz_store.cc:276] 0709 13:03:16.820185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.587998 (+767813us) service_pool.cc:224] Handling call
0709 13:03:17.588492 (+   494us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:17.612540 (+ 24048us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:17.612581 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:17.613235 (+   654us) log.cc:844] Serialized 26 byte log entry
0709 13:03:17.878209 (+264974us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:17.878268 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.878347 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:18.014263 (+135916us) raft_consensus.cc:1685] finished
0709 13:03:18.014283 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.014459 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1830,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":141,"thread_start_us":264357,"threads_started":1,"wal-append.queue_time_us":337568}
I20260709 13:03:18.015933  4983 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:18.016966  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.017726  4915 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:18.018049  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:18.018617  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:17.989915  4913 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:18.019853  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:17.991865  1646 rpcz_store.cc:269] 0709 13:03:16.449262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.449429 (+   167us) service_pool.cc:224] Handling call
0709 13:03:17.970034 (+1520605us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54001fefd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:17.970177 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3403219200}
I20260709 13:03:18.021027  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.021566  4913 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:18.021952  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:18.022353  1519 auto_rebalancer.cc:893] Removing replace marker failed: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:03:18.022866  1519 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:03:18.023212  1519 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:03:18.023653  1568 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:03:17.955955  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.027276  4984 raft_consensus.cc:993] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838: : Instructing follower f5f4b2707a9647e5bb99e065068dea2b to start an election
I20260709 13:03:18.005177  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.028355  4521 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:18.028738  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:18.029362  1642 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:18.029532  1639 raft_consensus.cc:2368] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7.
I20260709 13:03:18.008877  1567 raft_consensus.cc:2410] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:03:18.030583  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.031289  1639 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:18.032169  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.033212  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.033531  2345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 431) took 1190 ms. Trace:
I20260709 13:03:18.033793  1639 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:18.020022  4915 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:18.034132  2345 rpcz_store.cc:276] 0709 13:03:16.843100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.843243 (+   143us) service_pool.cc:224] Handling call
0709 13:03:16.843505 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:17.065482 (+221977us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:17.065535 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:17.065603 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:17.065698 (+    95us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.033153 (+967455us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b5400467654. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:18.033320 (+   167us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.033466 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509872896}
I20260709 13:03:18.037277  4865 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:18.038082  4865 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:18.038923  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:18.039215  4984 raft_consensus.cc:1081] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Signalling peer f5f4b2707a9647e5bb99e065068dea2b to start an election
I20260709 13:03:18.039659  1720 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:18.040822  4975 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:18.041615  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:18.042351  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 99821.4us
I20260709 13:03:18.042539  1720 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:18.043072  4865 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:18.043322  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:18.043673  1572 raft_consensus.cc:2410] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:03:18.007026  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:18.044134  1647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2271) took 1280 ms (client timeout 1637 ms). Trace:
W20260709 13:03:18.044543  1647 rpcz_store.cc:269] 0709 13:03:16.763690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.785930 (+ 22240us) service_pool.cc:224] Handling call
0709 13:03:18.043140 (+1257210us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b5800003098. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:03:18.044066 (+   926us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3050368}
I20260709 13:03:18.018817  1570 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:18.019989  4966 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:18.022181  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.004272  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.003811  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:18.047513  1645 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:18.043563  1538 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:18.048022  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 44319.9us
I20260709 13:03:18.048514  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.008075  4900 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:03:17.995272  5009 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.142s	user 0.003s	sys 0.000s
I20260709 13:03:18.050941  1568 raft_consensus.cc:1275] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:18.041033  1687 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:18.072971  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.074054  1687 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:18.074267  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.075273  4921 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:18.075205  1645 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:18.076012  4921 raft_consensus.cc:697] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:18.076718  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.077443  1645 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:03:18.077885  4992 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:18.078341  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:18.044456  1640 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:18.078478  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.079348  1713 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:18.081032  1645 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:03:18.042141  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 423) took 3320 ms. Trace:
I20260709 13:03:18.063529  4521 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:18.063982  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:18.064110  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:18.064471  4915 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:18.084900  1647 raft_consensus.cc:2376] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8. Re-sending same reply.
I20260709 13:03:18.089953  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1754) took 1477 ms. Trace:
I20260709 13:03:18.066684  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:18.069451  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.090497  1713 rpcz_store.cc:276] 0709 13:03:16.603884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.604029 (+   145us) service_pool.cc:224] Handling call
0709 13:03:18.078952 (+1474923us) spinlock_profiling.cc:243] Waited 1.32 ms on lock 0x7b580009db98. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60eee 0000000000375939 00007f824ddd41e0 00007f824de9d0c4 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:03:18.079115 (+   163us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.080962 (+  1847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.081589 (+   627us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3038592}
I20260709 13:03:18.091334  1646 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:18.069826  4900 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:18.004904  1611 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:18.079907  2314 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:18.082422  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.082366  1538 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:18.098714  1538 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:18.080601  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 161681us
I20260709 13:03:18.086323  2345 raft_consensus.cc:2393] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 4: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:18.065781  4919 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:18.110200  2314 leader_election.cc:400] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7.
I20260709 13:03:18.111147  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:18.034789  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:18.119135  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 114302us
I20260709 13:03:18.120419  1642 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:03:18.118237  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:18.124109  1570 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:18.107115  1640 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:03:18.125921  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:18.127470  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.128331  1647 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:18.130465  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:18.132002  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:18.133608  1685 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:18.101469  2312 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:18.141615  1538 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:18.143132  1538 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:18.139286  1611 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:18.142343  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:18.144690  1538 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:18.145134  1685 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:18.146107  4989 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:18.146550  4989 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:18.146587  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:18.147081  4989 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:18.146723  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:18.152285  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 63748.2us
I20260709 13:03:18.156466  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:18.156769  2314 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:18.158838  2314 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:18.163067  1565 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:03:18.087594  4915 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:18.088285  4521 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:18.175757  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:18.034153  4874 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:18.326254  4521 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:18.326400  4933 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:18.327628  4874 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:18.179909  1685 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:18.328855  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 149062us
I20260709 13:03:18.329262  1569 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 13: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:18.175803  4939 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:03:18.243963  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 99345us
W20260709 13:03:18.234305  2344 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 332) took 1275 ms (client timeout 1591 ms). Trace:
I20260709 13:03:18.312175  1611 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:18.312855  4959 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.170s	user 0.000s	sys 0.004s
I20260709 13:03:18.312657  4884 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:18.109160  4967 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 13:03:18.314918  4988 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:18.179170  4991 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:18.141288  4919 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:18.183142  4928 consensus_queue.cc:1048] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:03:18.145852  4975 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:18.077107  4921 consensus_queue.cc:237] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:18.084206  2346 rpcz_store.cc:276] 0709 13:03:14.721445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.721741 (+   296us) service_pool.cc:224] Handling call
0709 13:03:15.027755 (+306014us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:15.032940 (+  5185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.032986 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:15.033715 (+   729us) log.cc:844] Serialized 26 byte log entry
0709 13:03:15.236843 (+203128us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b18002173c0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:03:16.485926 (+1249083us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b18002173c0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a250d1
0709 13:03:16.486243 (+   317us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:16.486300 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.486394 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:16.486449 (+    55us) raft_consensus.cc:1685] finished
0709 13:03:16.486466 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.041937 (+1555471us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b54003f84d0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:18.042067 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24218,"raft.queue_time_us":6906,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":10207,"spinlock_wait_cycles":812595072,"thread_start_us":28716,"threads_started":1,"wal-append.queue_time_us":26724,"wal-append.run_cpu_time_us":2824,"wal-append.run_wall_time_us":1015417}
I20260709 13:03:18.171955  5017 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:03:18.317806  5015 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.199s	user 0.003s	sys 0.003s
I20260709 13:03:18.318403  4900 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:18.109668  4913 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:18.108709  4990 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:18.151466  5010 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.095s
I20260709 13:03:18.321002  4962 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:18.148205  4979 consensus_queue.cc:1048] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 13:03:18.117622  4970 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:18.148470  4927 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:18.327850  2345 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 14: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:18.330332  1685 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:18.427659  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 97423.8us
I20260709 13:03:18.428308  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:18.428462  1685 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:18.330861  1538 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:18.365886  5017 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:18.330662  4915 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:18.333858  4992 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:03:18.331593  2344 rpcz_store.cc:269] 0709 13:03:16.952158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.006248 (+ 54090us) service_pool.cc:224] Handling call
0709 13:03:18.066426 (+1060178us) spinlock_profiling.cc:243] Waited 4.67 ms on lock 0x7b580003a298. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60eee 0000000000375939 00007f824ddd41e0 00007f824de9d0c4 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:03:18.227738 (+161312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10734976}
I20260709 13:03:18.328819  4933 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:18.335682  4671 consensus_queue.cc:1048] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:03:18.357263  4919 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:18.344274  5016 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.139s	user 0.003s	sys 0.000s
I20260709 13:03:18.349807  4988 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:18.411401  1721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1751) took 1808 ms (client timeout 2186 ms). Trace:
I20260709 13:03:18.411759  1642 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:18.446992  1721 rpcz_store.cc:269] 0709 13:03:16.602919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.603093 (+   174us) service_pool.cc:224] Handling call
0709 13:03:18.411150 (+1808057us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400249354. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:18.411313 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2654592256}
I20260709 13:03:18.450603  5033 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:18.451802  5033 raft_consensus.cc:697] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:18.412163  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2267) took 1650 ms (client timeout 1547 ms). Trace:
I20260709 13:03:18.356175  4991 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:18.454188  4991 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:18.453954  1643 rpcz_store.cc:269] 0709 13:03:16.762003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.762206 (+   203us) service_pool.cc:224] Handling call
0709 13:03:18.411954 (+1649748us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b54001c0554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:18.412093 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":565904512}
W20260709 13:03:18.420753  1641 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 268) took 2225 ms (client timeout 1748 ms). Trace:
W20260709 13:03:18.455452  1641 rpcz_store.cc:269] 0709 13:03:16.185777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.200534 (+ 14757us) service_pool.cc:224] Handling call
0709 13:03:18.411221 (+2210687us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b54002db754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:18.411382 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2208784768}
I20260709 13:03:18.423797  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:18.456741  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:18.360765  4975 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:18.457950  4975 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:18.453372  5033 consensus_queue.cc:237] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:18.425696  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2266) took 1791 ms. Trace:
W20260709 13:03:18.461319  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 296737us
I20260709 13:03:18.461824  1648 rpcz_store.cc:276] 0709 13:03:16.634783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.730871 (+ 96088us) service_pool.cc:224] Handling call
0709 13:03:18.424779 (+1693908us) spinlock_profiling.cc:243] Waited 1.19 ms on lock 0x7b5800003098. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:03:18.425012 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.425246 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:18.425285 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:18.425337 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:18.425405 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.425473 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.425608 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2729216}
I20260709 13:03:18.463028  1537 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:18.426594  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:18.429762  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 99030.3us
I20260709 13:03:18.465529  1538 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:18.182605  4965 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:18.433754  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:18.437717  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 278957us
I20260709 13:03:18.468740  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:18.435384  1685 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:18.472445  5028 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:18.473019  5028 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.473387  5028 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:18.474174  4807 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:18.475056  4807 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:18.475903  4807 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:18.480772  1537 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:18.480880  2313 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:18.430385  4970 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:18.430697  4927 raft_consensus.cc:697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:18.480134  1685 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:18.432832  4962 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:18.432968  4913 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:18.440267  4967 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:18.437757  4992 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.433293  5017 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:18.494784  1642 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:18.441979  4919 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:18.478557  4988 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:18.510069  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 44620.8us
I20260709 13:03:18.519114  1538 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:18.493978  4897 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:18.523226  4521 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:18.524621  5023 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:03:18.495288  4970 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:18.521448  4897 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:18.488171  4939 raft_consensus.cc:697] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:18.526188  4897 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:18.525985  4927 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:18.494511  4913 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.527573  4913 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:18.527103  4970 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:18.487303  5025 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:18.527163  4939 consensus_queue.cc:237] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:18.491034  4962 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.508878  4992 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:18.557394  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 70471.6us
I20260709 13:03:18.562307  4884 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:18.563090  4884 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:18.563474  4884 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:18.563941  4884 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:18.564286  4962 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:18.568960   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:03:18.569504  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2268) took 1806 ms (client timeout 1914 ms). Trace:
W20260709 13:03:18.571687  1644 rpcz_store.cc:269] 0709 13:03:16.762655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.166986 (+404331us) service_pool.cc:224] Handling call
0709 13:03:18.569239 (+1402253us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b54000a5ad4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:18.569433 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1617992960}
I20260709 13:03:18.570601  4521 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:18.526223  4862 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:18.582109  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 399924us
I20260709 13:03:18.550916  4967 raft_consensus.cc:697] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:03:18.585866  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 94980.2us
I20260709 13:03:18.555152  5025 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:18.599076  1571 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:18.613170  1611 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:18.524662  5028 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:18.619511  1685 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:18.620128  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:18.592913  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 112109us
I20260709 13:03:18.622395  1648 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:18.583083  5006 raft_consensus.cc:3055] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:18.638479  1641 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: []
W20260709 13:03:18.646057  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 59148.7us
I20260709 13:03:18.560163  4975 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:18.594657  4521 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:18.652235  1648 raft_consensus.cc:1275] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:03:18.622946  4865 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.648592  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:18.603932  5025 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:18.652422  1685 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:18.661913  2313 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:18.628733  4900 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:18.629531  5039 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.644375  5006 raft_consensus.cc:740] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:18.672474  5024 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:18.672675  5039 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:18.632602  5026 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:18.673408  2313 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:18.674356  5026 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:18.675678  5039 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:18.675843  5024 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:18.680004  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:18.680471  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.681985  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.682647  1640 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:18.684067  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:18.673556  5006 consensus_queue.cc:260] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:18.686088  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.685308  5006 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:18.688297  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.689500  1640 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:18.690727  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.690766  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:18.694356  2345 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:18.696069  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:18.699258  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 50754.4us
I20260709 13:03:18.696973  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:18.698256  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:18.658221  4965 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:18.693506  1641 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:18.622973  4979 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:03:18.636101  4913 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:18.707942  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.710660  1639 raft_consensus.cc:2410] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:03:18.651014  4671 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:03:18.712752  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:18.715037  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:18.713503  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:18.677511  5026 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:18.716763  1644 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
W20260709 13:03:18.706660  4989 consensus_meta.cc:287] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.547s	user 0.012s	sys 0.000s
I20260709 13:03:18.701136  4965 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.723528  2345 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:18.724967  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:18.710731  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 191744us
I20260709 13:03:18.706573  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:18.701882  4521 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:18.744091  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.748373  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:18.768102  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:18.713128  4900 raft_consensus.cc:697] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:18.737501  1640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:39930
I20260709 13:03:18.770028  1612 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:18.742106  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 75569.1us
I20260709 13:03:18.771524  1612 leader_election.cc:400] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4.
W20260709 13:03:18.741834  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 272670us
I20260709 13:03:18.774175  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:18.716009  4984 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:18.670435  4865 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:18.759442  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:18.759524  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:18.772486  1612 leader_election.cc:403] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:03:18.737803  5036 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:03:18.772595  1686 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:18.783138  1612 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:18.706538  4919 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:18.732734  5034 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.736665  4965 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:18.770655  1640 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:18.803234  1640 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:18.803598  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:18.804467  1646 raft_consensus.cc:2376] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10. Re-sending same reply.
W20260709 13:03:18.792054  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 139861us
I20260709 13:03:18.807435  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.809613  5046 consensus_queue.cc:1059] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:18.781729  4984 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.798723  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:18.782404  5033 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:18.709288  4967 consensus_queue.cc:237] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:18.794435  4966 consensus_meta.cc:287] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.716s	user 0.013s	sys 0.002s
I20260709 13:03:18.820889  1640 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:18.827101  1573 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:03:18.827488  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.723028  4962 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:18.781697  5044 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:18.825628  4975 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:18.818078  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:18.794741  2346 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:18.820160  1538 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:18.821854  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:18.824604  1567 raft_consensus.cc:1240] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:03:18.736289  4989 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:18.826097  1648 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:18.843031  1567 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:03:18.844496  1567 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:03:18.796900  5010 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:18.817768  5024 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:18.793355  4983 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 3. Current term is 4.
I20260709 13:03:18.834182  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:18.817843  4984 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:18.840684  1718 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:18.845546  1538 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:18.846192  1640 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:18.794487  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:18.861497  4990 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:18.862102  5028 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:18.813822  5037 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:18.857270  1569 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
W20260709 13:03:18.861588  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 67216.2us
I20260709 13:03:18.864221  1538 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:18.814304  5034 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:18.825357  4992 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:18.839406  5041 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.843266  5026 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:18.881650  1538 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:18.854523  4966 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:18.878552  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:18.883086  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 76280.9us
I20260709 13:03:18.814010  4900 consensus_queue.cc:237] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:18.886514  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:18.825388  5017 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:18.887678  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: true dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:18.874123  4975 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:18.884366  4990 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:03:18.896158  1720 raft_consensus.cc:2393] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:18.897027  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:18.899523  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 104086us
I20260709 13:03:18.902110  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:18.897306  1646 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:18.901389  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:18.880766  5037 raft_consensus.cc:697] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:18.897933  4975 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:03:18.960685  5043 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:03:18.862614  5024 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:18.890364  5041 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:19.228003  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1795) took 1326 ms. Trace:
I20260709 13:03:19.228653  1572 rpcz_store.cc:276] 0709 13:03:17.901937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.914521 (+ 12584us) service_pool.cc:224] Handling call
0709 13:03:19.227729 (+1313208us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b5400012554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:19.227926 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":293204224}
W20260709 13:03:18.901839  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 89579.4us
W20260709 13:03:19.162781  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 274069us
W20260709 13:03:19.210369  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 328939us
I20260709 13:03:19.211030  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:19.211268  4931 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:19.211948  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:19.243304  1538 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:19.213326  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:19.160200  5010 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:19.214354  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:19.298465  1714 raft_consensus.cc:2368] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7.
I20260709 13:03:19.215854  5044 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:19.219170  4962 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:18.899024  5028 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:18.959496  5040 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:19.210420  2345 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 6: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:19.290968  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:19.223814  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 337000us
I20260709 13:03:19.212289  1612 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:19.302157  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:19.308274  1715 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:19.213629  4915 consensus_meta.cc:287] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.770s	user 0.010s	sys 0.001s
W20260709 13:03:19.311481  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 68278.2us
I20260709 13:03:19.312307  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:19.312558  1538 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:19.313119  1721 raft_consensus.cc:2368] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7.
I20260709 13:03:19.240295  4975 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:19.321969  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:19.220431  5033 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:19.404345  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:19.309168  4913 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:19.321668  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:19.415455  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:19.415894  5051 log.cc:927] Time spent T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 13:03:19.417126  5047 log.cc:927] Time spent T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.525s	user 0.000s	sys 0.014s
I20260709 13:03:19.416694  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:19.418468  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:19.419570  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:19.426985  1714 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 8: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:03:19.379045  5046 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.011s	sys 0.035s
W20260709 13:03:19.433013  5046 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.012s	sys 0.036s
I20260709 13:03:19.306171  5028 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:19.293555  4966 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.115s
I20260709 13:03:19.428047  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:19.310961  4919 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:19.321673  4935 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:03:19.521077  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 208622us
I20260709 13:03:19.437690  4665 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:19.532361  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:19.443485  4928 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:19.503032  5028 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:19.497452  4930 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:19.322310  4915 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:19.572813  1538 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:19.463361  5040 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:19.529595  4967 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.648365  1716 raft_consensus.cc:1240] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:03:19.314601  5010 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:19.656102  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 83434.4us
I20260709 13:03:19.521229  4865 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:19.694044  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:19.691812  1715 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:19.696180  1538 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:19.426615  5057 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:03:19.709074  1564 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:19.714601  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1783) took 1035 ms. Trace:
W20260709 13:03:19.708519  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 496432us
I20260709 13:03:19.651216  4989 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:19.715058  1643 rpcz_store.cc:276] 0709 13:03:18.678869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.679009 (+   140us) service_pool.cc:224] Handling call
0709 13:03:18.679362 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:18.713359 (+ 33997us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:18.713390 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:18.713918 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 13:03:18.773833 (+ 59915us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:18.773947 (+   114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.774020 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.420687 (+646667us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b48002619d8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ddaeaa1 00007f824dda5b37 00007f824dda59ca 00007f824de9d26f 00007f824de9cf37 00007f824deb155c 00007f824deb1508 00007f824deb1488 00007f824deb1443 00007f824deb06ec 00007f824cddaa96
0709 13:03:19.703143 (+282456us) raft_consensus.cc:1685] finished
0709 13:03:19.703168 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.714528 (+ 11360us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44940,"raft.queue_time_us":37900,"raft.run_cpu_time_us":2521,"raft.run_wall_time_us":697097,"spinlock_wait_cycles":23165696,"thread_start_us":38644,"threads_started":1,"wal-append.queue_time_us":82743}
I20260709 13:03:19.654479  4874 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:19.575470  5044 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:19.668794  4992 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:19.734237  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 217131us
I20260709 13:03:19.737006  1687 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:19.655457  4665 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:19.654045  4928 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:19.750466  1611 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.008s	sys 0.010s
I20260709 13:03:19.721170  5040 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:19.722409  4865 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:03:19.408557  5033 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:19.603737  4935 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:19.755649  1611 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.008s	sys 0.011s
I20260709 13:03:19.694557  4967 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:19.728497  4874 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:19.743999  4989 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:19.729415  5034 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:19.752530  5017 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:19.754695  5044 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:19.749569   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:19.767637  1721 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:19.718783  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:19.577646  5037 consensus_queue.cc:237] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:19.783171  2340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 436) took 1084 ms. Trace:
I20260709 13:03:19.804065  1721 raft_consensus.cc:1275] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:19.785532  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 481) took 1141 ms. Trace:
I20260709 13:03:19.767714  1687 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:19.815698  1564 rpcz_store.cc:276] 0709 13:03:18.644200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.646130 (+  1930us) service_pool.cc:224] Handling call
0709 13:03:18.646531 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:19.713860 (+1067329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.785269 (+ 71409us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54000f69d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:19.785459 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2443123584}
W20260709 13:03:19.791723  1471 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38662 (request call id 208) took 24082 ms (client timeout 15000 ms). Trace:
W20260709 13:03:19.816704  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 49083.6us
I20260709 13:03:19.727174  5067 consensus_queue.cc:1059] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
W20260709 13:03:19.772972  4991 consensus_meta.cc:287] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.318s	user 0.017s	sys 0.004s
I20260709 13:03:19.800316  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:19.795315  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55890 (request call id 353) took 1091 ms. Trace:
I20260709 13:03:19.799973  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:19.777179  4984 consensus_meta.cc:287] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.898s	user 0.013s	sys 0.004s
W20260709 13:03:19.769378  5025 consensus_meta.cc:287] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.068s	user 0.011s	sys 0.006s
I20260709 13:03:19.778824  4908 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:19.774593  5006 consensus_meta.cc:287] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.087s	user 0.016s	sys 0.008s
W20260709 13:03:19.776381  4965 consensus_meta.cc:287] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.957s	user 0.013s	sys 0.000s
I20260709 13:03:19.787024  4874 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:19.808946  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:19.806584  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 499688us
I20260709 13:03:19.857955  1717 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:19.804131  2340 rpcz_store.cc:276] 0709 13:03:18.699074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.699325 (+   251us) service_pool.cc:224] Handling call
0709 13:03:18.714475 (+ 15150us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.714713 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:18.869905 (+155192us) log.cc:844] Serialized 14 byte log entry
0709 13:03:19.225036 (+355131us) log.cc:844] Serialized 14 byte log entry
0709 13:03:19.780392 (+555356us) spinlock_profiling.cc:243] Waited 59.1 ms on lock 0x7b1800359b60. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d12c0a 00007f824f452219 00007f824cd5958f 00007f8245d98571 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd
0709 13:03:19.782185 (+  1793us) log.cc:844] Serialized 14 byte log entry
0709 13:03:19.782564 (+   379us) log.cc:844] Serialized 14 byte log entry
0709 13:03:19.782697 (+   133us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:19.782738 (+    41us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:19.782803 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.782969 (+   166us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.783097 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":571982,"prepare.queue_time_us":103911,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":882326,"raft.run_cpu_time_us":554,"raft.run_wall_time_us":56113,"spinlock_wait_cycles":297504256,"thread_start_us":169286,"threads_started":2,"wal-append.queue_time_us":30651}
W20260709 13:03:19.826045  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 107363us
I20260709 13:03:19.859920  1612 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:19.821516  1721 raft_consensus.cc:1240] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:19.789323  4895 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:19.816911  1471 rpcz_store.cc:269] 0709 13:02:55.709299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.710607 (+  1308us) service_pool.cc:224] Handling call
0709 13:02:56.156541 (+445934us) spinlock_profiling.cc:243] Waited 8.91 ms on lock 0x7b5400310010. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d61640 00007f824ec38b76 00007f824ed42c3e 00007f824d2032a4 00007f824d203232 00007f824d203161 00007f824d2030dc 00007f824d202392 00007f82471c048c 00007f82471bf8c6
0709 13:03:19.791630 (+23635089us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:14.471795 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:14.472605 (+   810us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:14.472657 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:03:14.472686 (+    29us) tablet.cc:662] Decoding operations
0709 13:03:14.473292 (+   606us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:14.473464 (+   172us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:14.473505 (+    41us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:03:14.473989 (+   484us) tablet.cc:701] Row locks acquired
0709 13:03:14.474017 (+    28us) write_op.cc:260] PREPARE: finished
0709 13:03:14.474239 (+   222us) write_op.cc:270] Start()
0709 13:03:14.474397 (+   158us) write_op.cc:276] Timestamp: P: 1783602194474168 usec, L: 0
0709 13:03:14.474428 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:03:14.475543 (+  1115us) log.cc:844] Serialized 929 byte log entry
0709 13:03:14.788804 (+313261us) op_driver.cc:464] REPLICATION: finished
0709 13:03:15.060170 (+271366us) write_op.cc:301] APPLY: starting
0709 13:03:15.060258 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:15.060420 (+   162us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:15.060562 (+   142us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:15.060981 (+   419us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:15.061189 (+   208us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:03:15.061237 (+    48us) write_op.cc:312] APPLY: finished
0709 13:03:15.061452 (+   215us) log.cc:844] Serialized 29 byte log entry
0709 13:03:15.061696 (+   244us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:15.061794 (+    98us) write_op.cc:454] Released schema lock
0709 13:03:15.061836 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:15.062357 (+   521us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:15.062394 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":61669,"reported_tablets":26,"spinlock_wait_cycles":20491136,"child_traces":[["op",{"apply.queue_time_us":261184,"apply.run_cpu_time_us":2134,"apply.run_wall_time_us":2145,"num_ops":2,"prepare.queue_time_us":78014,"prepare.run_cpu_time_us":4307,"prepare.run_wall_time_us":30186,"replication_time_us":284520,"thread_start_us":277296,"threads_started":2}]]}
I20260709 13:03:19.769517  4665 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:19.830000  4935 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:19.853075  4884 consensus_meta.cc:287] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.289s	user 0.009s	sys 0.021s
I20260709 13:03:19.818332  4939 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:03:19.774519  4988 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 1.241s	user 0.009s	sys 0.009s
W20260709 13:03:19.785063  4897 consensus_meta.cc:287] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.258s	user 0.015s	sys 0.006s
I20260709 13:03:19.782601  5066 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:19.836189  2347 rpcz_store.cc:276] 0709 13:03:18.703714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.703890 (+   176us) service_pool.cc:224] Handling call
0709 13:03:19.794469 (+1090579us) spinlock_profiling.cc:243] Waited 3.18 ms on lock 0x7b580003a298. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:03:19.794691 (+   222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:19.794923 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.794955 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:19.795003 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:19.795047 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.795108 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.795248 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7320320}
W20260709 13:03:19.788303  5058 log.cc:927] Time spent T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:03:19.836195  1573 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:19.788398  5044 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.821900  1687 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:19.782922  5040 raft_consensus.cc:697] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:19.785599  5059 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.787218  4989 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:19.854897  4984 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:19.793543  5010 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:19.828441  4966 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:19.865687  1721 raft_consensus.cc:1275] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:03:19.800499  1536 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.007s	sys 0.052s
I20260709 13:03:19.864230  4895 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:19.855967  4807 consensus_meta.cc:287] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.380s	user 0.015s	sys 0.018s
I20260709 13:03:19.831583  4682 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:03:19.890278  1612 leader_election.cc:400] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7.
W20260709 13:03:19.892047  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 70389.1us
I20260709 13:03:19.887288  4665 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.797199  5070 consensus_queue.cc:1048] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.122s
I20260709 13:03:19.863248  4965 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:19.912379  1719 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 8: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:03:19.903322  1536 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.007s	sys 0.053s
I20260709 13:03:19.892204  5044 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:19.850287  4991 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:19.916050  1687 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:19.894986  5071 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:19.896576  4989 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:19.907529  5024 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:19.907675  5055 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:19.885087  4884 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:19.882431  5075 consensus_queue.cc:1059] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:03:19.917683  2314 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:19.906104  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 48768.3us
W20260709 13:03:19.915310  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 165196us
I20260709 13:03:19.928483  2345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55890 (request call id 357) took 1174 ms. Trace:
I20260709 13:03:19.857034  5025 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:19.913749  4665 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:19.978296  1611 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:19.979228  4665 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:19.979894  4665 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:19.980302  4919 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:19.980662  4928 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:19.980489  4665 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:19.981271  4928 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:19.932955  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:19.979796  1611 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:19.933616  5064 log.cc:927] Time spent T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:03:19.938014  5076 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:19.938561  5072 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:19.906267  5041 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:19.941624  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:19.942181  1537 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.008s	sys 0.007s
W20260709 13:03:20.016829  1537 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.009s	sys 0.007s
W20260709 13:03:20.017140  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 777403us
I20260709 13:03:19.892043  4988 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:19.946986  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:20.025652  2340 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:19.951589  1569 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:19.927069  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1780) took 1265 ms. Trace:
I20260709 13:03:20.027087  1644 rpcz_store.cc:276] 0709 13:03:18.661496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.661714 (+   218us) service_pool.cc:224] Handling call
0709 13:03:18.662069 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.717287 (+ 55218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.926741 (+1209454us) spinlock_profiling.cc:243] Waited 54.4 ms on lock 0x7b54001c0554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:19.926991 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":125234048}
I20260709 13:03:19.953801  2345 rpcz_store.cc:276] 0709 13:03:18.754074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.759293 (+  5219us) service_pool.cc:224] Handling call
0709 13:03:19.928299 (+1169006us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b54003f84d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:19.928429 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":366920960}
I20260709 13:03:19.956211  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1821) took 1141 ms. Trace:
I20260709 13:03:19.957230  5033 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:19.942312  4865 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:19.957602  5026 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:20.032670  1611 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:19.987565  4915 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:19.952895  4930 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:19.892307  4897 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.051766  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:19.972474  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1775) took 1070 ms. Trace:
I20260709 13:03:19.974256  5049 raft_consensus.cc:3055] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:19.976811  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1797) took 2047 ms. Trace:
I20260709 13:03:19.908599  4807 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:19.983145  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.986967  1566 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 7: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:03:19.988619  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 71058.1us
I20260709 13:03:19.933185  5040 consensus_queue.cc:237] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.060352  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:20.060815  4988 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:20.061861  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:20.063076  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:19.944527  5067 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:20.063705  5067 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.064147  5067 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:19.991645  4975 consensus_meta.cc:287] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.530s	user 0.013s	sys 0.000s
I20260709 13:03:20.066092  4975 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:19.992404  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 73763.1us
W20260709 13:03:20.067929  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:19.942040  4964 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.002s	sys 0.011s
I20260709 13:03:19.897737  5059 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.072871  1722 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:19.996014  4970 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:19.911037  5010 consensus_queue.cc:1059] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:03:20.000612  4884 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:20.001250  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2243) took 1277 ms (client timeout 1573 ms). Trace:
I20260709 13:03:20.004133  1714 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:20.005137  1565 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:20.006026  5075 raft_consensus.cc:3055] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:20.005553  5028 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.007241  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:20.022290  1647 raft_consensus.cc:2410] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 13:03:20.072922  4964 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.002s	sys 0.012s
I20260709 13:03:20.086411  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1829) took 1227 ms. Trace:
W20260709 13:03:20.087142  1720 rpcz_store.cc:269] 0709 13:03:18.723980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.724623 (+   643us) service_pool.cc:224] Handling call
0709 13:03:20.001019 (+1276396us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54000541d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:20.001191 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":361800576}
I20260709 13:03:20.088871  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:19.959759  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:20.084118  1715 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:20.036468  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:20.091265  1646 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:20.028939  1570 rpcz_store.cc:276] 0709 13:03:18.815320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.815456 (+   136us) service_pool.cc:224] Handling call
0709 13:03:18.831476 (+ 16020us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:18.837275 (+  5799us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:18.837320 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:18.844134 (+  6814us) log.cc:844] Serialized 26 byte log entry
0709 13:03:19.173157 (+329023us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:19.173227 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.173332 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.955993 (+782661us) raft_consensus.cc:1685] finished
0709 13:03:19.956017 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.956153 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6004,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":16844,"thread_start_us":285096,"threads_started":1,"wal-append.queue_time_us":571953}
W20260709 13:03:20.047390  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 188922us
I20260709 13:03:20.052759  1717 rpcz_store.cc:276] 0709 13:03:18.901552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.211659 (+310107us) service_pool.cc:224] Handling call
0709 13:03:19.972271 (+760612us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b5400246154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:19.972404 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1416698112}
I20260709 13:03:20.054970  2349 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:03:20.050047  4682 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:20.031108  4979 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:20.054673  4930 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.016500  5072 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:20.104467  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1828) took 1249 ms. Trace:
I20260709 13:03:20.105057  1568 rpcz_store.cc:276] 0709 13:03:18.855571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.855723 (+   152us) service_pool.cc:224] Handling call
0709 13:03:18.856089 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:19.721765 (+865676us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.721797 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:19.722348 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 13:03:19.917613 (+195265us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:19.917679 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.979876 (+ 62197us) spinlock_profiling.cc:243] Waited 6.29 ms on lock 0x7b480036ffd8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ddaeaa1 00007f824dda5b37 00007f824dda59ca 00007f824de9d26f 00007f824de9cf37 00007f824deb155c 00007f824deb1508 00007f824deb1488 00007f824deb1443 00007f824deb06ec 00007f824cddaa96
0709 13:03:20.000389 (+ 20513us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b5400305a54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:20.000445 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:20.104197 (+103752us) raft_consensus.cc:1685] finished
0709 13:03:20.104214 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.104379 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":147601,"raft.run_cpu_time_us":8054,"raft.run_wall_time_us":484868,"spinlock_wait_cycles":1068363904,"thread_start_us":601853,"threads_started":2,"wal-append.queue_time_us":273308}
I20260709 13:03:20.059415  4967 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:20.064157  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:20.056030  2348 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:20.056789  5049 raft_consensus.cc:740] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:20.054715  1571 rpcz_store.cc:276] 0709 13:03:17.929235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.989785 (+ 60550us) service_pool.cc:224] Handling call
0709 13:03:18.598859 (+609074us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.604611 (+  5752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.976495 (+1371884us) spinlock_profiling.cc:243] Waited 609 ms on lock 0x7b54001b5650. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:19.976728 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1399802624}
I20260709 13:03:20.093473  1573 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 6: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:20.038954  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:20.119481  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:20.120198  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:20.120683  1571 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 13: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:20.121241  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:20.121836  1644 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:20.094466  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:20.122488  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:20.095175  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:20.098097  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:20.049337  5041 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:20.100518  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 46828.8us
I20260709 13:03:20.102386  5069 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.103143  5035 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:20.053303  4865 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:20.110105  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:20.087035  1569 rpcz_store.cc:276] 0709 13:03:18.859336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.875545 (+ 16209us) service_pool.cc:224] Handling call
0709 13:03:20.086151 (+1210606us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54002b9654. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:20.086319 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2434508160}
I20260709 13:03:20.112469  1716 raft_consensus.cc:2376] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10. Re-sending same reply.
I20260709 13:03:20.114125  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:20.042811  4935 consensus_queue.cc:1059] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:03:20.122797  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:03:20.158017  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45123.6us
I20260709 13:03:20.166899  5085 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:20.167827  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet eca30b43c26f420c965ee25f475581a0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:03:20.168252  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:20.169085  4865 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:20.087060  4807 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:20.169922  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:20.170212  5069 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.170751  1570 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:20.170895  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:20.172079  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:20.093878  4964 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.173095  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:20.083676  4897 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:20.018528  5076 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.179006  1642 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
W20260709 13:03:20.189289  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 93835.9us
I20260709 13:03:20.173522  4964 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.174249  5089 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:20.174952  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:20.175104  5073 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.084745  2344 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:20.082918  4917 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:20.116359  1612 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:20.213913  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:20.214592  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 98329.1us
I20260709 13:03:20.215111  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:20.172786  1570 raft_consensus.cc:1240] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:20.175875  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:20.177261  1717 raft_consensus.cc:2376] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9. Re-sending same reply.
I20260709 13:03:20.049191  5044 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.180611  1646 raft_consensus.cc:2376] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6. Re-sending same reply.
I20260709 13:03:20.186830  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:20.192528  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:20.223083  1538 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:20.223645  1642 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:20.172945  5069 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:20.167886  5085 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.088631  4915 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.100448  4682 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:20.196630  1643 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:03:20.085686  5059 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:20.090189  5075 raft_consensus.cc:740] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:20.113690  4913 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:20.100144  4979 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.210099  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1782) took 1532 ms. Trace:
I20260709 13:03:20.210165  1687 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:20.117038  5066 consensus_queue.cc:1059] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.156s
I20260709 13:03:20.115374  5090 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:20.169847  5035 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.266495  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:20.157905  4921 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 13:03:20.294771  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:20.270227  4884 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:20.296438  2347 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:20.267266  5049 consensus_queue.cc:260] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:20.294800  5090 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.300833  5049 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:20.302629  5028 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:20.302492  5090 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:20.269879  5078 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:03:20.238415  4991 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:20.273042  4984 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:20.274870  5091 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:03:20.306128  4991 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.306689  4991 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.306854  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1787) took 1089 ms. Trace:
I20260709 13:03:20.307701  1715 rpcz_store.cc:276] 0709 13:03:19.217793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.713106 (+495313us) service_pool.cc:224] Handling call
0709 13:03:19.720191 (+  7085us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:20.089976 (+369785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.306557 (+216581us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b54002c60d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:20.306779 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":836194688}
I20260709 13:03:20.309018  4991 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:20.277992  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:20.331511  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 121404us
I20260709 13:03:20.345913  1717 raft_consensus.cc:2376] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6. Re-sending same reply.
I20260709 13:03:20.292243  1568 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:20.292491  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:20.234015  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:20.225553  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:20.292990  4682 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:20.291399  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:20.289618  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:20.309401  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:20.288971  1648 rpcz_store.cc:276] 0709 13:03:18.677871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.678011 (+   140us) service_pool.cc:224] Handling call
0709 13:03:18.678368 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.842295 (+163927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.209743 (+1367448us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54002add54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:20.210021 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323852032}
W20260709 13:03:20.238622  5088 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:03:20.290871  4979 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:20.317098  4930 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:20.281256  5072 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:20.319221  2312 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:20.320801  1644 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:20.222052  5010 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:20.352807  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:20.355028  2313 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:20.376158  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 269834us
W20260709 13:03:20.376441  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 57312.5us
I20260709 13:03:20.373044  1612 leader_election.cc:400] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7.
I20260709 13:03:20.269836  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:20.432938  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:20.434114  1573 raft_consensus.cc:2393] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:20.219066  5073 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.374935  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:20.209818  5070 consensus_queue.cc:1059] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.110s
I20260709 13:03:20.212778  5041 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:20.428076  1718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2227) took 3233 ms (client timeout 1780 ms). Trace:
I20260709 13:03:20.429044  1713 raft_consensus.cc:1275] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:03:20.428901  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 56068.6us
I20260709 13:03:20.377152  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:20.317559  5076 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:20.359791  4865 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:20.436547  1717 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:20.437515  2312 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:20.441932  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 86977.3us
I20260709 13:03:20.454902  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:20.455607  2312 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:20.456872  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:20.235947  4964 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:20.445094  1648 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:20.328682  5066 consensus_queue.cc:1243] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:03:20.448441  1646 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:20.448472  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:20.462491  5084 consensus_queue.cc:1059] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:03:20.469827  1643 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:20.327466  1538 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:20.446731  1718 rpcz_store.cc:269] 0709 13:03:17.194880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.802312 (+607432us) service_pool.cc:224] Handling call
0709 13:03:20.427827 (+2625515us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540029a9d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:20.427989 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2346978560}
I20260709 13:03:20.375988  5078 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:20.471565  5078 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.450091  1612 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:20.318208  4919 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:20.462020  5074 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.298552  4913 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.467762  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:20.470593  5084 raft_consensus.cc:3055] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:20.438656  5025 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:20.383140  4966 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 13:03:20.465443  5037 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:20.298447  4915 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.424772  4935 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:20.479046  1612 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:20.479947  1569 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:20.472074  5078 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:20.488688  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:20.519580  1716 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:20.520150  1646 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:20.217190  5089 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.520903  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:20.471834  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 144474us
I20260709 13:03:20.452488  4970 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:20.484393  4991 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:20.507411  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:20.425104  4975 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:20.464627  4988 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:20.505334  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 148806us
I20260709 13:03:20.472041  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:20.527148  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 48197.4us
I20260709 13:03:20.640960  1612 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:20.490847  4900 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:20.503377  5074 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.519059  5070 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:20.478024  4967 consensus_queue.cc:1059] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:03:20.478597  4865 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.535003  1716 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:20.618749  1645 raft_consensus.cc:2410] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:20.584551  4930 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:20.600320  4966 raft_consensus.cc:3055] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:20.618801  1717 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
W20260709 13:03:20.613389  4874 consensus_meta.cc:287] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.742s	user 0.012s	sys 0.001s
I20260709 13:03:20.490688  4919 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:20.503922  5084 raft_consensus.cc:740] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:20.602860  5025 raft_consensus.cc:476] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 6 LEADER]: Not starting pre-election -- already a leader
W20260709 13:03:20.611526  5067 consensus_meta.cc:287] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.547s	user 0.007s	sys 0.006s
W20260709 13:03:20.621793  1719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2249) took 1741 ms (client timeout 1842 ms). Trace:
I20260709 13:03:20.622895  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1820) took 1812 ms. Trace:
W20260709 13:03:20.614760  4989 consensus_meta.cc:287] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.656s	user 0.013s	sys 0.000s
I20260709 13:03:20.599283  5069 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:20.569196  4965 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:20.611608  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:20.465729  5075 consensus_queue.cc:260] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.855580  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1832) took 1056 ms. Trace:
I20260709 13:03:20.808151  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:20.892357  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 251530us
I20260709 13:03:20.898312  1721 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:20.731467  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1780) took 1515 ms. Trace:
I20260709 13:03:20.899605  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:20.838537  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 439) took 1520 ms. Trace:
I20260709 13:03:20.896837  1567 rpcz_store.cc:276] 0709 13:03:19.799571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.799719 (+   148us) service_pool.cc:224] Handling call
0709 13:03:20.855214 (+1055495us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b54001b4754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:20.855480 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":791650560}
I20260709 13:03:20.895922  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:20.898615  1565 rpcz_store.cc:276] 0709 13:03:18.810670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.811017 (+   347us) service_pool.cc:224] Handling call
0709 13:03:18.811392 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:20.092195 (+1280803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.622581 (+530386us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400306954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:20.622817 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2745209600}
W20260709 13:03:20.898106  1719 rpcz_store.cc:269] 0709 13:03:18.880761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.885385 (+  4624us) service_pool.cc:224] Handling call
0709 13:03:20.621538 (+1736153us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400246654. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:20.621727 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2350056832}
I20260709 13:03:20.900792  2349 rpcz_store.cc:276] 0709 13:03:19.318034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.318386 (+   352us) service_pool.cc:224] Handling call
0709 13:03:19.427759 (+109373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:20.093440 (+665681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.838104 (+744664us) spinlock_profiling.cc:243] Waited 627 ms on lock 0x7b54005c4254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:20.838451 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1441901440}
W20260709 13:03:20.736738  5097 log.cc:927] Time spent T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:03:20.903051  5004 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.919123  4919 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:20.919833  5024 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.919575  5004 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.854305  5074 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:20.920439  5024 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:20.922261  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 278) took 2494 ms. Trace:
I20260709 13:03:20.922884  5024 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:20.912674  5067 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.926560  4665 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:20.926846  5067 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:20.912513  4874 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:20.928550  1564 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 5. Current term is 6. Ops: []
W20260709 13:03:20.921427  5049 consensus_meta.cc:287] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.620s	user 0.013s	sys 0.000s
I20260709 13:03:20.930003  1720 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 9: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:20.905154  4928 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:03:20.689285  5098 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.075s	user 0.002s	sys 0.000s
I20260709 13:03:20.900292  1714 rpcz_store.cc:276] 0709 13:03:19.216060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.518137 (+302077us) service_pool.cc:224] Handling call
0709 13:03:20.003886 (+485749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:20.090435 (+ 86549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.731144 (+640709us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b54000541d0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:20.731379 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1113770368}
I20260709 13:03:20.629256  4991 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:20.906577  4682 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:20.782060  5063 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:03:20.899127  4966 raft_consensus.cc:740] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:20.922716  1722 rpcz_store.cc:276] 0709 13:03:18.432007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.433857 (+  1850us) service_pool.cc:224] Handling call
0709 13:03:18.482415 (+ 48558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:20.073516 (+1591101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.921993 (+848477us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b5400247f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:20.922176 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3657392384}
I20260709 13:03:20.657809  4917 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:20.934729  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:20.701527  5044 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:20.898307  4913 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:20.735459  4908 raft_consensus.cc:3055] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:20.910475  2344 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:20.940618  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:20.951225  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:20.942374  1612 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:20.658317  5035 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:20.963016  1567 raft_consensus.cc:1240] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:20.963085  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:20.902242  5075 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:20.905431  4962 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.944881  4665 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:20.948375  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:20.947115  4921 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.937726  4900 consensus_queue.cc:1059] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:03:20.916637  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:20.702353  5070 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:20.960421  1718 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:20.968966  5106 consensus_queue.cc:1059] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:03:20.976264  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: true dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:20.976863  1718 raft_consensus.cc:3055] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:20.977165  1718 raft_consensus.cc:740] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:20.977934  1718 consensus_queue.cc:260] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.978261  4900 raft_consensus.cc:3055] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:20.978830  1718 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:20.978798  4900 raft_consensus.cc:740] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:20.965564  4913 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:20.969343  4928 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:20.963891  1572 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:03:20.963799  1612 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:20.905515  4931 consensus_queue.cc:1048] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:03:20.912817  4989 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.986001  2312 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:20.987661  1612 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:20.939801  4991 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:20.916575  5069 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.971010  1714 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:20.963927  4908 raft_consensus.cc:740] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:20.996014  4935 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:21.025054  5004 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:20.961477  1721 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
W20260709 13:03:21.000984  4979 consensus_meta.cc:287] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.568s	user 0.012s	sys 0.000s
I20260709 13:03:21.026412  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:21.026913  5110 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.027030  1721 raft_consensus.cc:2376] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6. Re-sending same reply.
I20260709 13:03:21.027593  5110 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:21.028128  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:21.029255  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:21.030721  1721 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:20.972621  4962 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.979643  4921 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.992570  5035 consensus_queue.cc:1059] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:20.993270  4917 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:20.993979  5070 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:20.994115  4900 consensus_queue.cc:260] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:20.986977  4682 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:20.986323  4913 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.025290  4935 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.026094  4979 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:21.030004  1647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1784) took 2350 ms (client timeout 2412 ms). Trace:
I20260709 13:03:20.964042  1565 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:20.985805  4665 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:21.030541  1718 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:21.014117  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1834) took 1206 ms. Trace:
I20260709 13:03:21.059019  1566 rpcz_store.cc:276] 0709 13:03:19.808363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.808650 (+   287us) service_pool.cc:224] Handling call
0709 13:03:21.013923 (+1205273us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400013e54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:21.014057 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295735680}
I20260709 13:03:20.929430  5078 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.057180  4935 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:21.014739  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 487) took 1058 ms. Trace:
I20260709 13:03:21.022507  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:21.060900  5041 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:20.991267  1612 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:21.008098  4895 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:21.062429  1645 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2293) took 3057 ms (client timeout 3341 ms). Trace:
I20260709 13:03:21.062970  4665 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:21.062932  1645 rpcz_store.cc:269] 0709 13:03:18.005054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.082127 (+ 77073us) service_pool.cc:224] Handling call
0709 13:03:21.062221 (+2980094us) spinlock_profiling.cc:243] Waited 2.51 s on lock 0x7b54004114d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:21.062355 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5784468736}
W20260709 13:03:21.062486  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 71304.1us
I20260709 13:03:21.064991  1612 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:21.065454  5089 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:21.066426  4913 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:21.066630  1639 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:03:21.066906  4682 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:21.046502  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:21.068547  1721 raft_consensus.cc:2368] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7.
I20260709 13:03:20.945298  5084 consensus_queue.cc:260] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:20.994879  4989 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:21.001721  5111 raft_consensus.cc:3055] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:21.051837  1717 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 8: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:21.025302  4927 raft_consensus.cc:3055] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:03:21.069747  5103 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.331s	user 0.006s	sys 0.000s
I20260709 13:03:21.007364  4874 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:20.997275  5073 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:20.996762  5085 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:21.056561  5054 raft_consensus.cc:3055] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:03:21.057381  1647 rpcz_store.cc:269] 0709 13:03:18.680061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.680192 (+   131us) service_pool.cc:224] Handling call
0709 13:03:21.029801 (+2349609us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400182c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:21.029950 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3073516416}
I20260709 13:03:21.062312  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1798) took 1033 ms. Trace:
I20260709 13:03:21.089015  1718 rpcz_store.cc:276] 0709 13:03:20.028611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.975979 (+947368us) service_pool.cc:224] Handling call
0709 13:03:21.062251 (+ 86272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.067214  1639 pending_rounds.cc:85] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 4
I20260709 13:03:21.091478  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1779) took 1876 ms. Trace:
I20260709 13:03:21.091842  1639 pending_rounds.cc:107] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 6.5
W20260709 13:03:21.012377  5095 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.559s	user 0.000s	sys 0.002s
I20260709 13:03:21.070573  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:21.096457  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:21.097195  1714 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:21.060951  1569 rpcz_store.cc:276] 0709 13:03:19.956807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.173884 (+217077us) service_pool.cc:224] Handling call
0709 13:03:20.174196 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:20.519051 (+344855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.014498 (+495447us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54001b5b54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:21.014683 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":702248704}
I20260709 13:03:21.071398  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:21.047242  5037 consensus_meta.cc:287] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.527s	user 0.010s	sys 0.004s
I20260709 13:03:21.025532  4908 consensus_queue.cc:260] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.078258  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:20.997812  4915 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:21.059612  4979 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:21.076247  5024 consensus_queue.cc:1059] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:03:21.107887  1612 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:21.110370  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:21.087568  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 484) took 1131 ms. Trace:
I20260709 13:03:21.074771  4962 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:21.123535  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:21.083662  5072 consensus_queue.cc:1175] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "066ee0109aba4465ba17fa65e14ef504" responder_term: 8 status { last_received { term: 7 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:21.088335  5054 raft_consensus.cc:740] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:21.124311  1573 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:03:21.092317  1713 rpcz_store.cc:276] 0709 13:03:19.215104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.215239 (+   135us) service_pool.cc:224] Handling call
0709 13:03:20.428637 (+1213398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:20.447571 (+ 18934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.091258 (+643687us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b540029a9d0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:21.091410 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2789978368}
I20260709 13:03:21.092712  1639 raft_consensus.cc:2889] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:21.080331  4927 raft_consensus.cc:740] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:21.082862  5084 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:21.103844  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:21.104776  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:21.137398  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.079427  5111 raft_consensus.cc:740] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:21.084924  2344 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:21.059551  4921 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:21.111063  1714 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:21.169785  1713 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:03:21.170588  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1805) took 1138 ms. Trace:
I20260709 13:03:21.170730  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:21.171099  1713 rpcz_store.cc:276] 0709 13:03:20.032502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.129886 (+1097384us) service_pool.cc:224] Handling call
0709 13:03:21.169519 (+ 39633us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.170334 (+   815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.170533 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.169840  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1803) took 1131 ms. Trace:
I20260709 13:03:21.172297  1640 rpcz_store.cc:276] 0709 13:03:20.038338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.039658 (+  1320us) service_pool.cc:224] Handling call
0709 13:03:20.039998 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:20.040459 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:20.040484 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:20.069026 (+ 28542us) log.cc:844] Serialized 26 byte log entry
0709 13:03:20.181975 (+112949us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:20.182015 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.182080 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:21.011751 (+829671us) spinlock_profiling.cc:243] Waited 6.12 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:21.153792 (+142041us) raft_consensus.cc:1685] finished
0709 13:03:21.153817 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:21.169760 (+ 15943us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10136,"raft.queue_time_us":3549,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":3316,"spinlock_wait_cycles":14070016,"thread_start_us":79378,"threads_started":1,"wal-append.queue_time_us":186039}
I20260709 13:03:21.105363  5037 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:21.111073  1564 rpcz_store.cc:276] 0709 13:03:19.955880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.956935 (+  1055us) service_pool.cc:224] Handling call
0709 13:03:19.957276 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:20.928944 (+971668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.087229 (+158285us) spinlock_profiling.cc:243] Waited 971 ms on lock 0x7b54001275d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:21.087499 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2233513216}
I20260709 13:03:21.116289  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1838) took 1056 ms. Trace:
I20260709 13:03:21.116600  5110 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:21.117152  1722 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:21.117517  5078 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:21.119292  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:21.119561  1569 raft_consensus.cc:1240] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: [7.3-7.3]
I20260709 13:03:21.120432  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:21.121527  1567 raft_consensus.cc:1240] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:21.121721  5017 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:21.122658  4665 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:21.124401  1565 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:21.124859  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:21.152591  1721 raft_consensus.cc:2393] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 6: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:21.219489  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:21.223578  1538 leader_election.cc:336] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.506s (SENT)
I20260709 13:03:21.260950  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2258) took 1180 ms. Trace:
I20260709 13:03:21.261145  1538 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:21.155084  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2252) took 1104 ms. Trace:
I20260709 13:03:21.165789  1718 raft_consensus.cc:3055] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:21.167068  4930 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:21.163192  4979 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.167613  4919 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.279368  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:21.168668  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:21.174793  1715 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:03:21.176890  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:21.290120  1640 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:21.177855  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:21.290315  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.179500  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:21.180365  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:21.291054  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 67567.4us
I20260709 13:03:21.297557  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:21.182977  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:21.189543  1570 rpcz_store.cc:276] 0709 13:03:20.060005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.216958 (+156953us) service_pool.cc:224] Handling call
0709 13:03:20.221056 (+  4098us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:20.325806 (+104750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:20.325854 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:20.377720 (+ 51866us) log.cc:844] Serialized 26 byte log entry
0709 13:03:20.498574 (+120854us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:20.498637 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.498737 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:21.047783 (+549046us) spinlock_profiling.cc:243] Waited 34.9 ms on lock 0x7b2400166988. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245c1e9c4 00007f8245c1cacb 00007f8245dd7dc9 00007f8245dd7a55 00007f8245dd709d 00007f8248a0910e 00007f8248a2088c 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221
0709 13:03:21.115947 (+ 68164us) raft_consensus.cc:1685] finished
0709 13:03:21.115970 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:21.116222 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":873,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":40620,"spinlock_wait_cycles":80157952,"thread_start_us":120258,"threads_started":1,"wal-append.queue_time_us":280836}
I20260709 13:03:21.104822  4682 consensus_queue.cc:1059] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:03:21.192948  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:21.200948  5004 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.198414  1643 raft_consensus.cc:1240] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: [7.3-7.3]
I20260709 13:03:21.200950  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 286) took 1177 ms. Trace:
I20260709 13:03:21.201529  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:21.209518  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1813) took 1171 ms. Trace:
I20260709 13:03:21.128276  5035 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:21.261408  1721 rpcz_store.cc:276] 0709 13:03:20.080674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.260124 (+1179450us) service_pool.cc:224] Handling call
0709 13:03:21.260410 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.260611 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.260642 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:21.260683 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:21.260726 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.260781 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:21.260890 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.270515  1716 rpcz_store.cc:276] 0709 13:03:20.050842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.154279 (+1103437us) service_pool.cc:224] Handling call
0709 13:03:21.154563 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.154754 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.154782 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:21.154822 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:21.154858 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.154910 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:21.155025 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.271621  1718 raft_consensus.cc:740] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:21.202793  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:21.259245  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:21.275710  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:21.276475  1568 raft_consensus.cc:2393] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 7: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:21.282675  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:21.283907  2349 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:21.111214  1612 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:21.196188  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2255) took 1116 ms. Trace:
I20260709 13:03:21.299130  1564 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:03:21.193197  2346 raft_consensus.cc:1240] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:21.260432  4665 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.194047  2344 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:21.313395  1641 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:21.307447  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:21.327028  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:21.341871  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 230732us
I20260709 13:03:21.321440  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:21.342707  1718 consensus_queue.cc:260] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.328060  1717 rpcz_store.cc:276] 0709 13:03:20.037723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.912615 (+874892us) service_pool.cc:224] Handling call
0709 13:03:21.209293 (+296678us) spinlock_profiling.cc:243] Waited 79.2 ms on lock 0x7b54001afed4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:21.209451 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182087936}
I20260709 13:03:21.343477  1612 leader_election.cc:400] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7.
I20260709 13:03:21.344710  1612 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:21.163546  4962 consensus_queue.cc:1059] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.140s
I20260709 13:03:21.216431  4915 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.340072  4665 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:21.329303  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:21.313510  1722 rpcz_store.cc:276] 0709 13:03:20.023376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.116657 (+1093281us) service_pool.cc:224] Handling call
0709 13:03:21.116987 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.200681 (+ 83694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.200895 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.173200  4708 consensus_queue.cc:1059] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.138s
I20260709 13:03:21.339905  1721 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:21.127876  5054 consensus_queue.cc:260] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:21.311790  4989 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:21.281200  4988 consensus_queue.cc:1059] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:03:21.331748  1713 rpcz_store.cc:276] 0709 13:03:20.079449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.195372 (+1115923us) service_pool.cc:224] Handling call
0709 13:03:21.195684 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.195867 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.195893 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:21.195934 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:21.195973 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.196024 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:21.196134 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:21.358253  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 64393.1us
I20260709 13:03:21.351048  4671 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.351490  5069 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:21.373980  5112 log.cc:927] Time spent T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.096s	user 0.000s	sys 0.002s
I20260709 13:03:21.374528  4671 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.360885  5034 consensus_queue.cc:1048] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:21.378978  5034 consensus_queue.cc:1059] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:21.313510  5111 consensus_queue.cc:260] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:21.285455  4979 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.287120  4930 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.348582  5035 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:21.338174  4874 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:21.339112  5085 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:21.301291  4919 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.327087  5004 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.383025  4708 raft_consensus.cc:3055] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:21.384164  4921 raft_consensus.cc:3055] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:21.365612  4915 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.386981  4671 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:21.391467  5065 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:21.391707  4708 raft_consensus.cc:740] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:21.398756  4921 raft_consensus.cc:740] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:21.390663  4874 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.392422  5085 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:21.420979  4921 consensus_queue.cc:260] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.425019  5085 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:21.425809  5075 raft_consensus.cc:3055] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:21.426405  5075 raft_consensus.cc:740] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:21.426744  4930 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:21.416859  4979 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:21.429162  4991 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.431427  5004 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:21.432834  5035 raft_consensus.cc:697] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:21.430374  5075 consensus_queue.cc:260] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.434525  5035 consensus_queue.cc:237] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.435705  5111 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:21.433234  5065 raft_consensus.cc:697] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:21.439842  5101 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.728s	user 0.004s	sys 0.004s
W20260709 13:03:21.441314  5100 log.cc:927] Time spent T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.706s	user 0.007s	sys 0.000s
I20260709 13:03:21.334162  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:21.335005  1565 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:21.329596  1567 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:21.337162  1569 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:21.343544  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:21.363175  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:21.519976  1641 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:21.526224  1642 raft_consensus.cc:2410] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 13:03:21.525219  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1808) took 1491 ms (client timeout 1925 ms). Trace:
W20260709 13:03:21.528172  1720 rpcz_store.cc:269] 0709 13:03:20.036404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.144743 (+108339us) service_pool.cc:224] Handling call
0709 13:03:21.524634 (+1379891us) spinlock_profiling.cc:243] Waited 713 ms on lock 0x7b54002995d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:21.524978 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1639961984}
I20260709 13:03:21.499421  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1840) took 1438 ms. Trace:
I20260709 13:03:21.335602  1572 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:21.530927  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 446) took 1215 ms. Trace:
I20260709 13:03:21.536931  4708 consensus_queue.cc:260] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:21.541023  4708 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:21.542289  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:21.544690  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:21.546532  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:21.548437  1564 raft_consensus.cc:2410] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:03:21.548265  4865 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:21.500813  1645 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:21.342314  1568 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:21.504436  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:21.508621  1644 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:21.510777  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1803) took 1478 ms. Trace:
W20260709 13:03:21.513005  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 148122us
W20260709 13:03:21.499758  1538 leader_election.cc:336] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.637s (SENT)
I20260709 13:03:21.552807  1718 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:21.555627  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:21.555990  1570 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:21.568130  4917 raft_consensus.cc:3055] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:03:21.571213  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.575681  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:21.530742  1571 rpcz_store.cc:276] 0709 13:03:20.060584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.223346 (+162762us) service_pool.cc:224] Handling call
0709 13:03:20.290869 (+ 67523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:20.298770 (+  7901us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:20.298799 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:20.299327 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 13:03:20.451765 (+152438us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:20.451811 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.452170 (+   359us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:21.440965 (+988795us) spinlock_profiling.cc:243] Waited 37.7 ms on lock 0x7b2400166988. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245c1e9c4 00007f8245c1cacb 00007f8245dd7dc9 00007f8245dd7a55 00007f8245dd709d 00007f8248a0910e 00007f8248a2088c 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221
0709 13:03:21.498438 (+ 57473us) raft_consensus.cc:1685] finished
0709 13:03:21.498519 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:21.499031 (+   512us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7483,"raft.queue_time_us":7604,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":14454,"spinlock_wait_cycles":86662912,"thread_start_us":151972,"threads_started":1,"wal-append.queue_time_us":315235}
I20260709 13:03:21.531694  2347 rpcz_store.cc:276] 0709 13:03:20.315249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.315565 (+   316us) service_pool.cc:224] Handling call
0709 13:03:20.315922 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:20.316300 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:20.316328 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:20.317065 (+   737us) log.cc:844] Serialized 26 byte log entry
0709 13:03:20.431389 (+114324us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:20.431433 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.431502 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:21.438451 (+1006949us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:21.530002 (+ 91551us) raft_consensus.cc:1685] finished
0709 13:03:21.530045 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:21.530482 (+   437us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1781,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":9532,"spinlock_wait_cycles":61826304,"thread_start_us":113904,"threads_started":1,"wal-append.queue_time_us":291995}
I20260709 13:03:21.499804  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1792) took 2785 ms. Trace:
I20260709 13:03:21.619604  1573 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:21.615814  1715 rpcz_store.cc:276] 0709 13:03:20.032010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.127466 (+1095456us) service_pool.cc:224] Handling call
0709 13:03:21.174266 (+ 46800us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.288523 (+114257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.510062 (+221539us) spinlock_profiling.cc:243] Waited 46.3 ms on lock 0x7b5400211950. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:21.510420 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":106579200}
I20260709 13:03:21.632673  4988 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:21.638180  1565 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:21.640228  4991 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:21.641824  5084 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:21.642915  5084 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:21.622148  1639 rpcz_store.cc:276] 0709 13:03:18.714159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.714835 (+   676us) service_pool.cc:224] Handling call
0709 13:03:21.066201 (+2351366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.130022 (+ 63821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.498897 (+368875us) spinlock_profiling.cc:243] Waited 2.35 s on lock 0x7b54004114d0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:21.499346 (+   449us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5407448832}
I20260709 13:03:21.649864  4967 consensus_queue.cc:1059] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:21.650745  1571 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:21.621110  4917 raft_consensus.cc:740] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:21.653370  1571 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:21.673159  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:21.674283  1573 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:03:21.678598  4921 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:21.680143  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 170665us
I20260709 13:03:21.682016  4988 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:21.683455  4874 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:21.684487  4983 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:21.685484  4983 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:21.688931  4983 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:21.431660  4966 consensus_queue.cc:260] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:21.690922  4966 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:21.691361  1537 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:21.693845  5046 consensus_queue.cc:1175] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cf2f71592be8465fb18e36e15d0b2f89" responder_term: 6 status { last_received { term: 5 index: 5 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:21.702533  4915 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:21.703900  4915 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:03:21.705600  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 206335us
I20260709 13:03:21.696678  5072 consensus_queue.cc:1059] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:21.700743  4927 consensus_queue.cc:260] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:21.708269  4927 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:21.710150  5024 consensus_queue.cc:1175] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "130bc5ff39b847b1b3204626f284f838" responder_term: 11 status { last_received { term: 10 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:21.711443  1537 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:21.710203  5065 consensus_queue.cc:237] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.712534  4874 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:21.713438  4874 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.717392  5017 raft_consensus.cc:3055] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:21.718024  5017 raft_consensus.cc:740] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:21.720381  5017 consensus_queue.cc:260] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:21.728415  5017 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:21.748750  1566 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:21.760417  1641 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:21.767623  4908 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:21.768246  4919 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:21.773608  5046 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.775018  5046 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.750281  4967 raft_consensus.cc:3055] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:21.778008  4967 raft_consensus.cc:740] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:21.778402  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:21.779386  1568 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:03:21.758957  4915 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:21.797137  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:21.779661  4967 consensus_queue.cc:260] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:21.814128  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 102952us
W20260709 13:03:21.815963  4935 consensus_meta.cc:287] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.756s	user 0.018s	sys 0.000s
W20260709 13:03:21.816551  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.817130  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.817759  5054 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:21.826965  5075 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:21.830145  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 148920us
I20260709 13:03:21.832777  1611 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:21.815828  1537 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:21.791965  4988 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:21.838258  1645 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:21.729285  4917 consensus_queue.cc:260] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.846012  1566 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:03:21.847132  4865 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:03:21.847759  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.851190  5110 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.852885  4682 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:21.807205  4874 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:21.862449  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 173823us
I20260709 13:03:21.878664  1567 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:21.878809  4865 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.880298  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:21.882048  1569 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 8. Current term is 9. Ops: []
W20260709 13:03:21.882195  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 73481.8us
I20260709 13:03:21.883351  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:21.884207  1714 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:03:21.898465  1570 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:21.816591  4935 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:21.914947  1571 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:03:21.912155  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 365121us
W20260709 13:03:21.925750  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.930023  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.931038  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.893955  4915 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:21.936620  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.937564  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 228079us
I20260709 13:03:21.898048  4682 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:03:21.940526  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.942302  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.875003  4988 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:03:21.870164  5110 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:21.941008  5072 raft_consensus.cc:3055] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:21.958815  1567 raft_consensus.cc:1240] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:21.964545  5035 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:21.965562  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 75182.5us
I20260709 13:03:21.965879  4900 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:21.969224  1570 raft_consensus.cc:1275] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:03:21.973922  5046 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:21.975948  5124 log.cc:927] Time spent T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.090s	user 0.003s	sys 0.000s
W20260709 13:03:21.975353  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 159922us
I20260709 13:03:21.978672  1537 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:21.980410  1537 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:21.990232  4991 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:21.990814  4991 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.991255  4991 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:21.956555  5072 raft_consensus.cc:740] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:21.994163  4874 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:21.994984  4874 raft_consensus.cc:697] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:21.995086  5072 consensus_queue.cc:260] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:21.996287  4874 consensus_queue.cc:237] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:21.996928  5072 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:22.001794  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 169228us
I20260709 13:03:21.945243  4865 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:22.008500  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.025856  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 45683us
I20260709 13:03:22.028081  1572 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:22.005992  1571 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:21.975260  4967 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:22.035514  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 66236.6us
I20260709 13:03:21.950958  5084 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:22.038581  1566 raft_consensus.cc:1275] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:03:22.039781  1566 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:22.041066  1566 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:22.041395  1564 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
W20260709 13:03:22.046947  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.025270  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:22.022967  4917 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:22.025873  4935 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:22.024755  5127 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:22.073889  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:22.081089  1571 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:22.088699  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:22.089818  1720 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:22.092833  5110 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:22.094573  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:22.095397  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.096357  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.093966  5127 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:22.100539  5127 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:22.115221  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.142766  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 189648us
I20260709 13:03:22.001379  4682 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:22.237761  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:22.239912  1720 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:03:22.197058  1460 maintenance_manager.cc:419] P 220fdfc07a3c4ce28ca7333445633466: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033399
W20260709 13:03:22.242012  4966 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.550s	user 0.020s	sys 0.006s
W20260709 13:03:22.198437  5125 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.002s
I20260709 13:03:22.114430  1646 raft_consensus.cc:1240] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:22.202296  5120 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:22.237733  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:22.235039  2313 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:22.242548  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 172004us
W20260709 13:03:22.251256  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.241181  4682 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:22.261464  1565 consensus_meta.cc:287] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.622s	user 0.006s	sys 0.013s
I20260709 13:03:22.262367  1565 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:22.267445  1611 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:22.270488  1573 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: []
W20260709 13:03:22.263389  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 80721us
W20260709 13:03:22.272835  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.275341  1718 consensus_meta.cc:287] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.646s	user 0.015s	sys 0.004s
W20260709 13:03:22.257786  4927 consensus_meta.cc:287] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.548s	user 0.011s	sys 0.008s
I20260709 13:03:22.277149  1537 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:22.263054  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1821) took 1130 ms. Trace:
W20260709 13:03:22.258520  5017 consensus_meta.cc:287] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.529s	user 0.011s	sys 0.008s
I20260709 13:03:22.283373  1565 rpcz_store.cc:276] 0709 13:03:21.132226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.574609 (+442383us) service_pool.cc:224] Handling call
0709 13:03:22.262983 (+688374us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:22.267963  4921 consensus_meta.cc:287] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.583s	user 0.009s	sys 0.013s
I20260709 13:03:22.274122  2313 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:22.274899  5084 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:22.289724  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.295918  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.297176  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:22.285516  4682 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:22.299000  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.301170  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:22.303158  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.303056  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:22.303408  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.304582  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:22.308094  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:35480 (request call id 8) took 1340 ms. Trace:
I20260709 13:03:22.311134  1644 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:22.274542  4935 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.313500  1716 rpcz_store.cc:276] 0709 13:03:20.967535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.329180 (+361645us) service_pool.cc:224] Handling call
0709 13:03:22.307358 (+978178us) spinlock_profiling.cc:243] Waited 931 ms on lock 0x7b540038fed4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e8b735 00007f8248e839fa 00007f824de4b4c9 00007f82473c7c54 00007f82473c7be2 00007f82473c7b11 00007f82473c7a8c 00007f82473c6d42 00007f82471c048c 00007f82471bf8c6
0709 13:03:22.307816 (+   458us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2141482752}
W20260709 13:03:22.325870  1568 consensus_meta.cc:287] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.546s	user 0.016s	sys 0.001s
I20260709 13:03:22.326409  1568 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
W20260709 13:03:22.329044  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.330555  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.331390  2348 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:03:22.331708  1643 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:22.332763  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.326057  1714 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:03:22.333794  4915 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:03:22.334273  4908 consensus_meta.cc:287] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.566s	user 0.010s	sys 0.023s
I20260709 13:03:22.334522  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.335232  1640 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:22.336077  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:22.338049  4665 consensus_meta.cc:287] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.966s	user 0.011s	sys 0.020s
I20260709 13:03:22.338587  1647 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:22.338670  4665 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:22.339926  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1844) took 1204 ms. Trace:
I20260709 13:03:22.340430  1573 rpcz_store.cc:276] 0709 13:03:21.135770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.675208 (+539438us) service_pool.cc:224] Handling call
0709 13:03:22.267319 (+592111us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.277004 (+  9685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.339603 (+ 62599us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b5400405bd0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:22.339835 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1350729984}
I20260709 13:03:22.330826  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.341737  1718 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:22.342099  1639 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
W20260709 13:03:22.342736  5062 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:03:22.343497  1642 raft_consensus.cc:2368] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8.
I20260709 13:03:22.338188  4991 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:22.338179  1643 raft_consensus.cc:2393] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 6: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:22.343075  1644 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:22.346896  1611 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:22.336125  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.348431  1648 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:22.342875  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1819) took 1158 ms. Trace:
I20260709 13:03:22.350560  1646 rpcz_store.cc:276] 0709 13:03:21.184496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.307115 (+122619us) service_pool.cc:224] Handling call
0709 13:03:21.338896 (+ 31781us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.260950 (+922054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.342523 (+ 81573us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b54001f3454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:22.342796 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1521341184}
I20260709 13:03:22.347054  4991 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:22.356576  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:22.342792  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1811) took 2305 ms. Trace:
W20260709 13:03:22.368106  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.326272  1538 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.211s (SENT) [suppressed 48 similar messages]
I20260709 13:03:22.322328  4988 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:22.369541  1718 rpcz_store.cc:276] 0709 13:03:20.037215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.135605 (+1098390us) service_pool.cc:224] Handling call
0709 13:03:21.165547 (+ 29942us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.552027 (+386480us) spinlock_profiling.cc:243] Waited 5.95 ms on lock 0x7b58007ae1d0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248de10aa 00007f8248e63447 00007f8248e7bfe7 00007f8248e7d7de 00007f8248e87b49 00007f8248e88416 00007f8248e84d8f 00007f8248e8483e 00007f824de49419 00007f82473c0bd4
0709 13:03:22.342290 (+790263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.342689 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13677440}
I20260709 13:03:22.382831  1647 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
W20260709 13:03:22.392712  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 78017.4us
I20260709 13:03:22.374783  4915 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.405301  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:22.407752  4915 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:22.387254  4988 raft_consensus.cc:697] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:22.410688  4988 consensus_queue.cc:237] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:22.413162  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet eca30b43c26f420c965ee25f475581a0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:22.407789  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55888 (request call id 452) took 1220 ms. Trace:
W20260709 13:03:22.418859  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.419562  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.384984  4908 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:22.418241  2348 rpcz_store.cc:276] 0709 13:03:21.187502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.187655 (+   153us) service_pool.cc:224] Handling call
0709 13:03:22.392764 (+1205109us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540065b3d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:22.407669 (+ 14905us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2330965632}
W20260709 13:03:22.429279  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.447219  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.406414  4665 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:22.450349  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:22.464589  1571 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:03:22.402141  4935 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:22.474341  1647 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:22.472775  1686 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:03:22.490646  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.489399  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.495079  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.502611  1611 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:22.498333  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 244867us
W20260709 13:03:22.504506  4900 consensus_meta.cc:287] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.538s	user 0.013s	sys 0.005s
W20260709 13:03:22.505589  5130 log.cc:927] Time spent T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.113s	user 0.001s	sys 0.001s
I20260709 13:03:22.502329  5062 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:22.513130  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.510511  1642 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:22.516096  1641 raft_consensus.cc:2410] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:03:22.516135  4665 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:22.523556  4665 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.530850  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.531849  1642 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 4.
I20260709 13:03:22.515662  5062 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.536798  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:22.534098  1648 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:22.555981  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.537932  4665 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:22.540238  5062 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:22.568535  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.563732  5127 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:22.594983  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.604772  1647 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:22.559803  1469 catalog_manager.cc:5697] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 3 to 6, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 6 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:22.610970  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:22.612159  5127 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:22.614588  5127 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:22.616583  1611 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:22.616757  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 496) took 1114 ms. Trace:
I20260709 13:03:22.617419  1570 rpcz_store.cc:276] 0709 13:03:21.502525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.970506 (+467981us) service_pool.cc:224] Handling call
0709 13:03:21.971741 (+  1235us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.973245 (+  1504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.973362 (+   117us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:22.095131 (+121769us) log.cc:844] Serialized 26 byte log entry
0709 13:03:22.359135 (+264004us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:22.359199 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.359301 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.616337 (+257036us) raft_consensus.cc:1685] finished
0709 13:03:22.616364 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.616607 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":318,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":264,"thread_start_us":263308,"threads_started":1,"wal-append.queue_time_us":449999}
I20260709 13:03:22.621264  1647 raft_consensus.cc:2376] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8. Re-sending same reply.
I20260709 13:03:22.621104  4682 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.609797  5120 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.627563  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:22.639700  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.624045  4915 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:22.644419  5138 log.cc:927] Time spent T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:03:22.647833  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.642997  1469 catalog_manager.cc:5697] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 4 to 5, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:22.636049  4935 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:22.631157  5120 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:22.629463  4682 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:22.669806  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:22.661880  5072 consensus_meta.cc:287] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.659s	user 0.018s	sys 0.002s
W20260709 13:03:22.679903  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 76132.5us
I20260709 13:03:22.683645  1469 catalog_manager.cc:5697] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 8 to 9, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 9 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:22.687318  1469 catalog_manager.cc:5697] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 3 to 6, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 6 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:22.691072  5075 consensus_meta.cc:287] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.863s	user 0.011s	sys 0.009s
I20260709 13:03:22.692396  1611 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:22.689985  1469 catalog_manager.cc:5697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 4 to 6, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 6 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:22.700702  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:22.708014  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.669684  4966 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:03:22.721123  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.717463  1469 catalog_manager.cc:5697] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 4 to 5, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
W20260709 13:03:22.725888  5137 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:03:22.700830  4900 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:22.701087  5075 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:22.729122  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.731601  5142 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:22.731791  1469 catalog_manager.cc:5697] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 1 to 6, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 6 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:22.728722  4900 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:22.727982  4935 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.733517  4900 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:22.733721  1469 catalog_manager.cc:5697] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 4 to 6, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 6 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:03:22.734679  4935 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:22.735627  1469 catalog_manager.cc:5697] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:22.734052  4967 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:22.733543  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.737368  1469 catalog_manager.cc:5697] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 2 to 3, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:22.747670  1469 catalog_manager.cc:5697] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 5 to 6, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 6 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:22.750281  1469 catalog_manager.cc:5697] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:03:22.760568  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 50735.7us
I20260709 13:03:22.761549  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:22.753225  1469 catalog_manager.cc:5697] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 5 to 6, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 6 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:22.730037  4921 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:22.764087  5135 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.191s	user 0.001s	sys 0.001s
I20260709 13:03:22.780954  5065 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:22.781900  5065 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:22.783859  1570 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:03:22.784421  5065 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:22.785318  5065 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:22.758306  4967 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:22.789876  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.790949  1646 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:22.770262  4908 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:22.792917  5062 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:22.793692  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:22.795109  1570 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:22.795296  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:22.796073  2348 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:22.780582  4921 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:22.798053  5062 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:22.798805  4921 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:22.799355  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:22.799350  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:22.801187  2348 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:22.801129  1570 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 6: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:22.805811  5120 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:22.806963  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:22.807860  1570 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:22.809041  4908 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:22.720595  4966 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:22.808883  4665 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:22.800529  1469 catalog_manager.cc:5697] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 4 to 7, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 7 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:22.813267  4935 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:22.814816  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.821238  4682 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:22.802075  5075 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:22.825269  1469 catalog_manager.cc:5697] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 4 to 7, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 7 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:22.825877  5065 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:22.831244  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:22.793874  4967 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:22.835868  2348 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:03:22.838800  1572 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
W20260709 13:03:22.793830  5136 log.cc:927] Time spent T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.163s	user 0.000s	sys 0.002s
I20260709 13:03:22.809989  4908 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:22.842104  1469 catalog_manager.cc:5697] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 4 to 6, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 6 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:22.845335  1686 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:22.857185  4935 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.858033  4967 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.811096  5142 raft_consensus.cc:697] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:22.861720  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:22.864527  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:22.866539  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:22.866839  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:22.868155  1647 raft_consensus.cc:2376] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8. Re-sending same reply.
I20260709 13:03:22.870499  1641 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:22.836074  5120 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:22.873003  4917 consensus_meta.cc:287] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.802s	user 0.012s	sys 0.005s
W20260709 13:03:22.884343  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.892668  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 657934us
I20260709 13:03:22.895373  4966 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:22.868220  5062 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:22.870303  4967 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:22.882285  5065 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:22.891537  4935 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:22.897615  5075 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:22.897857  5145 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:22.918042  5145 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:22.918625  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:22.918906  2312 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:22.920128  4935 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:22.889848  1646 raft_consensus.cc:2376] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8. Re-sending same reply.
I20260709 13:03:22.920747  4935 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:22.921875  5145 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:22.921983  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:22.922724  1571 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:03:22.924150  2312 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:22.925096  5072 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:22.925490  5072 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.925917  5072 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:22.898169  1537 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:22.931195  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:22.932157  4874 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:22.932657  4874 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:22.933073  4874 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:22.933669  4915 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:22.900480  4908 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:22.934609  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:22.935388  1639 raft_consensus.cc:2368] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8.
I20260709 13:03:22.901897  4665 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:22.902046  5120 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:22.939019  5120 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:22.902168  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.938373  1537 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:22.940164  1643 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
W20260709 13:03:22.904417  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.942128  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.884996  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:22.913540  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:22.943477  1647 raft_consensus.cc:2410] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:03:22.913942  1644 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:22.880956  1469 catalog_manager.cc:5697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 4 to 6, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 6 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:22.910117  1648 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:22.908296  5062 raft_consensus.cc:697] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:22.941247  4874 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:22.910827  1570 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:22.900661  5046 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:22.967698  2312 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:22.968921  1469 catalog_manager.cc:5697] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 5 to 6, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 6 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
W20260709 13:03:22.972770  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.974092  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.975018  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.965091  4900 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:22.977926  5062 consensus_queue.cc:237] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:22.981138  4921 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.982502  4921 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:22.912576  5142 consensus_queue.cc:237] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:22.988056  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:22.990113  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:22.916225  4967 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:22.996140  1570 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 10: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:22.949584  4966 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.000380  1469 catalog_manager.cc:5697] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 4 to 5, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:23.001937  1568 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:23.003898  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:23.014209  4967 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.018115  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:23.018039  4967 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:23.019004  1569 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:23.019130  1645 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:22.993973  4935 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:23.022101  4935 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.022601  4935 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:23.028184  4874 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:23.033226  1569 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:22.981323  4900 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:23.016028  1469 catalog_manager.cc:5697] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 3 to 5, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
W20260709 13:03:23.048468  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 110568us
I20260709 13:03:23.038008  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:23.050355  1645 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:23.053059  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.052896  4927 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.054654  4927 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:23.055845  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:23.059381  1469 catalog_manager.cc:5697] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 4 to 11, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 11 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:23.056589  2349 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:22.996912  4915 raft_consensus.cc:697] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:23.076830  1469 catalog_manager.cc:5697] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:23.084033  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:22.994271  5046 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:23.099746  1718 raft_consensus.cc:2376] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10. Re-sending same reply.
I20260709 13:03:23.095516  5151 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:03:23.079658  5134 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:03:23.104627  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:23.106341  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:23.104627  1537 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:23.014976  4966 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:23.110532  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:23.112404  1715 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:23.113143  1469 catalog_manager.cc:5697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 5 to 7, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 7 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
W20260709 13:03:23.111902  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 80112.2us
I20260709 13:03:23.114537  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.117429  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:23.043009  4988 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:03:23.120481  1644 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:23.119678  1564 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:23.106410  5151 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:23.132606  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.137876  1644 raft_consensus.cc:1275] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:03:23.134059  1469 catalog_manager.cc:5697] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 1 to 5, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:23.143915  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:23.144778  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.145041  1648 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:23.145534  1718 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:23.120085  4967 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:23.079149  4915 consensus_queue.cc:237] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:23.135941  5151 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:23.162612  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:23.179610  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:23.172360  1469 catalog_manager.cc:5697] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:23.188714  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:23.189561  1646 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:23.190568  1719 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:03:23.192763  1721 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:03:23.194478  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:23.195791  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:23.196431  1718 raft_consensus.cc:2376] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6. Re-sending same reply.
I20260709 13:03:23.137241  5072 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:23.182315  1570 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:03:23.203154  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:23.206980  1643 raft_consensus.cc:1275] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
W20260709 13:03:23.209436  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.211596  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.212316  1720 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:03:23.210217  2312 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:23.207659  1643 pending_rounds.cc:85] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 2
I20260709 13:03:23.182688  4900 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.220224  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:23.220932  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.221413  2349 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:23.183487  5065 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:23.226650  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:23.227401  1647 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:23.225674  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.221621  1469 catalog_manager.cc:5697] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:23.229583  1715 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:03:23.202203  4927 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:23.217447  1643 pending_rounds.cc:107] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:03:23.234536  1643 raft_consensus.cc:2889] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:23.217459  5160 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.237869  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:23.237926  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:23.235443  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.282639  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:23.283470  1647 raft_consensus.cc:2376] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10. Re-sending same reply.
W20260709 13:03:23.287725  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 68675.3us
I20260709 13:03:23.288122  1720 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:23.291606   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:23.293794  4921 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:23.295681  5046 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:23.297047  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.230712  4900 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:23.298796  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:23.299539  2347 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:23.299908  4900 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:23.300474  5072 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:23.300534  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:23.301347  1647 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:23.305246  1716 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:23.307096  1639 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 4.
W20260709 13:03:23.306571  2314 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.714s (SENT)
I20260709 13:03:23.314343  1645 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:23.317016  5160 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:23.320214  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.321846  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:23.286839  4927 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:23.322779  1643 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
I20260709 13:03:23.345532  1648 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:23.350485  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:23.352727  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.355297  1647 raft_consensus.cc:2410] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:03:23.359966  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:23.364421  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 154334us
I20260709 13:03:23.368268  4935 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:23.368916  4966 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:23.398048  1648 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:23.403137  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:23.403798  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:23.405217  2347 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
W20260709 13:03:23.371063  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 83372.7us
I20260709 13:03:23.344805  4927 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:23.413650  2314 leader_election.cc:336] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.633s (SENT)
I20260709 13:03:23.426426  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:23.427294  1570 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:23.424274  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 68418us
W20260709 13:03:23.424263  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 120697us
W20260709 13:03:23.438048  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.438445  5062 consensus_queue.cc:1059] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:23.439666  1611 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:23.441053  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 217664us
I20260709 13:03:23.441516  5146 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:23.443324  5146 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:23.443791  5146 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.444242  5146 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:23.444893  5161 raft_consensus.cc:3055] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:23.446376  4921 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:23.446628  4966 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:23.450048  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:23.423022  4927 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:23.443528  4967 consensus_queue.cc:1059] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:23.450736  1565 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:23.446626  5161 raft_consensus.cc:740] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:23.474220  5161 consensus_queue.cc:260] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:23.475700  5161 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:23.475553  1569 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:03:23.454403  1573 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:23.441084  1611 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:23.446748  1687 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:23.489323  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:23.484310  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 48265.7us
I20260709 13:03:23.486901  4935 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:23.480162  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:23.483363  1538 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:23.485328  5162 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.424031  5155 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:23.488592  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 74982.2us
I20260709 13:03:23.478858  1611 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:23.504115  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:23.506520  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:23.510145  1571 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:23.520478  1569 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:23.513621  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:23.517107  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.518146  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:23.523523  1720 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:23.521438  1564 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:03:23.527315  1573 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
W20260709 13:03:23.535838  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 57107.5us
I20260709 13:03:23.536231  1565 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:03:23.536715  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:23.537519  1572 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
W20260709 13:03:23.538602  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 91987.8us
W20260709 13:03:23.539839  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 56581.5us
I20260709 13:03:23.542460  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:23.540575  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:23.543164  1564 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:23.542814  1611 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:23.550316  1567 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:23.551578  1687 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:23.562402  1570 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:03:23.566380  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.567589  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.536347  4921 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:23.577979  1685 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.546s (SENT)
I20260709 13:03:23.583113  2312 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:23.593235  2312 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:23.562947  4900 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:23.600220  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 48775.6us
I20260709 13:03:23.601902  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.603067  1720 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 10: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:23.545722  5142 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:23.542848  5164 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:23.611974  1572 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:03:23.606477  4900 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.612993  1721 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:23.613145  4900 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:23.614683  1687 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:23.615921  5169 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:23.606832  5017 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:23.607576  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.617736  5017 raft_consensus.cc:697] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:23.618156  1722 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:23.618857  5017 consensus_queue.cc:237] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:23.621740  1685 leader_election.cc:336] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.579s (SENT)
I20260709 13:03:23.621934  4900 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:23.599614  4935 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:03:23.630182  1538 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:23.611199  5165 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:23.623775  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:23.628254  5155 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:03:23.638562  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.545047  5120 consensus_queue.cc:1059] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:03:23.593870  5160 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:23.638785  1567 raft_consensus.cc:2410] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:23.654902  1572 raft_consensus.cc:606] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:23.639686  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 90972.5us
I20260709 13:03:23.655506  1572 raft_consensus.cc:614] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:23.658390  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.664232  1722 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:23.649499  4915 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:23.649163  5155 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:23.653946  5167 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:23.650615  5169 raft_consensus.cc:697] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:23.650281  5165 raft_consensus.cc:697] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:03:23.723871  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 93828.9us
I20260709 13:03:23.726949  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:23.721822  5142 raft_consensus.cc:697] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:23.732084  1538 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:23.694809  4915 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:23.748951  1564 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:23.698311  5167 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:23.760490  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:23.778456  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 68274.7us
I20260709 13:03:23.750340  4915 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:23.537259  5162 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:23.731179  4927 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:23.783025  1719 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:23.762467  5167 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:23.778442  5151 consensus_meta.cc:287] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.580s	user 0.009s	sys 0.004s
I20260709 13:03:23.795976  5152 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:23.758519  5160 consensus_queue.cc:1048] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:03:23.796646  5152 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.797222  5152 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:23.793848  4900 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:23.780671  5178 raft_consensus.cc:3055] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:03:23.830488  5170 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:03:23.792117  5120 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.842756  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:23.804764  5178 raft_consensus.cc:740] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:23.825385  5062 raft_consensus.cc:3055] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:23.860281  1538 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:23.864182  5175 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:23.864959  5175 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:23.865326  1538 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:23.867468  5150 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:23.802994  5151 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:23.887769  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.882694  5150 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:23.894526  5150 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:23.897006  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.889334  4874 raft_consensus.cc:2764] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:03:23.898005  1721 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:23.898904  4935 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.900081  4935 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:23.902637  1538 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:23.904752  1538 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:23.863246  5062 raft_consensus.cc:740] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:23.899250  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.847939  5180 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:23.913830  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 256116us
I20260709 13:03:23.916329  1611 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:23.855324  4988 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.920143  1719 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:23.844655  5120 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:23.917836  1611 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:23.924928  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.926867  1720 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:23.923347  1538 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:23.938678  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.935623  1611 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:23.918526  4874 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:23.918344  5154 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:23.943876  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:23.950115  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:23.818631  5165 consensus_queue.cc:237] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:23.955591  1611 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:23.817112  5169 consensus_queue.cc:237] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:24.009546  2349 raft_consensus.cc:2368] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 6. Current term is 7.
I20260709 13:03:24.011145  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:23.857431  5142 consensus_queue.cc:237] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:23.986464  1611 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:24.015470  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:24.016494  1721 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:23.925073  5065 consensus_queue.cc:1059] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 13:03:24.014532  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:23.943487  5186 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:24.026791  2348 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:23.954246  4874 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:23.985024  4900 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:24.035275  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.921739  5180 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:24.028875  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:23.985688  5183 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:23.935377  4988 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:23.985954  5154 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:24.024602  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:24.038551  1612 connection.cc:256] RPC call timeout handler was delayed by -1.0062918959999934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1876s for a response
I20260709 13:03:24.039577  4874 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:24.054661  1721 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:24.058094  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:24.035437  5186 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:24.058765  1572 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:24.039326  4900 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:24.057862  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:24.062435  1644 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:03:24.063273  1645 raft_consensus.cc:1240] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:24.064299  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:24.065023  1639 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:03:24.067006  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:24.068200  1641 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:03:24.069470  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:24.070384  1643 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:24.071323  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:24.072464  1646 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 7: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:24.043421  5183 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.073661  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:24.074612  1642 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
W20260709 13:03:24.075882  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 159110us
I20260709 13:03:24.076222  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:24.074666  1647 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:24.080415  1538 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:23.945452  5062 consensus_queue.cc:260] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:24.049446  5154 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:24.081820  5183 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:24.074276  5186 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:24.098316  4927 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:24.025878  5178 consensus_queue.cc:260] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:24.101040  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.104383  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:24.043622  5175 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:24.094049  5181 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 13:03:24.159094  1639 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:24.157348  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:24.186046  1646 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:03:24.160301  1612 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:24.161463  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:24.187675  1570 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:24.164266  5062 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:24.167577  5190 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.168394  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:24.172119  4988 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:24.174285  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:24.175443  5187 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:03:24.176399  5120 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:24.158596  5191 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:24.196875  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:24.199231  1572 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:24.157480  5162 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:24.081888  4900 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:24.220356  1641 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:24.222807  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 142547us
I20260709 13:03:24.217845  5175 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:24.220757  1644 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:03:24.224485  1643 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
W20260709 13:03:24.224695  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 238321us
I20260709 13:03:24.225049  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:24.245169  1648 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:03:24.059378  5192 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:03:24.227675  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 67496.4us
I20260709 13:03:24.221102  5178 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:24.228432  5189 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:24.148936  5164 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:24.247027  5151 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:24.251184  1538 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:24.235374  5191 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.246809  5175 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:24.269660  5120 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:24.269675  5175 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:24.273769  5162 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:24.285051  4935 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:24.238785  2312 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:24.242949  1567 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:24.247201  1612 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:24.227563  5190 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:24.250584  5192 raft_consensus.cc:697] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:24.275177  5191 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:24.342964  5176 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.550s	user 0.009s	sys 0.002s
I20260709 13:03:24.371212  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:24.400787  1640 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:24.362174  5142 raft_consensus.cc:3055] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:24.418736  5142 raft_consensus.cc:740] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:24.419639  5142 consensus_queue.cc:260] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:24.363535  5192 consensus_queue.cc:237] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:24.421185  5142 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:24.454295  2314 leader_election.cc:336] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.623s (SENT)
I20260709 13:03:24.504424  2314 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:24.509516  5017 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:24.510248  5127 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:24.567024  5200 log.cc:927] Time spent T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.209s	user 0.002s	sys 0.000s
I20260709 13:03:24.618255  5193 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:24.618978  5193 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:24.622149  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 383479us
I20260709 13:03:24.623111  1611 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:24.624142  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 378191us
W20260709 13:03:24.622305  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 375198us
I20260709 13:03:24.628825  1570 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:24.359057  5195 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:24.361133  5188 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:03:24.357559  5179 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:24.639719  1569 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
W20260709 13:03:24.656167  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.657474  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:24.639931  1641 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
W20260709 13:03:24.645731  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 394524us
I20260709 13:03:24.634184  2312 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:24.651855  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:24.510951  5127 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:24.691152  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 61423.4us
I20260709 13:03:24.661393  1565 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 8
W20260709 13:03:24.661835  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 207618us
W20260709 13:03:24.651930  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 900943us
I20260709 13:03:24.662336  1538 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:24.710289  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1830) took 1240 ms. Trace:
I20260709 13:03:24.632511  5017 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:24.663576  1640 raft_consensus.cc:2410] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:03:24.695852  5183 consensus_meta.cc:287] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.517s	user 0.014s	sys 0.005s
I20260709 13:03:24.716087  5183 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:24.640736  5197 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:24.710786  1714 rpcz_store.cc:276] 0709 13:03:23.470222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.507075 (+ 36853us) service_pool.cc:224] Handling call
0709 13:03:23.507460 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.507695 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:23.585932 (+ 78237us) log.cc:844] Serialized 14 byte log entry
0709 13:03:23.824088 (+238156us) spinlock_profiling.cc:243] Waited 3.21 ms on lock 0x7b180042a300. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a2523a
0709 13:03:24.709930 (+885842us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:24.709985 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:24.710039 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.710103 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:24.710227 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":885557,"prepare.queue_time_us":134759,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"spinlock_wait_cycles":7385472,"thread_start_us":112941,"threads_started":2,"wal-append.queue_time_us":150254}
I20260709 13:03:24.720121  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:24.733451  1716 raft_consensus.cc:2368] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 6. Current term is 7.
I20260709 13:03:24.720696  5120 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.735266  1687 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:24.721249  5150 consensus_meta.cc:287] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.826s	user 0.025s	sys 0.025s
I20260709 13:03:24.721210  5201 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.719988  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:24.722128  4874 consensus_meta.cc:287] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.620s	user 0.009s	sys 0.008s
I20260709 13:03:24.725358  5198 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:24.729967  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 67774.7us
I20260709 13:03:24.692152  4935 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:24.647536  5065 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
W20260709 13:03:24.744010  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.774848  5154 consensus_meta.cc:287] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.593s	user 0.018s	sys 0.000s
I20260709 13:03:24.775391  5154 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:24.747032  5175 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:24.648407  5195 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:24.777213  1572 raft_consensus.cc:1240] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5. Ops: []
W20260709 13:03:24.758466  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 124467us
W20260709 13:03:24.776440  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.779150  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.780364  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.772770  5185 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:24.707455  4988 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 6, Time since last communication: 0.043s
W20260709 13:03:24.836316  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 56449.7us
I20260709 13:03:24.756985  4935 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.737155  5179 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.839399  5151 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.852182  5179 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:24.734704  5164 raft_consensus.cc:2764] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 13:03:24.851440  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1843) took 1171 ms. Trace:
W20260709 13:03:24.863955  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet eca30b43c26f420c965ee25f475581a0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:03:24.865168  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 86411.7us
I20260709 13:03:24.866647  1612 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:24.870218  5180 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:24.872401  5180 raft_consensus.cc:2764] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:03:24.873997  5127 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:24.878263  5193 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:24.878571  5195 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:24.879456  5202 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 13:03:24.884841  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.750655  5150 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:24.806815  5190 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:24.898365  1565 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
W20260709 13:03:24.897421  2347 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.019s	sys 0.018s
W20260709 13:03:24.919798  2347 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.019s	sys 0.018s
I20260709 13:03:24.850348  5185 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.855645  4935 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:24.857406  1722 rpcz_store.cc:276] 0709 13:03:23.679964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.693490 (+ 13526us) service_pool.cc:224] Handling call
0709 13:03:23.706684 (+ 13194us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:23.838074 (+131390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:23.838121 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:23.856102 (+ 17981us) log.cc:844] Serialized 26 byte log entry
0709 13:03:24.036325 (+180223us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:24.036368 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.036436 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:24.851011 (+814575us) raft_consensus.cc:1685] finished
0709 13:03:24.851049 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:24.851303 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17882,"raft.queue_time_us":910508,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":16841,"thread_start_us":190910,"threads_started":2,"wal-append.queue_time_us":319585}
I20260709 13:03:24.931870  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55890 (request call id 365) took 1272 ms. Trace:
I20260709 13:03:24.742909  5120 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:24.932572  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 201973us
I20260709 13:03:24.934188  2347 rpcz_store.cc:276] 0709 13:03:23.659801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.659990 (+   189us) service_pool.cc:224] Handling call
0709 13:03:23.693766 (+ 33776us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.694030 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:24.244360 (+550330us) log.cc:844] Serialized 14 byte log entry
0709 13:03:24.920313 (+675953us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:24.920389 (+    76us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:24.931589 (+ 11200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.931672 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:24.931804 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":787050,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"thread_start_us":1072174,"threads_started":2}
W20260709 13:03:24.932072  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 62499.7us
I20260709 13:03:24.757768  4874 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:24.921433  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.939987  1564 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:24.944526  1685 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.503s (SENT)
I20260709 13:03:24.948163  1687 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:24.945900  1685 leader_election.cc:336] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.642s (SENT)
I20260709 13:03:24.934562  5185 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:24.953750  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:24.890506  5151 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:24.950974  5152 consensus_meta.cc:287] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.153s	user 0.015s	sys 0.021s
I20260709 13:03:24.985970  5152 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:24.987033  4874 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:24.760538  5201 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:24.991293  5186 consensus_meta.cc:287] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.762s	user 0.009s	sys 0.001s
I20260709 13:03:24.991955  5201 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:24.989313  5152 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:24.807118  5198 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:24.996889  5198 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:24.997340  5120 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:24.999049  5062 consensus_meta.cc:287] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.772s	user 0.014s	sys 0.014s
I20260709 13:03:24.999574  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:24.999909  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.000406  1718 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:25.003539  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:24.969151  5174 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.006s	sys 0.023s
W20260709 13:03:25.009006  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 51725.8us
W20260709 13:03:25.013007  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.013049  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:25.013548  1612 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:03:25.014248  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.010957  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:25.010409  1538 connection.cc:256] RPC call timeout handler was delayed by -1.2605281130000208s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20030000000000003s for a response
I20260709 13:03:25.015337  1569 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:24.975689  5161 consensus_meta.cc:287] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.499s	user 0.021s	sys 0.018s
W20260709 13:03:24.975979  5146 consensus_meta.cc:287] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.530s	user 0.021s	sys 0.014s
W20260709 13:03:25.016347  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 68302us
W20260709 13:03:25.008955  5155 consensus_meta.cc:287] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.283s	user 0.023s	sys 0.017s
W20260709 13:03:25.017510  1538 leader_election.cc:336] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.559s (SENT)
I20260709 13:03:25.031216  1715 raft_consensus.cc:2410] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 8 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 2.
W20260709 13:03:25.022677  5142 consensus_meta.cc:287] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.000s
I20260709 13:03:25.023039  5210 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:24.970310  5195 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:03:25.024885  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:24.957995  5180 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:24.987033  5179 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:25.017009  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.056906  1565 raft_consensus.cc:2393] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:03:25.026795  2314 leader_election.cc:336] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.057s (SENT)
I20260709 13:03:25.031090  1538 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:25.057722  2314 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:25.050597  5210 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:25.059605  5165 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:25.060454  5165 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:25.061046  5210 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:25.061008  5165 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:25.063602  5165 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:25.064621  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:25.065346  1564 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:25.012354  5174 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.084s	user 0.006s	sys 0.024s
I20260709 13:03:24.975836  5175 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:25.069908  5152 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:25.033037  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 206363us
W20260709 13:03:25.070405  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 124545us
I20260709 13:03:25.034590  1611 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:25.071415  5197 raft_consensus.cc:697] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:25.072687  1685 leader_election.cc:336] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.706s (SENT)
I20260709 13:03:25.073204  1685 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:25.074079  5161 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:03:25.075853  5181 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.345s	user 0.000s	sys 0.002s
I20260709 13:03:25.077144  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1832) took 1606 ms. Trace:
W20260709 13:03:25.076553  1685 leader_election.cc:336] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.739s (SENT)
I20260709 13:03:25.079392  1685 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:25.026818  5167 consensus_meta.cc:287] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.208s	user 0.014s	sys 0.000s
W20260709 13:03:25.038524  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 67440.2us
I20260709 13:03:25.030635  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.085767  1568 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:24.971277  5017 raft_consensus.cc:697] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:25.028223  1687 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:25.027448  5160 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:25.055127  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:25.058622  1538 leader_election.cc:336] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.568s (SENT)
I20260709 13:03:25.034199  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:24.991878  5186 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.079496  1564 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:25.037590  1569 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:25.028090  5154 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:25.077509  1713 rpcz_store.cc:276] 0709 13:03:23.471055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.471489 (+   434us) service_pool.cc:224] Handling call
0709 13:03:23.472055 (+   566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:23.472592 (+   537us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:23.642661 (+170069us) log.cc:844] Serialized 14 byte log entry
0709 13:03:24.652024 (+1009363us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:24.700733 (+ 48709us) log.cc:844] Serialized 26 byte log entry
0709 13:03:24.701213 (+   480us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:24.701262 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.701344 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:25.075554 (+374210us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b2400430178. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245c1e9c4 00007f8245c1cacb 00007f8245dd7dc9 00007f8245dd7a55 00007f8245dd709d 00007f8248a0910e 00007f8248a2088c 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221
0709 13:03:25.076810 (+  1256us) raft_consensus.cc:1685] finished
0709 13:03:25.076930 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.077084 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":882208,"prepare.queue_time_us":112965,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":4161,"raft.queue_time_us":932219,"raft.run_cpu_time_us":392,"raft.run_wall_time_us":399,"spinlock_wait_cycles":54131584,"thread_start_us":233223,"threads_started":2,"wal-append.queue_time_us":142593}
I20260709 13:03:25.090797  1722 raft_consensus.cc:2393] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:03:25.089931  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 55418.7us
I20260709 13:03:25.098111  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.101038  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.054203  5216 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:25.082418  5120 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:25.117837  1685 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.769s (SENT)
I20260709 13:03:25.033102  5155 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.102327  5175 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:25.031951  5146 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.135365  5214 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.139729  5214 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:25.135988  4935 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:25.141791  5062 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:25.143687  4935 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:25.111423  5161 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:03:25.146792  1685 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.786s (SENT)
W20260709 13:03:25.147544  1685 leader_election.cc:336] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.854s (SENT)
W20260709 13:03:25.148381  1685 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.869s (SENT)
I20260709 13:03:25.148763  1685 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:25.149022  5151 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:25.149093  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 94037.8us
I20260709 13:03:25.150494  1612 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:03:25.151086  4900 consensus_meta.cc:287] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.902s	user 0.008s	sys 0.012s
I20260709 13:03:25.092303  5167 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.113121  5120 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:25.151852  5142 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:03:25.133049  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 104924us
W20260709 13:03:25.113740  4915 consensus_meta.cc:287] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.317s	user 0.015s	sys 0.022s
I20260709 13:03:25.084709  5180 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.181980  4915 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.183836  5180 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:25.184577  4915 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:25.162163  5161 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:25.163280  5167 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:25.164029  5183 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:25.165050  5186 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:25.211010  1715 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1835) took 1708 ms (client timeout 1740 ms). Trace:
W20260709 13:03:25.211663  1715 rpcz_store.cc:269] 0709 13:03:23.502501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.502764 (+   263us) service_pool.cc:224] Handling call
0709 13:03:25.210684 (+1707920us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b5400247a54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:25.210923 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3473588480}
W20260709 13:03:25.212505  5191 consensus_meta.cc:287] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.849s	user 0.013s	sys 0.005s
W20260709 13:03:25.168022  5206 log.cc:927] Time spent T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:03:25.169294  5146 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:25.156000  5142 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:25.213608  5150 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:25.200207  4900 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.172709  5185 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:25.138643  1538 leader_election.cc:336] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.704s (SENT)
W20260709 13:03:25.214664  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 89429.8us
W20260709 13:03:25.215358  1538 leader_election.cc:336] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.936s (SENT)
I20260709 13:03:25.215816  1538 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:25.216053  5185 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:25.173848  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.175580  5155 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:25.156610  5175 raft_consensus.cc:697] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:25.218092  1564 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:25.218137  5150 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:03:25.219179  1538 leader_election.cc:336] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.169s (SENT)
I20260709 13:03:25.218920  5175 consensus_queue.cc:237] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.219759  1538 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:25.220903  5192 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:25.157992  1687 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:25.184787  1570 raft_consensus.cc:2393] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 7: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:25.222311  5185 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:03:25.190621  1537 connection.cc:256] RPC call timeout handler was delayed by -2.14803815099998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:03:25.191676  5214 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:25.222950  1537 connection.cc:256] RPC call timeout handler was delayed by -1.7074553330000413s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
W20260709 13:03:25.223353  1537 connection.cc:256] RPC call timeout handler was delayed by -1.1197553330000574s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2169s for a response
I20260709 13:03:25.193024  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:25.226265  5224 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.169s	user 0.000s	sys 0.002s
I20260709 13:03:25.227437  1570 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:25.193568  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.229171  5185 raft_consensus.cc:697] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:25.159252  5120 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:25.230676  1611 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:25.230857  4988 consensus_queue.cc:1059] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:25.231977  1611 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:25.232102  4988 raft_consensus.cc:3055] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:25.232615  4988 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:25.232816  5174 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:25.233347  5174 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.233728  5174 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:25.233810  5179 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:25.230734  4927 consensus_queue.cc:1059] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:03:25.234596  5179 raft_consensus.cc:697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:25.235476  4927 raft_consensus.cc:3055] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:25.236017  4927 raft_consensus.cc:740] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:25.165112  1612 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:25.213779  5142 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:25.172079  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:25.237383  2347 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
W20260709 13:03:25.222184  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 64378.9us
I20260709 13:03:25.240204  1687 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:25.241086  1687 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:25.241871  1687 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:25.187332  5162 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.242239  1611 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:25.072566  5197 consensus_queue.cc:237] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.242694  5162 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.243176  5146 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:25.243300  4988 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:25.193189  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.244730  5162 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:25.244844  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 79816.2us
I20260709 13:03:25.235575  5179 consensus_queue.cc:237] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.243979  5146 raft_consensus.cc:697] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:25.227358  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:25.256857  1564 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:03:25.258308  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:25.259259  1573 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:25.259433  5219 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:03:25.204051  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 55692.5us
I20260709 13:03:25.245839  1612 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:25.243211  1537 leader_election.cc:400] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 6. Current term is 7.
I20260709 13:03:25.263171  1537 leader_election.cc:403] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:03:25.264078  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.265177  1565 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:25.256302  5186 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:25.266541  1612 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:25.247138  5017 consensus_queue.cc:237] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:25.270176  5150 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 6. Current term is 7.
I20260709 13:03:25.242259  5185 consensus_queue.cc:237] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.138458  5201 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:25.250802  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:25.270550  4988 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:25.274370  1569 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:25.276264  1567 raft_consensus.cc:1275] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:25.276690  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:25.220974  1538 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.211s (SENT)
W20260709 13:03:25.273005  1685 leader_election.cc:336] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.048s (SENT)
I20260709 13:03:25.278152  1538 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:25.278461  1687 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:25.221544  5178 consensus_meta.cc:287] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.960s	user 0.013s	sys 0.000s
I20260709 13:03:25.274091  5195 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:25.289855  5211 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:03:25.282464  5146 consensus_queue.cc:237] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.286741  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:25.290633  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:25.291502  1566 raft_consensus.cc:2368] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8.
I20260709 13:03:25.287907  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:25.292869  1572 raft_consensus.cc:2393] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 7: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:25.294026  5198 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.252267  5191 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.294480  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.294696  5198 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.298115  5191 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:25.298631  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:25.310506  1572 raft_consensus.cc:2376] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10. Re-sending same reply.
I20260709 13:03:25.311332  4900 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:25.311472  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.274413  5209 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:25.312160  1572 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:25.312371  5209 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:25.299548  5198 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:25.312780  5209 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:25.277330  1573 raft_consensus.cc:2376] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7. Re-sending same reply.
I20260709 13:03:25.300369  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.313889  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.314453  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:25.315264  1573 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:25.318270  1687 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:25.315073  1570 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
W20260709 13:03:25.320245  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 47273.7us
W20260709 13:03:25.321139  1685 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.070s (SENT)
I20260709 13:03:25.324301  1612 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:25.326731  1685 leader_election.cc:336] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.102s (SENT)
I20260709 13:03:25.325953  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:25.326157  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 105221us
I20260709 13:03:25.322312  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.325417  5160 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:25.328295  5160 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:25.328673  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:25.329097  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.329836  1566 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
W20260709 13:03:25.330160  1538 leader_election.cc:336] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.219s (SENT)
I20260709 13:03:25.330691  1570 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:25.330698  1538 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:25.334342  1611 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:25.335709  5191 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:03:25.337195  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.337414  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.337690  4900 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:25.338151  4900 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.245537  4927 consensus_queue.cc:260] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:25.338552  4900 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:25.339203  4927 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:25.339280  5216 consensus_queue.cc:1059] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:25.340381  5216 consensus_queue.cc:1243] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:03:25.340408  4967 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.341226  4967 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.321907  4915 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:25.321856  5167 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:25.343410  5167 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.343811  5167 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:25.343914  4967 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:25.328719  5160 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:25.351676  4921 raft_consensus.cc:3055] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:25.352283  4921 raft_consensus.cc:740] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:25.353904  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.355052  1571 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:25.356982  1569 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:25.327314  1685 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:25.323082  5161 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:25.361328  2313 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:25.370370  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:25.324240  5214 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:25.376250  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:25.379148  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.378108  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:25.380386  1569 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:25.370926  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 61352.9us
I20260709 13:03:25.275466  5174 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.325781  5211 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:25.377357  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:25.402674  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:25.402674  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:25.410925  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:25.378187  5162 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:25.377971  5198 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:25.353248  4921 consensus_queue.cc:260] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.418291  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.418144  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:25.438498  1715 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:25.421976  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.433248  1564 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:03:25.425380  1569 raft_consensus.cc:2376] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7. Re-sending same reply.
I20260709 13:03:25.382396  1611 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:25.378364  5161 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:03:25.429384  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 102690us
I20260709 13:03:25.429396  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.431196  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:25.416291  4935 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:03:25.441280  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 58991.2us
I20260709 13:03:25.418157  4966 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:03:25.442553  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 112422us
I20260709 13:03:25.461541  1648 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
W20260709 13:03:25.464288  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.464968  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.372586  5120 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.425065  5211 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:25.475660  2312 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:25.444108  1722 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:25.476612  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:25.467594  1538 leader_election.cc:336] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.102s (SENT)
I20260709 13:03:25.417557  5214 raft_consensus.cc:697] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:25.491112  1641 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:25.444768  5179 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:25.396140  5142 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:25.479609  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:25.508966  1640 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:25.465683  4935 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.482127  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.485623  1612 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:25.485855  1721 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:25.487926  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.489084  1648 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:03:25.517149  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.477187  1570 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:25.500840  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.518105  1719 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:25.448398  5161 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:25.481742  5120 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:25.489743  5174 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:25.532574  2313 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:25.514350  1716 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:25.477484  1566 raft_consensus.cc:2376] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7. Re-sending same reply.
I20260709 13:03:25.471433  5169 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.510145  5210 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:25.523172  1646 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 8. Current term is 9. Ops: []
W20260709 13:03:25.523386  5235 log.cc:927] Time spent T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:03:25.524613  1713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:03:25.492645  5211 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:25.526145  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:25.478390  4988 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:25.525856  1717 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:03:25.521149  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.604321  1639 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:25.538074  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.603832  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:25.492249  4967 consensus_queue.cc:1059] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:03:25.541602  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.541692  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.621292  1565 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:25.520008  4935 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:25.546200  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:25.603581  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:25.529204  4921 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:25.535391  2313 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:25.609054  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:25.628521  1572 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:25.603152  5210 raft_consensus.cc:697] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:25.610044  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:25.611447  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.547436  5017 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:25.613010  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.613261  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.614135  1646 raft_consensus.cc:1240] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:25.614210  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:25.614557  1687 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:25.545814  5180 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:25.622646  1647 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:25.622592  1640 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:25.624267  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 138729us
I20260709 13:03:25.541159  5179 raft_consensus.cc:697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:25.631726  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 96462us
W20260709 13:03:25.632222  2314 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.606s (SENT)
W20260709 13:03:25.606911  5228 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:03:25.656585  2314 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:25.665529  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 118624us
I20260709 13:03:25.666414  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.668392  1565 raft_consensus.cc:2393] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:25.669160  1687 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:25.677564  1564 raft_consensus.cc:3055] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:25.678148  1564 raft_consensus.cc:740] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:25.678972  1564 consensus_queue.cc:260] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:25.680054  1564 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:25.681795  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.683005  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.684202  1571 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:25.684381  5215 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:25.630888  4874 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.686592  4874 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.638536  5017 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.691344  5017 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:25.696090  1641 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:25.699113  1611 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:25.703549  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:25.704598  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:25.705313  2347 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:25.548866  5160 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:25.710880  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.542155  5169 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:25.646865  4927 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:25.725731  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 225522us
I20260709 13:03:25.685439  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.731554  1571 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:25.734716  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.604043  4988 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:25.738377  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:25.735129  1564 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:03:25.743607  1564 pending_rounds.cc:85] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 9
I20260709 13:03:25.665500  5219 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:25.744799  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:25.748545  1573 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:25.689443  5142 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:25.748924  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.745354  1564 pending_rounds.cc:107] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:03:25.762095  1564 raft_consensus.cc:2889] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:25.762217  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.754818  1572 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:25.774361  1640 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:25.716054  5178 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:25.779800  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.780836  1566 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
I20260709 13:03:25.693451  5192 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:03:25.748169  5215 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:25.809829  4966 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.812510  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.813745  1718 raft_consensus.cc:2393] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:25.813916  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:25.814528  1719 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:25.809963  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 112572us
I20260709 13:03:25.816084  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.818264  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:25.823107  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.824381  1565 raft_consensus.cc:2393] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:25.817546  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:25.817586  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:25.652851  5214 consensus_queue.cc:237] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.800575  5219 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:03:25.782091  5210 consensus_queue.cc:237] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:25.804404  4874 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:25.830478  5178 raft_consensus.cc:697] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:25.834426  5215 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:25.838945  5179 consensus_queue.cc:237] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.838471  5240 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:25.879060  5215 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:25.839331  5241 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:25.880175  5241 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.880565  5241 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:25.882946  5178 consensus_queue.cc:237] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.886314  5175 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:25.891593  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.835345  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:25.893453  1611 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:25.907953  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 165345us
I20260709 13:03:25.910982  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:25.911449  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:25.912498  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:25.853670  5209 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.540s	user 0.008s	sys 0.003s
I20260709 13:03:25.901821  1721 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:25.904045  1719 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:25.810909  4966 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.915802  1611 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:25.860210  5186 consensus_meta.cc:287] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.594s	user 0.010s	sys 0.005s
I20260709 13:03:25.913749  1717 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:25.840437  5238 log.cc:927] Time spent T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:03:25.919157  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.917938  5201 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:25.922906  1565 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:25.930418  5183 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:25.934046  5245 consensus_queue.cc:1059] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:25.936523  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.937445  1717 raft_consensus.cc:2393] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:25.937613  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:25.925835  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:25.895202  5219 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:25.940037  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.844283  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:25.946180  1572 raft_consensus.cc:1275] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:25.939476  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.948218  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.842798  5169 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:25.904918  1722 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:25.843283  4988 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:25.855649  5160 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:25.930114  5186 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:25.919811  5209 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:25.940801  1647 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:25.926599  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.938277  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:25.979918  1717 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:25.979980  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.984879  1722 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:25.896068  5180 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.953094  2348 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
W20260709 13:03:25.957512  1715 consensus_meta.cc:287] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.512s	user 0.009s	sys 0.002s
I20260709 13:03:25.968089  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.969313  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.990111  1715 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:25.989711  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.991285  1716 raft_consensus.cc:2393] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 8: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:25.992386  1715 raft_consensus.cc:1275] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:25.888782  5175 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:25.986637  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:26.008463  1722 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:03:26.009084  1719 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:26.011224  1719 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:26.005545  1537 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:25.974624  5201 raft_consensus.cc:697] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:26.017081  1719 raft_consensus.cc:1275] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:03:26.000964  5185 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.001s	sys 0.012s
I20260709 13:03:26.018720  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:25.992264  5183 raft_consensus.cc:697] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:26.022351  1722 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:26.026607  1718 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: []
W20260709 13:03:26.029084  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 108806us
I20260709 13:03:26.007992  5160 raft_consensus.cc:3055] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:26.042582  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:26.043316  1722 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
W20260709 13:03:26.049857  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 234342us
I20260709 13:03:25.988888  4988 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:03:26.003028  5240 raft_consensus.cc:697] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:03:26.025950  5185 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.002s	sys 0.012s
I20260709 13:03:25.985541  5247 consensus_queue.cc:1059] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.086s
W20260709 13:03:26.063316  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 63986.5us
I20260709 13:03:26.076041  1537 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:25.969720  5219 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:26.021744  5191 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:26.062995  4900 consensus_meta.cc:287] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.724s	user 0.014s	sys 0.002s
I20260709 13:03:26.041342  5155 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:26.007153  5180 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:26.084911  2314 leader_election.cc:336] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.983s (SENT)
I20260709 13:03:26.089309  5175 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:26.090476  5242 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:03:26.054373  5160 raft_consensus.cc:740] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:26.099630  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:26.118546  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 144681us
I20260709 13:03:26.129983  5252 consensus_queue.cc:1059] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:26.130787  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:26.131811  5247 raft_consensus.cc:3055] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:26.077085  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:26.077183  5169 consensus_queue.cc:1059] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:03:26.260846  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:26.080713  5150 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:03:26.117391  5167 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.773s	user 0.012s	sys 0.004s
I20260709 13:03:26.298389  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:26.357590  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.485213  1719 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:26.459906  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 383989us
W20260709 13:03:26.461221  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 376355us
I20260709 13:03:26.262025  1640 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:26.460355  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 272088us
I20260709 13:03:26.262936  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:26.471063  1642 raft_consensus.cc:2376] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6. Re-sending same reply.
I20260709 13:03:26.474854  1564 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:03:26.474915  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:26.468900  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 239211us
I20260709 13:03:26.482044  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:26.482436  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:26.484263  1567 raft_consensus.cc:1275] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:26.498196  1568 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
W20260709 13:03:26.468277  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.263906  1646 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:26.490572  1641 raft_consensus.cc:2393] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:26.490119  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:26.490988  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 200604us
I20260709 13:03:26.491549  1648 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:26.486752  1537 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:26.415542  5161 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.877s	user 0.015s	sys 0.000s
I20260709 13:03:26.504449  1572 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:26.490341  2340 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
W20260709 13:03:26.333036  4935 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.692s	user 0.013s	sys 0.001s
W20260709 13:03:26.384552  5017 consensus_meta.cc:287] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.692s	user 0.011s	sys 0.005s
I20260709 13:03:26.509692  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:26.543393  5198 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:26.543787  5017 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:26.544683  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:26.545822  5146 consensus_queue.cc:1048] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:26.546393  1640 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:26.547354  5146 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:26.537987  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:26.510656  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:26.511157  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:26.549355  2348 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:26.549790  1565 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:26.550021  1647 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:26.122747  5192 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.276782  5197 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:26.522693  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:26.550127  1717 consensus_meta.cc:287] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.567s	user 0.005s	sys 0.006s
I20260709 13:03:26.551746  1717 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:26.522782  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:26.538347  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:26.485170  4921 consensus_meta.cc:287] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.847s	user 0.014s	sys 0.004s
W20260709 13:03:26.554157  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 93037.8us
I20260709 13:03:26.338542  5241 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:26.474328  5247 raft_consensus.cc:740] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:26.505334  5195 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.490340  5175 consensus_queue.cc:1048] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:03:26.137544  5183 consensus_queue.cc:237] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:26.265353  4966 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:26.563879  1720 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
W20260709 13:03:26.537621  2344 consensus_meta.cc:287] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.548s	user 0.009s	sys 0.000s
I20260709 13:03:26.566498  2344 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
W20260709 13:03:26.569841  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 83244.3us
I20260709 13:03:26.571166  1537 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:26.134733  5209 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:26.150624  5240 consensus_queue.cc:237] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:26.259981  5186 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.095615  4900 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:26.094713  5191 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:26.525627  5155 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:26.135425  5160 consensus_queue.cc:260] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:26.539875  5161 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:26.528095  1573 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:26.536293  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:26.135573  5180 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:26.535975  5211 consensus_meta.cc:287] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.927s	user 0.014s	sys 0.000s
I20260709 13:03:26.582113  5195 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:26.597733  5211 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:26.543219  1572 raft_consensus.cc:1275] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:03:26.601449  1721 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:26.518307  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:26.538501  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:26.601509  1687 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:26.557173  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:26.526192  5244 log.cc:927] Time spent T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.684s	user 0.005s	sys 0.001s
I20260709 13:03:26.572499  1647 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:26.340098  5219 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:26.610291  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:26.611748  1717 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:26.581997  1719 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:26.538277  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:26.635605  1643 raft_consensus.cc:2368] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10.
I20260709 13:03:26.585804  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:26.637619  1718 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:26.553699  5249 raft_consensus.cc:3055] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:26.589742  1648 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:26.191038  5201 consensus_queue.cc:237] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:26.594105  1641 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:26.589495  5150 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:26.556880  4935 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:26.599131  5247 consensus_queue.cc:260] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:26.604583  1646 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 9: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:26.556736  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:26.605589  1687 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:26.579329  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:26.614022  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:26.668294  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:26.510011  5167 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:26.563154  5197 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:26.563599  5192 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:26.562024  5248 consensus_queue.cc:1059] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 13:03:26.588438  5186 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:26.590161  5252 consensus_queue.cc:1243] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:27.162351  1647 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:26.704123  2314 leader_election.cc:336] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.151s (SENT)
W20260709 13:03:27.120613  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 549553us
I20260709 13:03:27.168237  2314 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:27.163547  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:27.134755  1611 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.017s	sys 0.017s
I20260709 13:03:27.163645  1570 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:27.165504  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:27.173287  1646 raft_consensus.cc:2393] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:03:27.170423  1611 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.019s	sys 0.017s
I20260709 13:03:27.176684  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:27.179857  1719 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:27.181377  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:26.981344  1713 raft_consensus.cc:606] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:27.177184  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 634163us
I20260709 13:03:27.188524  1719 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:27.197551  1640 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.002s	sys 0.052s
W20260709 13:03:27.198909  1640 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.003s	sys 0.052s
W20260709 13:03:27.170899  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 466719us
I20260709 13:03:27.169765  1717 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:26.980293  5264 net_util.cc:396] Time spent resolve address for 127.0.107.194: real 0.301s	user 0.000s	sys 0.000s
I20260709 13:03:27.285171  1714 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:03:27.286633  1538 leader_election.cc:336] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.675s (SENT)
W20260709 13:03:27.394933  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 108375us
I20260709 13:03:26.611981  4966 consensus_queue.cc:1059] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.693s
I20260709 13:03:27.398823  4966 consensus_queue.cc:1243] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:03:27.399418  4966 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:03:27.398197  1538 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.837s (SENT) [suppressed 53 similar messages]
W20260709 13:03:27.302927  1571 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.038s	sys 0.040s
I20260709 13:03:27.399960  4966 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:27.394933  1714 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2285) took 1976 ms (client timeout 2383 ms). Trace:
W20260709 13:03:27.402576  1571 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.044s	sys 0.051s
W20260709 13:03:27.402192  1714 rpcz_store.cc:269] 0709 13:03:25.419008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.419218 (+   210us) service_pool.cc:224] Handling call
0709 13:03:27.394664 (+1975446us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b540038fed4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:27.394862 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4233883904}
I20260709 13:03:27.389564  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1898) took 1941 ms. Trace:
I20260709 13:03:27.374758  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:27.406672  1642 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:27.206151  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:27.356345  2314 leader_election.cc:336] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.526s (SENT)
W20260709 13:03:27.411038  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 54790.5us
I20260709 13:03:27.411818  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1871) took 1134 ms. Trace:
I20260709 13:03:27.412261  1722 rpcz_store.cc:276] 0709 13:03:26.277277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.277547 (+   270us) service_pool.cc:224] Handling call
0709 13:03:26.278114 (+   567us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:26.290567 (+ 12453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:26.290603 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:26.469016 (+178413us) log.cc:844] Serialized 26 byte log entry
0709 13:03:26.742648 (+273632us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:26.742693 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.742772 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.411577 (+668805us) raft_consensus.cc:1685] finished
0709 13:03:27.411602 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.411759 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7360,"raft.queue_time_us":12271,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":5352,"thread_start_us":193439,"threads_started":1,"wal-append.queue_time_us":893155}
W20260709 13:03:27.417300  2314 leader_election.cc:336] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.138s (SENT)
I20260709 13:03:27.405241  1569 rpcz_store.cc:276] 0709 13:03:25.448606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.468078 (+ 19472us) service_pool.cc:224] Handling call
0709 13:03:25.468850 (+   772us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:25.473570 (+  4720us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:25.473616 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:25.501319 (+ 27703us) log.cc:844] Serialized 26 byte log entry
0709 13:03:25.715896 (+214577us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:25.715950 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.716043 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:26.460098 (+744055us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:27.305226 (+845128us) raft_consensus.cc:1685] finished
0709 13:03:27.305250 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.389139 (+ 83889us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4694,"raft.queue_time_us":4242,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":23764,"spinlock_wait_cycles":32237952,"thread_start_us":98683,"threads_started":1,"wal-append.queue_time_us":284050}
I20260709 13:03:27.418044  5150 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:26.669746  5249 raft_consensus.cc:740] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:03:27.013499  5211 net_util.cc:396] Time spent resolve address for 127.0.107.196: real 0.289s	user 0.000s	sys 0.000s
I20260709 13:03:27.172904  5155 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.430120  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:27.173012  5127 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:27.433466  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 264711us
I20260709 13:03:27.287123  1644 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:27.238431  1719 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
W20260709 13:03:27.436048  1686 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.861s	user 0.013s	sys 0.019s
W20260709 13:03:27.470976  1686 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.013s	sys 0.020s
I20260709 13:03:27.404168  5247 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:27.436457  1564 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.016s	sys 0.017s
W20260709 13:03:27.473717  1564 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.016s	sys 0.017s
I20260709 13:03:27.474963  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1882) took 1056 ms. Trace:
I20260709 13:03:27.444567  1570 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:27.475404  1719 rpcz_store.cc:276] 0709 13:03:26.418912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.180978 (+762066us) service_pool.cc:224] Handling call
0709 13:03:27.474894 (+293916us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:27.447259  1641 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:03:27.448223  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:27.479033  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:27.422780  5150 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:27.377182  5169 raft_consensus.cc:3055] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:27.377878  4967 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:27.450970  1717 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
W20260709 13:03:27.456282  1718 consensus_meta.cc:287] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.768s	user 0.009s	sys 0.007s
I20260709 13:03:27.499910  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1876) took 1083 ms. Trace:
I20260709 13:03:27.499840  1718 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:27.501200  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1879) took 1083 ms. Trace:
I20260709 13:03:27.501775  1718 rpcz_store.cc:276] 0709 13:03:26.417958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.559756 (+141798us) service_pool.cc:224] Handling call
0709 13:03:27.500950 (+941194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:27.012588  5214 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:27.032085  5178 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:27.209992  5257 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.479938  1646 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 9: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:27.480270  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2373) took 1027 ms. Trace:
I20260709 13:03:27.515172  1641 rpcz_store.cc:276] 0709 13:03:26.454671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.509514 (+ 54843us) service_pool.cc:224] Handling call
0709 13:03:27.480199 (+970685us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:27.418818  5197 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:27.302520  5267 log.cc:927] Time spent T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.235s	user 0.000s	sys 0.000s
W20260709 13:03:27.483711  1715 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.898s	user 0.056s	sys 0.064s
W20260709 13:03:27.532613  1715 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.056s	sys 0.064s
W20260709 13:03:27.497041  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.429169  5167 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:27.534678  5167 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:27.535423  5167 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:27.500695  1717 rpcz_store.cc:276] 0709 13:03:26.416856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.630289 (+213433us) service_pool.cc:224] Handling call
0709 13:03:27.499807 (+869518us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:27.503782  1685 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.022s	sys 0.017s
W20260709 13:03:27.538347  1685 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.022s	sys 0.017s
I20260709 13:03:27.472700  5269 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:27.516196  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:27.546504  1648 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:03:27.471472  5256 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:03:27.538740  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 870419us
I20260709 13:03:27.484472  5169 raft_consensus.cc:740] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:27.499502  4967 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.504048  4935 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:27.513166  5241 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:27.580955  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:27.596982  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 991244us
I20260709 13:03:27.608549  1687 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:27.582792  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:27.619632  1647 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:27.623827  1648 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 9: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:27.508240  5178 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:27.640897  5216 raft_consensus.cc:3055] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:27.641481  5216 raft_consensus.cc:740] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:27.641534  5178 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:27.643501  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:27.644120  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2377) took 1081 ms. Trace:
I20260709 13:03:27.565615  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:27.541458  5257 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:27.634267  5268 log.cc:927] Time spent T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:03:27.635092  5186 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:27.595865  5269 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:27.636626  4966 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:27.637147  5278 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.638175  5277 raft_consensus.cc:3055] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:27.638464  5169 consensus_queue.cc:260] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:27.639621  5150 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:27.644258  1641 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:27.580919  4967 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:27.480921  5155 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:27.644892  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:27.677977  1685 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:27.749850  1646 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:27.681221  5245 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:03:27.766535  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 290501us
I20260709 13:03:27.645712  1647 rpcz_store.cc:276] 0709 13:03:26.562492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.667393 (+104901us) service_pool.cc:224] Handling call
0709 13:03:27.644037 (+976644us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:27.581779  5256 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:27.748462  5277 raft_consensus.cc:740] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:27.863572  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1881) took 1445 ms. Trace:
I20260709 13:03:27.868331  1537 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:27.874991  1714 rpcz_store.cc:276] 0709 13:03:26.418588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.422028 (+1003440us) service_pool.cc:224] Handling call
0709 13:03:27.422421 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:27.422642 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:27.422927 (+   285us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.863146 (+440219us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:27.863223 (+    77us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:27.863277 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.863353 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.863503 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":993,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":13365,"thread_start_us":391403,"threads_started":1}
I20260709 13:03:27.876050  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:27.863960  2314 leader_election.cc:336] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.234s (SENT)
W20260709 13:03:27.845850  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 392022us
I20260709 13:03:27.681049  5195 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:27.634301  5192 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:27.754427  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 275700us
I20260709 13:03:27.674149  5241 consensus_queue.cc:1048] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.198s
I20260709 13:03:27.654610  5191 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:27.871834  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:27.895939  1646 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:27.896034  1644 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:27.897068  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:27.898204  1646 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:27.872535  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 194659us
I20260709 13:03:27.901108  1685 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:27.837529  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:27.884128  2314 leader_election.cc:336] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.173s (SENT)
I20260709 13:03:27.856483  5180 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.903062  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:27.613350  5249 consensus_queue.cc:260] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:27.862033  5214 raft_consensus.cc:697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 8 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:27.867612  5142 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:27.871752  5197 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.920760  5197 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:27.922966  5197 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:27.912282  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 44106.3us
I20260709 13:03:27.926124  1537 leader_election.cc:400] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 6. Current term is 7.
I20260709 13:03:27.926877  1537 leader_election.cc:403] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:27.915292  1641 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:27.911340  5180 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:27.916950  5260 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 1.449s	user 0.007s	sys 0.017s
I20260709 13:03:27.762064  5278 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:27.920102  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 535) took 1402 ms. Trace:
I20260709 13:03:27.919893  5142 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:27.937055  1564 rpcz_store.cc:276] 0709 13:03:26.517380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.518343 (+   963us) service_pool.cc:224] Handling call
0709 13:03:26.518818 (+   475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:26.555109 (+ 36291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:26.555140 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:26.565001 (+  9861us) log.cc:844] Serialized 26 byte log entry
0709 13:03:27.474214 (+909213us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:27.474267 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.474352 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.919503 (+445151us) raft_consensus.cc:1685] finished
0709 13:03:27.919527 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.919924 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22033,"raft.queue_time_us":24186,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"thread_start_us":863161,"threads_started":1,"wal-append.queue_time_us":1287441}
I20260709 13:03:27.927958  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2378) took 1342 ms. Trace:
I20260709 13:03:27.947652  1641 rpcz_store.cc:276] 0709 13:03:26.585373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.914458 (+1329085us) service_pool.cc:224] Handling call
0709 13:03:27.914944 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:27.927604 (+ 12660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.927879 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:27.889926  5183 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.004s	sys 0.032s
W20260709 13:03:27.939110  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 327120us
W20260709 13:03:27.935873  1571 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.000s	sys 0.017s
W20260709 13:03:27.942976  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.896418  5167 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:27.956231  1648 raft_consensus.cc:2376] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9. Re-sending same reply.
I20260709 13:03:27.646819  5216 consensus_queue.cc:260] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:27.951807  1722 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.019s	sys 0.021s
I20260709 13:03:27.978541  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:27.931529  5169 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:27.989625  1642 consensus_meta.cc:287] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.571s	user 0.012s	sys 0.000s
I20260709 13:03:28.014346  1642 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
W20260709 13:03:28.014720  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 88812.6us
I20260709 13:03:28.015514  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:27.948951  5183 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.005s	sys 0.032s
I20260709 13:03:28.011332  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57572 (request call id 18) took 2504 ms. Trace:
W20260709 13:03:28.010704  1571 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.000s	sys 0.022s
W20260709 13:03:28.001765  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.015550  1537 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:28.016793  1642 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:28.009698  1612 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:27.936655  5150 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:28.019004  1718 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:28.020960  1713 rpcz_store.cc:276] 0709 13:03:25.479036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.507579 (+ 28543us) service_pool.cc:224] Handling call
0709 13:03:27.976241 (+2468662us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54004079d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e7f853 00007f824de4d60a 00007f82473d37d4 00007f82473d3762 00007f82473d3691 00007f82473d360c 00007f82473d28c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:27.976356 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2764897536}
I20260709 13:03:27.954614  5288 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:28.035651  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 539) took 1486 ms. Trace:
W20260709 13:03:27.978840  5236 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.521s	user 0.016s	sys 0.041s
I20260709 13:03:27.929979  5280 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.109s
I20260709 13:03:28.011365  5197 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.038276  1571 rpcz_store.cc:276] 0709 13:03:26.549313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.550549 (+  1236us) service_pool.cc:224] Handling call
0709 13:03:26.573460 (+ 22911us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.574682 (+  1222us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:27.403289 (+828607us) log.cc:844] Serialized 14 byte log entry
0709 13:03:28.034719 (+631430us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:28.034820 (+   101us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:28.034885 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.035044 (+   159us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.035504 (+   460us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11770,"prepare.queue_time_us":823379,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"thread_start_us":1432248,"threads_started":2}
I20260709 13:03:27.953233  5287 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:28.045181  1612 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:28.047423  5285 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:28.055392  5285 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:27.953787  5286 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 6. Current term is 7.
I20260709 13:03:28.017892  5245 raft_consensus.cc:697] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:28.036563  5211 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:28.070927  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:28.050429  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:28.077720  1713 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:28.078138  1714 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:28.079413  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1861) took 2164 ms. Trace:
I20260709 13:03:28.079975  1715 rpcz_store.cc:276] 0709 13:03:25.945360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.013653 (+ 68293us) service_pool.cc:224] Handling call
0709 13:03:26.015447 (+  1794us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:26.015894 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:26.015929 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:26.029342 (+ 13413us) log.cc:844] Serialized 26 byte log entry
0709 13:03:26.310251 (+280909us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:26.551130 (+240879us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.533200 (+982070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.533311 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.882553 (+349242us) spinlock_profiling.cc:243] Waited 6.04 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:28.073244 (+190691us) raft_consensus.cc:1685] finished
0709 13:03:28.079157 (+  5913us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.079341 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32349,"prepare.queue_time_us":398339,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":18757,"raft.queue_time_us":40001,"raft.run_cpu_time_us":401,"raft.run_wall_time_us":21627,"spinlock_wait_cycles":13887744,"thread_start_us":1418548,"threads_started":3,"wal-append.queue_time_us":401931}
I20260709 13:03:28.082620  5142 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:28.083454  5174 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:28.085137  5256 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:28.027043  5192 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:28.071408  5214 consensus_queue.cc:237] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:28.086146  5017 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:28.110283  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55890 (request call id 375) took 2186 ms. Trace:
I20260709 13:03:28.114446  5269 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:28.090762  5155 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:27.916868  5277 consensus_queue.cc:260] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:28.137329  2349 rpcz_store.cc:276] 0709 13:03:25.923842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.923980 (+   138us) service_pool.cc:224] Handling call
0709 13:03:25.924263 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.121321 (+197058us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:26.121367 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:26.121421 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:26.121478 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.110028 (+1988550us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b5400549054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:28.110080 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.110198 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":452591488}
I20260709 13:03:28.106555  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2294) took 2288 ms. Trace:
W20260709 13:03:28.095762  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2372) took 1831 ms (client timeout 1643 ms). Trace:
I20260709 13:03:28.104781  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:28.156302  1648 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
W20260709 13:03:28.119381  1538 leader_election.cc:336] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.003s (SENT)
I20260709 13:03:28.064039  5285 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:28.049937  5191 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:28.099282  5257 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:28.060981  1642 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5.
W20260709 13:03:28.163134  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 147686us
I20260709 13:03:28.163981  1537 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:28.164495  1642 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:28.165267  1537 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:28.125267  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:03:28.128283  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 53573.2us
W20260709 13:03:28.128898  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 227895us
I20260709 13:03:28.127938  5215 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.168691  1612 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:28.169677  1648 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:03:28.171614  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:28.173703  1685 connection.cc:256] RPC call timeout handler was delayed by -3.0471281790000262s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18580000000000002s for a response
W20260709 13:03:28.175752  1685 connection.cc:256] RPC call timeout handler was delayed by -2.9679737019999948s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2028s for a response
I20260709 13:03:28.065220  5249 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:28.175994  5286 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:28.176355  1612 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:28.176589  5286 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:28.177037  5286 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:28.177824  5285 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:28.178634  5285 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:28.154300  1721 rpcz_store.cc:276] 0709 13:03:25.818235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.979771 (+161536us) service_pool.cc:224] Handling call
0709 13:03:26.005138 (+ 25367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.602398 (+597260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.106247 (+1503849us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b5400212854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:28.106457 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1370920192}
I20260709 13:03:28.130388  5256 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:03:28.180307  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:28.181156  1721 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:28.182054  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:28.155246  1644 rpcz_store.cc:269] 0709 13:03:26.331608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.482220 (+150612us) service_pool.cc:224] Handling call
0709 13:03:28.095512 (+1613292us) spinlock_profiling.cc:243] Waited 734 ms on lock 0x7b54001fe0d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:28.095644 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1689158016}
I20260709 13:03:28.182829  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:28.183559  2348 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:28.159271  5195 consensus_queue.cc:1048] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:03:28.179308  5285 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:28.182238  5290 log.cc:927] Time spent T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:03:28.186061  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:28.186563  5195 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:28.186087  5285 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:28.188405  5195 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:28.168622  5017 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.190140  5017 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:28.189368  5285 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:28.054173  5197 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:28.194212  5215 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:28.196696  5197 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:28.196812  5215 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:28.160266  5155 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.198680  5277 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:28.198964  1685 connection.cc:256] RPC call timeout handler was delayed by -2.8941737019999891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.211s for a response
I20260709 13:03:28.161594  5243 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
W20260709 13:03:28.199923  1685 connection.cc:256] RPC call timeout handler was delayed by -2.8336001319999582s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22010000000000002s for a response
W20260709 13:03:28.200384  1685 connection.cc:256] RPC call timeout handler was delayed by -1.703197264000039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
W20260709 13:03:28.051255  5236 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.627s	user 0.021s	sys 0.042s
W20260709 13:03:28.200681  1685 connection.cc:256] RPC call timeout handler was delayed by -1.3467972640000312s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1945s for a response
W20260709 13:03:28.201220  1685 connection.cc:256] RPC call timeout handler was delayed by -1.3458128450000117s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3307s for a response
W20260709 13:03:28.201629  1685 connection.cc:256] RPC call timeout handler was delayed by -1.2936972640000022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20040000000000002s for a response
I20260709 13:03:28.129972  5192 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:28.202234  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:28.203775  5192 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:28.204074  5161 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:28.207705  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2291) took 2394 ms (client timeout 2042 ms). Trace:
W20260709 13:03:28.208284  1720 rpcz_store.cc:269] 0709 13:03:25.812711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.815822 (+  3111us) service_pool.cc:224] Handling call
0709 13:03:28.207362 (+2391540us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b540020a154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:28.207578 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1718397696}
I20260709 13:03:28.210350  1716 raft_consensus.cc:1275] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:03:28.175359  5245 consensus_queue.cc:237] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:28.177858  5292 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:28.213210  1713 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:28.172287  2340 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:28.197816  5155 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:28.228165  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 56262.6us
I20260709 13:03:28.214170  1646 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:28.215860  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1880) took 1797 ms. Trace:
I20260709 13:03:28.229132  1537 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:28.229290  1722 rpcz_store.cc:276] 0709 13:03:26.418284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.424324 (+1006040us) service_pool.cc:224] Handling call
0709 13:03:27.424729 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:27.424959 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:28.008795 (+583836us) log.cc:844] Serialized 14 byte log entry
0709 13:03:28.215465 (+206670us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:28.215520 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:28.215568 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.215632 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.215785 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":59701,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":60,"raft.queue_time_us":107,"thread_start_us":766576,"threads_started":2}
I20260709 13:03:28.210510  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:28.166577  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:03:28.231992  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 54332.7us
I20260709 13:03:28.197480  5197 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:28.215961  1720 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:28.234128  1612 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:28.239936  2348 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:28.240527  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:28.241238  2340 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:28.199247  5278 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:28.212287  5287 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:28.244355  1612 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:28.216563  5017 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:28.254182  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:28.260695  1537 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:28.268536  2348 raft_consensus.cc:2393] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:28.261897  1686 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:03:28.244686  5248 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.903s	user 0.008s	sys 0.012s
I20260709 13:03:28.253095  5287 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.271837  1537 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:28.270390  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:28.271661  1612 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:28.280002  1537 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:28.238566  5195 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:28.238384  5197 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.282625  1612 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:28.285487  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.269994  2346 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 7: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:28.278873  5174 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:28.286978  5142 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:28.251274  5191 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:28.287849  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:28.278705  5248 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.944s	user 0.008s	sys 0.012s
I20260709 13:03:28.300290  1612 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:28.284580  5286 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:28.249346  5161 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.297504  2347 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:28.308101  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:28.298069  1722 raft_consensus.cc:1275] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:28.263411  5192 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:28.279160  5297 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.286230  5278 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:28.284020  5195 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:28.310652  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 173667us
I20260709 13:03:28.315075  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:28.320215  1567 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 7: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:28.278268  5285 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:28.296471  5197 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:28.323642  1685 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.549s (SENT)
I20260709 13:03:28.324162  1685 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:28.324370  1612 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:28.325388  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:28.281649  5287 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:28.334936  1722 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 9: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:28.336221  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:28.337854  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:28.338652  1722 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:03:28.302800  5257 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:28.344722  5286 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:28.345752  5286 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:28.346482  5286 raft_consensus.cc:697] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:28.347690  5286 consensus_queue.cc:237] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:28.349233  1612 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:28.350598  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:28.349853  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 69928.4us
I20260709 13:03:28.323972  5195 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:28.355428  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:28.362457  1571 raft_consensus.cc:2393] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:28.311627  5183 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:28.321076  5278 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:28.365589  1612 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:28.368093  1722 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:28.371239  1537 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:28.376879  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:28.353683  5192 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:28.314955  5161 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:28.350930  5142 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:03:28.351682  5178 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:28.385097  1722 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:28.388454  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:28.389514  1722 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 8: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:28.392374  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:28.393837  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 83827.1us
W20260709 13:03:28.382802  4967 consensus_meta.cc:287] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.574s	user 0.009s	sys 0.005s
I20260709 13:03:28.366665  5257 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:28.407158  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1853) took 3017 ms (client timeout 2201 ms). Trace:
I20260709 13:03:28.406088  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:28.323546  5297 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:28.390295  5285 consensus_queue.cc:1059] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:03:28.799466  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:28.837303  2340 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.592s	user 0.020s	sys 0.002s
W20260709 13:03:28.473891  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.804388  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:28.969759  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.851058  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:03:28.753816  1713 consensus_meta.cc:287] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.530s	user 0.008s	sys 0.006s
I20260709 13:03:28.378528  5278 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:28.800055  1566 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:28.991671  2349 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:28.972308  1565 raft_consensus.cc:2410] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 11 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 13:03:28.986006  1537 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.004s	sys 0.024s
W20260709 13:03:28.973379  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.519600  1719 raft_consensus.cc:1275] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:28.432318  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:28.774642  5300 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
W20260709 13:03:28.996585  1537 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.004s	sys 0.024s
W20260709 13:03:28.998221  1564 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.007s	sys 0.033s
I20260709 13:03:28.999318  1571 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 7: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:28.998234  2340 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:28.993706  1713 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:28.998164  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:28.409044  1643 rpcz_store.cc:269] 0709 13:03:25.377707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.377829 (+   122us) service_pool.cc:224] Handling call
0709 13:03:28.394904 (+3017075us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400099054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:28.395044 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2886224128}
W20260709 13:03:28.640892  5295 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
I20260709 13:03:29.093974  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:28.423169  5174 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:03:29.008988  1612 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.004s	sys 0.024s
I20260709 13:03:28.758355  5311 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.794203  5017 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:28.992018  5286 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.012s	sys 0.015s
W20260709 13:03:28.761626  5299 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.353s	user 0.000s	sys 0.001s
I20260709 13:03:28.600989  5308 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:28.937282  5245 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.008s	sys 0.031s
W20260709 13:03:29.033049  1538 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.008s	sys 0.003s
I20260709 13:03:28.762174  5307 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:29.097653  1572 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 12.
I20260709 13:03:28.782874  5161 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:29.140161  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:29.140834  1566 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
W20260709 13:03:29.121122  5286 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.013s	sys 0.021s
W20260709 13:03:28.763130  5305 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.246s	user 0.000s	sys 0.000s
W20260709 13:03:29.114646  1612 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.008s	sys 0.028s
W20260709 13:03:29.097273  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 723298us
W20260709 13:03:29.153690  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 788159us
I20260709 13:03:28.408888  5257 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.512969  5298 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:28.461556  5302 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:28.956915  5195 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.576s	user 0.012s	sys 0.001s
I20260709 13:03:28.512316  5306 consensus_queue.cc:1059] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.588s
I20260709 13:03:28.898537  5304 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:28.796816  5155 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:28.437140  5192 raft_consensus.cc:697] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:03:29.136718  1685 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.778s	user 0.011s	sys 0.013s
W20260709 13:03:29.180229  1685 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.822s	user 0.011s	sys 0.013s
W20260709 13:03:29.138931  1538 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.012s	sys 0.007s
W20260709 13:03:29.098381  1564 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.007s	sys 0.041s
I20260709 13:03:29.071475  5250 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.094460  5285 raft_consensus.cc:3055] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:28.513151  5296 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.767s
W20260709 13:03:29.135727  5245 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.012s	sys 0.035s
I20260709 13:03:29.113260  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1894) took 1069 ms. Trace:
W20260709 13:03:29.132378  5277 consensus_meta.cc:287] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.917s	user 0.009s	sys 0.005s
I20260709 13:03:29.165117  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:29.166265  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 187559us
I20260709 13:03:29.175900  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:29.173840  5257 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:29.178333  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:29.123831  5311 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:29.181362  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 786625us
I20260709 13:03:29.156219  5161 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.184105  1713 rpcz_store.cc:276] 0709 13:03:28.044102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.163486 (+119384us) service_pool.cc:224] Handling call
0709 13:03:29.113164 (+949678us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.203016  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2383) took 2008 ms. Trace:
W20260709 13:03:29.175009  5183 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.034s	sys 0.045s
I20260709 13:03:29.125349  5017 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:29.136878  5308 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:29.213554  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.409987  4967 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:29.180756  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 822425us
I20260709 13:03:29.208052  1717 raft_consensus.cc:1275] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:29.165967  5307 raft_consensus.cc:697] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:29.177201  5142 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:03:29.206966  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:29.227279  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:29.212774  5183 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.034s	sys 0.046s
W20260709 13:03:29.216985  5287 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.862s	user 0.013s	sys 0.000s
I20260709 13:03:29.182703  5285 raft_consensus.cc:740] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:29.217375  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:29.223129  1685 leader_election.cc:336] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.945s (SENT)
I20260709 13:03:29.236053  1685 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:29.182693  5272 consensus_queue.cc:1048] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:03:29.207099  1566 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:29.211409  1647 rpcz_store.cc:276] 0709 13:03:27.174830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.905207 (+730377us) service_pool.cc:224] Handling call
0709 13:03:27.916903 (+ 11696us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:27.917253 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.917282 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:28.098308 (+181026us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b2c000120e8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824777ea11 00007f824778e372 00007f8248ec0a01 00007f8248e8543a 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2
0709 13:03:28.098822 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 13:03:28.313178 (+214356us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:28.313226 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.313295 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:29.177806 (+864511us) raft_consensus.cc:1685] finished
0709 13:03:29.177829 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.182433 (+  4604us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":135,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":113,"spinlock_wait_cycles":27529472,"thread_start_us":213913,"threads_started":1,"wal-append.queue_time_us":322058}
I20260709 13:03:29.210613  5320 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:29.179993  5195 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:29.263321  5327 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.263823  1713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:03:29.264432  1713 raft_consensus.cc:606] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:29.264988  5195 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:29.177433  5302 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:29.265833  5180 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:29.266232  5307 consensus_queue.cc:237] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.267714  1720 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:29.268379  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 308) took 1020 ms. Trace:
I20260709 13:03:29.268657  5302 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:29.237175  1685 leader_election.cc:336] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.004s (SENT)
I20260709 13:03:29.270145  5296 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:29.270799  1685 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.028s (SENT)
I20260709 13:03:29.239722  4967 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:29.272284  1685 leader_election.cc:336] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.110s (SENT)
I20260709 13:03:29.272791  4967 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.237380  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1858) took 3323 ms. Trace:
I20260709 13:03:29.273330  4967 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.273825  1716 rpcz_store.cc:276] 0709 13:03:25.913980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.007860 (+ 93880us) service_pool.cc:224] Handling call
0709 13:03:28.209800 (+2201940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.211140 (+  1340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.237129 (+1025989us) spinlock_profiling.cc:243] Waited 2.19 s on lock 0x7b540020a150. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:29.237309 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5039717248}
I20260709 13:03:29.276142  4967 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:29.241015  5277 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:29.243213  5316 log.cc:927] Time spent T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:03:29.243690  5319 consensus_queue.cc:1059] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:29.279235  5309 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:29.278733  5192 consensus_queue.cc:237] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.279894  5309 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.241148  5155 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:29.244421  5312 consensus_queue.cc:1059] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:29.282164  2312 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:29.283066  5308 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:29.245280  5301 consensus_queue.cc:1059] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:29.286914  5311 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:29.270759  5296 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:29.288439  5246 log.cc:927] Time spent T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 3.226s	user 0.003s	sys 0.043s
I20260709 13:03:29.289835  5321 raft_consensus.cc:3055] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:29.290279  4900 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:29.290411  5321 raft_consensus.cc:740] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:29.290517  5296 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:29.278286  5285 consensus_queue.cc:260] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.292241  5285 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:29.291916  5321 consensus_queue.cc:260] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.282399  5297 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:29.294502  5297 raft_consensus.cc:3055] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:03:29.294812  5191 consensus_meta.cc:287] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.993s	user 0.006s	sys 0.008s
I20260709 13:03:29.294963  5297 raft_consensus.cc:740] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:29.295499  5191 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:29.204421  5250 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.283336  2312 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:03:29.297495  5315 log.cc:927] Time spent T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:03:29.298193  2312 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:29.298426  5250 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:29.203660  5298 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.299125  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1918) took 1009 ms. Trace:
I20260709 13:03:29.250595  5287 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:29.299557  1569 rpcz_store.cc:276] 0709 13:03:28.290203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.290338 (+   135us) service_pool.cc:224] Handling call
0709 13:03:28.290652 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.358471 (+ 67819us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:28.358520 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:28.358587 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:28.358635 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.298862 (+940227us) spinlock_profiling.cc:243] Waited 64.1 ms on lock 0x7b54001b3d54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:29.298913 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.299055 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3302,"raft.queue_time_us":73306,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":75118,"spinlock_wait_cycles":147458816}
I20260709 13:03:29.235561  5178 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:29.183283  5304 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.307859  5310 raft_consensus.cc:3055] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:29.308275  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1916) took 1028 ms. Trace:
I20260709 13:03:29.308632  5310 raft_consensus.cc:740] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:29.216511  5161 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.308905  1564 rpcz_store.cc:276] 0709 13:03:28.280201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.280326 (+   125us) service_pool.cc:224] Handling call
0709 13:03:28.280714 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:28.374235 (+ 93521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:28.374289 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:28.385957 (+ 11668us) log.cc:844] Serialized 26 byte log entry
0709 13:03:29.181788 (+795831us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:29.181850 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.181937 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:29.308025 (+126088us) raft_consensus.cc:1685] finished
0709 13:03:29.308050 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.308208 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":64444,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"thread_start_us":833993,"threads_started":2,"wal-append.queue_time_us":845873}
I20260709 13:03:29.237473  5324 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:29.299573  5298 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.318614  5298 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:29.230022  5183 raft_consensus.cc:697] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:29.320082  5183 consensus_queue.cc:237] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.305009  5160 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:29.311959  5324 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:29.324715  5324 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:29.325330  5324 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:29.230707  5174 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:29.307963  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 326) took 1913 ms. Trace:
I20260709 13:03:29.302595  5243 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:29.268675  1720 rpcz_store.cc:276] 0709 13:03:28.248167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.267177 (+1019010us) service_pool.cc:224] Handling call
0709 13:03:29.267545 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.268116 (+   571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.268305 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:29.314635  1685 leader_election.cc:336] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.471s (SENT)
I20260709 13:03:29.315415  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1868) took 3770 ms. Trace:
I20260709 13:03:29.295397  5017 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:29.304584  5320 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:29.327824  5309 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:29.327975  5302 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:29.232780  1612 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:29.334900  1715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:03:29.384020  1715 raft_consensus.cc:606] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:29.384660  1715 raft_consensus.cc:614] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:29.301450  5287 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:29.337528  1648 rpcz_store.cc:276] 0709 13:03:27.394379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.156992 (+762613us) service_pool.cc:224] Handling call
0709 13:03:28.166300 (+  9308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:28.170272 (+  3972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.307715 (+1137443us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b5400183b50. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:29.307899 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7891840}
I20260709 13:03:29.349800  1685 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:29.346588  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 193169us
W20260709 13:03:29.381786  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.347781  1639 rpcz_store.cc:276] 0709 13:03:25.544739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.624638 (+ 79899us) service_pool.cc:224] Handling call
0709 13:03:25.640683 (+ 16045us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:25.675824 (+ 35141us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:25.675862 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:25.676399 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 13:03:25.924870 (+248471us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:25.924929 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.925532 (+   603us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:29.287272 (+3361740us) spinlock_profiling.cc:243] Waited 4.32 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:29.312898 (+ 25626us) raft_consensus.cc:1685] finished
0709 13:03:29.312921 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.315340 (+  2419us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33556,"raft.queue_time_us":40944,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":127,"spinlock_wait_cycles":9944448,"thread_start_us":219571,"threads_started":1,"wal-append.queue_time_us":325114}
I20260709 13:03:29.383458  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:35480 (request call id 11) took 1152 ms. Trace:
I20260709 13:03:29.386812  1713 rpcz_store.cc:276] 0709 13:03:28.204956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.265523 (+1060567us) service_pool.cc:224] Handling call
0709 13:03:29.357010 (+ 91487us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:29.388870  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 90768.2us
I20260709 13:03:29.428329  1720 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:29.389726  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 157084us
I20260709 13:03:29.432070  1612 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:29.312070  5161 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:29.373270  5320 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:29.337473  5327 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.383376  5302 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.393018  5298 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:29.347327  5243 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:29.464257  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 221503us
I20260709 13:03:29.434208  1537 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:03:29.435678  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 121088us
I20260709 13:03:29.436286  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:29.466347  1685 leader_election.cc:336] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.683s (SENT)
I20260709 13:03:29.466414  1569 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:03:29.466823  1685 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:29.440871  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:29.462249  1566 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:29.465778  1538 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:29.471472  2312 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:29.476207  1720 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:29.468452  5243 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:29.500387  2314 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:29.501626  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:03:29.492575  1685 leader_election.cc:336] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 3.307s (SENT)
W20260709 13:03:29.495468  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 61375.3us
W20260709 13:03:29.496117  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.502408  2347 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:29.492900  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 313) took 1207 ms. Trace:
I20260709 13:03:29.502879  1537 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:29.503131  1720 rpcz_store.cc:276] 0709 13:03:28.285554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.389138 (+1103584us) service_pool.cc:224] Handling call
0709 13:03:29.428179 (+ 39041us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.490527 (+ 62348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.492823 (+  2296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.502429  1685 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:29.483423  5161 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:29.481138  5324 consensus_queue.cc:1059] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:03:29.511727  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:29.464341  5302 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:29.514635  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:29.516021  2340 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 3.
I20260709 13:03:29.436417  5309 consensus_queue.cc:1059] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:03:29.498426  5192 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:29.521472  1645 raft_consensus.cc:1240] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:29.531163  1566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:51464
I20260709 13:03:29.545482  1566 raft_consensus.cc:606] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:29.545977  1566 raft_consensus.cc:614] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:29.531028  5332 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:29.534140  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:29.548743  1720 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 11: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:29.509869  5167 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:29.498852  5017 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:29.501376  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:29.501896  5307 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:29.534798  5318 log.cc:927] Time spent T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 13:03:29.383957  5310 consensus_queue.cc:260] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:29.561599  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.553540  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 309) took 1286 ms. Trace:
I20260709 13:03:29.501376  5150 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:03:29.492354  5183 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:29.560685  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet eca30b43c26f420c965ee25f475581a0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:29.519543  5306 raft_consensus.cc:2764] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:03:29.543053  5179 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.584756  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:29.632037  1639 raft_consensus.cc:2393] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:29.585259  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:29.563400  5197 consensus_meta.cc:287] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.211s	user 0.010s	sys 0.006s
I20260709 13:03:29.435470  5297 consensus_queue.cc:260] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:29.605531  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:29.635473  1646 raft_consensus.cc:2410] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:03:29.635694  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:29.608742  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:29.638477  1648 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:29.609802  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:29.639542  1643 raft_consensus.cc:2376] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5. Re-sending same reply.
I20260709 13:03:29.611646  5312 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.613175  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:29.643800  1644 raft_consensus.cc:2376] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8. Re-sending same reply.
I20260709 13:03:29.613875  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:29.615967  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2371) took 3352 ms. Trace:
I20260709 13:03:29.617393  4921 raft_consensus.cc:3055] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:29.623682  1612 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:29.625201  5276 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.626669  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 318) took 2416 ms. Trace:
I20260709 13:03:29.553489  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:29.563855  5017 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.658104   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:29.675690  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:29.677683  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:29.679960  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:29.680662  1644 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:03:29.642874  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:29.682027  1646 raft_consensus.cc:2376] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7. Re-sending same reply.
I20260709 13:03:29.650641  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:29.683727  1720 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 9: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:29.656896  5254 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.567129  5307 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:29.600093  5292 consensus_meta.cc:287] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.367s	user 0.016s	sys 0.004s
I20260709 13:03:29.651314  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:29.703215  1714 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:29.664901  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:29.585299  5142 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.670846  5167 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.660198  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:29.672881  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 168534us
I20260709 13:03:29.661561  1641 rpcz_store.cc:276] 0709 13:03:27.209986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.962106 (+752120us) service_pool.cc:224] Handling call
0709 13:03:28.110851 (+148745us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.111199 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:28.111225 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:28.111268 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:28.111299 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.111353 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.626474 (+1515121us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54002dbc50. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:29.626603 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4005,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":79,"spinlock_wait_cycles":341137280}
I20260709 13:03:29.645952  1640 rpcz_store.cc:276] 0709 13:03:26.263925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.547184 (+283259us) service_pool.cc:224] Handling call
0709 13:03:26.547505 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.547764 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:27.202575 (+654811us) log.cc:844] Serialized 14 byte log entry
0709 13:03:29.615389 (+2412814us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b1800452080. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a2523a
0709 13:03:29.615602 (+   213us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:29.615655 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:29.615713 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.615780 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.615909 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1060221,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":116,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":151,"spinlock_wait_cycles":31634176,"thread_start_us":1058220,"threads_started":2,"wal-append.queue_time_us":750072,"wal-append.run_cpu_time_us":1666,"wal-append.run_wall_time_us":1001680}
I20260709 13:03:29.631625  5174 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:29.715298  1648 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:03:29.728482  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:29.693255  1538 leader_election.cc:336] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.518s (SENT)
W20260709 13:03:29.705332  5333 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:03:29.652082  4921 raft_consensus.cc:740] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:29.672487  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:29.699563  5272 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.707150  5309 consensus_queue.cc:1243] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:03:29.649690  5197 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.722398  1537 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:29.719316  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:29.729326  1641 raft_consensus.cc:2376] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11. Re-sending same reply.
I20260709 13:03:29.730865  1643 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 3.
I20260709 13:03:29.737814  1537 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:29.715814  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:29.738346  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:29.707166  5307 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:29.731997  1538 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:29.706043  5247 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.688755  5017 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:29.734614  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 204399us
I20260709 13:03:29.710690  5292 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:29.735904  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:29.737574  1714 raft_consensus.cc:2410] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:29.738502  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:29.717000  5142 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.743458  1648 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 9: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:29.754223  1647 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:03:29.663829  5312 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.641439  5179 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.823215  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:29.727094  5167 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:29.732730  5201 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.690s	user 0.019s	sys 0.019s
I20260709 13:03:29.800040  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:29.730794  5330 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.300s
W20260709 13:03:29.766673  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.714509  5257 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.754580  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:29.813042  1639 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:29.813699  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:29.820940  5313 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:03:29.756139  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 62939.6us
W20260709 13:03:29.894608  1538 leader_election.cc:336] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.643s (SENT)
W20260709 13:03:29.895799  1538 leader_election.cc:336] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.667s (SENT)
I20260709 13:03:29.896234  1538 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:29.681556  5276 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.706547  5254 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.754599  5272 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.865772  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:29.757886  5150 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:29.690810  5178 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:03:29.882589  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:29.920593  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:29.890301  1714 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:29.921334  1640 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:29.922283  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:29.822045  5342 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.255s
W20260709 13:03:29.885921  1685 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.716s (SENT)
I20260709 13:03:29.892428  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:29.932861  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 46995.2us
I20260709 13:03:29.760470  5309 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:03:29.890643  5201 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.857s	user 0.020s	sys 0.020s
I20260709 13:03:29.753892  5308 consensus_queue.cc:1059] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.144s
I20260709 13:03:29.879317  5325 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.917172  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:29.922997  1714 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:29.946837  1639 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:29.947548  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:29.891819  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.929562  5344 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.880102  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:29.930423  5335 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.379s	user 0.000s	sys 0.002s
I20260709 13:03:29.812736  5247 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:29.935660  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 423430us
I20260709 13:03:29.817268  5296 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 13:03:29.897617  5340 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:29.934262  1685 leader_election.cc:336] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.739s (SENT)
I20260709 13:03:30.019733  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:29.955107  2347 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:03:29.945290  5289 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.964130  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:30.021333  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 87087.7us
I20260709 13:03:29.942341  5313 raft_consensus.cc:697] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:29.872705  4921 consensus_queue.cc:260] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:29.936352  5150 raft_consensus.cc:697] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:29.901079  5328 consensus_queue.cc:1059] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.355s
I20260709 13:03:29.900579  5332 raft_consensus.cc:697] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:30.000183  5347 consensus_queue.cc:1059] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:03:30.001802  5351 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:30.014472  1721 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:30.023041  1714 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:30.050190  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:29.931048  5327 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:30.051021  1714 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:30.054884  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:30.056084  1714 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
W20260709 13:03:30.035095  1685 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.780s (SENT)
I20260709 13:03:29.960585  5312 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:30.058552  1685 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:30.057608  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:30.062448  1714 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:29.954521  5325 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:30.064209  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 168438us
W20260709 13:03:30.064997  1538 leader_election.cc:336] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.670s (SENT)
I20260709 13:03:30.066740  1538 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:30.032729  5160 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:30.051496  5337 log.cc:927] Time spent T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.330s	user 0.002s	sys 0.000s
W20260709 13:03:30.084519  5339 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.268s	user 0.000s	sys 0.001s
I20260709 13:03:30.086198  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:30.089208  5150 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:30.090643  5341 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 13:03:30.110913  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:30.112078  5167 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:30.112587  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 318) took 1746 ms. Trace:
I20260709 13:03:30.050485  5289 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:30.073426  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:30.065529  5298 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:30.059506  5338 log.cc:927] Time spent T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.338s	user 0.000s	sys 0.002s
I20260709 13:03:30.005841  5344 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:30.082430  4921 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:30.109915  5179 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:30.110152  5276 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:30.113036  1715 rpcz_store.cc:276] 0709 13:03:28.366556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.393033 (+1026477us) service_pool.cc:224] Handling call
0709 13:03:29.425293 (+ 32260us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:29.439001 (+ 13708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.439052 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:29.469098 (+ 30046us) log.cc:844] Serialized 26 byte log entry
0709 13:03:29.642119 (+173021us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:29.677361 (+ 35242us) log.cc:844] Serialized 14 byte log entry
0709 13:03:29.677743 (+   382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.677873 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:30.112352 (+434479us) raft_consensus.cc:1685] finished
0709 13:03:30.112375 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.112523 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":184,"prepare.queue_time_us":47954,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":26618,"raft.run_cpu_time_us":342,"raft.run_wall_time_us":25921,"thread_start_us":127911,"threads_started":1,"wal-append.queue_time_us":258080}
I20260709 13:03:30.112725  2347 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 11: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:03:30.120576  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 47323.2us
W20260709 13:03:30.097874  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.112918  5197 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:30.129237  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:30.132776  1715 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
W20260709 13:03:30.142838  5346 log.cc:927] Time spent T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:03:30.144302  5289 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:30.132026  5167 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:30.146476  5167 raft_consensus.cc:697] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:30.147677  5167 consensus_queue.cc:237] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:30.095036  5312 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:30.142937  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:30.102633  5309 raft_consensus.cc:697] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:30.110948  5142 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:30.162904  5357 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:30.163633  5357 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:30.164523  5183 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.532s	user 0.006s	sys 0.004s
I20260709 13:03:30.164968  5183 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:30.169766  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1864) took 4626 ms. Trace:
I20260709 13:03:30.170440  1645 rpcz_store.cc:276] 0709 13:03:25.543624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.543775 (+   151us) service_pool.cc:224] Handling call
0709 13:03:25.606519 (+ 62744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.545280 (+3938761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.169279 (+623999us) spinlock_profiling.cc:243] Waited 3.91 s on lock 0x7b54002aadd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:30.169628 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9003770240}
I20260709 13:03:30.190500  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1888) took 2228 ms. Trace:
I20260709 13:03:30.191700  1719 rpcz_store.cc:276] 0709 13:03:27.962153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.963845 (+  1692us) service_pool.cc:224] Handling call
0709 13:03:27.964240 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.998722 (+1034482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.172007 (+1173285us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b540002dfd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:30.190407 (+ 18400us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1276055936}
I20260709 13:03:30.060391  5351 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:30.204499  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:30.238125  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:30.218565  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:30.266371  1718 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:30.239266  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:30.227758  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:30.228139  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:30.268472  1572 raft_consensus.cc:2410] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:30.268638  1566 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 7: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
W20260709 13:03:30.232316  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 150232us
I20260709 13:03:30.218106  1715 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:30.270565  1538 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:30.268709  1571 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
W20260709 13:03:30.225732  1685 leader_election.cc:336] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.997s (SENT)
W20260709 13:03:30.274890  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49245.7us
W20260709 13:03:30.206585  5285 consensus_meta.cc:287] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.914s	user 0.009s	sys 0.010s
W20260709 13:03:30.254441  5243 consensus_meta.cc:287] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.738s	user 0.014s	sys 0.000s
I20260709 13:03:30.223371  5257 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:30.154340  5312 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:30.134845  5313 consensus_queue.cc:237] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:30.154990  5356 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:30.286845  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:30.289857  1567 raft_consensus.cc:1240] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:30.302822  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:30.250476  5325 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:30.248939  5334 log.cc:927] Time spent T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.356s	user 0.000s	sys 0.011s
I20260709 13:03:30.238909  5357 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:30.293898  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:30.276890  1719 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:30.315053  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:30.281939  1648 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:30.296482  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:30.304538  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:30.318061  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:30.319290  1570 raft_consensus.cc:2410] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:03:30.303854  1565 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:30.310340  5254 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:30.288645  5272 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:30.312065  1567 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:30.272509  1538 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:30.321192  1612 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.005s	sys 0.010s
W20260709 13:03:30.310590  5302 consensus_meta.cc:287] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.777s	user 0.013s	sys 0.000s
I20260709 13:03:30.323078  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:30.333448  1719 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 9: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:30.324627  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:30.333112  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:30.335189  1570 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:30.326287  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:30.329577  1612 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.005s	sys 0.010s
I20260709 13:03:30.327967  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:30.337970  1567 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:03:30.339234  1567 raft_consensus.cc:3055] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:30.339576  1567 raft_consensus.cc:740] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:30.339246  1570 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:30.329492  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:30.343542  1572 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 7: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:30.202126  5332 consensus_queue.cc:237] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:30.325412  5245 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.005s	sys 0.012s
I20260709 13:03:30.320885  5361 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:30.348341  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 674241us
I20260709 13:03:30.344770  5355 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:30.346652  1567 consensus_queue.cc:260] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:30.298405  5243 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:30.300523  5309 consensus_queue.cc:237] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:30.349519  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:30.393882  1612 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:30.396994  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:30.456990  1565 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:30.393867  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:30.326867  5362 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:03:30.462195  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 68585.8us
W20260709 13:03:30.347342  5349 log.cc:927] Time spent T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.190s	user 0.002s	sys 0.000s
I20260709 13:03:30.481586  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:30.341799  5254 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.462500  1645 raft_consensus.cc:2376] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8. Re-sending same reply.
I20260709 13:03:30.483757  1640 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:30.467334  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:30.470800  5257 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.473775  5312 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:30.495059  1570 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:30.495204  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:30.496006  1720 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:30.464084  5278 consensus_meta.cc:287] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.370s	user 0.013s	sys 0.004s
I20260709 13:03:30.478956  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1885) took 2538 ms. Trace:
W20260709 13:03:30.346558  5245 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.005s	sys 0.012s
W20260709 13:03:30.463959  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.489259  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:30.419286  5307 consensus_meta.cc:287] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.660s	user 0.012s	sys 0.001s
I20260709 13:03:30.502099  1717 rpcz_store.cc:276] 0709 13:03:27.941523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.941903 (+   380us) service_pool.cc:224] Handling call
0709 13:03:27.942275 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.223643 (+1281368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.478668 (+1255025us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400248954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:30.478878 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2900286848}
I20260709 13:03:30.504231  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:30.505141  1717 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:30.403012  5344 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:30.502823  2344 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:03:30.402489  5351 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:30.342960  5302 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:30.379618  5272 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:30.538702  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.475548  1714 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
W20260709 13:03:30.526940  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 254563us
W20260709 13:03:30.539633  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 50693.8us
I20260709 13:03:30.540220  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:30.355485  5355 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.545094  1612 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:30.497247  5183 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:30.534215  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:30.492776  5174 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:30.555792  5186 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:30.357362  5361 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:30.559966  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.535722  5362 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:03:30.540849  5320 consensus_meta.cc:287] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.071s	user 0.016s	sys 0.000s
I20260709 13:03:30.497119  5254 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:30.570209  1717 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:30.545223  5142 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:30.508066  5278 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:30.500013  5257 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:30.552521  5355 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:30.562716  5186 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:30.512166  5307 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:30.580050  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.584122  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:30.576087  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 547) took 1956 ms. Trace:
I20260709 13:03:30.552680  5174 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:30.609229  1722 raft_consensus.cc:2393] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 8: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:30.594107  1720 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:30.556049  5197 consensus_queue.cc:1048] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:03:30.547191  5313 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:30.598872  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.613314  1538 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:30.557822  5272 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:30.606734  1611 connection.cc:256] RPC call timeout handler was delayed by -1.3448434409999663s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2005s for a response
I20260709 13:03:30.604326  1568 rpcz_store.cc:276] 0709 13:03:28.619665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.151886 (+532221us) service_pool.cc:224] Handling call
0709 13:03:29.153251 (+  1365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:29.178306 (+ 25055us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.178337 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:29.628795 (+450458us) spinlock_profiling.cc:243] Waited 5.71 ms on lock 0x7b2c00000658. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824777ea11 00007f824778e372 00007f8248ec0a01 00007f8248e8543a 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2
0709 13:03:29.653272 (+ 24477us) log.cc:844] Serialized 26 byte log entry
0709 13:03:29.984435 (+331163us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:29.984502 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.984623 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:30.571347 (+586724us) raft_consensus.cc:1685] finished
0709 13:03:30.571372 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.575999 (+  4627us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13060,"raft.queue_time_us":32264,"spinlock_wait_cycles":21767168,"thread_start_us":259824,"threads_started":1,"wal-append.queue_time_us":384093}
I20260709 13:03:30.617741  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:30.569199  5183 consensus_queue.cc:1048] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
W20260709 13:03:30.632066  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.685217  1573 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:30.603721  5186 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:30.616894  1611 connection.cc:256] RPC call timeout handler was delayed by -1.2134434409999812s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21509999999999999s for a response
W20260709 13:03:30.683673  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 70470.5us
W20260709 13:03:30.692876  1611 connection.cc:256] RPC call timeout handler was delayed by -1.0916386220000049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19870000000000002s for a response
I20260709 13:03:30.692198  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:30.610551  5174 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:30.693704  1538 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:30.576546  5320 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:30.703109  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:30.703892  1568 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:30.684248  5017 consensus_meta.cc:287] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.866s	user 0.015s	sys 0.002s
I20260709 13:03:30.684820  5357 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:30.714452  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 165649us
I20260709 13:03:30.695724  5285 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.725890  1639 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:30.694970  5362 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:30.705940  5150 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:30.699343  5243 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:30.726035  5340 consensus_meta.cc:287] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.685s	user 0.014s	sys 0.005s
I20260709 13:03:30.710505  1538 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:30.683386  5313 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:30.750581  5150 raft_consensus.cc:697] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:30.752518  5150 consensus_queue.cc:237] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:30.755194  1568 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:30.757167  5179 consensus_queue.cc:1048] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:03:30.759675  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 49249.5us
I20260709 13:03:30.738204  5247 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:30.715211  5017 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:30.748721  5340 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:30.726631  5357 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:30.767941  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:30.760494  5364 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 13:03:30.730441  5285 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:30.768234  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:30.785172  5254 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:30.789218  5278 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:30.771989  5344 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:30.791842  5017 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:30.793812  5357 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:30.794590  1713 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:30.840834  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1905) took 1323 ms. Trace:
I20260709 13:03:30.780647  5312 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.795321  1642 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:03:30.796128  5377 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.862977  5312 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:30.863307  5377 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:30.772482  5360 raft_consensus.cc:2764] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:03:30.797971  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:30.796033  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:30.807045  1612 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:30.812364  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1904) took 1294 ms. Trace:
I20260709 13:03:30.815022  5313 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:30.814858  5186 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:30.815953  5375 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:30.816751  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.820917  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:03:30.878638  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 72296.2us
I20260709 13:03:30.834666  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:30.834847  5370 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:03:30.836283  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2315) took 1024 ms. Trace:
I20260709 13:03:30.771926  5378 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.841316  1713 rpcz_store.cc:276] 0709 13:03:29.518114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.793992 (+1275878us) service_pool.cc:224] Handling call
0709 13:03:30.794365 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.840497 (+ 46132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.840722 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:30.794654  5307 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:30.818117  5369 log.cc:927] Time spent T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 13:03:30.840222  5257 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:30.858829  5371 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:03:30.859103  5320 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:30.899470  1641 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:30.779525  5302 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:30.860651  5272 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:30.863847  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:30.904752  1642 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 9: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:30.906055  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:30.798744  5372 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:30.866918  1643 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:30.907460  5372 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:30.907374  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:30.908895  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:30.910245  1643 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:30.866863  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:30.914047  1646 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:30.926664  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1902) took 1424 ms. Trace:
W20260709 13:03:30.927728  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.867007  1720 rpcz_store.cc:276] 0709 13:03:29.517412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.613071 (+1095659us) service_pool.cc:224] Handling call
0709 13:03:30.613500 (+   429us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:30.614262 (+   762us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:30.614293 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:30.614903 (+   610us) log.cc:844] Serialized 52 byte log entry
0709 13:03:30.749418 (+134515us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:30.784473 (+ 35055us) log.cc:844] Serialized 14 byte log entry
0709 13:03:30.810294 (+ 25821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.810983 (+   689us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:30.811232 (+   249us) raft_consensus.cc:1685] finished
0709 13:03:30.811265 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.811940 (+   675us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":420,"raft.run_cpu_time_us":397,"raft.run_wall_time_us":78518,"thread_start_us":49929,"threads_started":2,"wal-append.queue_time_us":90487}
I20260709 13:03:30.872265  1566 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:30.865375  5312 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:30.875814  1568 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:30.834493  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:30.936156  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.936319  1646 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:03:30.883016  1715 rpcz_store.cc:276] 0709 13:03:29.812144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.142619 (+330475us) service_pool.cc:224] Handling call
0709 13:03:30.836108 (+693489us) spinlock_profiling.cc:243] Waited 61.4 ms on lock 0x7b5400210f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:30.836226 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141240704}
I20260709 13:03:30.880488  1612 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:30.837414  5367 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.835762  5340 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:30.900833  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1906) took 1372 ms. Trace:
I20260709 13:03:30.972760  1713 rpcz_store.cc:276] 0709 13:03:29.518507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.889954 (+1371447us) service_pool.cc:224] Handling call
0709 13:03:30.890233 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.890565 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:30.890583 (+    18us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:30.890612 (+    29us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:30.890634 (+    22us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.890675 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.890769 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":124,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":9904}
I20260709 13:03:30.927485  1642 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:03:30.975647  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:30.976725  1642 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:30.927103  1717 rpcz_store.cc:276] 0709 13:03:29.505995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.605511 (+1099516us) service_pool.cc:224] Handling call
0709 13:03:30.607212 (+  1701us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:30.607769 (+   557us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:30.607822 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:30.697889 (+ 90067us) log.cc:844] Serialized 26 byte log entry
0709 13:03:30.898674 (+200785us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:30.898766 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.898885 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:30.926404 (+ 27519us) raft_consensus.cc:1685] finished
0709 13:03:30.926423 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.926605 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105,"spinlock_wait_cycles":173537664,"thread_start_us":163355,"threads_started":1,"wal-append.queue_time_us":182832}
I20260709 13:03:30.888733  5313 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:30.898420  5324 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.969734  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:30.929984  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1907) took 1407 ms. Trace:
I20260709 13:03:30.978725  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:31.004940  1643 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:31.005381  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:31.005726  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1912) took 1477 ms. Trace:
I20260709 13:03:30.996798  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:31.006412  1646 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:31.004711  1720 rpcz_store.cc:276] 0709 13:03:29.522550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.929106 (+1406556us) service_pool.cc:224] Handling call
0709 13:03:30.929421 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.929618 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:30.929666 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:30.929713 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:30.929753 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.929809 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.929928 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:31.006134  1717 rpcz_store.cc:276] 0709 13:03:29.528186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.004912 (+1476726us) service_pool.cc:224] Handling call
0709 13:03:31.005212 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.005386 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.005409 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:31.005447 (+    38us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:31.005515 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.005565 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.005677 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:31.008821  1717 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:03:30.966166  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 85747.4us
I20260709 13:03:30.840480  5344 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:31.009333  1645 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:03:31.009509  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2303) took 1206 ms. Trace:
I20260709 13:03:31.018950  5313 raft_consensus.cc:697] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:31.023378  5324 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:31.023556  5257 consensus_queue.cc:1059] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:30.884625  5375 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:30.887832  5378 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:31.077152  1721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1895) took 3012 ms (client timeout 2151 ms). Trace:
I20260709 13:03:31.031886  5377 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:31.032888  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1901) took 1531 ms. Trace:
I20260709 13:03:31.036806  5307 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:31.003742  5367 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:31.038708  5186 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:31.040012  5174 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:31.043700  1717 rpcz_store.cc:276] 0709 13:03:29.803115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.008220 (+1205105us) service_pool.cc:224] Handling call
0709 13:03:31.008500 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.009262 (+   762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.009453 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:31.044750  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 310078us
W20260709 13:03:31.067219  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.067289  1643 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 12.
W20260709 13:03:31.286383  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.322131  1646 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:31.323622  1611 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:31.324043  1717 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:31.324623  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2304) took 1520 ms. Trace:
I20260709 13:03:31.322435  1719 rpcz_store.cc:276] 0709 13:03:29.502000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.353764 (+851764us) service_pool.cc:224] Handling call
0709 13:03:30.354192 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:30.382576 (+ 28384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:30.382634 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:30.383449 (+   815us) log.cc:844] Serialized 26 byte log entry
0709 13:03:30.584984 (+201535us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:30.585040 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.585120 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:31.032626 (+447506us) raft_consensus.cc:1685] finished
0709 13:03:31.032647 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.032821 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1659,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":46381,"thread_start_us":97385,"threads_started":1,"wal-append.queue_time_us":267447}
I20260709 13:03:31.324949  1717 rpcz_store.cc:276] 0709 13:03:29.803869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.321770 (+1517901us) service_pool.cc:224] Handling call
0709 13:03:31.323869 (+  2099us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.324415 (+   546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.324575 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:31.325932  1719 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:31.326107  1717 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:31.077778  1721 rpcz_store.cc:269] 0709 13:03:28.065189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.181778 (+116589us) service_pool.cc:224] Handling call
0709 13:03:31.076919 (+2895141us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b54003002d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:31.077052 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4194656512}
I20260709 13:03:31.327003  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2314) took 1515 ms. Trace:
I20260709 13:03:31.004643  5320 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:31.128326  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:31.327731  1717 rpcz_store.cc:276] 0709 13:03:29.811872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.325714 (+1513842us) service_pool.cc:224] Handling call
0709 13:03:31.325952 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.326798 (+   846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.326950 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:31.327003  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2312) took 1515 ms. Trace:
W20260709 13:03:31.328860  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:31.335614  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.292665  5216 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:31.334967  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:31.118384  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.337612  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:31.338276  1570 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:31.339134  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:31.340065  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.340217  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:31.340799  1570 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:31.335055  1719 rpcz_store.cc:276] 0709 13:03:29.811336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.325407 (+1514071us) service_pool.cc:224] Handling call
0709 13:03:31.325782 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.326798 (+  1016us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.326950 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:31.342391  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:31.344769  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1911) took 1818 ms. Trace:
W20260709 13:03:31.263849  5383 log.cc:927] Time spent T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.198s	user 0.002s	sys 0.000s
I20260709 13:03:31.350987  1713 rpcz_store.cc:276] 0709 13:03:29.526636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.002029 (+1475393us) service_pool.cc:224] Handling call
0709 13:03:31.002667 (+   638us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:31.032905 (+ 30238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.032940 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:31.033656 (+   716us) log.cc:844] Serialized 52 byte log entry
0709 13:03:31.171637 (+137981us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:31.171766 (+   129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.171980 (+   214us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:31.344494 (+172514us) raft_consensus.cc:1685] finished
0709 13:03:31.344516 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.344690 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18598,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":133,"thread_start_us":131482,"threads_started":1,"wal-append.queue_time_us":303815}
I20260709 13:03:31.077484  5378 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:31.315965  5324 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:31.321136  5310 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:31.310740  5344 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:31.354928  1571 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:31.363543  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:31.378567  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:31.379568  1570 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:31.380859  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:31.384348  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 488) took 1930 ms. Trace:
I20260709 13:03:31.356238  5375 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:31.387508  1537 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:40670 dropped due to backpressure: service queue is full with 50 items
I20260709 13:03:31.385218  2344 rpcz_store.cc:276] 0709 13:03:29.453950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.454075 (+   125us) service_pool.cc:224] Handling call
0709 13:03:29.483447 (+ 29372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:30.511936 (+1028489us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.383905 (+871969us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540065b3d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:31.384235 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2343686528}
I20260709 13:03:31.345974  5247 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:31.419648  1569 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:31.392063  5320 raft_consensus.cc:697] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:31.388469  5391 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.421072  5150 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.405325  5174 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:31.405779  5285 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:31.390758  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 43748.7us
I20260709 13:03:31.397930  5062 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.413547  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1908) took 1890 ms. Trace:
I20260709 13:03:31.424134  1611 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:31.424690  1715 rpcz_store.cc:276] 0709 13:03:29.523372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.941788 (+1418416us) service_pool.cc:224] Handling call
0709 13:03:30.942228 (+   440us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:30.998001 (+ 55773us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:30.998055 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:30.998941 (+   886us) log.cc:844] Serialized 52 byte log entry
0709 13:03:31.170550 (+171609us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:31.329409 (+158859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.329513 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:31.413230 (+ 83717us) raft_consensus.cc:1685] finished
0709 13:03:31.413258 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.413473 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":30378,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":7391,"thread_start_us":50994,"threads_started":1,"wal-append.queue_time_us":354728}
W20260709 13:03:31.417069  5355 consensus_meta.cc:287] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.810s	user 0.011s	sys 0.000s
I20260709 13:03:31.419798  5247 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:31.331259  5307 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:31.423691  5062 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:31.430532  5307 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:31.431299  1714 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:31.431838  5174 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:31.432694  5384 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.246s	user 0.003s	sys 0.000s
I20260709 13:03:31.434379  5247 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:31.422087  5150 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:31.436080  5304 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:31.436865  5174 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:31.398613  5377 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.438491  5377 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:31.439139  1611 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:31.440061  5375 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:31.440583  1611 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:31.441465  5375 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:31.435029  5257 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:31.448000  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1928) took 3022 ms. Trace:
I20260709 13:03:31.448434  1573 rpcz_store.cc:276] 0709 13:03:28.425217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.425382 (+   165us) service_pool.cc:224] Handling call
0709 13:03:29.603702 (+1178320us) spinlock_profiling.cc:243] Waited 3.65 ms on lock 0x7b5800011198. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:03:29.603901 (+   199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.685712 (+1081811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.447713 (+762001us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54002242d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:31.447900 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2494893056}
I20260709 13:03:31.421188  5391 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:31.460731  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:31.462246  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:31.403735  5367 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:31.501605  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:31.502686  1573 raft_consensus.cc:2410] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:03:31.502403  1611 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:31.503840  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:31.504458  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.504940  1573 raft_consensus.cc:2410] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:31.009737  5340 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:31.467556  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.421831  5320 consensus_queue.cc:237] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:31.523173  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.441350  5377 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:31.544783  5340 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:31.188695  5313 consensus_queue.cc:237] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:31.555756  5186 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:31.561890  5391 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:31.564272  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 243115us
W20260709 13:03:31.572525  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:31.586373  1685 leader_election.cc:336] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.858s (SENT)
I20260709 13:03:31.557152  5186 consensus_queue.cc:1059] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:03:31.466634  5355 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:31.630131  1570 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:31.637251  5257 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:29.596616  1716 rpcz_store.cc:276] 0709 13:03:28.267340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.275689 (+1008349us) service_pool.cc:224] Handling call
0709 13:03:29.276306 (+   617us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:29.276833 (+   527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.276868 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:29.277531 (+   663us) log.cc:844] Serialized 26 byte log entry
0709 13:03:29.389282 (+111751us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:29.549854 (+160572us) log.cc:844] Serialized 14 byte log entry
0709 13:03:29.552628 (+  2774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.552730 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:29.552877 (+   147us) raft_consensus.cc:1685] finished
0709 13:03:29.552896 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.553462 (+   566us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":180686,"raft.queue_time_us":681,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":10329,"spinlock_wait_cycles":29974784,"thread_start_us":194004,"threads_started":2,"wal-append.queue_time_us":73139}
I20260709 13:03:31.471470  1537 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:40670 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1916)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1912)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1910)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1942)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 556)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1938)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1935)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1936)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1907)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1936)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 560)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1926)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1928)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1924)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1904)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1906)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1908)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 552)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 553)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 554)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1931)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1932)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1933)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1934)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1937)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1946)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1917)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1918)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1919)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1920)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1923)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 559)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 561)
I20260709 13:03:31.639887  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 310) took 3372 ms. Trace:
I20260709 13:03:31.640290  1722 rpcz_store.cc:276] 0709 13:03:28.267665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.405693 (+138028us) service_pool.cc:224] Handling call
0709 13:03:31.639673 (+3233980us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b5400212d54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:31.639829 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3767096064}
W20260709 13:03:31.642043  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 330) took 2020 ms (client timeout 2254 ms). Trace:
W20260709 13:03:31.642493  1648 rpcz_store.cc:269] 0709 13:03:29.621479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.865366 (+243887us) service_pool.cc:224] Handling call
0709 13:03:31.641858 (+1776492us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b54000cf3d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:31.641979 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":597163776}
I20260709 13:03:31.646754  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:31.648497  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:31.649953  1721 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:31.642807  5179 raft_consensus.cc:3055] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:31.654989  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:31.655433  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2317) took 1838 ms. Trace:
I20260709 13:03:31.671494  1721 rpcz_store.cc:276] 0709 13:03:29.817029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.327952 (+1510923us) service_pool.cc:224] Handling call
0709 13:03:31.649745 (+321793us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.650482 (+   737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.653425 (+  2943us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b5400212d50. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:31.653578 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":680248064}
I20260709 13:03:31.655740  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:31.662236  1718 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:03:31.663277  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:31.675123  1715 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:03:31.666292  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:31.676628  1713 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
W20260709 13:03:31.609720  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 495913us
I20260709 13:03:31.679006  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:31.684707  2349 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:31.684952  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:31.686241  1573 raft_consensus.cc:2410] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:03:31.629197  5355 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:31.652187  5247 consensus_queue.cc:1059] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:03:31.700374  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:31.704772  1685 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:31.741614  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1910) took 1144 ms. Trace:
I20260709 13:03:31.742332  1573 rpcz_store.cc:276] 0709 13:03:30.597067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.700111 (+1103044us) service_pool.cc:224] Handling call
0709 13:03:31.741506 (+ 41395us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:31.744760  1685 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:31.670442  5179 raft_consensus.cc:740] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:31.759996  1538 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:31.760073  2314 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:31.806617  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 46658.1us
I20260709 13:03:31.807461  2314 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:31.765691  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.804260  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:31.810461  1639 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
W20260709 13:03:31.763069  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.818454  1537 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:31.818719  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2410) took 1022 ms. Trace:
I20260709 13:03:31.824199  1641 rpcz_store.cc:276] 0709 13:03:30.795885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.796644 (+   759us) service_pool.cc:224] Handling call
0709 13:03:30.796943 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.903514 (+106571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.818357 (+914843us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54000a69d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:31.818637 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":235295616}
I20260709 13:03:31.824821  1537 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:31.819696  1569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1888) took 1532 ms (client timeout 1997 ms). Trace:
I20260709 13:03:31.826478  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:31.814054  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 308434us
W20260709 13:03:31.826893  1569 rpcz_store.cc:269] 0709 13:03:30.287766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.288065 (+   299us) service_pool.cc:224] Handling call
0709 13:03:31.819447 (+1531382us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400406ad4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:31.819584 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2534194816}
I20260709 13:03:31.749056  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:31.808647  2314 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:31.792021  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:31.782881  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:31.786705  1643 raft_consensus.cc:1240] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:31.797151  1642 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:31.799126  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:31.769815  5287 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:31.801537  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:31.802435  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:31.682967  5294 consensus_queue.cc:1059] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.097s
I20260709 13:03:31.757397  5308 consensus_queue.cc:1059] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:03:31.770743  5216 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:31.777794  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:31.831312  2314 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:03:31.746162  1685 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:40670 dropped due to backpressure: service queue is full with 50 items
W20260709 13:03:31.889161  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 57946.9us
I20260709 13:03:31.829666  5332 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:31.833776  5340 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:31.797120  5348 raft_consensus.cc:3055] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:31.894210  1685 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:31.816963  5328 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.869231  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:31.855538  5287 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.870353  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:31.870548  5247 raft_consensus.cc:3055] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:31.914835  5287 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:31.914906  5247 raft_consensus.cc:740] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:31.916028  5247 consensus_queue.cc:260] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:31.917333  5247 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:31.870661  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:31.872404  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:31.920831  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 174729us
I20260709 13:03:31.921514  1644 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:31.921924  1685 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:31.922245  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:31.871702  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:31.873576  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:31.804248  5313 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:31.822031  5355 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:31.817436  5367 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.925922  5355 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.817407  5192 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.832743  5250 raft_consensus.cc:3055] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:31.833029  5301 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:03:31.916045  5402 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:03:31.895358  2314 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:31.844740  5397 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.815539  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:31.944012  5397 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:31.944334  2341 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:31.931717  5355 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:31.938760  5367 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:31.817586  5312 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:31.862785  5354 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:31.845014  5324 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:31.867851  5304 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:31.884537  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:31.962790  2345 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:31.911548  1721 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:31.918678  5361 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:31.919694  5400 log.cc:927] Time spent T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:03:31.898676  5216 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:31.770025  5245 consensus_queue.cc:1059] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:03:31.907167  5017 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:31.897472  5340 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:31.904937  5348 raft_consensus.cc:740] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:03:31.942031  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 48509.1us
I20260709 13:03:31.978883  2314 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:31.924213  5294 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:31.938627  5301 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:31.932217  5250 raft_consensus.cc:740] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:31.941004  5192 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:31.985335  5297 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:31.986523  5150 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:31.987651  5192 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:31.914294  5328 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:31.965477  5399 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.160s	user 0.002s	sys 0.000s
W20260709 13:03:31.965682  5398 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.089s	user 0.002s	sys 0.001s
I20260709 13:03:31.998572  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:31.999301  5397 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:31.967116  5216 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:31.984550  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:32.000524  5250 consensus_queue.cc:260] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:32.001969  5250 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:32.010126  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:31.980943  5340 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:31.936237  5313 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:31.985361  5301 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:31.965903  5312 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:32.020778  5062 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:03:31.961488  5324 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:32.022118  5324 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:32.022336  5287 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:32.024605  5287 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:32.026257  5367 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:32.027294  5348 consensus_queue.cc:260] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:32.028604  5348 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:32.030267  5321 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:32.034098  5328 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:32.034654  5307 raft_consensus.cc:3055] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:32.035100  5307 raft_consensus.cc:740] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:32.036203  5307 consensus_queue.cc:260] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:31.958463  5332 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:31.892899  5179 consensus_queue.cc:260] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:31.962962  5254 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:32.042670  5312 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:32.043874  5353 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:32.044690  5353 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:32.044764  5179 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:32.045372  5353 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:32.040653  5254 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.050484  5254 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:32.044070  5313 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:32.037501  5307 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:32.053325  5372 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:32.057521  5340 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:32.070253  5355 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:32.079389  5353 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:32.080235  5391 raft_consensus.cc:3055] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:32.080717  5391 raft_consensus.cc:740] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:32.081629  5391 consensus_queue.cc:260] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:32.082402  5353 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:32.082654  5391 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:32.084625  5410 log.cc:927] Time spent T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.100s	user 0.002s	sys 0.000s
I20260709 13:03:32.080196  5355 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:32.120569  1459 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.670s	user 0.018s	sys 0.025s
I20260709 13:03:32.128134   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:31.865401  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:32.135637  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.137176  1569 raft_consensus.cc:2410] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:03:31.834903  1537 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:32.138470  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1932) took 1009 ms. Trace:
I20260709 13:03:32.139081  1569 rpcz_store.cc:276] 0709 13:03:31.128936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.828156 (+699220us) service_pool.cc:224] Handling call
0709 13:03:32.138227 (+310071us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:31.806474  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 46579.3us
I20260709 13:03:32.141858  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:32.143340  1569 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:32.141105  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 306167us
I20260709 13:03:32.146770  1573 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
W20260709 13:03:32.156567  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.159479  1537 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:32.167112  1459 maintenance_manager.cc:643] P 220fdfc07a3c4ce28ca7333445633466: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 9.833s	user 0.224s	sys 0.105s Metrics: {"bytes_written":30690,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":265157,"dirs.run_cpu_time_us":728,"dirs.run_wall_time_us":35952,"drs_written":1,"lbm_read_time_us":455,"lbm_reads_lt_1ms":4,"lbm_write_time_us":179531,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":7,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"spinlock_wait_cycles":10379008,"thread_start_us":348721,"threads_started":2,"update_count":26,"wal-append.queue_time_us":521892}
W20260709 13:03:32.176167  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.180507  1460 maintenance_manager.cc:419] P 220fdfc07a3c4ce28ca7333445633466: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7266 bytes on disk
I20260709 13:03:31.871623  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:32.196780  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:32.199038  2313 leader_election.cc:336] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.581s (SENT)
I20260709 13:03:32.182808  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:31.930662  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:32.200809  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 221857us
I20260709 13:03:32.202924  2314 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:32.203215  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:32.204365  1648 raft_consensus.cc:2376] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 5. Re-sending same reply.
I20260709 13:03:32.200876  1644 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:32.188650  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1913) took 2659 ms. Trace:
I20260709 13:03:32.189941  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:32.209751  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:32.211028  1645 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 7: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:32.212270  1648 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:32.212561  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:32.190495  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 321) took 2179 ms. Trace:
I20260709 13:03:32.208418  1720 rpcz_store.cc:276] 0709 13:03:29.529565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.007196 (+1477631us) service_pool.cc:224] Handling call
0709 13:03:31.007437 (+   241us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:31.041237 (+ 33800us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.041286 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:31.060273 (+ 18987us) log.cc:844] Serialized 26 byte log entry
0709 13:03:31.565240 (+504967us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:31.565287 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.565364 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:32.186606 (+621242us) raft_consensus.cc:1685] finished
0709 13:03:32.186660 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:32.188287 (+  1627us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33282,"raft.queue_time_us":35614,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":10558,"thread_start_us":360544,"threads_started":1,"wal-append.queue_time_us":647592}
I20260709 13:03:31.923177  1643 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:32.217094  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:32.218137  1645 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:32.217555  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:32.219156  1643 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:32.218993  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:32.219519  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:32.185576  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1914) took 1822 ms. Trace:
I20260709 13:03:32.192312  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:32.226838  1721 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 8: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:32.227850  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:32.228451  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:32.220791  1645 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:32.208451  2314 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:32.213909  1719 rpcz_store.cc:276] 0709 13:03:30.010133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.342432 (+1332299us) service_pool.cc:224] Handling call
0709 13:03:31.350938 (+  8506us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:31.361750 (+ 10812us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.361784 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:31.383751 (+ 21967us) log.cc:844] Serialized 26 byte log entry
0709 13:03:31.609399 (+225648us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:31.609462 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.609579 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:32.188599 (+579020us) raft_consensus.cc:1685] finished
0709 13:03:32.188677 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:32.189811 (+  1134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2873,"raft.queue_time_us":32039,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":29609,"thread_start_us":174542,"threads_started":1,"wal-append.queue_time_us":361581}
I20260709 13:03:32.244897  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:32.258093  5312 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.272975  5312 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:32.258370  5328 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:32.277997  5328 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:32.278088  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:32.278635  5328 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:32.279493  1720 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:32.261823  5340 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:32.263490  5356 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:32.282773  5356 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.283388  5356 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:32.263345  5359 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.286607  5359 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:32.294842  5359 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:32.265764  1639 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:32.301714  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:32.303064  1639 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
W20260709 13:03:32.270922  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 111865us
I20260709 13:03:32.277405  5312 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:32.281172  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:32.265157  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:32.315590  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:32.320493  1459 maintenance_manager.cc:643] P 220fdfc07a3c4ce28ca7333445633466: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.003s	user 0.000s	sys 0.001s Metrics: {"cfile_init":1,"lbm_read_time_us":821,"lbm_reads_lt_1ms":4}
I20260709 13:03:32.220752  1717 rpcz_store.cc:276] 0709 13:03:30.363351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.352209 (+988858us) service_pool.cc:224] Handling call
0709 13:03:31.352771 (+   562us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:31.372691 (+ 19920us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.372720 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:31.382387 (+  9667us) log.cc:844] Serialized 26 byte log entry
0709 13:03:31.582904 (+200517us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:31.582945 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.583018 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:32.184663 (+601645us) raft_consensus.cc:1685] finished
0709 13:03:32.184716 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:32.185208 (+   492us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":879,"raft.queue_time_us":30051,"raft.run_cpu_time_us":462,"raft.run_wall_time_us":5023,"thread_start_us":163963,"threads_started":1,"wal-append.queue_time_us":299819}
W20260709 13:03:32.337610  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 156934us
I20260709 13:03:32.337869  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:32.339615  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:32.341876  1719 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:32.343864  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 421854us
I20260709 13:03:32.353981  5415 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:32.354893  5415 raft_consensus.cc:697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 12 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:32.356689  5415 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:32.358655  1537 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:32.363773  1537 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:32.365774  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.366626  5359 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:32.367255  5359 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:32.367797  5359 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:32.368253  1537 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:32.369798  5340 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:32.370462  5340 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.370913  5340 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:32.364651  5312 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:32.377208  5312 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:32.377306  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:32.377897  5312 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:32.378505  1717 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:32.379168  5197 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:32.380057  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 172171us
I20260709 13:03:32.379839  5197 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:32.382220  2314 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:32.369658  5353 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:32.384603  5342 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:32.390444  1720 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:32.391918  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:32.392995  2344 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:03:32.379846  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:32.395567  1717 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
W20260709 13:03:32.397286  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.392005  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:32.398550  5324 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:32.400228  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.399684  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:32.401336  1720 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:32.360491  1645 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:32.384640  5197 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:32.421096  1569 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:03:32.422587  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:32.423861  1643 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:32.423347  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 556) took 1600 ms. Trace:
I20260709 13:03:32.424860  1569 rpcz_store.cc:276] 0709 13:03:30.822597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.140870 (+1318273us) service_pool.cc:224] Handling call
0709 13:03:32.423281 (+282411us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:32.402477  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:32.425700  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:32.414165  1571 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1944) took 1640 ms (client timeout 1693 ms). Trace:
W20260709 13:03:32.411123  1568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1913) took 1693 ms (client timeout 1858 ms). Trace:
W20260709 13:03:32.412477  1570 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1915) took 1653 ms (client timeout 2100 ms). Trace:
I20260709 13:03:32.411553  5285 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:03:32.430673  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:32.421865  5367 consensus_queue.cc:1059] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:03:32.424583  1721 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
W20260709 13:03:32.427942  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:32.434834  1566 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1887) took 2118 ms (client timeout 1516 ms). Trace:
W20260709 13:03:32.428346  1570 rpcz_store.cc:269] 0709 13:03:30.759291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.380478 (+621187us) service_pool.cc:224] Handling call
0709 13:03:32.411943 (+1031465us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b54002ba054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:32.412227 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":570892032}
I20260709 13:03:32.427445  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:32.412932  5345 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:32.426573  1571 rpcz_store.cc:269] 0709 13:03:30.773391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.819532 (+ 46141us) service_pool.cc:224] Handling call
0709 13:03:32.413691 (+1594159us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b5400405bd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:32.413901 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1092304640}
I20260709 13:03:32.437572  5331 consensus_queue.cc:1059] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:03:32.437826  5345 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:32.438841  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2402) took 2818 ms. Trace:
I20260709 13:03:32.439548  1567 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:32.440068  2313 leader_election.cc:336] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.985s (SENT)
I20260709 13:03:32.440472  5345 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:32.441146  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2414) took 1644 ms. Trace:
W20260709 13:03:32.427918  1568 rpcz_store.cc:269] 0709 13:03:30.718063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.766699 (+ 48636us) service_pool.cc:224] Handling call
0709 13:03:32.410543 (+1643844us) spinlock_profiling.cc:243] Waited 56.9 ms on lock 0x7b5400013e54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:32.410858 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130902784}
I20260709 13:03:32.439781  5197 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:32.438694  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:32.422600  5324 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:32.443872  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:32.435508  1566 rpcz_store.cc:269] 0709 13:03:30.287323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.287490 (+   167us) service_pool.cc:224] Handling call
0709 13:03:32.405106 (+2117616us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b54002b8754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:32.405396 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1182848384}
I20260709 13:03:32.444593  5197 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.439428  5367 raft_consensus.cc:3055] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:32.446233  5197 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:32.446457  1571 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 9: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:32.447011  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:32.447520  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1940) took 1755 ms. Trace:
I20260709 13:03:32.448249  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1934) took 1243 ms. Trace:
I20260709 13:03:32.446467  5367 raft_consensus.cc:740] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:32.449258  5301 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:32.441610  1646 rpcz_store.cc:276] 0709 13:03:30.796900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.007172 (+210272us) service_pool.cc:224] Handling call
0709 13:03:31.321961 (+314789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.322958 (+   997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.440624 (+1117666us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b54000fffd0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:32.440879 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":723358976}
I20260709 13:03:32.450955  5367 consensus_queue.cc:260] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:32.451483  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:32.446249  2312 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:32.454483  1646 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:32.455144  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:32.455098  1714 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1916) took 2090 ms (client timeout 2334 ms). Trace:
I20260709 13:03:32.455175  5417 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:32.458250  5417 raft_consensus.cc:697] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 13 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:32.458473  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:32.459319  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:32.459650  1568 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:32.439402  1640 rpcz_store.cc:276] 0709 13:03:29.619928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.936226 (+316298us) service_pool.cc:224] Handling call
0709 13:03:29.959188 (+ 22962us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.492883 (+533695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.438160 (+1945277us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b5400099054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:32.438525 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1205741952}
I20260709 13:03:32.448344  1571 rpcz_store.cc:276] 0709 13:03:30.692347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.438218 (+1745871us) service_pool.cc:224] Handling call
0709 13:03:32.447418 (+  9200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:32.461760  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:32.462703  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:32.463464  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:32.463920  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:32.464542  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:32.465128  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:32.456983  1714 rpcz_store.cc:269] 0709 13:03:30.364000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.539934 (+175934us) service_pool.cc:224] Handling call
0709 13:03:32.454321 (+1914387us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b5400210554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:32.454796 (+   475us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2030792064}
I20260709 13:03:32.478693  5361 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:32.479220  5361 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:32.479652  5361 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:32.447584  1647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 336) took 2824 ms (client timeout 2016 ms). Trace:
W20260709 13:03:32.480696  1647 rpcz_store.cc:269] 0709 13:03:29.623284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.701023 (+ 77739us) service_pool.cc:224] Handling call
0709 13:03:32.446839 (+2745816us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b54002db754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:32.447319 (+   480us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8645888}
I20260709 13:03:32.463466  1571 raft_consensus.cc:2376] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9. Re-sending same reply.
W20260709 13:03:32.482695  5247 consensus_meta.cc:287] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.554s	user 0.005s	sys 0.015s
I20260709 13:03:32.481845  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1936) took 1247 ms. Trace:
I20260709 13:03:32.469290  1572 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 7: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:32.448575  5331 raft_consensus.cc:3055] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:32.484052  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1933) took 1299 ms. Trace:
I20260709 13:03:32.484160  5331 raft_consensus.cc:740] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:32.484611  1572 rpcz_store.cc:276] 0709 13:03:31.184933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.430176 (+1245243us) service_pool.cc:224] Handling call
0709 13:03:32.483946 (+ 53770us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:32.482262  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:32.470729  1722 raft_consensus.cc:3055] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:32.459672  5417 consensus_queue.cc:237] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:32.464749  1645 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:32.486930  1571 rpcz_store.cc:276] 0709 13:03:31.234691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.462298 (+1227607us) service_pool.cc:224] Handling call
0709 13:03:32.481751 (+ 19453us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:32.401407  5356 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:32.468514  1718 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:03:32.448949  1570 rpcz_store.cc:276] 0709 13:03:31.205268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.444829 (+1239561us) service_pool.cc:224] Handling call
0709 13:03:32.448056 (+  3227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:32.482944  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:32.460146  1716 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:32.470242  5357 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.468509  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:32.460202  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1935) took 1242 ms. Trace:
W20260709 13:03:32.481813  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 113807us
I20260709 13:03:32.487421  1722 raft_consensus.cc:740] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:32.501775  1568 rpcz_store.cc:276] 0709 13:03:31.217551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.458056 (+1240505us) service_pool.cc:224] Handling call
0709 13:03:32.460107 (+  2051us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:32.444447  1648 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:32.502791  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:32.504599  1644 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:32.502962  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:32.511009  5285 consensus_queue.cc:1059] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:32.512457  5345 raft_consensus.cc:3055] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:32.512907  5345 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:32.454780  5301 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:32.425071  5328 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:32.520414  1647 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:32.524839  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:32.524876  1643 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:32.522948  1640 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
W20260709 13:03:32.528527  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 486422us
I20260709 13:03:32.530769  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:32.517203  5367 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:32.538789  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:32.540763  2313 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:32.533654  5017 raft_consensus.cc:697] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:32.542562  5415 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:32.540828  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:32.543457  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:32.543392  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:32.544306  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 195084us
I20260709 13:03:32.535355  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 560) took 1189 ms. Trace:
I20260709 13:03:32.545049  1571 rpcz_store.cc:276] 0709 13:03:31.345601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.499246 (+1153645us) service_pool.cc:224] Handling call
0709 13:03:32.535271 (+ 36025us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:32.539654  1570 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:03:32.548821  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:32.550025  1572 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:32.550738  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:32.572135  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:32.546409  1722 consensus_queue.cc:260] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:32.566417  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1936) took 1975 ms. Trace:
I20260709 13:03:32.573678  1572 rpcz_store.cc:276] 0709 13:03:30.579898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.486363 (+1906465us) service_pool.cc:224] Handling call
0709 13:03:32.555109 (+ 68746us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:32.586513  5216 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:32.590194  1566 raft_consensus.cc:2410] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:03:32.590864  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1907) took 2053 ms. Trace:
I20260709 13:03:32.591351  1569 raft_consensus.cc:2410] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:03:32.592260  1644 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
W20260709 13:03:32.592940  1569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1938) took 1990 ms (client timeout 2560 ms). Trace:
I20260709 13:03:32.595295  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:32.598285  1640 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:32.599906  1722 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:32.601856  5179 consensus_meta.cc:287] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.527s	user 0.002s	sys 0.019s
I20260709 13:03:32.602851  1647 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:03:32.604408  5294 raft_consensus.cc:3055] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:32.604883  1647 raft_consensus.cc:1275] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:32.605060  5294 raft_consensus.cc:740] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:32.605785  1721 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
W20260709 13:03:32.593505  1569 rpcz_store.cc:269] 0709 13:03:30.602706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.444356 (+1841650us) service_pool.cc:224] Handling call
0709 13:03:32.592597 (+148241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:32.591235  1566 rpcz_store.cc:276] 0709 13:03:30.537253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.468022 (+1930769us) service_pool.cc:224] Handling call
0709 13:03:32.590774 (+122752us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:32.615474  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.619190  1716 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:03:32.590885  1718 raft_consensus.cc:1275] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:32.623008  1646 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
W20260709 13:03:32.623688  1685 leader_election.cc:336] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.201s (SENT)
I20260709 13:03:32.624956  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:32.607509  5321 consensus_meta.cc:287] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.576s	user 0.016s	sys 0.004s
I20260709 13:03:32.627365  5331 consensus_queue.cc:260] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:32.630880  5216 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:32.632261  1641 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:03:32.632463  1647 raft_consensus.cc:1240] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:32.634040  1645 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
W20260709 13:03:32.657286  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45911.5us
I20260709 13:03:32.636440  1648 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:32.522902  5357 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:32.646440  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:32.629362  1573 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1942) took 1898 ms (client timeout 2334 ms). Trace:
I20260709 13:03:32.656983  1640 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:32.669426  1643 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
W20260709 13:03:32.665436  1573 rpcz_store.cc:269] 0709 13:03:30.730529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.776940 (+1046411us) service_pool.cc:224] Handling call
0709 13:03:32.629104 (+852164us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b54002233d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:32.629282 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":577200000}
W20260709 13:03:32.667959  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 103260us
W20260709 13:03:32.691916  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.542994  5017 consensus_queue.cc:237] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:32.696425  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:32.699887  2314 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:32.696099  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:32.703076  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1945) took 1923 ms. Trace:
I20260709 13:03:32.703738  1568 rpcz_store.cc:276] 0709 13:03:30.780066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.538099 (+1758033us) service_pool.cc:224] Handling call
0709 13:03:32.702990 (+164891us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:32.709048  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:32.700804  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:32.722112  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:32.719913  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:32.725684  5252 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:03:32.732314  1687 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.136s (SENT) [suppressed 42 similar messages]
I20260709 13:03:32.733671  2314 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:32.734867  5324 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:32.735354  5303 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:32.735522  5324 raft_consensus.cc:697] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:32.736050  5303 raft_consensus.cc:697] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:32.736792  5324 consensus_queue.cc:237] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:32.738250  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:32.743228  5250 consensus_meta.cc:287] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.741s	user 0.011s	sys 0.007s
W20260709 13:03:32.745921  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 201265us
I20260709 13:03:32.746646  5356 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:32.747084  1537 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:32.748306  1645 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:32.752238  1645 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:03:32.726539  5252 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:32.761303  1566 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:32.740053  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:32.708618  1646 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:32.745937  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:32.759943  1647 raft_consensus.cc:2410] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:03:32.725818  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:32.762219  1717 raft_consensus.cc:1240] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:03:32.667426  5340 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:32.787596  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:32.792244  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.792627  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:32.796006  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:32.797091  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 71714us
I20260709 13:03:32.795118  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1926) took 1774 ms. Trace:
I20260709 13:03:32.800127  1571 rpcz_store.cc:276] 0709 13:03:31.020865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.680854 (+1659989us) service_pool.cc:224] Handling call
0709 13:03:32.795030 (+114176us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:32.807544  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 70807.1us
I20260709 13:03:32.811293  1645 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:32.700460  5285 consensus_queue.cc:1059] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 13:03:32.841616  5285 consensus_queue.cc:1243] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:03:32.842296  5285 raft_consensus.cc:3055] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:32.842706  5285 raft_consensus.cc:740] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:32.843413  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2412) took 2046 ms. Trace:
I20260709 13:03:32.843775  5285 consensus_queue.cc:260] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:32.845309  5328 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:32.848596  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:32.850790  1717 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:32.854537  5361 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:32.825964  5359 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:32.857739  5361 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:32.858951  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1875) took 1022 ms. Trace:
I20260709 13:03:32.859272  5359 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:32.859509  1640 rpcz_store.cc:276] 0709 13:03:31.835943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.599495 (+763552us) service_pool.cc:224] Handling call
0709 13:03:32.647967 (+ 48472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:32.682846 (+ 34879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.858546 (+175700us) spinlock_profiling.cc:243] Waited 47.9 ms on lock 0x7b54002dc150. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:32.858716 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110143744}
W20260709 13:03:32.869577  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.870875  1643 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:32.820060  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:32.820536  5294 consensus_queue.cc:260] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:32.821843  5254 consensus_meta.cc:287] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.771s	user 0.012s	sys 0.010s
W20260709 13:03:32.822191  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.826316  1647 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:32.825942  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:32.827026  1573 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:32.827095  5351 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:32.827862  5307 consensus_meta.cc:287] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.623s	user 0.009s	sys 0.009s
I20260709 13:03:32.827039  1648 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:32.834453  5345 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:32.888558  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1874) took 1045 ms. Trace:
W20260709 13:03:32.815344  1687 connection.cc:256] RPC call timeout handler was delayed by -2.0694943169999647s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22050000000000003s for a response
I20260709 13:03:32.806356  5301 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:32.836374  1538 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:32.839835  5348 consensus_meta.cc:287] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.811s	user 0.010s	sys 0.008s
I20260709 13:03:32.840775  5331 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:32.908344  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:32.909835  1687 connection.cc:256] RPC call timeout handler was delayed by -1.761351048999984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30150000000000005s for a response
W20260709 13:03:32.910874  1687 connection.cc:256] RPC call timeout handler was delayed by -1.5797517720000087s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1811s for a response
W20260709 13:03:32.911218  1687 connection.cc:256] RPC call timeout handler was delayed by -1.5642381050000154s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1756s for a response
W20260709 13:03:32.911502  1687 connection.cc:256] RPC call timeout handler was delayed by -1.3365172690000122s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19990000000000002s for a response
W20260709 13:03:32.911787  1687 connection.cc:256] RPC call timeout handler was delayed by -1.3173580040000274s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2037s for a response
W20260709 13:03:32.912132  1687 connection.cc:256] RPC call timeout handler was delayed by -1.2264054900000474s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21000000000000002s for a response
W20260709 13:03:32.912792  1687 connection.cc:256] RPC call timeout handler was delayed by -1.1623925499999928s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22010000000000002s for a response
I20260709 13:03:32.818907  1641 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:32.916836  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1924) took 1148 ms. Trace:
I20260709 13:03:32.843947  1642 rpcz_store.cc:276] 0709 13:03:30.796409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.005544 (+209135us) service_pool.cc:224] Handling call
0709 13:03:31.005815 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.855772 (+849957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.842651 (+986879us) spinlock_profiling.cc:243] Waited 791 ms on lock 0x7b54002ab7d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:32.843151 (+   500us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1819533696}
I20260709 13:03:32.845490  5285 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:32.882513  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1928) took 1861 ms. Trace:
I20260709 13:03:32.737293  5303 consensus_queue.cc:237] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:32.902669  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 115273us
I20260709 13:03:32.922566  1648 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:32.907398  1571 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:32.908623  1719 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:32.857117  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:32.889557  1639 rpcz_store.cc:276] 0709 13:03:31.835543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.596409 (+760866us) service_pool.cc:224] Handling call
0709 13:03:32.596974 (+   565us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:32.597698 (+   724us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:32.597729 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:32.598511 (+   782us) log.cc:844] Serialized 26 byte log entry
0709 13:03:32.877305 (+278794us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:32.877354 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.877441 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:32.877494 (+    53us) raft_consensus.cc:1685] finished
0709 13:03:32.877507 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:32.881242 (+  3735us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":278161,"threads_started":1,"wal-append.queue_time_us":238379}
I20260709 13:03:32.920218  1573 rpcz_store.cc:276] 0709 13:03:31.021404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.765329 (+1743925us) service_pool.cc:224] Handling call
0709 13:03:32.882442 (+117113us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:32.913546  1687 connection.cc:256] RPC call timeout handler was delayed by -1.0752580040000339s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23060000000000003s for a response
W20260709 13:03:32.925179  1687 connection.cc:256] RPC call timeout handler was delayed by -1.0380092560000094s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1694s for a response
W20260709 13:03:32.925727  1687 connection.cc:256] RPC call timeout handler was delayed by -1.0306568379999703s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2871s for a response
W20260709 13:03:32.926436  1687 connection.cc:256] RPC call timeout handler was delayed by -1.0183130660000188s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17020000000000002s for a response
I20260709 13:03:32.917331  1716 rpcz_store.cc:276] 0709 13:03:31.760016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.502960 (+742944us) service_pool.cc:224] Handling call
0709 13:03:32.565147 (+ 62187us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:32.626329 (+ 61182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.907757 (+281428us) spinlock_profiling.cc:243] Waited 42.8 ms on lock 0x7b540002e9d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:32.908309 (+   552us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98457472}
W20260709 13:03:32.942270  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.951589  5294 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:32.951773  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:32.952556  1571 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:32.954080  1568 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:32.956504  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:32.957342  5252 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:32.958019  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:32.958948  1573 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:32.959437  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1924) took 1939 ms. Trace:
I20260709 13:03:32.960772  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2334) took 1144 ms. Trace:
I20260709 13:03:32.961787  1719 rpcz_store.cc:276] 0709 13:03:31.785559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.314925 (+529366us) service_pool.cc:224] Handling call
0709 13:03:32.927796 (+612871us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:32.957316  1568 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:32.965553  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1944) took 1019 ms. Trace:
I20260709 13:03:32.968175  2340 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
W20260709 13:03:32.978672  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.959774  1573 rpcz_store.cc:276] 0709 13:03:31.020097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.924796 (+1904699us) service_pool.cc:224] Handling call
0709 13:03:32.959370 (+ 34574us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:32.979251  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:32.976397  5421 log.cc:927] Time spent T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:03:32.971383  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:32.982134  5313 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.982998  5313 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:32.984898  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.984526  1719 rpcz_store.cc:276] 0709 13:03:31.946374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.963426 (+1017052us) service_pool.cc:224] Handling call
0709 13:03:32.964593 (+  1167us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:32.965046 (+   453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:32.965080 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:32.965162 (+    82us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:32.965210 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.965276 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:32.965442 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:32.985450  1568 raft_consensus.cc:1240] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:32.994887  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1904) took 2661 ms. Trace:
I20260709 13:03:32.996583  1568 rpcz_store.cc:276] 0709 13:03:30.340007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.973027 (+2633020us) service_pool.cc:224] Handling call
0709 13:03:32.985065 (+ 12038us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:32.990296 (+  5231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.994144 (+  3848us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:32.999536  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 163332us
I20260709 13:03:33.000762  1538 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:33.001689  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 552) took 2441 ms. Trace:
I20260709 13:03:32.899219  5254 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:32.924297  5216 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.010119  1715 raft_consensus.cc:2393] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 9: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:33.002804  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:33.010540  5216 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:32.982355  1571 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:32.949628  1645 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:33.012755  1568 rpcz_store.cc:276] 0709 13:03:30.560752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.999588 (+2438836us) service_pool.cc:224] Handling call
0709 13:03:33.000361 (+   773us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.000831 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:33.000879 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:33.001101 (+   222us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:33.001163 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.001261 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.001511 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:33.027908  1720 consensus_meta.cc:287] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.531s	user 0.012s	sys 0.000s
I20260709 13:03:33.028564  1720 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:33.029232  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1932) took 1123 ms. Trace:
I20260709 13:03:33.029608  1720 rpcz_store.cc:276] 0709 13:03:31.906367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.399185 (+492818us) service_pool.cc:224] Handling call
0709 13:03:33.029156 (+629971us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:33.030615  1572 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 4.
W20260709 13:03:33.034713  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.035352  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.016234  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:33.038651  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 87352.4us
W20260709 13:03:33.039520  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.043246  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:33.044046  1722 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:03:33.044853  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2331) took 1317 ms. Trace:
I20260709 13:03:33.049391  5186 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:33.051231  5254 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:33.051930  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1908) took 2492 ms. Trace:
I20260709 13:03:33.053699  5192 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:33.055605  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 554) took 2494 ms. Trace:
I20260709 13:03:33.053362  1573 rpcz_store.cc:276] 0709 13:03:30.559543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.983262 (+2423719us) service_pool.cc:224] Handling call
0709 13:03:32.984014 (+   752us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.050852 (+ 66838us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:33.050989 (+   137us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:33.051076 (+    87us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:33.051177 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.051546 (+   369us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.051729 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42987,"raft.queue_time_us":69642,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":256}
I20260709 13:03:33.057667  5192 raft_consensus.cc:697] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:33.045264  1722 rpcz_store.cc:276] 0709 13:03:31.727738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.469432 (+741694us) service_pool.cc:224] Handling call
0709 13:03:32.469918 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.044495 (+574577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.044758 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:33.065362  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1906) took 2528 ms. Trace:
I20260709 13:03:32.920115  5301 raft_consensus.cc:697] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:33.069244  1716 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:33.078972  1571 rpcz_store.cc:276] 0709 13:03:30.498869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.981565 (+2482696us) service_pool.cc:224] Handling call
0709 13:03:32.982123 (+   558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.026627 (+ 44504us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.027089 (+   462us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:33.069914  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 89269.5us
I20260709 13:03:33.071563  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:33.063704  1568 rpcz_store.cc:276] 0709 13:03:30.561418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.027309 (+2465891us) service_pool.cc:224] Handling call
0709 13:03:33.054076 (+ 26767us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.054532 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:33.054632 (+   100us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:33.055073 (+   441us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:33.055137 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.055220 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.055386 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:33.082749  5247 consensus_queue.cc:1059] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:03:33.083105  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 80474.6us
I20260709 13:03:33.084584  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 553) took 2523 ms. Trace:
I20260709 13:03:33.085099  1569 rpcz_store.cc:276] 0709 13:03:30.561092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.001259 (+2440167us) service_pool.cc:224] Handling call
0709 13:03:33.041460 (+ 40201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.084137 (+ 42677us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:33.084168 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:33.084244 (+    76us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:33.084294 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.084362 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.084516 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23296}
I20260709 13:03:33.095527  5307 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:33.096328  5307 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.096815  5307 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:33.099160  5216 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:33.106647  5313 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:33.110661  1573 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:03:33.111515  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1931) took 2549 ms. Trace:
I20260709 13:03:33.111869  1573 rpcz_store.cc:276] 0709 13:03:30.562266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.064896 (+2502630us) service_pool.cc:224] Handling call
0709 13:03:33.110415 (+ 45519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:33.111214 (+   799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.111454 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:33.114290  5301 consensus_queue.cc:237] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:33.060941  5197 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.614s	user 0.015s	sys 0.000s
I20260709 13:03:33.118062  5197 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:33.062256  5357 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:33.129859  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 114409us
W20260709 13:03:33.131994  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 46406us
W20260709 13:03:33.143060  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 89139.8us
I20260709 13:03:32.903723  5351 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:33.151250  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 404374us
I20260709 13:03:33.160008  1716 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:03:33.163226  5192 consensus_queue.cc:237] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:33.174491  1565 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:33.143417  5357 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:33.177980  5357 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.178473  5357 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:33.181311  1538 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:33.140482  5353 consensus_meta.cc:287] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.736s	user 0.013s	sys 0.000s
I20260709 13:03:33.189129  1687 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:33.193971  1567 consensus_meta.cc:287] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.753s	user 0.016s	sys 0.000s
I20260709 13:03:33.193564  5353 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:33.212860  1567 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:03:33.213259  1567 pending_rounds.cc:85] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 4
I20260709 13:03:33.213469  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1936) took 1269 ms. Trace:
I20260709 13:03:33.214169  5250 consensus_queue.cc:1059] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:03:33.219556  5250 consensus_queue.cc:1243] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:33.218976  1718 rpcz_store.cc:276] 0709 13:03:31.944167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.626998 (+682831us) service_pool.cc:224] Handling call
0709 13:03:32.627725 (+   727us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:32.628390 (+   665us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:32.628427 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:32.629221 (+   794us) log.cc:844] Serialized 26 byte log entry
0709 13:03:32.879291 (+250070us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:32.879329 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.879399 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:33.213169 (+333770us) raft_consensus.cc:1685] finished
0709 13:03:33.213199 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.213392 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":249571,"threads_started":1,"wal-append.queue_time_us":580768}
I20260709 13:03:33.213506  1567 pending_rounds.cc:107] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 8.5
W20260709 13:03:33.214169  1685 leader_election.cc:336] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.887s (SENT)
I20260709 13:03:33.222175  1567 raft_consensus.cc:2889] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:33.223308  1685 leader_election.cc:336] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.956s (SENT)
W20260709 13:03:33.223824  5306 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:03:33.224063  1685 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:33.222044  1537 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:33.223430  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1902) took 2907 ms. Trace:
I20260709 13:03:33.217964  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1922) took 1458 ms. Trace:
I20260709 13:03:33.225467  1567 rpcz_store.cc:276] 0709 13:03:30.316295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.338886 (+ 22591us) service_pool.cc:224] Handling call
0709 13:03:30.339123 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:32.408596 (+2069473us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b580073a0d0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248de10aa 00007f8248e63447 00007f8248e7bfe7 00007f8248e7d7de 00007f8248e87b49 00007f8248e88416 00007f8248e84d8f 00007f8248e8483e 00007f824de49419 00007f82473c0bd4
0709 13:03:33.222990 (+814394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.223343 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4064577792}
I20260709 13:03:33.211158   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:33.214934  5216 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:33.228680  5216 raft_consensus.cc:697] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:33.230044  5216 consensus_queue.cc:237] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:33.231563  5312 consensus_meta.cc:287] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.853s	user 0.016s	sys 0.000s
I20260709 13:03:33.232054  5312 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:33.233611  5287 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:33.220697  1571 raft_consensus.cc:1275] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:33.234946  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1933) took 2672 ms. Trace:
I20260709 13:03:33.235342  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1932) took 2672 ms. Trace:
I20260709 13:03:33.241410  1571 rpcz_store.cc:276] 0709 13:03:30.562679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.139435 (+2576756us) service_pool.cc:224] Handling call
0709 13:03:33.220163 (+ 80728us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.235074 (+ 14911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.235286 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:33.234797  1685 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.022s (SENT)
I20260709 13:03:33.232467  5355 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.235427  1572 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:33.225955  1714 rpcz_store.cc:276] 0709 13:03:31.759402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.479243 (+719841us) service_pool.cc:224] Handling call
0709 13:03:32.479836 (+   593us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:32.506294 (+ 26458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:32.506341 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:32.610107 (+103766us) log.cc:844] Serialized 26 byte log entry
0709 13:03:32.916688 (+306581us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:32.916748 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.916859 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:33.217711 (+300852us) raft_consensus.cc:1685] finished
0709 13:03:33.217735 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.217891 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6913,"raft.run_cpu_time_us":480,"raft.run_wall_time_us":55028,"thread_start_us":209160,"threads_started":1,"wal-append.queue_time_us":540016}
W20260709 13:03:33.237720  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 71978.6us
I20260709 13:03:33.235345  1568 rpcz_store.cc:276] 0709 13:03:30.563211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.158770 (+2595559us) service_pool.cc:224] Handling call
0709 13:03:33.234245 (+ 75475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.234558 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:33.234585 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:33.234681 (+    96us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:33.234721 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.234772 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.234894 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:33.243048  1685 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:33.236574  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 84195.1us
W20260709 13:03:33.250742  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.258773  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1937) took 2659 ms. Trace:
I20260709 13:03:33.213475  1687 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:33.259486  1572 rpcz_store.cc:276] 0709 13:03:30.599881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.234825 (+2634944us) service_pool.cc:224] Handling call
0709 13:03:33.235234 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.258383 (+ 23149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.258685 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:33.259613  1538 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:33.182883  5247 consensus_queue.cc:1243] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer cf2f71592be8465fb18e36e15d0b2f89 log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:03:33.273397  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.267976  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1935) took 1324 ms. Trace:
I20260709 13:03:33.218186  5340 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:33.235651  5313 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:33.291610  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 56971.6us
I20260709 13:03:33.295774  1612 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:33.213981  5197 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:33.299978  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1934) took 1009 ms. Trace:
W20260709 13:03:33.311149  1685 leader_election.cc:336] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.107s (SENT)
I20260709 13:03:33.242568  5359 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:33.314779  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.317448  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 95598.3us
I20260709 13:03:33.319103  1612 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:33.308717  5285 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:33.306546  5356 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:33.285215  1721 rpcz_store.cc:276] 0709 13:03:31.943810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.607463 (+663653us) service_pool.cc:224] Handling call
0709 13:03:32.624604 (+ 17141us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:32.625150 (+   546us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:32.771416 (+146266us) log.cc:844] Serialized 14 byte log entry
0709 13:03:33.267441 (+496025us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:33.267510 (+    69us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:33.267569 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.267655 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.267877 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":220576,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":43820,"thread_start_us":475077,"threads_started":2}
W20260709 13:03:33.320448  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 265147us
I20260709 13:03:33.311048  5247 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:33.320952  5287 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:33.335932  1567 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:33.338675  1564 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
W20260709 13:03:33.339663  5428 log.cc:927] Time spent T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.058s	user 0.001s	sys 0.001s
I20260709 13:03:33.328768  5285 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.341081  5285 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:33.342581  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 351) took 1539 ms. Trace:
I20260709 13:03:33.343176  1644 rpcz_store.cc:276] 0709 13:03:31.802802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.593206 (+790404us) service_pool.cc:224] Handling call
0709 13:03:32.594200 (+   994us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:32.619051 (+ 24851us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:32.619094 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:32.630312 (+ 11218us) log.cc:844] Serialized 26 byte log entry
0709 13:03:32.802575 (+172263us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:33.068825 (+266250us) log.cc:844] Serialized 14 byte log entry
0709 13:03:33.341832 (+273007us) spinlock_profiling.cc:243] Waited 41.2 ms on lock 0x7b540032f3a0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e00733 00007f8248e85dbf 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c
0709 13:03:33.341912 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.342034 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:33.342102 (+    68us) raft_consensus.cc:1685] finished
0709 13:03:33.342252 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.342462 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8773,"prepare.queue_time_us":411022,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":38,"raft.queue_time_us":20550,"raft.run_cpu_time_us":468,"raft.run_wall_time_us":37351,"spinlock_wait_cycles":94785792,"thread_start_us":329028,"threads_started":2,"wal-append.queue_time_us":300961}
I20260709 13:03:33.330971  5340 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:03:33.348443  1538 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:33.352265  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.310472  1713 rpcz_store.cc:276] 0709 13:03:31.934317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.595408 (+661091us) service_pool.cc:224] Handling call
0709 13:03:32.595947 (+   539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:32.596527 (+   580us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:32.597294 (+   767us) log.cc:844] Serialized 14 byte log entry
0709 13:03:32.841380 (+244086us) spinlock_profiling.cc:243] Waited 23 ms on lock 0x7b18002f4640. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:03:32.943204 (+101824us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:32.943264 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:32.943322 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.943387 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:32.943527 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114548,"prepare.queue_time_us":16851,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":59,"raft.queue_time_us":40366,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":94,"spinlock_wait_cycles":52837248,"thread_start_us":282917,"threads_started":1,"wal-append.queue_time_us":625449}
W20260709 13:03:33.341014  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 127693us
I20260709 13:03:33.308652  5313 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:33.322491  5359 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.285408  5355 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:33.375988  5186 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:33.381038  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 69933us
I20260709 13:03:33.395714  4966 raft_consensus.cc:3055] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:33.397279  5252 raft_consensus.cc:3055] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:03:33.397864  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.398387  5345 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:33.406414  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 64884.4us
I20260709 13:03:33.368469  5312 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:33.410955  1687 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:33.411806  5340 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:33.361923  5197 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:33.413213  1612 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:33.415220  5313 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:33.376899  5372 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:33.488456  2340 raft_consensus.cc:1240] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:33.330416  5356 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.423645  4966 raft_consensus.cc:740] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:33.424185  5186 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:33.425597  5331 consensus_meta.cc:287] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.509s	user 0.012s	sys 0.002s
W20260709 13:03:33.468721  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.478410  5351 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:33.429467  5301 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.427302  5359 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:33.426550  5252 raft_consensus.cc:740] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:33.490075  5197 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:33.506412  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 93551.5us
I20260709 13:03:33.503149  5356 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:33.505455  5340 raft_consensus.cc:697] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:33.541926  1568 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:03:33.545264  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1923) took 2525 ms. Trace:
I20260709 13:03:33.545849  1568 rpcz_store.cc:276] 0709 13:03:31.019696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.537842 (+2518146us) service_pool.cc:224] Handling call
0709 13:03:33.539643 (+  1801us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:33.544782 (+  5139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.545174 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:33.547639  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 289014us
W20260709 13:03:33.548326  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.543812  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:33.497181  5372 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.499325  5313 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:33.556465  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 131365us
I20260709 13:03:33.558636  1568 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:33.559249  2349 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:33.532193  5285 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:33.559656  4966 consensus_queue.cc:260] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:33.538646  5197 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:33.535053  5301 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:33.565160  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 559) took 2511 ms. Trace:
I20260709 13:03:33.570156  1572 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:33.532531  5328 consensus_queue.cc:1059] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:03:33.567426  5351 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:33.571403  1568 rpcz_store.cc:276] 0709 13:03:31.053438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.546922 (+2493484us) service_pool.cc:224] Handling call
0709 13:03:33.555377 (+  8455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.564806 (+  9429us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.565088 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:33.560250  5307 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:33.577534  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.577517  1537 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:33.558717  5372 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:33.588514  5186 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:33.590588  1573 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:33.592236  5307 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:33.593861  5250 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:33.594070  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:33.594957  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:33.597550  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:33.598188  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:33.615294  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:33.600142  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.616496  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.615245  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:33.609112  1612 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:33.618352  1643 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:03:33.598583  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 187782us
I20260709 13:03:33.620783  1647 raft_consensus.cc:2368] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12.
I20260709 13:03:33.574869  5351 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:33.625865  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:33.626652  1642 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:33.627398  1639 raft_consensus.cc:2410] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:03:33.598346  4966 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:33.629776  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 55570.7us
I20260709 13:03:33.631626  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:33.630897  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:33.634214  1640 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 7: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:33.634688  1646 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:33.608198  5287 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:33.625520  5294 consensus_queue.cc:1059] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:03:33.640458  1687 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:33.641402  5294 raft_consensus.cc:3055] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:33.642040  5294 raft_consensus.cc:740] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:33.642603  5247 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:33.643443  5247 raft_consensus.cc:697] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:33.643230  5294 consensus_queue.cc:260] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:33.644621  1687 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:33.645179  5294 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:33.644826  5247 consensus_queue.cc:237] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:33.646201  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:33.642154  1565 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1905) took 3287 ms (client timeout 2205 ms). Trace:
I20260709 13:03:33.579193  5313 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:33.638713  5340 consensus_queue.cc:237] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:33.681381  5254 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:33.684978  1611 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:33.687610  5443 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:33.692262  5443 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:33.659106  5285 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:33.662230  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:33.699795  5443 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:33.666031  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:33.651104  1612 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:33.667232  5301 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:33.630249  5250 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:33.619444  5186 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:33.631912  5351 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:33.675500  5413 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:33.688339  5254 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:03:33.689797  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 116339us
I20260709 13:03:33.690431  5313 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:33.626186  5312 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:33.696163  5287 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:33.657996  5269 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:33.650604  1565 rpcz_store.cc:269] 0709 13:03:30.355006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.487664 (+132658us) service_pool.cc:224] Handling call
0709 13:03:33.641813 (+3154149us) spinlock_profiling.cc:243] Waited 2.17 s on lock 0x7b5400225bd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:33.642006 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4997278976}
I20260709 13:03:33.709398  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1934) took 3139 ms. Trace:
I20260709 13:03:33.730593  1567 rpcz_store.cc:276] 0709 13:03:30.563546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.226853 (+2663307us) service_pool.cc:224] Handling call
0709 13:03:33.227219 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.337459 (+110240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.702502 (+365043us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54001270d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:33.702703 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249473792}
W20260709 13:03:33.710799  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.734464  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 83442.3us
I20260709 13:03:33.717332  5186 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.738091  5186 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:33.743244  1567 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:03:33.760226  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1958) took 1417 ms. Trace:
I20260709 13:03:33.760736  1567 rpcz_store.cc:276] 0709 13:03:32.353934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.731563 (+1377629us) service_pool.cc:224] Handling call
0709 13:03:33.742917 (+ 11354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:33.759867 (+ 16950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.760150 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:33.743168  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1943) took 1797 ms. Trace:
I20260709 13:03:33.742494  1537 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:33.773749  1566 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:33.778445  1567 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:33.736650  5269 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:33.783047  5269 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:33.785717  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 138050us
I20260709 13:03:33.787422  1611 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:33.795336  5250 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:33.798938  5287 consensus_queue.cc:1059] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:33.800002  5287 consensus_queue.cc:1243] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer cf2f71592be8465fb18e36e15d0b2f89 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:03:33.800704  5287 raft_consensus.cc:3055] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:33.801103  5287 raft_consensus.cc:740] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:33.764400  1717 rpcz_store.cc:276] 0709 13:03:31.946140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.927628 (+981488us) service_pool.cc:224] Handling call
0709 13:03:32.971525 (+ 43897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:32.972347 (+   822us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:32.973395 (+  1048us) log.cc:844] Serialized 14 byte log entry
0709 13:03:33.737540 (+764145us) log.cc:844] Serialized 14 byte log entry
0709 13:03:33.742741 (+  5201us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:33.742804 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:33.742868 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.742940 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.743095 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":255804,"prepare.queue_time_us":975,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":92,"raft.queue_time_us":269556,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":312,"thread_start_us":437310,"threads_started":2,"wal-append.queue_time_us":416836}
I20260709 13:03:33.734247  5313 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:33.776222  1685 leader_election.cc:336] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.136s (SENT)
I20260709 13:03:33.810817  1685 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:33.771261  5357 consensus_meta.cc:287] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.592s	user 0.010s	sys 0.005s
I20260709 13:03:33.759509  5413 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.812999  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 562) took 1402 ms. Trace:
I20260709 13:03:33.815975  1567 rpcz_store.cc:276] 0709 13:03:32.410968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.762202 (+1351234us) service_pool.cc:224] Handling call
0709 13:03:33.778239 (+ 16037us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.811982 (+ 33743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.812906 (+   924us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:33.802129  5391 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:33.826958  5391 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.827178  5447 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:33.827492  5391 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:33.797293  5356 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:33.829788  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:33.830816  5356 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:33.831773  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1917) took 3009 ms. Trace:
I20260709 13:03:33.832204  1573 rpcz_store.cc:276] 0709 13:03:30.822174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.401524 (+2579350us) service_pool.cc:224] Handling call
0709 13:03:33.402265 (+   741us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.609537 (+207272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.831539 (+222002us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b5400405bd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:33.831713 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":432492800}
I20260709 13:03:33.833812  1571 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:33.835108  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1946) took 3014 ms. Trace:
I20260709 13:03:33.835471  1571 rpcz_store.cc:276] 0709 13:03:30.821142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.291447 (+2470305us) service_pool.cc:224] Handling call
0709 13:03:33.327883 (+ 36436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.834238 (+506355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.834823 (+   585us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b54002b8c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:33.835027 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1163265408}
W20260709 13:03:33.822985  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 46847.3us
I20260709 13:03:33.742367  5328 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 13:03:33.833621  1573 raft_consensus.cc:1240] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:33.841750  5448 consensus_queue.cc:1048] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:03:33.842383  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1962) took 1285 ms. Trace:
I20260709 13:03:33.842823  1573 rpcz_store.cc:276] 0709 13:03:32.557105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.833123 (+1276018us) service_pool.cc:224] Handling call
0709 13:03:33.833453 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.842060 (+  8607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.842302 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:33.844079  5445 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:33.843670  5252 consensus_queue.cc:260] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:33.844961  5252 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:33.818917  5357 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:33.847714  5357 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:33.869686  5252 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:03:33.893959  5197 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:33.896217  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 153810us
I20260709 13:03:33.899503  5197 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:33.899507  1537 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:33.900781  5355 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:33.901648  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.902228  5357 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:33.901911  5359 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:33.903828  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.902833  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1919) took 3013 ms. Trace:
I20260709 13:03:33.910492  1572 rpcz_store.cc:276] 0709 13:03:30.889107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.433181 (+2544074us) service_pool.cc:224] Handling call
0709 13:03:33.433843 (+   662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.570934 (+137091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.902412 (+331478us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b5400190fd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:33.902716 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312993024}
W20260709 13:03:33.878026  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 65792.9us
I20260709 13:03:33.857271  5331 raft_consensus.cc:3055] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:33.823796  5192 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:33.921895  1685 leader_election.cc:336] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.172s (SENT)
I20260709 13:03:33.921938  1572 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:33.923751  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1965) took 1076 ms. Trace:
I20260709 13:03:33.924247  1572 rpcz_store.cc:276] 0709 13:03:32.847647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.912079 (+1064432us) service_pool.cc:224] Handling call
0709 13:03:33.921695 (+  9616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.923380 (+  1685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.923647 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:33.930846  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1960) took 1500 ms. Trace:
I20260709 13:03:33.931501  1567 rpcz_store.cc:276] 0709 13:03:32.430207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.817952 (+1387745us) service_pool.cc:224] Handling call
0709 13:03:33.892658 (+ 74706us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.926076 (+ 33418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:33.926124 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:33.926197 (+    73us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:33.930431 (+  4234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.930538 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.930759 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:33.947677  5186 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:33.949052  1643 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:33.950932  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:33.951653  1569 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:33.951999  1717 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 9: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:33.954445  5186 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:33.955560  1564 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1885) took 3719 ms (client timeout 1887 ms). Trace:
I20260709 13:03:33.956306  1612 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:33.955881  1570 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:33.958191  5450 consensus_queue.cc:1059] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:03:33.959671  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.960166  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.961184  1567 raft_consensus.cc:1240] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:03:33.962417  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1967) took 1114 ms. Trace:
I20260709 13:03:33.964692  1567 rpcz_store.cc:276] 0709 13:03:32.848242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.937472 (+1089230us) service_pool.cc:224] Handling call
0709 13:03:33.960916 (+ 23444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:33.962111 (+  1195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.962351 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:33.919589  5331 raft_consensus.cc:740] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:03:33.978941  1566 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 1954) took 1665 ms (client timeout 2212 ms). Trace:
W20260709 13:03:33.955982  1564 rpcz_store.cc:269] 0709 13:03:30.236579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.237815 (+  1236us) service_pool.cc:224] Handling call
0709 13:03:33.955315 (+3717500us) spinlock_profiling.cc:243] Waited 2.92 s on lock 0x7b54002ba554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:33.955483 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6713158912}
W20260709 13:03:33.979548  1566 rpcz_store.cc:269] 0709 13:03:32.313426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.941305 (+627879us) service_pool.cc:224] Handling call
0709 13:03:33.978740 (+1037435us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b5400306954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:33.978871 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1140210048}
I20260709 13:03:33.960201  5359 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:33.955155  5186 consensus_queue.cc:1059] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
W20260709 13:03:33.966516  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.974061  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.977556  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1918) took 3109 ms. Trace:
I20260709 13:03:33.874752  5413 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:33.985706  1564 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:03:33.957484  5313 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:33.988576  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1920) took 3099 ms. Trace:
I20260709 13:03:33.996959  2349 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:03:33.997071  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 209781us
I20260709 13:03:33.997368  1570 rpcz_store.cc:276] 0709 13:03:30.889431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.471593 (+2582162us) service_pool.cc:224] Handling call
0709 13:03:33.955484 (+483891us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.957746 (+  2262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.988351 (+ 30605us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b54002ba550. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:33.988506 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1111491968}
I20260709 13:03:33.996829  1537 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:34.002883  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.993445  1569 rpcz_store.cc:276] 0709 13:03:30.867995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.406793 (+2538798us) service_pool.cc:224] Handling call
0709 13:03:33.407432 (+   639us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.952296 (+544864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.977198 (+ 24902us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b5400307354. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:33.977472 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1251220864}
I20260709 13:03:33.994297  1564 pending_rounds.cc:85] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 9
I20260709 13:03:34.009835  1564 pending_rounds.cc:107] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:03:34.010368  1566 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:03:34.010368  1564 raft_consensus.cc:2889] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:34.013999  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 92207.5us
I20260709 13:03:34.010998  1566 pending_rounds.cc:85] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 4
I20260709 13:03:34.016675  1566 pending_rounds.cc:107] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:03:34.017453  1566 raft_consensus.cc:2889] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:34.019249  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:34.020028  1717 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:34.022490  1567 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:34.026588  1569 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:34.029557  1570 raft_consensus.cc:3055] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:34.030278  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:34.030519  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1968) took 1182 ms. Trace:
I20260709 13:03:34.040143  1567 rpcz_store.cc:276] 0709 13:03:32.848646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.966308 (+1117662us) service_pool.cc:224] Handling call
0709 13:03:34.009382 (+ 43074us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:34.030328 (+ 20946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.030471 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:34.030058  1570 raft_consensus.cc:740] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:34.049453  1717 raft_consensus.cc:2393] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 8: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:34.051342  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:34.057265  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 60549us
I20260709 13:03:34.058939  1719 raft_consensus.cc:2410] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:03:34.062453  1537 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:34.072532  5269 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:34.072741  5186 raft_consensus.cc:3055] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:34.074813  1471 catalog_manager.cc:5697] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 6 to 8, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 8 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:34.078929  1717 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:34.077561  1471 catalog_manager.cc:5697] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:34.054560  5417 consensus_queue.cc:1059] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:03:34.058964  5359 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.086345  5359 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:34.082026  5186 raft_consensus.cc:740] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:34.089392  5186 consensus_queue.cc:260] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:34.083104  5192 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:34.085687  5417 raft_consensus.cc:3055] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:34.094516  5417 raft_consensus.cc:740] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:34.094168  1569 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:34.095800  5417 consensus_queue.cc:260] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:34.098310  5417 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:34.098940  1569 raft_consensus.cc:3055] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:34.102164  1569 raft_consensus.cc:740] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:03:34.104709  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 55690.3us
W20260709 13:03:34.105731  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:34.106598  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:34.108978  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 46650.7us
I20260709 13:03:34.078366  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 561) took 2619 ms. Trace:
I20260709 13:03:33.996773  5287 consensus_queue.cc:260] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:34.114588  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:34.116936  1566 raft_consensus.cc:1275] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:03:34.118266  1717 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:34.116685  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1964) took 1269 ms. Trace:
I20260709 13:03:34.117806  1566 pending_rounds.cc:85] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 6
I20260709 13:03:34.124425  1566 pending_rounds.cc:107] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:03:34.124728  1566 raft_consensus.cc:2889] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:34.124832  5458 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:34.125576  5458 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:34.125340  1471 catalog_manager.cc:5697] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:34.127581  1471 catalog_manager.cc:5697] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 6 to 7, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 7 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:34.128736  5458 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:34.102543  5448 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:03:34.110154  1568 rpcz_store.cc:276] 0709 13:03:31.459045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.618787 (+2159742us) service_pool.cc:224] Handling call
0709 13:03:33.711086 (+ 92299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.724878 (+ 13792us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:33.983318 (+258440us) log.cc:844] Serialized 14 byte log entry
0709 13:03:34.077259 (+ 93941us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:34.077479 (+   220us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:34.077628 (+   149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.077827 (+   199us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.078182 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":367010,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":780,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":256904,"threads_started":2}
I20260709 13:03:34.135099  1714 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:03:34.136996  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 117992us
I20260709 13:03:34.137198  1537 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:34.029544  5294 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:03:34.033990  5353 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 13:03:34.122516  1570 consensus_queue.cc:260] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:34.135417  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:34.133387  1569 consensus_queue.cc:260] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:34.122816  1573 rpcz_store.cc:276] 0709 13:03:32.847153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.843574 (+996421us) service_pool.cc:224] Handling call
0709 13:03:33.844024 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:33.844438 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:33.844473 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:33.845179 (+   706us) log.cc:844] Serialized 26 byte log entry
0709 13:03:34.025296 (+180117us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:34.025336 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.025426 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:34.116422 (+ 90996us) raft_consensus.cc:1685] finished
0709 13:03:34.116446 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.116619 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":179732,"threads_started":1,"wal-append.queue_time_us":245310}
I20260709 13:03:34.135020  5269 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:34.130110  5186 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:34.159073  1611 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:34.161603  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:34.129611  1471 catalog_manager.cc:5697] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index 2 to 4, term changed from 3 to 6, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 6 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:34.159212  5247 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:34.159003  5269 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:34.168351  5306 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:34.130915  5359 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:34.172705  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:34.168193  1471 catalog_manager.cc:5697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 6 to 7, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 7 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:34.147195  5287 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:34.061514  5372 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:34.186583  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1966) took 1329 ms. Trace:
I20260709 13:03:34.187296  1572 rpcz_store.cc:276] 0709 13:03:32.847931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.925080 (+1077149us) service_pool.cc:224] Handling call
0709 13:03:33.939067 (+ 13987us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.982144 (+ 43077us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:34.075239 (+ 93095us) log.cc:844] Serialized 14 byte log entry
0709 13:03:34.177229 (+101990us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:34.177290 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:34.177356 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.177433 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.177566 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":110426,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":5083,"thread_start_us":159723,"threads_started":2}
W20260709 13:03:34.196226  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 76650.3us
I20260709 13:03:34.090188  5331 consensus_queue.cc:260] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:34.203156  1685 leader_election.cc:336] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.811s (SENT)
I20260709 13:03:34.199949  1570 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:34.205807  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 237445us
I20260709 13:03:34.202263  1471 catalog_manager.cc:5697] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:34.215595  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:34.183113  5301 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:34.216897  1714 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:34.168386  5247 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:34.231195  5247 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:34.238464  5458 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:34.242169  1537 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:34.245797  1569 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:34.245896  5269 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:34.246909  5269 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:34.247584  5269 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:34.247957  5372 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:34.250766  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 91869us
I20260709 13:03:34.251680  1611 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:34.196149  5447 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:34.254360  5447 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:34.254757  5447 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:34.255118  5447 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:34.258697  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.263840  1715 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:34.278034  2313 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:34.278723  5443 consensus_meta.cc:287] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.578s	user 0.010s	sys 0.010s
I20260709 13:03:34.279374  5443 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:34.282104  5413 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:34.282486  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 79433us
I20260709 13:03:34.284564  1611 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:34.287142  1611 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:34.238579  1469 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38650 (request call id 197) took 26310 ms (client timeout 15000 ms). Trace:
I20260709 13:03:34.290650  1569 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:03:34.300637  1569 pending_rounds.cc:85] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 6
I20260709 13:03:34.301146  1569 pending_rounds.cc:107] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:03:34.293162  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:34.301503  1569 raft_consensus.cc:2889] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:34.306550  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 64778.7us
I20260709 13:03:34.317121  1537 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:34.264456  5458 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.320559  1471 catalog_manager.cc:5697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:34.325691  1611 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:34.308385  5413 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.316505  5359 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:03:34.333101  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 162702us
I20260709 13:03:34.329931  1537 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:34.325461  5351 consensus_meta.cc:287] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.593s	user 0.012s	sys 0.003s
I20260709 13:03:34.335932  1471 catalog_manager.cc:5697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:34.341795  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.342904  1569 raft_consensus.cc:2393] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 4: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:34.324987  5313 raft_consensus.cc:697] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:34.329026  5312 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:34.363550  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.333307  5359 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:34.324982  5458 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:34.337447  5269 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:34.349488  5357 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:34.374878  5472 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:34.356844  5391 consensus_meta.cc:287] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.528s	user 0.012s	sys 0.004s
I20260709 13:03:34.376778  1611 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:34.349233  5443 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:34.378608  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:34.377390  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 47577.2us
I20260709 13:03:34.382836  1471 catalog_manager.cc:5697] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:34.389432  1685 leader_election.cc:336] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.663s (SENT)
I20260709 13:03:34.390177  1685 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:34.391135  5464 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:34.391412  5450 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:34.392022  5450 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.392432  5450 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:34.393167  5458 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:34.394061  5269 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:34.396463  5192 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:34.397507  5301 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:34.397928  5252 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:34.338845  5413 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:34.398746  5252 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.349268  5457 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:34.354998  5312 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.398161  5301 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:34.400442  1611 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:34.398073  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.408489  1720 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:34.399312  5252 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:34.410748  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:34.412393  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet eca30b43c26f420c965ee25f475581a0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:34.412710  5313 consensus_queue.cc:237] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:34.414460  5462 log.cc:927] Time spent T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 13:03:34.416254  5454 log.cc:927] Time spent T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:03:34.415235  5192 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.419741  5192 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:34.426268  5443 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.420666  1471 catalog_manager.cc:5697] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:34.426903  5443 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:34.348352  5351 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:34.430258  5443 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:34.383919  5357 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.430994  5351 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:34.431169  5357 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:34.432209  1564 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:34.433799  1611 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:34.434810  1611 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:34.437157  1611 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:34.437796  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.438060  5443 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:34.455255  5465 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.456218  5465 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:34.458447  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.438220  5472 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:34.401671  5391 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:34.376777  5359 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:34.465902  1687 leader_election.cc:336] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.755s (SENT)
I20260709 13:03:34.439225  5461 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:34.439442  5457 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:34.452072  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1952) took 2149 ms. Trace:
W20260709 13:03:34.457816  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49035.8us
I20260709 13:03:34.477458  1565 rpcz_store.cc:276] 0709 13:03:32.303070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.729796 (+1426726us) service_pool.cc:224] Handling call
0709 13:03:33.730326 (+   530us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:33.772301 (+ 41975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:33.772336 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:33.825947 (+ 53611us) log.cc:844] Serialized 52 byte log entry
0709 13:03:34.148057 (+322110us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:34.148096 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.407800 (+259704us) spinlock_profiling.cc:243] Waited 29.6 ms on lock 0x7b54001b4254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:34.407856 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:34.441337 (+ 33481us) raft_consensus.cc:1685] finished
0709 13:03:34.441356 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.451937 (+ 10581us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":515,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":37817,"spinlock_wait_cycles":68061440,"thread_start_us":223434,"threads_started":1,"wal-append.queue_time_us":304282}
I20260709 13:03:34.453474  5443 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.402915  5312 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:34.482479  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45416.1us
W20260709 13:03:34.288815  1469 rpcz_store.cc:269] 0709 13:03:07.928570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.933757 (+  5187us) service_pool.cc:224] Handling call
0709 13:03:34.238239 (+26304482us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:23.728303 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:23.861593 (+133290us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:23.889750 (+ 28157us) write_op.cc:435] Acquired schema lock
0709 13:03:23.912592 (+ 22842us) tablet.cc:662] Decoding operations
0709 13:03:24.289778 (+377186us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:24.289876 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:24.289910 (+    34us) tablet.cc:685] Acquiring locks for 26 operations
0709 13:03:24.659242 (+369332us) tablet.cc:701] Row locks acquired
0709 13:03:24.659298 (+    56us) write_op.cc:260] PREPARE: finished
0709 13:03:24.659493 (+   195us) write_op.cc:270] Start()
0709 13:03:24.659640 (+   147us) write_op.cc:276] Timestamp: P: 1783602204659409 usec, L: 0
0709 13:03:24.659663 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:03:24.690067 (+ 30404us) log.cc:844] Serialized 10271 byte log entry
0709 13:03:25.384358 (+694291us) op_driver.cc:464] REPLICATION: finished
0709 13:03:25.824836 (+440478us) write_op.cc:301] APPLY: starting
0709 13:03:25.865774 (+ 40938us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:25.869160 (+  3386us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:25.869327 (+   167us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:26.339711 (+470384us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:26.340322 (+   611us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=52
0709 13:03:26.340393 (+    71us) write_op.cc:312] APPLY: finished
0709 13:03:26.416990 (+ 76597us) log.cc:844] Serialized 176 byte log entry
0709 13:03:27.680266 (+1263276us) spinlock_profiling.cc:243] Waited 43 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8245d7b0ac 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b
0709 13:03:28.201031 (+520765us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:28.201294 (+   263us) write_op.cc:454] Released schema lock
0709 13:03:28.201387 (+    93us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:28.251723 (+ 50336us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:28.269161 (+ 17438us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":77289,"reported_tablets":54,"child_traces":[["op",{"apply.queue_time_us":356052,"apply.run_cpu_time_us":80874,"apply.run_wall_time_us":2446313,"num_ops":26,"prepare.queue_time_us":225381,"prepare.run_cpu_time_us":62060,"prepare.run_wall_time_us":1200906,"raft.queue_time_us":169976,"raft.run_cpu_time_us":41440,"raft.run_wall_time_us":153204,"replication_time_us":724525,"spinlock_wait_cycles":98944896,"thread_start_us":2354068,"threads_started":5,"wal-append.queue_time_us":2010802,"wal-append.run_cpu_time_us":25574,"wal-append.run_wall_time_us":2385731}]]}
W20260709 13:03:34.485037  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:34.459635  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 282342us
W20260709 13:03:34.490857  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.481400  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.479938  1471 catalog_manager.cc:5697] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index 2 to 6, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 6 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:34.472119  5461 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.525982  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.486176  5357 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:34.490342  5443 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:34.527298  5413 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:34.531010  5467 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:34.536500  5391 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:34.537945  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 51014.5us
I20260709 13:03:34.539281  5471 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:34.539661  1471 catalog_manager.cc:5697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 5 to 8, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 8 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:34.546541  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:34.546589  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 236824us
W20260709 13:03:34.553532  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.553706  1570 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:34.459417  5458 consensus_queue.cc:1059] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:03:34.559716  5306 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:34.474339  5351 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 13:03:34.570331  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1972) took 1415 ms. Trace:
I20260709 13:03:34.570904  1570 rpcz_store.cc:276] 0709 13:03:33.155435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.998894 (+843459us) service_pool.cc:224] Handling call
0709 13:03:34.021689 (+ 22795us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:34.569990 (+548301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.570261 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:34.573125  1471 catalog_manager.cc:5697] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 6 to 7, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 7 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:34.547127  5461 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:34.579084  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:34.582444  1471 catalog_manager.cc:5697] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 6 to 7, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 7 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:34.550554  5247 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:03:34.592187  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 151409us
I20260709 13:03:34.604820  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:34.605597  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:34.608423  1687 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.860s (SENT)
I20260709 13:03:34.611039  1687 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:34.616745  1471 catalog_manager.cc:5697] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:34.572008  5467 raft_consensus.cc:697] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:34.614609  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 315515us
I20260709 13:03:34.582515  5192 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:34.599185  5457 raft_consensus.cc:3055] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:34.606545  5458 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.573381  5447 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:34.576041  5450 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:34.588964  5301 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:34.649458  1471 catalog_manager.cc:5697] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 3 to 8, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 8 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:34.639272  5413 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:34.654518  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:34.657421  1570 raft_consensus.cc:2376] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9. Re-sending same reply.
W20260709 13:03:34.649012  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 80951.2us
I20260709 13:03:34.630481  5254 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:34.603132  5306 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:34.668246  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 59857.3us
I20260709 13:03:34.635973  5457 raft_consensus.cc:740] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:34.674679  5254 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:34.614359  5391 consensus_queue.cc:1059] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:03:34.677748  5306 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:34.699162  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:34.650523  5458 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:34.627526  5247 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:34.681000  5475 consensus_queue.cc:1059] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:34.686815  5301 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:34.688378  5443 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:34.691370  5474 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:03:34.694689  5447 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:34.698031  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.733441  1567 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 7: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:34.734582  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.735469  1567 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 7: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:34.694892  5252 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:34.736610  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:34.700624  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1963) took 1854 ms. Trace:
I20260709 13:03:34.738346  5465 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:34.737499  1571 rpcz_store.cc:276] 0709 13:03:32.846463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.836387 (+989924us) service_pool.cc:224] Handling call
0709 13:03:33.836710 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.940349 (+103639us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:34.171226 (+230877us) log.cc:844] Serialized 14 byte log entry
0709 13:03:34.485859 (+314633us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:34.487005 (+  1146us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:34.487078 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.700308 (+213230us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54001b6554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:34.700364 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.700552 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":151975,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":3828,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":38376,"spinlock_wait_cycles":237682432,"thread_start_us":441932,"threads_started":2,"wal-append.queue_time_us":363108}
W20260709 13:03:34.707432  1687 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.022s (SENT)
I20260709 13:03:34.711917  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.704269  5372 consensus_queue.cc:1059] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:03:34.704624  5473 consensus_queue.cc:1059] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:03:34.698734  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.754206  1566 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
W20260709 13:03:34.720846  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 48160.1us
I20260709 13:03:34.749552  5357 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:34.719528  5476 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:03:34.756485  2313 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:34.768126  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:34.761297  1572 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:34.770460  1566 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:34.774494  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.780544  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.781706  1572 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:34.695746  5464 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:34.777182  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 73366.4us
I20260709 13:03:34.696105  5467 consensus_queue.cc:237] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:34.768736  5269 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.802253  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.811712  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.705107  5186 consensus_queue.cc:1059] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.094s
W20260709 13:03:34.802040  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 94649.5us
I20260709 13:03:34.808811  1566 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:34.784901  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.782527  5254 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:34.813199  1572 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 7: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:34.801358  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.776217  5450 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:34.831060  1573 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:03:34.831459  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 75041.7us
I20260709 13:03:34.833230  1567 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:34.798617  5475 raft_consensus.cc:3055] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:03:34.832441  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.837776  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:34.835350  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 72905.1us
I20260709 13:03:34.840692  1570 raft_consensus.cc:2393] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:34.797608  5465 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:34.842043  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.842697  1566 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:34.843604  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.801290  5473 raft_consensus.cc:3055] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:34.846542  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:34.849854  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:34.851220  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.851981  1566 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:34.852048  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.847131  1713 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:34.854152  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.849658  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.855711  1715 pending_rounds.cc:85] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 9
I20260709 13:03:34.855980  1572 raft_consensus.cc:2410] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:34.856094  1715 pending_rounds.cc:107] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:03:34.856663  1715 raft_consensus.cc:2889] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:34.859030  1571 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:34.850904  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.854805  1567 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:34.696656  5457 consensus_queue.cc:260] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:34.871402  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.875456  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.837002  5475 raft_consensus.cc:740] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:34.845391  5465 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.884236  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.830155  5269 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:34.886029  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:34.846421  5473 raft_consensus.cc:740] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:34.887616  1572 raft_consensus.cc:2376] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8. Re-sending same reply.
I20260709 13:03:34.857517  5413 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:34.890091  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.878609  5197 raft_consensus.cc:3055] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:03:34.903066  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.904345  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:34.885233  1566 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:34.902127  2349 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:34.877571  5450 consensus_queue.cc:1059] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.030s
W20260709 13:03:34.917434  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.902743  5197 raft_consensus.cc:740] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:34.923235  5458 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:34.923789  5447 raft_consensus.cc:3055] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:34.924441  5485 consensus_queue.cc:1059] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:34.926937  5372 consensus_queue.cc:1059] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:34.916446  5457 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:34.957667  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45697.2us
I20260709 13:03:34.930131  1564 raft_consensus.cc:1275] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:03:34.907749  5312 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:34.900619  5287 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.886664  5465 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:34.950441  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:34.952111  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 53700.7us
I20260709 13:03:34.927525  5247 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:34.961866  1566 raft_consensus.cc:2393] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:34.904444  5413 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:34.948904  5447 raft_consensus.cc:740] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:34.981062  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.983023  1719 raft_consensus.cc:1275] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:34.994400  1721 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:35.000319  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:35.003237  1569 raft_consensus.cc:1240] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:03:35.001410  1717 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:34.904070  5475 consensus_queue.cc:260] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:34.969702  5443 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:34.943564  5254 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:03:35.027877  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:35.030007  2344 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:35.031080  5483 raft_consensus.cc:3055] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:35.031091  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:35.031644  5483 raft_consensus.cc:740] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:35.031752  5269 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:34.976967  5287 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:35.032662  5483 consensus_queue.cc:260] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:35.033130  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:35.033728  5484 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.120s
I20260709 13:03:35.034855  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:35.035622  1716 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:34.993562  5473 consensus_queue.cc:260] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:35.034180  1722 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:35.033972  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:35.044291  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:35.038902  2340 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
W20260709 13:03:35.029292  5359 consensus_meta.cc:287] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.541s	user 0.011s	sys 0.001s
I20260709 13:03:35.072222  1722 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:35.073990  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:35.075903  1722 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:35.037545  5473 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:35.078781  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.008749  5450 consensus_queue.cc:1048] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
W20260709 13:03:35.079687  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.085429  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:35.079453  2312 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:03:35.093842  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.984825  5486 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:03:35.097713  1722 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:35.103931  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:35.060725  5475 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:35.108031  1713 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:35.112672  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.112941  1719 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:35.118098  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:35.080188  5359 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:35.128134  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:35.130738  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:35.038134  5197 consensus_queue.cc:260] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:35.136401  5197 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:35.137622  5470 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.130s
I20260709 13:03:35.151489  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:35.151850  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:35.151336  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:35.150946  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:35.157269  1647 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:35.130802  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:35.159289  2340 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:35.118435  5476 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:35.147969  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.140264  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:35.170991  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:35.172415  1714 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:35.095765  5461 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:35.176370  1713 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:35.124452  5483 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:35.197387  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:35.199887  1641 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:35.196911  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:35.150345  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:35.201344  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:35.202428  1648 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:35.208654  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:35.209720  1639 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:35.210770  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:35.211812  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:35.212417  1646 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:35.078213  5447 consensus_queue.cc:260] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:35.224648  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:35.225450  1713 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:35.148820  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:35.173509  5476 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.153110  5487 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.233443  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:35.147008  5481 consensus_queue.cc:1059] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.158s
I20260709 13:03:35.151341  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:35.254029  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:35.256990  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:35.267333  1639 raft_consensus.cc:2410] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:35.267445  1719 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:03:35.258064  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:35.242090  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 237404us
I20260709 13:03:35.259255  5312 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:35.271234  1641 raft_consensus.cc:2393] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:35.280299  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 168730us
W20260709 13:03:35.284757  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 134744us
I20260709 13:03:35.238138  5476 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:35.297407  1646 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:35.299012  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:35.313244  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.315557  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:35.315814  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:35.317786  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:35.319195  1713 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:35.324970  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:35.249470  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:35.342456  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:35.343822  1641 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:35.277200  5301 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:35.382234  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:35.384192  1716 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:35.388052  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:35.393348  1641 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
W20260709 13:03:35.413285  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.440850  1641 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:03:35.465692  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:35.469112  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 89522.2us
I20260709 13:03:35.315438  5372 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.138s
I20260709 13:03:35.490248  1716 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:03:35.315557  5487 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:35.344928  5254 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.120s
I20260709 13:03:35.440440  5287 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:35.381165  5301 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:35.518002  5359 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:35.518668  5331 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:35.518954  5359 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:35.519668  5359 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:35.495812  5372 consensus_queue.cc:1243] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer 066ee0109aba4465ba17fa65e14ef504 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:03:35.501319  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:35.523839  1641 raft_consensus.cc:2393] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:35.383773  2342 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:35.528903  1716 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:03:35.517894  5287 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.510423  5500 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.527330  1714 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:35.533864  1646 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:03:35.570415  1644 raft_consensus.cc:2410] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:35.586136  2340 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:35.568897  1648 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:03:35.513535  5301 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:35.602057  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:35.600826  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:35.610724  2344 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.552s	user 0.010s	sys 0.006s
I20260709 13:03:35.620309  1647 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:35.619692  2344 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:03:35.643395  5484 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.644223  5484 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:35.644670  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 214098us
I20260709 13:03:35.567912  5287 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:35.663169  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:35.623916  5461 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:35.637625  5497 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.684633  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:35.657076  5359 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:35.588776  5500 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:35.698740  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:35.696851  5465 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.717s	user 0.012s	sys 0.001s
W20260709 13:03:35.718156  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:35.718883  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.720091  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:35.723673  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:35.707037  5484 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:35.718834  5457 consensus_meta.cc:287] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.746s	user 0.012s	sys 0.004s
I20260709 13:03:35.721235  1641 raft_consensus.cc:2376] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8. Re-sending same reply.
I20260709 13:03:35.737458  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:35.740756  2344 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:35.725270  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:35.714241  5513 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.741837  1716 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:35.699952  5497 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:35.747227  1642 raft_consensus.cc:2410] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:03:35.752214  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:35.753257  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.755558  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:35.757066  5487 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:35.757858  5508 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 13:03:35.758613  5501 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.167s	user 0.001s	sys 0.001s
I20260709 13:03:35.723174  5461 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.761962  5461 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:35.771232  2349 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: [9.8-11.9]
I20260709 13:03:35.772146  5500 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:35.773218  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 573) took 1068 ms. Trace:
I20260709 13:03:35.773697  1567 rpcz_store.cc:276] 0709 13:03:34.704639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.876759 (+172120us) service_pool.cc:224] Handling call
0709 13:03:34.880998 (+  4239us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:34.881261 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:34.881288 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:34.888961 (+  7673us) log.cc:844] Serialized 26 byte log entry
0709 13:03:35.284314 (+395353us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:35.284350 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:35.284451 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:35.772933 (+488482us) raft_consensus.cc:1685] finished
0709 13:03:35.772971 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:35.773134 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":343324,"threads_started":1,"wal-append.queue_time_us":564991}
I20260709 13:03:35.810459  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:35.813848  1641 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:35.821200  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:35.848146  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:35.849146  1646 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
W20260709 13:03:35.773677  5510 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:03:35.884258  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.886036   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:35.917236  5519 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.918040  5519 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:35.926204  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:35.926420  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:35.927076  5497 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:35.927814  1647 raft_consensus.cc:2410] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:03:35.944866  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:35.927075  1716 raft_consensus.cc:2393] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 9: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:35.946202  1640 raft_consensus.cc:2410] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:03:35.755473  5513 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:35.721980  5465 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:35.884063  5483 consensus_meta.cc:287] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.662s	user 0.011s	sys 0.004s
I20260709 13:03:35.929198  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:35.947788  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:36.001596  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:35.981002  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.986096  1643 raft_consensus.cc:2410] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:03:35.990896  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:36.006165  1641 raft_consensus.cc:2393] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:35.991935  1714 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:35.967052  2341 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 8: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:35.996858  1640 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:03:36.033926  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:36.027572  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1955) took 1028 ms. Trace:
I20260709 13:03:36.037871  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:35.925616  5475 consensus_meta.cc:287] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.812s	user 0.012s	sys 0.003s
I20260709 13:03:35.813877  5500 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:03:35.975800  5473 consensus_meta.cc:287] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.870s	user 0.011s	sys 0.004s
W20260709 13:03:35.980134  5476 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.655s	user 0.012s	sys 0.000s
I20260709 13:03:36.055003  1716 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:36.058555  1612 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:36.059193  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:35.964532  5513 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:35.973345  5287 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:36.068660  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:36.070438  1640 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:36.060678  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:36.060668  1639 raft_consensus.cc:2410] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:36.060935  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:36.075189  1647 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:35.955614  5461 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:35.955039  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:36.065358  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:36.066280  1612 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:36.075873  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:36.036443  1718 rpcz_store.cc:276] 0709 13:03:34.998927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.999119 (+   192us) service_pool.cc:224] Handling call
0709 13:03:35.022567 (+ 23448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:35.060608 (+ 38041us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:35.279301 (+218693us) log.cc:844] Serialized 14 byte log entry
0709 13:03:36.026709 (+747408us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:36.026778 (+    69us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:36.027229 (+   451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.027321 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:36.027493 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10877,"prepare.queue_time_us":425532,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":41,"raft.queue_time_us":113177,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":3379,"spinlock_wait_cycles":7936,"thread_start_us":853409,"threads_started":3,"wal-append.queue_time_us":371671}
I20260709 13:03:36.082114  1612 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:36.077461  5254 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:36.057891  5186 raft_consensus.cc:3055] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:03:36.091590  1687 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.034s	sys 0.075s
I20260709 13:03:36.092029  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:36.091710  1719 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
W20260709 13:03:36.092958  1687 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.035s	sys 0.075s
I20260709 13:03:36.078642  5475 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:36.098189  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.097599  1612 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:36.090876  5473 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:36.063637  5483 consensus_queue.cc:1048] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:03:36.063431  5497 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
W20260709 13:03:36.103390  1537 connection.cc:256] RPC call timeout handler was delayed by -1.9268927919999896s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1603s for a response
W20260709 13:03:36.093192  5306 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.012s	sys 0.025s
W20260709 13:03:36.108556  1537 connection.cc:256] RPC call timeout handler was delayed by -1.82159279199999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17200000000000001s for a response
I20260709 13:03:36.101614  1612 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:36.109212  1537 connection.cc:256] RPC call timeout handler was delayed by -1.5074927919999936s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2069s for a response
W20260709 13:03:36.109607  1537 connection.cc:256] RPC call timeout handler was delayed by -1.4354927919999909s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2149s for a response
I20260709 13:03:36.109912  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:36.112565  1645 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:36.111449  1612 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:36.088944  5465 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:36.092141  5186 raft_consensus.cc:740] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:36.096702  5461 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:36.120718  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:36.101619  5475 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.127585  5475 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:36.100385  5513 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:36.130149  5513 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:36.130430  5186 consensus_queue.cc:260] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:36.132491  5331 consensus_meta.cc:287] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.613s	user 0.013s	sys 0.003s
W20260709 13:03:36.115082  5306 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.012s	sys 0.025s
I20260709 13:03:36.098507  5472 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:36.071604  5476 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:36.088202  5254 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:36.103140  5484 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:36.121999  5461 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.134959  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1963) took 1106 ms. Trace:
I20260709 13:03:36.151595  1713 rpcz_store.cc:276] 0709 13:03:35.028700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.322789 (+294089us) service_pool.cc:224] Handling call
0709 13:03:35.323261 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:35.346986 (+ 23725us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:35.347886 (+   900us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:35.396782 (+ 48896us) log.cc:844] Serialized 26 byte log entry
0709 13:03:35.631575 (+234793us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:35.631636 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:35.631741 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:36.134699 (+502958us) raft_consensus.cc:1685] finished
0709 13:03:36.134726 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:36.134887 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7029,"raft.queue_time_us":23027,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":28489,"thread_start_us":135658,"threads_started":1,"wal-append.queue_time_us":591941}
I20260709 13:03:36.116089  2346 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:36.156607  1565 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:36.080250  5500 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:36.116539  5372 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:36.118249  5486 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:36.171155  5413 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:36.173383  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 572) took 1821 ms. Trace:
I20260709 13:03:36.174008  1569 rpcz_store.cc:276] 0709 13:03:34.351484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.362006 (+ 10522us) service_pool.cc:224] Handling call
0709 13:03:34.382727 (+ 20721us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:35.003803 (+621076us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.172950 (+1169147us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b5400013e54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:36.173256 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1426480000}
I20260709 13:03:36.178406  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:36.180377  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:36.181412  1639 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:36.197100  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:36.160843  5312 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:36.198693  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 104080us
I20260709 13:03:36.160328  5461 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:36.207135  1640 raft_consensus.cc:2410] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:03:36.208295  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 92496.2us
I20260709 13:03:36.212603  1612 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:36.160502  5512 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:36.216542  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:36.218336  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.192093  5484 raft_consensus.cc:697] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:36.221525  1639 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:03:36.228736  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:36.228933  5473 raft_consensus.cc:697] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:36.241886  5525 consensus_queue.cc:1059] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:36.242488  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2446) took 1083 ms. Trace:
I20260709 13:03:36.242756  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:36.243040  5413 raft_consensus.cc:3055] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:36.243642  5413 raft_consensus.cc:740] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:36.244724  5484 consensus_queue.cc:237] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:36.244839  5413 consensus_queue.cc:260] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:36.246131  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.248339  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:36.249612  1639 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:36.243072  1642 rpcz_store.cc:276] 0709 13:03:35.159425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.159559 (+   134us) service_pool.cc:224] Handling call
0709 13:03:36.242241 (+1082682us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b54002add54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:36.242383 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1255082752}
I20260709 13:03:36.225772  5486 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:36.288477  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:36.292949  1571 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 7: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:36.213531  5312 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:36.292726  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:36.295583  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.222831  5512 raft_consensus.cc:697] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:36.295339  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:36.335871  1572 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:36.247006  5413 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:36.302255  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:36.304266  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:36.337918  1570 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:36.308382  5476 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:36.225414  5497 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 13:03:36.310204  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:36.239675  5500 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:36.342496  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:36.342548  1564 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:36.290045  5471 consensus_queue.cc:1059] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:03:36.241395  5254 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:36.353816  5486 raft_consensus.cc:697] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:36.354557  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:36.354421  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:36.355247  1642 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
W20260709 13:03:36.355458  1565 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 1940) took 1841 ms (client timeout 1659 ms). Trace:
I20260709 13:03:36.356144  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:36.311188  5498 raft_consensus.cc:3055] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:36.288194  5513 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:36.312543  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:36.326604  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.356015  1640 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:03:36.361683  1567 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:36.360596  5498 raft_consensus.cc:740] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:36.355968  5486 consensus_queue.cc:237] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:36.364228  5498 consensus_queue.cc:260] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:36.340071  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:36.311590  1537 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:36.358726  1685 connection.cc:256] RPC call timeout handler was delayed by -1.1908880120000163s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2223s for a response
I20260709 13:03:36.343361  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:36.367805  1569 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:36.311554  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:36.369055  1573 raft_consensus.cc:2393] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:03:36.355829  1565 rpcz_store.cc:269] 0709 13:03:34.546533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.560520 (+ 13987us) service_pool.cc:224] Handling call
0709 13:03:36.355267 (+1794747us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b5400013954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:36.355394 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3568320256}
I20260709 13:03:36.308760  5461 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:36.363389  5252 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:36.371476  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:36.385555  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:36.374199  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:36.309937  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 62490.2us
W20260709 13:03:36.368677  1685 connection.cc:256] RPC call timeout handler was delayed by -1.1161880120000092s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23060000000000003s for a response
I20260709 13:03:36.335376  5372 raft_consensus.cc:697] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:36.386994  1565 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:36.387825  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 175056us
I20260709 13:03:36.389000  1571 raft_consensus.cc:2393] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:36.390223  1568 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:36.382176  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:36.393433  1569 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:36.393942  5252 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:36.404726  5252 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:36.405701  5513 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:36.413703  1565 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
W20260709 13:03:36.414780  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:36.391515  1685 leader_election.cc:336] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.100s (SENT)
I20260709 13:03:36.427105  1685 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:36.417832  1640 raft_consensus.cc:1240] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:36.418478  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:36.386149  1570 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:36.424597  1564 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:36.425100  1612 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:36.458519  1612 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:36.462345  1564 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:03:36.464715  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:36.465464  1573 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:36.468747  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:36.357275  5512 consensus_queue.cc:237] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:36.469776  5197 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:36.470328  5471 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:36.470772  5471 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:36.471174  5471 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:36.471572  5534 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:36.472541  5534 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.475378  5519 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:36.480926  1641 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:03:36.396096  5481 raft_consensus.cc:3055] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:36.465237  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:36.470273  5197 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.492036  1720 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:03:36.493383  2314 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:36.498570  2314 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:36.489516  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1964) took 1460 ms. Trace:
W20260709 13:03:36.496031  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 184597us
I20260709 13:03:36.500170  2314 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:36.473163  5534 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:36.519960  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:36.522162  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 130791us
I20260709 13:03:36.522799  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:36.513494  1569 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:36.518484  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:36.524188  1572 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:36.524706  1570 raft_consensus.cc:2410] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:36.518245  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:36.522933  1685 connection.cc:256] RPC call timeout handler was delayed by -1.0648880119999831s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2363s for a response
I20260709 13:03:36.525800  1721 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
W20260709 13:03:36.526304  1685 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 3.015s (SENT)
I20260709 13:03:36.526893  1685 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:36.302294  5473 consensus_queue.cc:237] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:36.458199  5536 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:36.532140  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.524706  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:36.537446  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:36.506675  5197 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:36.543608  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 123213us
I20260709 13:03:36.544252  1641 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:03:36.555408  1640 raft_consensus.cc:1240] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:36.547886  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:36.555325  1687 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:36.556571  1566 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:36.548974  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:36.552668  1564 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:36.491016  5481 raft_consensus.cc:740] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:36.533958  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:36.552984  1572 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:03:36.521735  5447 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:36.516052  5472 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:36.563388  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.509909  5353 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:36.558202  1567 raft_consensus.cc:2376] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9. Re-sending same reply.
I20260709 13:03:36.516598  5254 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:36.516216  5465 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:03:36.577263  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 51013.1us
I20260709 13:03:36.577280  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:36.602450  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:36.603976  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:36.608935  1641 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: [8.4-10.5]
I20260709 13:03:36.611728  1640 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:36.611958  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 55034.3us
I20260709 13:03:36.613826  1640 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:36.588852  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:36.620998  1572 raft_consensus.cc:2393] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 9: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:03:36.620639  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 65558us
I20260709 13:03:36.628580  1722 raft_consensus.cc:2410] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:36.640168  1572 pending_rounds.cc:85] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 3
I20260709 13:03:36.640525  1572 pending_rounds.cc:107] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:03:36.640760  1572 raft_consensus.cc:2889] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:36.589087  5465 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:36.645750  1647 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:36.623708  5331 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.655105  1647 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:36.560314  5536 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:36.624269  5524 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:36.623950  5525 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:36.603143  5472 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:36.645005  5252 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:36.682144  1564 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:36.579751  5372 consensus_queue.cc:237] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:36.682405  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 69539.8us
I20260709 13:03:36.653587  5465 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:36.684317  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.684720  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:36.681231  1647 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:36.688446  1640 raft_consensus.cc:1275] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:03:36.695501  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:36.704617  1639 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:36.743682  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2469) took 1008 ms. Trace:
I20260709 13:03:36.749925  1569 raft_consensus.cc:2410] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:03:36.750571  1566 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:36.744285  1639 rpcz_store.cc:276] 0709 13:03:35.735014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.225548 (+490534us) service_pool.cc:224] Handling call
0709 13:03:36.743590 (+518042us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:36.704829  1564 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:36.770152  1639 raft_consensus.cc:1240] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:03:36.771648  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:36.784715  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 365) took 1010 ms. Trace:
I20260709 13:03:36.757161  1573 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:36.694566  5391 raft_consensus.cc:3055] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:36.785919  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:36.764830  1612 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:36.788556  1566 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:36.785434  1639 rpcz_store.cc:276] 0709 13:03:35.774352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.769331 (+994979us) service_pool.cc:224] Handling call
0709 13:03:36.769906 (+   575us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.778483 (+  8577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.784638 (+  6155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:36.662550  5331 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:36.792387  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 359) took 1020 ms. Trace:
I20260709 13:03:36.688395  5357 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:36.687399  1720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:03:36.702512  5252 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:36.649485  5481 consensus_queue.cc:260] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:36.793282  1647 rpcz_store.cc:276] 0709 13:03:35.771436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.768385 (+996949us) service_pool.cc:224] Handling call
0709 13:03:36.768744 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:36.784957 (+ 16213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.785030 (+    73us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:36.785156 (+   126us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:36.791234 (+  6078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.791458 (+   224us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:36.792043 (+   585us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2091,"raft.queue_time_us":31279,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":6392}
I20260709 13:03:36.970213  1569 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:36.998540  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:37.013860  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 213220us
W20260709 13:03:36.798154  1687 leader_election.cc:336] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.687s (SENT)
I20260709 13:03:36.760856  5527 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:37.035283  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:37.036119  1717 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:03:37.017143  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 381746us
I20260709 13:03:37.015493  1720 raft_consensus.cc:606] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:37.045351  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 247142us
W20260709 13:03:37.048411  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:37.055939  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.042295  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:37.077167  1564 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:37.086686  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:37.088137  1573 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:37.079103  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:37.109222  1564 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:36.784451  5461 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:37.113107  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2441) took 1963 ms (client timeout 2306 ms). Trace:
I20260709 13:03:37.114204  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:36.791661  5391 raft_consensus.cc:740] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:36.832429  5483 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:37.129925  5483 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.130460  5483 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:37.131680  5543 log.cc:927] Time spent T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
W20260709 13:03:37.113579  1643 rpcz_store.cc:269] 0709 13:03:35.149526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.150692 (+  1166us) service_pool.cc:224] Handling call
0709 13:03:37.112859 (+1962167us) spinlock_profiling.cc:243] Waited 827 ms on lock 0x7b5400183154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:37.113040 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1901799168}
W20260709 13:03:37.119472  1687 leader_election.cc:336] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.781s (SENT)
W20260709 13:03:37.134763  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 63741.1us
W20260709 13:03:37.135488  1687 leader_election.cc:336] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.820s (SENT)
I20260709 13:03:37.120745  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:37.126197  5531 log.cc:927] Time spent T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.345s	user 0.002s	sys 0.000s
I20260709 13:03:37.126479  5532 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:37.129141  5548 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:37.133836  5269 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:03:37.042250  5525 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:03:37.198756  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 163390us
I20260709 13:03:37.042997  5357 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:37.207520  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:37.031175  5538 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.128s
I20260709 13:03:37.046083  5527 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.130138  5488 raft_consensus.cc:3055] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:03:37.250833  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 213420us
W20260709 13:03:37.263721  1572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.000s	sys 0.008s
W20260709 13:03:37.264459  1572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.000s	sys 0.008s
W20260709 13:03:37.266978  1538 connection.cc:256] RPC call timeout handler was delayed by -1.2007585790000235s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21389999999999998s for a response
W20260709 13:03:37.267726  1538 connection.cc:256] RPC call timeout handler was delayed by -1.1428277609999782s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2818s for a response
I20260709 13:03:37.175643  5532 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:37.235864  5548 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:37.134146  5544 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 13:03:37.247990  1713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.003s	sys 0.003s
W20260709 13:03:37.316615  1713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.004s	sys 0.004s
I20260709 13:03:37.270766  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:37.319473  1568 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:03:37.268201  1538 connection.cc:256] RPC call timeout handler was delayed by -1.1231350419999444s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15710000000000002s for a response
W20260709 13:03:37.320392  1538 connection.cc:256] RPC call timeout handler was delayed by -1.0478522229999498s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20379999999999998s for a response
W20260709 13:03:37.320786  1538 connection.cc:256] RPC call timeout handler was delayed by -1.0240277610000135s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29500000000000004s for a response
W20260709 13:03:37.308319  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 62325.8us
W20260709 13:03:37.321491  1641 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.005s	sys 0.009s
W20260709 13:03:37.323014  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:37.229576  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 197622us
I20260709 13:03:37.203375  5417 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.099s
I20260709 13:03:37.325237  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:37.327456  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:37.358026  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 221943us
W20260709 13:03:37.271606  5484 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.000s	sys 0.008s
I20260709 13:03:37.269379  5513 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:37.359076  1687 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.876s (SENT)
I20260709 13:03:37.299031  1564 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
W20260709 13:03:37.369136  5484 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.002s	sys 0.012s
I20260709 13:03:37.249787  5527 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:37.323375  1641 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.005s	sys 0.009s
W20260709 13:03:37.368990  1687 leader_election.cc:336] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.136s (SENT)
I20260709 13:03:37.248975  5488 raft_consensus.cc:740] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:37.254629  5552 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:37.350018  5532 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:37.374454  1538 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:37.395188  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.265147  5525 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:37.416793  1538 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:37.382431  5549 log.cc:927] Time spent T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.552s	user 0.002s	sys 0.000s
W20260709 13:03:37.418848  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 55196.6us
I20260709 13:03:37.420063  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:37.366449  5472 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:37.433346  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 103231us
I20260709 13:03:37.452021  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:37.453363  1720 raft_consensus.cc:2410] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 13 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
W20260709 13:03:37.427211  5537 log.cc:927] Time spent T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.472s	user 0.001s	sys 0.001s
I20260709 13:03:37.315690  5391 consensus_queue.cc:260] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:37.459779  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 90950.5us
W20260709 13:03:37.460558  1614 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.008s	sys 0.033s
W20260709 13:03:37.465044  1614 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.008s	sys 0.033s
W20260709 13:03:37.461510  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.381127  5513 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:37.486316  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:37.496587  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.506537  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:37.407696  5552 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:37.527988  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 80089.4us
I20260709 13:03:37.472533  5570 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:37.538674  2312 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:37.496295  5550 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.509050  5567 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.525043  5391 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:37.545627  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:37.553392  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 224399us
W20260709 13:03:37.532734  5413 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.188s	user 0.006s	sys 0.007s
I20260709 13:03:37.522449  5331 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:37.529883  5565 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:37.610325  5560 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.352s	user 0.002s	sys 0.000s
I20260709 13:03:37.611083  5579 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:37.611929  5572 consensus_queue.cc:1059] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
W20260709 13:03:37.613425  5556 log.cc:927] Time spent T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.381s	user 0.000s	sys 0.002s
W20260709 13:03:37.614071  5562 log.cc:927] Time spent T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 13:03:37.547451  5551 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.616513  1647 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:37.616864  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1995) took 1089 ms. Trace:
I20260709 13:03:37.646098  1570 rpcz_store.cc:276] 0709 13:03:36.527682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.528007 (+   325us) service_pool.cc:224] Handling call
0709 13:03:36.533551 (+  5544us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.534862 (+  1311us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:36.653097 (+118235us) log.cc:844] Serialized 14 byte log entry
0709 13:03:36.729201 (+ 76104us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.729899 (+   698us) log.cc:844] Serialized 26 byte log entry
0709 13:03:36.730066 (+   167us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:36.730096 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.730177 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:37.616600 (+886423us) raft_consensus.cc:1685] finished
0709 13:03:37.616622 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:37.616777 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":234917,"raft.queue_time_us":20290,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":31866,"spinlock_wait_cycles":31317888,"thread_start_us":140084,"threads_started":2,"wal-append.queue_time_us":354529}
I20260709 13:03:37.630641  1717 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:37.650935  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:37.542981  5546 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:37.651965  1717 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:37.653347  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 236622us
W20260709 13:03:37.655762  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 416431us
W20260709 13:03:37.659495  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.599828  5252 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:37.616604  5331 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:37.651093  5353 consensus_meta.cc:287] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 1.046s	user 0.016s	sys 0.000s
I20260709 13:03:37.736860  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:37.738431  1721 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:37.739276  1612 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:39935 timed out after 2.477s (SENT) [suppressed 61 similar messages]
I20260709 13:03:37.569132  5550 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:37.697672  5497 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.802s	user 0.000s	sys 0.011s
I20260709 13:03:37.616953  5413 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.117s
I20260709 13:03:37.732311  5331 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:37.743074  5570 raft_consensus.cc:697] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:37.749586  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:37.592509  5567 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:37.656385  5551 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:37.688194  5565 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:37.713399  5546 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:36.499682  1715 rpcz_store.cc:276] 0709 13:03:35.028940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.340164 (+311224us) service_pool.cc:224] Handling call
0709 13:03:35.439831 (+ 99667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:35.440078 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:36.247191 (+807113us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b18003322c0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f824c87d3c7 00007f8248e8f9a7 00007f8248e9f0bf
0709 13:03:36.247354 (+   163us) log.cc:844] Serialized 14 byte log entry
0709 13:03:36.489166 (+241812us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:36.489232 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:36.489278 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.489336 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:36.489458 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":147129,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":66400,"spinlock_wait_cycles":30121216,"thread_start_us":336299,"threads_started":2}
W20260709 13:03:38.042640  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:37.891607  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 358739us
I20260709 13:03:37.755426  1538 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:38.059015  1717 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:38.024044  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1998) took 1394 ms. Trace:
W20260709 13:03:37.922946  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.061158  1612 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:38.060209  1572 rpcz_store.cc:276] 0709 13:03:36.630053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.639677 (+  9624us) service_pool.cc:224] Handling call
0709 13:03:36.639970 (+   293us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:37.264909 (+624939us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:37.264944 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:37.265594 (+   650us) log.cc:844] Serialized 52 byte log entry
0709 13:03:37.567830 (+302236us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:37.568652 (+   822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.569160 (+   508us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:38.023718 (+454558us) raft_consensus.cc:1685] finished
0709 13:03:38.023750 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.023959 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":53174,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":46930,"thread_start_us":813212,"threads_started":2,"wal-append.queue_time_us":687892}
I20260709 13:03:37.789886  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:37.861969  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:38.062538  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:37.932893  5197 consensus_meta.cc:287] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.357s	user 0.006s	sys 0.009s
I20260709 13:03:38.070526  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1937) took 2320 ms. Trace:
I20260709 13:03:38.074473  1720 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:37.754344  1648 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:03:37.753085  5497 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.001s	sys 0.011s
W20260709 13:03:38.091894  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:37.937022  5447 consensus_meta.cc:287] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.318s	user 0.016s	sys 0.000s
W20260709 13:03:37.816340  5475 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.652s	user 0.016s	sys 0.007s
W20260709 13:03:38.080829  5532 consensus_meta.cc:287] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.663s	user 0.011s	sys 0.002s
I20260709 13:03:38.130641  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2335) took 2623 ms. Trace:
I20260709 13:03:37.799633  5550 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:38.100761  1645 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2444) took 2950 ms (client timeout 1582 ms). Trace:
I20260709 13:03:38.107295  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 579) took 1077 ms. Trace:
I20260709 13:03:37.789265  5585 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.357s
I20260709 13:03:37.751528  5548 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:37.895916  5577 log.cc:927] Time spent T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 13:03:38.093042  5553 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.102121  5603 consensus_queue.cc:1059] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.048s
W20260709 13:03:37.893682  5465 consensus_meta.cc:287] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 1.118s	user 0.008s	sys 0.009s
W20260709 13:03:38.100451  1747 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:38281 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:38281 timed out after 15.000s (SENT)
I20260709 13:03:37.762535  5353 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:03:38.090955  5524 consensus_meta.cc:287] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.403s	user 0.009s	sys 0.006s
I20260709 13:03:37.938921  5481 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:38.203150  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:38.163587  5579 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:37.789397  5554 log.cc:927] Time spent T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.391s	user 0.000s	sys 0.002s
I20260709 13:03:38.155540  1714 rpcz_store.cc:276] 0709 13:03:35.494921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.585175 (+ 90254us) service_pool.cc:224] Handling call
0709 13:03:35.991555 (+406380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:36.007985 (+ 16430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.118087 (+2110102us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b5400209c50. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:38.118308 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":933511040}
I20260709 13:03:38.157220  1640 rpcz_store.cc:276] 0709 13:03:35.749846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.615735 (+865889us) service_pool.cc:224] Handling call
0709 13:03:36.616137 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:36.734757 (+118620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.940123 (+1205366us) spinlock_profiling.cc:243] Waited 71.9 ms on lock 0x7b54000a73d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:38.070403 (+130280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165255936}
I20260709 13:03:38.158507  1646 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:38.065232  5331 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:38.164343  5532 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:38.219450  5532 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:37.933939  5584 log.cc:927] Time spent T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.080s	user 0.000s	sys 0.002s
I20260709 13:03:38.121184  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:38.103369  5534 consensus_meta.cc:287] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.561s	user 0.012s	sys 0.006s
W20260709 13:03:38.131340  5575 log.cc:927] Time spent T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.442s	user 0.002s	sys 0.000s
I20260709 13:03:38.098794  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 1994) took 1541 ms. Trace:
I20260709 13:03:38.130311  5578 consensus_queue.cc:1059] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.096s
I20260709 13:03:37.984428  5312 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:38.080466  5522 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:38.249961  2346 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:03:38.250587  5522 raft_consensus.cc:697] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:38.251461  1646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1938) took 2501 ms (client timeout 1931 ms). Trace:
W20260709 13:03:38.252194  1646 rpcz_store.cc:269] 0709 13:03:35.750082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.891055 (+140973us) service_pool.cc:224] Handling call
0709 13:03:38.251170 (+2360115us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b54001fefd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:38.251369 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4809200000}
I20260709 13:03:38.252332  5522 consensus_queue.cc:237] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:38.129613  5471 consensus_meta.cc:287] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 1.658s	user 0.007s	sys 0.011s
W20260709 13:03:38.156123  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 68331.8us
I20260709 13:03:38.163580  1571 rpcz_store.cc:276] 0709 13:03:37.029789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.106138 (+1076349us) service_pool.cc:224] Handling call
0709 13:03:38.106552 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.106816 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:38.106855 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:38.106916 (+    61us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:38.106973 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.107045 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.107209 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:38.060340  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:38.256979  1717 raft_consensus.cc:2393] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:03:38.161980  1645 rpcz_store.cc:269] 0709 13:03:35.150450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.151079 (+   629us) service_pool.cc:224] Handling call
0709 13:03:38.100508 (+2949429us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b54001f3454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:38.100657 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2115209088}
W20260709 13:03:38.257889  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.076748  5552 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:37.998775  5513 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:38.105059  5591 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:38.092671  5596 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:38.171850  1721 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:38.097846  5443 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.934582  5525 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:37.801932  2341 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 374) took 2654 ms (client timeout 2477 ms). Trace:
I20260709 13:03:38.202077  5524 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:38.197175  5465 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:38.071928  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 316813us
W20260709 13:03:37.986171  5483 consensus_meta.cc:287] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.855s	user 0.008s	sys 0.005s
I20260709 13:03:38.207701  1720 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
W20260709 13:03:37.789000  5568 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.160s	user 0.000s	sys 0.002s
I20260709 13:03:38.226490  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:38.254995  5546 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:38.231468  1722 rpcz_store.cc:276] 0709 13:03:36.557698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.558219 (+   521us) service_pool.cc:224] Handling call
0709 13:03:38.098521 (+1540302us) spinlock_profiling.cc:243] Waited 38.5 ms on lock 0x7b5400213754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:38.098717 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88477568}
I20260709 13:03:38.058647  5536 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:38.150329  5475 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:38.206157  5527 consensus_meta.cc:287] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.799s	user 0.010s	sys 0.004s
I20260709 13:03:38.236891  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:38.238016  5598 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:03:38.191236  5586 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:38.196642  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:38.248116  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:38.159921  5197 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:38.296878  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:38.298513  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:38.298714  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:38.299175  5465 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:38.299477  1722 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:38.287729  5483 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:38.300140  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:38.300561  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:38.301215  1722 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:38.263286  1646 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:03:38.221115  5461 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.254954  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:38.271250  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.224916  5587 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.232990  5413 raft_consensus.cc:3055] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:38.275291  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:38.309446  1721 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:38.310236  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2338) took 1274 ms. Trace:
I20260709 13:03:38.283951  1714 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:38.279276  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:38.311105  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2342) took 1196 ms. Trace:
I20260709 13:03:38.311564  1714 rpcz_store.cc:276] 0709 13:03:37.115147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.226083 (+1110936us) service_pool.cc:224] Handling call
0709 13:03:38.311044 (+ 84961us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:38.311935  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:38.312170  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:38.312564  1642 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:38.312490  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:38.229686  1715 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:38.222530  5579 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:38.319115  1611 connection.cc:256] RPC call timeout handler was delayed by -1.5672086370000216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1511s for a response
W20260709 13:03:38.319726  1611 connection.cc:256] RPC call timeout handler was delayed by -1.4637086370000247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
W20260709 13:03:38.320106  1611 connection.cc:256] RPC call timeout handler was delayed by -1.4034086369999841s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1693s for a response
I20260709 13:03:38.320214  1685 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:38.320547  1611 connection.cc:256] RPC call timeout handler was delayed by -1.338608636999993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1765s for a response
W20260709 13:03:38.320858  1611 connection.cc:256] RPC call timeout handler was delayed by -1.2675086369999917s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1844s for a response
W20260709 13:03:38.321123  1611 connection.cc:256] RPC call timeout handler was delayed by -1.0821086369999762s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.205s for a response
W20260709 13:03:38.300400  5588 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:03:38.150188  5570 consensus_queue.cc:237] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:38.269541  1713 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.018s	sys 0.055s
I20260709 13:03:38.301594  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:03:38.323010  1713 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.019s	sys 0.056s
I20260709 13:03:38.323428  2347 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 8: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:38.323807  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1991) took 1828 ms. Trace:
I20260709 13:03:38.322582  1611 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:38.263478  5471 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:38.300884  5586 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:38.306645  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1990) took 1814 ms. Trace:
I20260709 13:03:38.302326  5565 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:38.286825  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:38.295140  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:38.310911  1721 rpcz_store.cc:276] 0709 13:03:37.035992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.274847 (+1238855us) service_pool.cc:224] Handling call
0709 13:03:38.310151 (+ 35304us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:38.335652  1720 raft_consensus.cc:2410] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:03:38.336263  5546 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:38.312952  1717 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:38.334087  5594 consensus_queue.cc:1059] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:03:38.336862  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:38.337620  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:38.338232  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:38.338996  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:38.339299  5488 consensus_queue.cc:260] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:38.339965  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:38.340278  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:38.340468  5488 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:38.341189  1717 raft_consensus.cc:2393] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:38.314111  1714 raft_consensus.cc:2410] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:38.328956  5525 consensus_queue.cc:1059] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:03:38.343092  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:38.315696  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:38.246040  5534 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:38.324132  1713 rpcz_store.cc:276] 0709 13:03:36.495493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.495717 (+   224us) service_pool.cc:224] Handling call
0709 13:03:36.534366 (+ 38649us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:36.567476 (+ 33110us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.567525 (+    49us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:36.579035 (+ 11510us) log.cc:844] Serialized 78 byte log entry
0709 13:03:37.317142 (+738107us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:37.581599 (+264457us) log.cc:844] Serialized 14 byte log entry
0709 13:03:38.323445 (+741846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.323526 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:38.323567 (+    41us) raft_consensus.cc:1685] finished
0709 13:03:38.323603 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.323745 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36255,"prepare.queue_time_us":297827,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"raft.queue_time_us":573676,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":17081,"thread_start_us":1626076,"threads_started":3,"wal-append.queue_time_us":751843}
I20260709 13:03:38.200572  5553 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:38.325577  5595 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.318981  5465 consensus_queue.cc:1059] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:03:38.326495  5550 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:38.276440  5591 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:38.278360  5443 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:38.357438  5578 raft_consensus.cc:3055] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:38.373171  5578 raft_consensus.cc:740] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:38.310369  5461 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:38.374704  5578 consensus_queue.cc:260] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:38.330909  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:38.379294  5524 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:03:38.380257  5391 consensus_meta.cc:287] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.797s	user 0.011s	sys 0.000s
I20260709 13:03:38.332391  1719 rpcz_store.cc:276] 0709 13:03:36.492482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.492658 (+   176us) service_pool.cc:224] Handling call
0709 13:03:36.528733 (+ 36075us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.557751 (+ 29018us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.557788 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.577708 (+ 19920us) log.cc:844] Serialized 26 byte log entry
0709 13:03:36.671508 (+ 93800us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:36.671582 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.671686 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:38.306388 (+1634702us) raft_consensus.cc:1685] finished
0709 13:03:38.306410 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.306577 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7813,"raft.queue_time_us":48762,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":30145,"thread_start_us":28067,"threads_started":1,"wal-append.queue_time_us":277823}
I20260709 13:03:38.382303  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:38.383680  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 578) took 1354 ms. Trace:
I20260709 13:03:38.279194  5596 raft_consensus.cc:697] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:38.356390  5550 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:38.386475  5596 consensus_queue.cc:237] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:38.388100  5550 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:38.389098  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 379) took 1064 ms. Trace:
I20260709 13:03:38.389374  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:38.395308  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1930) took 2660 ms. Trace:
I20260709 13:03:38.384317  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:38.346554  2349 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:38.318068  5413 raft_consensus.cc:740] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:38.345233  1538 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:38.270761  2341 rpcz_store.cc:269] 0709 13:03:35.147982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.148400 (+   418us) service_pool.cc:224] Handling call
0709 13:03:37.801556 (+2653156us) spinlock_profiling.cc:243] Waited 736 ms on lock 0x7b5400800054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:37.801769 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1691864448}
I20260709 13:03:38.325928  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:38.353832  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:38.354013  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:38.359084  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:38.361411  2340 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: []
W20260709 13:03:38.325371  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 50094us
I20260709 13:03:38.326382  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:38.295797  5527 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:38.341492  2342 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:38.357287  5594 raft_consensus.cc:3055] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:38.319229  5587 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:38.336879  5546 raft_consensus.cc:697] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:38.399377  1720 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:38.325213  1611 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:38.332821  5551 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:38.623392  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:38.384385  1569 rpcz_store.cc:276] 0709 13:03:37.029427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.382232 (+1352805us) service_pool.cc:224] Handling call
0709 13:03:38.382900 (+   668us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.383154 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:38.383193 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:38.383252 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:38.383302 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.383397 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.383585 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:38.533622  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:38.608021  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 113218us
I20260709 13:03:38.503875  2348 raft_consensus.cc:2410] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:03:38.396555  1641 rpcz_store.cc:276] 0709 13:03:35.735493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.610411 (+874918us) service_pool.cc:224] Handling call
0709 13:03:36.610939 (+   528us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:37.376735 (+765796us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:37.376785 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:37.386614 (+  9829us) log.cc:844] Serialized 26 byte log entry
0709 13:03:37.593758 (+207144us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:37.596720 (+  2962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.596855 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:38.394982 (+798127us) raft_consensus.cc:1685] finished
0709 13:03:38.395011 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.395224 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8184,"raft.queue_time_us":73322,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":15273,"thread_start_us":911028,"threads_started":2,"wal-append.queue_time_us":637670}
I20260709 13:03:38.545019  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1992) took 2047 ms. Trace:
I20260709 13:03:38.629546  1645 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:38.629101  1716 rpcz_store.cc:276] 0709 13:03:36.498007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.498143 (+   136us) service_pool.cc:224] Handling call
0709 13:03:36.498512 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:36.498755 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:36.533904 (+ 35149us) log.cc:844] Serialized 14 byte log entry
0709 13:03:37.402898 (+868994us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b180042ebc0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a2523a 00007f8248e8fa2a
0709 13:03:38.544079 (+1141181us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b180042ebc0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287
0709 13:03:38.544362 (+   283us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:38.544452 (+    90us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:38.544546 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.544696 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.544915 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35148,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":1351,"raft.queue_time_us":547745,"raft.run_cpu_time_us":479,"raft.run_wall_time_us":479,"spinlock_wait_cycles":33755776,"thread_start_us":346067,"threads_started":2,"wal-append.queue_time_us":1205904}
I20260709 13:03:38.604218  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:38.600493  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:38.632252  1715 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:38.632751  1714 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:38.601364  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:38.490316  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.538682  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 585) took 1504 ms. Trace:
W20260709 13:03:38.626143  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 304561us
I20260709 13:03:38.636399  1685 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:38.637485  1717 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:38.627807  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:38.640153  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2346) took 1314 ms. Trace:
I20260709 13:03:38.640592  1717 rpcz_store.cc:276] 0709 13:03:37.326006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.346253 (+1020247us) service_pool.cc:224] Handling call
0709 13:03:38.640089 (+293836us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:38.641058  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.635043  1565 rpcz_store.cc:276] 0709 13:03:37.034656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.133488 (+ 98832us) service_pool.cc:224] Handling call
0709 13:03:37.134071 (+   583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.315511 (+181440us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:37.357467 (+ 41956us) log.cc:844] Serialized 14 byte log entry
0709 13:03:37.738875 (+381408us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:37.738931 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:37.738975 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.538230 (+799255us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b5400305f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:38.538313 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.538565 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25960,"prepare.queue_time_us":46331,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":238811,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"spinlock_wait_cycles":416742144,"thread_start_us":380648,"threads_started":2,"wal-append.queue_time_us":378084}
W20260709 13:03:38.641548  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:38.635509  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 299719us
I20260709 13:03:38.645244  2345 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 8: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:38.639665  1568 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:38.655951  1569 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:03:38.658214  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:38.660507  1571 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:38.663072  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:38.655192  2314 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:38.357182  5465 raft_consensus.cc:3055] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:03:38.670600  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:38.679289  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 66102us
I20260709 13:03:38.670897  1722 raft_consensus.cc:2410] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
W20260709 13:03:38.496213  5558 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 1.181s	user 0.004s	sys 0.002s
I20260709 13:03:38.675563  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:38.444013  5557 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.540s	user 0.008s	sys 0.000s
W20260709 13:03:38.679129  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 354005us
I20260709 13:03:38.686028  1566 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:03:38.687024  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2007) took 1886 ms. Trace:
I20260709 13:03:38.687408  1611 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:38.683892  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.685308  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2349) took 1215 ms. Trace:
I20260709 13:03:38.675081  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:38.688076  1566 rpcz_store.cc:276] 0709 13:03:36.801021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.638507 (+1837486us) service_pool.cc:224] Handling call
0709 13:03:38.685528 (+ 47021us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.686675 (+  1147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.686950 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:38.729400  1722 rpcz_store.cc:276] 0709 13:03:37.469405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.300291 (+830886us) service_pool.cc:224] Handling call
0709 13:03:38.685224 (+384933us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:38.690119  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.690385  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:38.733928  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 63757us
I20260709 13:03:38.733310  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:38.769918  1722 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:38.665463  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:38.663941  2347 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:38.683153  1612 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:38.606371  5578 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:38.739550  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 84437.7us
I20260709 13:03:38.350548  5606 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:38.601727  5601 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.730437  1572 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:38.741561  1714 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:38.693404  5565 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.783670  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:38.784772  1714 raft_consensus.cc:2376] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9. Re-sending same reply.
W20260709 13:03:38.743734  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 56446.1us
W20260709 13:03:38.744870  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 58824.3us
I20260709 13:03:38.789223  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:38.790436  1714 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:38.530912  5567 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:38.602576  5553 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:38.487305  5534 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:38.664371  2343 raft_consensus.cc:2376] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12. Re-sending same reply.
I20260709 13:03:38.657514  5592 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:38.778291  1639 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:38.663934  5252 consensus_queue.cc:1059] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.120s
W20260709 13:03:38.779776  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 96834.3us
W20260709 13:03:38.751133  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 114714us
I20260709 13:03:38.807258  1685 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:38.594300  5197 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:38.485340  5331 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:38.736393  5525 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.692189  5513 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:38.633556  5594 raft_consensus.cc:740] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:38.685184  5551 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:38.766450  5590 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:38.801549  1611 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:38.680140  5465 raft_consensus.cc:740] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:38.803936  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:38.758033  5598 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:38.763549  5497 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:38.765517  5443 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:38.805007  1612 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:38.769912  5579 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:38.818346  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 1999) took 2161 ms. Trace:
I20260709 13:03:38.771371  5532 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:38.820680  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.826061  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 584) took 1792 ms. Trace:
I20260709 13:03:38.866968  1573 rpcz_store.cc:276] 0709 13:03:37.034253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.089061 (+ 54808us) service_pool.cc:224] Handling call
0709 13:03:37.089487 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.097843 (+  8356us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:37.143325 (+ 45482us) log.cc:844] Serialized 14 byte log entry
0709 13:03:38.393376 (+1250051us) spinlock_profiling.cc:243] Waited 2.58 ms on lock 0x7b180044ad60. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a2523a 00007f8248e8fa2a
0709 13:03:38.825578 (+432202us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:38.825679 (+   101us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:38.825747 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.825837 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.825994 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28722,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":16549,"spinlock_wait_cycles":5939456,"thread_start_us":431841,"threads_started":1}
I20260709 13:03:38.827116  1715 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:03:38.870472  1714 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
W20260709 13:03:38.872020  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.747296  5524 consensus_queue.cc:1059] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:03:38.353657  5595 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:38.875571  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:38.875927  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:38.834674  5546 consensus_queue.cc:237] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:38.876667  1715 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:38.877959  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:38.878592  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:38.878825  5595 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:38.879280  1715 raft_consensus.cc:2410] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:38.829326  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:38.834035  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:38.396081  2346 rpcz_store.cc:276] 0709 13:03:37.324844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.327016 (+  2172us) service_pool.cc:224] Handling call
0709 13:03:37.425326 (+ 98310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.250612 (+825286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.388634 (+138022us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b54004759d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:38.388862 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1783988096}
I20260709 13:03:38.880319  1572 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:38.798640  5550 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:38.810240  5592 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:38.835635  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1989) took 2344 ms. Trace:
W20260709 13:03:38.886693  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 81826.3us
I20260709 13:03:38.850349  1685 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:38.856649  1567 rpcz_store.cc:276] 0709 13:03:36.657203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.657358 (+   155us) service_pool.cc:224] Handling call
0709 13:03:36.680615 (+ 23257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:36.680872 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:37.198556 (+517684us) log.cc:844] Serialized 14 byte log entry
0709 13:03:37.718473 (+519917us) log.cc:844] Serialized 14 byte log entry
0709 13:03:38.542592 (+824119us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b18003f7a80. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287 00007f8248ea0208
0709 13:03:38.817937 (+275345us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:38.817996 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:38.818058 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.818138 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.818283 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15373,"prepare.queue_time_us":175081,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":100,"raft.queue_time_us":355267,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":217,"spinlock_wait_cycles":24628480,"thread_start_us":1234656,"threads_started":4,"wal-append.queue_time_us":376887}
I20260709 13:03:38.875864  5587 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:38.881438  1719 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:38.812702  5586 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:38.875520  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:38.894767  1714 raft_consensus.cc:2410] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:03:38.895939  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:38.896697  1714 raft_consensus.cc:2376] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15. Re-sending same reply.
I20260709 13:03:38.836050  5591 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:38.882071  1573 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:38.886863  1718 rpcz_store.cc:276] 0709 13:03:36.491353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.491532 (+   179us) service_pool.cc:224] Handling call
0709 13:03:36.536102 (+ 44570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.682088 (+145986us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.682126 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.683008 (+   882us) log.cc:844] Serialized 26 byte log entry
0709 13:03:36.969944 (+286936us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:37.408645 (+438701us) log.cc:844] Serialized 14 byte log entry
0709 13:03:37.567573 (+158928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.568198 (+   625us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:38.495024 (+926826us) spinlock_profiling.cc:243] Waited 4.4 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:38.828202 (+333178us) raft_consensus.cc:1685] finished
0709 13:03:38.835446 (+  7244us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.835580 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":162732,"prepare.queue_time_us":625429,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":5385,"raft.queue_time_us":679109,"raft.run_cpu_time_us":411,"raft.run_wall_time_us":48877,"spinlock_wait_cycles":10114432,"thread_start_us":665428,"threads_started":4,"wal-append.queue_time_us":430709}
I20260709 13:03:38.890097  5550 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:38.848130  5497 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.897568  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:38.825810  5603 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:03:38.903257  1722 raft_consensus.cc:2410] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:03:38.810580  5565 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:38.827575  5525 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:38.827822  5513 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:38.898551  5551 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:38.869457  5553 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:38.907424  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:38.836560  5590 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:38.915079  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 47886.4us
I20260709 13:03:38.915246  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:38.950809  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2356) took 1010 ms. Trace:
I20260709 13:03:38.951205  1715 rpcz_store.cc:276] 0709 13:03:37.940353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.886141 (+945788us) service_pool.cc:224] Handling call
0709 13:03:38.950728 (+ 64587us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:38.941394  5519 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.013s	sys 0.005s
I20260709 13:03:38.811781  5413 consensus_queue.cc:260] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:38.883620  5197 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:38.894301  5592 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:38.929831  5586 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:38.888018  5471 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:38.918143  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 116814us
I20260709 13:03:38.927634  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:38.916386  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 66183.4us
I20260709 13:03:38.946460  1714 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:38.925824  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2353) took 1221 ms. Trace:
W20260709 13:03:38.882788  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 95305.2us
W20260709 13:03:38.937573  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.934254  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:40.534538  1645 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:40.538609  5473 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:40.539275  5473 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:40.542454  5512 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.542986  5512 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:40.545162  5512 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:39.069684  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2475) took 1835 ms. Trace:
I20260709 13:03:40.545822  1639 rpcz_store.cc:276] 0709 13:03:37.234404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.360793 (+126389us) service_pool.cc:224] Handling call
0709 13:03:37.385263 (+ 24470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.800403 (+1415140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.069312 (+268909us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400182c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:39.069564 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3203448192}
I20260709 13:03:39.308257  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:40.547559  1715 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 9: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:40.548208  1715 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2345) took 3297 ms (client timeout 3576 ms). Trace:
W20260709 13:03:40.548594  1715 rpcz_store.cc:269] 0709 13:03:37.250488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.307925 (+2057437us) service_pool.cc:224] Handling call
0709 13:03:40.548138 (+1240213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:39.226347  5357 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:38.932099  5331 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:39.154462  5461 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:38.955042  5475 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:39.198135  5465 consensus_queue.cc:260] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:38.927615  5497 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:39.246063  5498 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:38.934927  5513 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:39.124073  5483 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:38.965492  5519 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.013s	sys 0.005s
I20260709 13:03:38.932693  5525 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:38.922174  5550 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:40.584817  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:40.586824  5473 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:38.942251  5551 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.591171  5551 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:38.990631  5586 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:39.060818  5565 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:38.953310  5587 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:39.080475  5592 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:40.599133  5592 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:38.957554  5590 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:39.023191  5597 consensus_queue.cc:1059] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:03:40.608018  1645 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
W20260709 13:03:40.608652  1645 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 376) took 1681 ms (client timeout 1826 ms). Trace:
I20260709 13:03:39.069000  5606 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:38.800508  5601 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:39.238332  5594 consensus_queue.cc:260] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:40.615437  5594 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:40.616032  1564 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
W20260709 13:03:39.326526  5611 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.400s	user 0.001s	sys 0.003s
I20260709 13:03:38.930603  5615 consensus_queue.cc:1059] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.106s
W20260709 13:03:39.433374  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.612026  5497 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:40.625957  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:39.440543  2340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 509) took 1971 ms. Trace:
W20260709 13:03:39.713181  1687 leader_election.cc:336] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.993s (SENT)
I20260709 13:03:39.771200  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55890 (request call id 393) took 1507 ms. Trace:
I20260709 13:03:40.004592  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:40.027730  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.629840  2340 rpcz_store.cc:276] 0709 13:03:37.469018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.470908 (+  1890us) service_pool.cc:224] Handling call
0709 13:03:37.509336 (+ 38428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.631378 (+1122042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.440132 (+808754us) spinlock_profiling.cc:243] Waited 852 ms on lock 0x7b5400474d54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:39.440448 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1959204096}
I20260709 13:03:40.096448  1611 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:40.633433  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 537067us
I20260709 13:03:40.633872  5630 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:40.634379  1611 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:40.634967  5594 consensus_queue.cc:1059] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:40.635915  5606 raft_consensus.cc:3055] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:40.636300  5606 raft_consensus.cc:740] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:40.637064  1611 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:40.131218  1719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 349) took 1861 ms (client timeout 2022 ms). Trace:
W20260709 13:03:40.645007  1719 rpcz_store.cc:269] 0709 13:03:38.270225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.384065 (+113840us) service_pool.cc:224] Handling call
0709 13:03:40.131027 (+1746962us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b5400053f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:40.131168 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":585874176}
I20260709 13:03:40.645398  5590 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:40.646387  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:40.647147  1719 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
W20260709 13:03:40.647804  1719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2390) took 1747 ms (client timeout 2061 ms). Trace:
I20260709 13:03:40.648046  5590 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:40.648402  1719 rpcz_store.cc:269] 0709 13:03:38.900218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.646068 (+1745850us) service_pool.cc:224] Handling call
0709 13:03:40.647711 (+  1643us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.641346  5550 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:40.139328  2314 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.248s	user 0.029s	sys 0.058s
W20260709 13:03:40.650295  2314 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.759s	user 0.029s	sys 0.059s
W20260709 13:03:40.650709  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 1.75978e+06us
I20260709 13:03:40.650902  5550 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:40.175066  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:40.654476  1717 raft_consensus.cc:2410] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:03:40.655274  1717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2377) took 2364 ms (client timeout 2105 ms). Trace:
W20260709 13:03:40.655964  1717 rpcz_store.cc:269] 0709 13:03:38.290669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.173051 (+1882382us) service_pool.cc:224] Handling call
0709 13:03:40.655183 (+482132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.198661  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:40.658444  1716 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:40.659819  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2352) took 3013 ms (client timeout 2818 ms). Trace:
W20260709 13:03:40.660284  1716 rpcz_store.cc:269] 0709 13:03:37.646359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.198286 (+2551927us) service_pool.cc:224] Handling call
0709 13:03:40.659723 (+461437us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.203606  5527 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:40.216531  5552 raft_consensus.cc:3055] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:03:40.228963  5612 log.cc:927] Time spent T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.308s	user 0.003s	sys 0.003s
W20260709 13:03:40.236930  5613 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.350s	user 0.002s	sys 0.004s
I20260709 13:03:40.313675  5578 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.320050  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:40.325795  5614 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 1.423s	user 0.004s	sys 0.013s
I20260709 13:03:40.677848  1720 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:40.675673  1470 catalog_manager.cc:5697] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 6 to 8, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 8 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:40.680203  1470 catalog_manager.cc:5697] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 4 to 6, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 6 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:40.336055  1538 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.485s	user 0.019s	sys 0.048s
W20260709 13:03:40.681880  1538 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.831s	user 0.019s	sys 0.049s
I20260709 13:03:40.677527  1611 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:40.682597  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 1.83156e+06us
I20260709 13:03:40.683576  5475 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:40.684154  5475 raft_consensus.cc:697] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:40.348627  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:40.354537  5610 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 1.370s	user 0.007s	sys 0.017s
W20260709 13:03:40.402932  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 1.47544e+06us
I20260709 13:03:40.392910  5186 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:40.408018  2370 debug-util.cc:398] Leaking SignalData structure 0x7b08002e6260 after lost signal to thread 431
W20260709 13:03:40.687865  2370 debug-util.cc:398] Leaking SignalData structure 0x7b080030aac0 after lost signal to thread 434
I20260709 13:03:40.688450  5331 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:03:40.689064  2370 debug-util.cc:398] Leaking SignalData structure 0x7b08001efa60 after lost signal to thread 1505
W20260709 13:03:40.690487  2370 debug-util.cc:398] Leaking SignalData structure 0x7b08002f0ae0 after lost signal to thread 1597
W20260709 13:03:40.690791  2370 debug-util.cc:398] Leaking SignalData structure 0x7b08002f0ac0 after lost signal to thread 1598
I20260709 13:03:40.691229  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 372) took 2945 ms. Trace:
I20260709 13:03:40.409384  5457 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:40.692035  5457 raft_consensus.cc:697] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:40.692953  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:40.685231  5475 consensus_queue.cc:237] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:40.694887  2345 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:03:40.418975  5608 log.cc:927] Time spent T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.575s	user 0.004s	sys 0.011s
W20260709 13:03:40.696744  2370 debug-util.cc:398] Leaking SignalData structure 0x7b08002e0280 after lost signal to thread 1672
I20260709 13:03:40.439184  1685 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:40.444680  5546 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.512s	user 0.015s	sys 0.016s
W20260709 13:03:40.697862  5546 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.772s	user 0.016s	sys 0.016s
I20260709 13:03:40.448542  1714 raft_consensus.cc:2410] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:03:40.698637  5578 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:40.699928  1714 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2373) took 2494 ms (client timeout 2570 ms). Trace:
W20260709 13:03:40.700351  1714 rpcz_store.cc:269] 0709 13:03:38.206190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.897341 (+691151us) service_pool.cc:224] Handling call
0709 13:03:40.699853 (+1802512us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:40.445396  5596 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.483s	user 0.009s	sys 0.016s
I20260709 13:03:40.497541  5481 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.498803  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:40.500092  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:40.701496  5596 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.748s	user 0.010s	sys 0.016s
I20260709 13:03:40.501355  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:40.501523  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:40.511049  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 370) took 2766 ms. Trace:
I20260709 13:03:40.710345  5461 consensus_queue.cc:1059] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:40.691637  1640 rpcz_store.cc:276] 0709 13:03:37.745448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.224563 (+479115us) service_pool.cc:224] Handling call
0709 13:03:38.258934 (+ 34371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:38.259543 (+   609us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:38.259591 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:38.278589 (+ 18998us) log.cc:844] Serialized 26 byte log entry
0709 13:03:38.659945 (+381356us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:38.763882 (+103937us) log.cc:844] Serialized 14 byte log entry
0709 13:03:38.764142 (+   260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.764252 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.227798 (+1463546us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:40.677377 (+449579us) raft_consensus.cc:1685] finished
0709 13:03:40.690896 (+ 13519us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.691159 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26801,"prepare.queue_time_us":168188,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":20230,"raft.queue_time_us":7790,"raft.run_cpu_time_us":413,"raft.run_wall_time_us":19410,"spinlock_wait_cycles":304961408,"thread_start_us":336161,"threads_started":1,"wal-append.queue_time_us":536113}
I20260709 13:03:40.512308  1722 rpcz_store.cc:276] 0709 13:03:37.704656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.732951 (+1028295us) service_pool.cc:224] Handling call
0709 13:03:38.925755 (+192804us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.711859  5461 raft_consensus.cc:3055] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:40.712070  5578 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:40.712488  5461 raft_consensus.cc:740] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:40.513392  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:40.526345  1713 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2383) took 2052 ms (client timeout 1580 ms). Trace:
I20260709 13:03:40.567377  5465 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:40.713670  1713 rpcz_store.cc:269] 0709 13:03:38.474527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.526079 (+2051552us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:40.526244 (+   165us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:40.714442  5331 consensus_queue.cc:1059] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:40.583331  2312 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:40.714856  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:40.715289  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:40.715812  1713 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
W20260709 13:03:40.715921  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 132720us
I20260709 13:03:40.716425  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2395) took 1529 ms. Trace:
I20260709 13:03:40.574219  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:40.594802  5586 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:40.600144  1565 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:40.718134  1715 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2394) took 1531 ms (client timeout 1556 ms). Trace:
W20260709 13:03:40.718700  1715 rpcz_store.cc:269] 0709 13:03:39.186739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.573956 (+1387217us) service_pool.cc:224] Handling call
0709 13:03:40.718078 (+144122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.719079  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:40.719740  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:40.596585  5587 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:40.720619  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 586) took 2091 ms. Trace:
I20260709 13:03:38.835273  5604 consensus_queue.cc:1048] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.082s
W20260709 13:03:40.609058  1645 rpcz_store.cc:269] 0709 13:03:38.927088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.927273 (+   185us) service_pool.cc:224] Handling call
0709 13:03:40.608584 (+1681311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.710923  1644 rpcz_store.cc:276] 0709 13:03:37.744888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.102816 (+357928us) service_pool.cc:224] Handling call
0709 13:03:38.103172 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:38.103575 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:38.103608 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:38.104324 (+   716us) log.cc:844] Serialized 26 byte log entry
0709 13:03:38.274824 (+170500us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:38.472287 (+197463us) log.cc:844] Serialized 14 byte log entry
0709 13:03:40.460763 (+1988476us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b5400472fa0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e00733 00007f8248e85dbf 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c
0709 13:03:40.460848 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.460979 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.461357 (+   378us) raft_consensus.cc:1685] finished
0709 13:03:40.461516 (+   159us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.510975 (+ 49459us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":50051,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":16042,"raft.run_cpu_time_us":688,"raft.run_wall_time_us":8862,"spinlock_wait_cycles":508045056,"thread_start_us":359665,"threads_started":2,"wal-append.queue_time_us":323145,"wal-append.run_cpu_time_us":27955,"wal-append.run_wall_time_us":1800244}
I20260709 13:03:40.612121  5413 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:40.723191  5578 raft_consensus.cc:2764] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:03:40.612990  1718 raft_consensus.cc:2410] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:40.716511  1722 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:40.724717  1718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2359) took 2622 ms (client timeout 2698 ms). Trace:
I20260709 13:03:40.677615  5550 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:03:40.620216  1564 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2012) took 1997 ms (client timeout 1538 ms). Trace:
W20260709 13:03:40.726377  1564 rpcz_store.cc:269] 0709 13:03:38.622994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.623109 (+   115us) service_pool.cc:224] Handling call
0709 13:03:40.620001 (+1996892us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b5400224cd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:40.620133 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4574374272}
W20260709 13:03:40.724752  1722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2361) took 2621 ms (client timeout 3257 ms). Trace:
W20260709 13:03:40.630107  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 916974us
I20260709 13:03:40.722326  5587 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:40.621811  5473 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.728099  5473 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:40.728580  5331 consensus_queue.cc:1243] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:03:40.729099  5596 raft_consensus.cc:3055] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:40.729933  5596 raft_consensus.cc:740] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:40.631309  5497 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:40.731278  5596 consensus_queue.cc:260] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:40.637204  5606 consensus_queue.cc:260] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:40.732111  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2363) took 2627 ms. Trace:
I20260709 13:03:40.639533  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:40.641113  5630 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:40.111340  5624 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.641723  5551 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:40.731472  5473 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:40.744975  5624 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:40.746615  5551 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:40.112187  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:40.751096  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.630107  2348 rpcz_store.cc:276] 0709 13:03:38.263425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.311667 (+ 48242us) service_pool.cc:224] Handling call
0709 13:03:39.770788 (+1459121us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b5400467654. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:39.771065 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":438770560}
I20260709 13:03:40.753978  5606 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:40.630534  1721 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
W20260709 13:03:40.756800  1721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2378) took 2452 ms (client timeout 2093 ms). Trace:
W20260709 13:03:40.757386  1721 rpcz_store.cc:269] 0709 13:03:38.304846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.004172 (+1699326us) service_pool.cc:224] Handling call
0709 13:03:40.756704 (+752532us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.758929  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:40.759665  1721 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:40.740401  5587 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.733865  5578 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.664700  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:40.762590  5587 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:40.763306  5578 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:40.764038  1719 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
W20260709 13:03:40.764604  1719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2393) took 1578 ms (client timeout 2015 ms). Trace:
W20260709 13:03:40.764977  1719 rpcz_store.cc:269] 0709 13:03:39.186441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.664378 (+1477937us) service_pool.cc:224] Handling call
0709 13:03:40.764531 (+100153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.767145  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:40.767412  5578 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:40.768379  1719 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:40.708030  1470 catalog_manager.cc:5697] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:40.661458  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:40.770172  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2339) took 3733 ms (client timeout 4148 ms). Trace:
W20260709 13:03:40.770632  1716 rpcz_store.cc:269] 0709 13:03:37.036342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.661092 (+3624750us) service_pool.cc:224] Handling call
0709 13:03:40.770100 (+109008us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.770757  1470 catalog_manager.cc:5697] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 6 to 7, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 7 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:40.771873  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:40.772539  1716 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
W20260709 13:03:40.665936  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:40.775178  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.666980  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:40.776737  1717 raft_consensus.cc:2376] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8. Re-sending same reply.
W20260709 13:03:40.777534  1717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2392) took 1591 ms (client timeout 1969 ms). Trace:
W20260709 13:03:40.778219  1717 rpcz_store.cc:269] 0709 13:03:39.185768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.657336 (+1471568us) service_pool.cc:224] Handling call
0709 13:03:40.777465 (+120129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.779407  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:40.780032  1717 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:40.662240  5552 raft_consensus.cc:740] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:40.761989  5550 raft_consensus.cc:697] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:40.713917  5461 consensus_queue.cc:260] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:40.782507  5552 consensus_queue.cc:260] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:40.783174  5461 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:40.782858  5550 consensus_queue.cc:237] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:40.784078  5552 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:40.674598  5483 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:03:40.686043  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.694051  5457 consensus_queue.cc:237] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:40.788295  1470 catalog_manager.cc:5697] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 8 to 9, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 9 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:40.411619  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 363) took 4638 ms (client timeout 4150 ms). Trace:
I20260709 13:03:40.739504  1687 rpcz_store.cc:276] 0709 13:03:38.104398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.732050 (+2627652us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:40.791442  1642 rpcz_store.cc:269] 0709 13:03:35.773130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.355887 (+582757us) service_pool.cc:224] Handling call
0709 13:03:40.411318 (+4055431us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b54000cf3d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:40.411530 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4495070464}
I20260709 13:03:40.695317  2340 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 8: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
W20260709 13:03:40.697382  2370 debug-util.cc:398] Leaking SignalData structure 0x7b08002e0260 after lost signal to thread 1673
I20260709 13:03:40.702051  5481 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:40.704144  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 371) took 2959 ms. Trace:
I20260709 13:03:40.705139  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:40.701443  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:40.800625  2370 debug-util.cc:398] Leaking SignalData structure 0x7b08003dcf00 after lost signal to thread 1746
W20260709 13:03:40.706514  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:40.801971  1612 connection.cc:256] RPC call timeout handler was delayed by -1.5473779520000335s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 13:03:40.803071  2370 debug-util.cc:398] Leaking SignalData structure 0x7b080017ebc0 after lost signal to thread 2373
W20260709 13:03:40.803814  2370 debug-util.cc:398] Leaking SignalData structure 0x7b080010b740 after lost signal to thread 2374
I20260709 13:03:40.804059  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:40.805251  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:40.707360  1611 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:40.808900  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 101643us
I20260709 13:03:40.812256  1611 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:40.709384  1720 raft_consensus.cc:2410] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:03:40.813263  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2372) took 2618 ms (client timeout 2215 ms). Trace:
W20260709 13:03:40.813684  1720 rpcz_store.cc:269] 0709 13:03:38.195652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.319727 (+2124075us) service_pool.cc:224] Handling call
0709 13:03:40.813182 (+493455us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:40.714699  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 275599us
I20260709 13:03:40.814249  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:40.815191  1639 raft_consensus.cc:2410] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:03:40.710265  2343 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:40.721892  5527 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:40.611474  2344 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
W20260709 13:03:40.725255  1718 rpcz_store.cc:269] 0709 13:03:38.102760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.907084 (+804324us) service_pool.cc:224] Handling call
0709 13:03:40.724504 (+1817420us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b5400210f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:40.724658 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3838123392}
I20260709 13:03:40.618034  5601 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:40.727187  1471 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38666 (request call id 214) took 17922 ms (client timeout 15000 ms). Trace:
I20260709 13:03:40.632362  5592 raft_consensus.cc:3055] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:40.638590  5498 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.720989  1565 rpcz_store.cc:276] 0709 13:03:38.629700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.880664 (+250964us) service_pool.cc:224] Handling call
0709 13:03:38.881014 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.718567 (+1837553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.720389 (+  1822us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b54001b5b54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:40.720562 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3953466112}
I20260709 13:03:40.716948  1713 rpcz_store.cc:276] 0709 13:03:39.187073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.714564 (+1527491us) service_pool.cc:224] Handling call
0709 13:03:40.716365 (+  1801us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.832778  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:40.832715  1645 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:03:40.834232  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:40.833061  1715 raft_consensus.cc:2410] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:03:40.739800  1568 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:03:40.837255  1713 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:40.834630  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:40.838256  1641 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:40.838372  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:40.831000  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:40.838964  1713 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:40.739531  1722 rpcz_store.cc:269] 0709 13:03:38.103382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.711756 (+2608374us) service_pool.cc:224] Handling call
0709 13:03:40.724688 (+ 12932us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:40.839737  2344 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55888 (request call id 510) took 3051 ms (client timeout 1789 ms). Trace:
I20260709 13:03:40.840227  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:40.840817  5357 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.801297  1714 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
W20260709 13:03:40.842555  1714 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 356) took 1550 ms (client timeout 1998 ms). Trace:
W20260709 13:03:40.843168  1687 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42276 dropped due to backpressure: service queue is full with 50 items
I20260709 13:03:40.843662  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:40.844751  1640 raft_consensus.cc:2410] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:03:40.843266  1714 rpcz_store.cc:269] 0709 13:03:39.292111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.701130 (+1409019us) service_pool.cc:224] Handling call
0709 13:03:40.842496 (+141366us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.845958  1647 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:40.796371  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:40.826283  1470 catalog_manager.cc:5697] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 2 to 10, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 10 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:40.850307  2349 raft_consensus.cc:2393] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 5: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:40.822017  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:40.815428  1685 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:40.840950  1722 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:40.842619  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:40.825766  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:40.717976  2312 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:40.842706  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:40.845870  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:40.854435  1714 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:40.788431  5587 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:40.859659  1641 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:03:40.800091  1648 rpcz_store.cc:276] 0709 13:03:37.745215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.171568 (+426353us) service_pool.cc:224] Handling call
0709 13:03:38.194210 (+ 22642us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:38.206628 (+ 12418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:38.206659 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:38.213225 (+  6566us) log.cc:844] Serialized 26 byte log entry
0709 13:03:38.640373 (+427148us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:38.684834 (+ 44461us) log.cc:844] Serialized 14 byte log entry
0709 13:03:38.685263 (+   429us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.685475 (+   212us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.235825 (+1550350us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:40.401200 (+165375us) spinlock_profiling.cc:243] Waited 6.22 ms on lock 0x7b1800376ea0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:03:40.703908 (+302708us) raft_consensus.cc:1685] finished
0709 13:03:40.703933 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.704081 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8043,"prepare.queue_time_us":123856,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":56,"raft.queue_time_us":3816,"raft.run_cpu_time_us":474,"raft.run_wall_time_us":18170,"spinlock_wait_cycles":382674048,"thread_start_us":362419,"threads_started":1,"wal-append.queue_time_us":555362}
W20260709 13:03:40.861810  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 143923us
I20260709 13:03:40.852856  1643 raft_consensus.cc:2393] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:40.832322  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:40.862459  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:40.863992  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:40.864411  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49228.5us
I20260709 13:03:40.867318  1641 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:03:40.831656  1717 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:40.830927  1721 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
W20260709 13:03:40.869853  1721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 350) took 2599 ms (client timeout 3451 ms). Trace:
I20260709 13:03:40.870276  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:40.870448  1721 rpcz_store.cc:269] 0709 13:03:38.270767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.758470 (+2487703us) service_pool.cc:224] Handling call
0709 13:03:40.869777 (+111307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.869860  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:40.840124  2344 rpcz_store.cc:269] 0709 13:03:37.788712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.789527 (+   815us) service_pool.cc:224] Handling call
0709 13:03:40.839549 (+3050022us) spinlock_profiling.cc:243] Waited 2.53 s on lock 0x7b5400600254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:40.839667 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5830480640}
W20260709 13:03:40.873692  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 52532.9us
W20260709 13:03:40.874151  1537 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.504s (SENT)
I20260709 13:03:40.863454  5590 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:40.875308  5590 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.875505  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:40.876482  2346 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:40.858598  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:40.872637  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:40.827286  5528 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.887136  1713 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:03:40.828469  5353 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.858750  2341 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:40.851899  1644 raft_consensus.cc:2393] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:40.855432  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:40.841603  5357 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:40.894589  1714 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:03:40.880730  1685 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:40.833520  5551 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:40.883755  1722 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:40.902449  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:40.886291  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:40.875737  5590 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:40.891933  5353 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:40.915544  5357 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:40.916257  5413 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:40.890435  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 78274.3us
I20260709 13:03:40.884486  5624 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:40.843209  5592 raft_consensus.cc:740] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:40.871663  1569 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:40.891060  5502 consensus_queue.cc:1059] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:03:40.891331  5483 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:40.824213  1471 rpcz_store.cc:269] 0709 13:03:22.805455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.805768 (+   313us) service_pool.cc:224] Handling call
0709 13:03:22.926578 (+120810us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b5400310010. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d61640 00007f824ec38b76 00007f824ed42c3e 00007f824d2032a4 00007f824d203232 00007f824d203161 00007f824d2030dc 00007f824d202392 00007f82471c048c 00007f82471bf8c6
0709 13:03:40.727113 (+17800535us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:35.074991 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:35.075858 (+   867us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:35.075933 (+    75us) write_op.cc:435] Acquired schema lock
0709 13:03:35.075957 (+    24us) tablet.cc:662] Decoding operations
0709 13:03:35.077590 (+  1633us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:35.107099 (+ 29509us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:35.113900 (+  6801us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:03:35.223854 (+109954us) tablet.cc:701] Row locks acquired
0709 13:03:35.246540 (+ 22686us) write_op.cc:260] PREPARE: finished
0709 13:03:35.289985 (+ 43445us) write_op.cc:270] Start()
0709 13:03:35.379785 (+ 89800us) write_op.cc:276] Timestamp: P: 1783602215280225 usec, L: 0
0709 13:03:35.382046 (+  2261us) op_driver.cc:348] REPLICATION: starting
0709 13:03:35.522539 (+140493us) log.cc:844] Serialized 6847 byte log entry
0709 13:03:36.126173 (+603634us) op_driver.cc:464] REPLICATION: finished
0709 13:03:36.246267 (+120094us) write_op.cc:301] APPLY: starting
0709 13:03:36.246384 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:36.880394 (+634010us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:36.985977 (+105583us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:39.063837 (+2077860us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:39.067622 (+  3785us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=34,mrs_lookups=0
0709 13:03:39.067679 (+    57us) write_op.cc:312] APPLY: finished
0709 13:03:39.068342 (+   663us) log.cc:844] Serialized 156 byte log entry
0709 13:03:39.082236 (+ 13894us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:39.082492 (+   256us) write_op.cc:454] Released schema lock
0709 13:03:39.082603 (+   111us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:39.083328 (+   725us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:39.083362 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":120366,"reported_tablets":53,"spinlock_wait_cycles":2496896,"child_traces":[["op",{"apply.queue_time_us":94010,"apply.run_cpu_time_us":68857,"apply.run_wall_time_us":2837041,"cfile_cache_hit":66,"cfile_cache_hit_bytes":69003,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6218,"cfile_init":2,"lbm_read_time_us":21623,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":11,"num_ops":17,"prepare.queue_time_us":40342,"prepare.run_cpu_time_us":42246,"prepare.run_wall_time_us":612299,"raft.queue_time_us":156667,"raft.run_cpu_time_us":29341,"raft.run_wall_time_us":85849,"replication_time_us":743604,"thread_start_us":342742,"threads_started":5,"wal-append.queue_time_us":237473,"wal-append.run_cpu_time_us":12530,"wal-append.run_wall_time_us":2313665}]]}
I20260709 13:03:40.935828  1470 catalog_manager.cc:5697] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 5 to 8, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 8 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:40.883040  1612 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:40.902215  1685 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:40.905719  1719 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
W20260709 13:03:40.906644  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:40.890924  1537 leader_election.cc:336] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.569s (SENT)
I20260709 13:03:40.908483  1646 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:40.905969  1718 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:03:40.844532  1687 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42276 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2030)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 359)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 361)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2017)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2021)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 357)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2029)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2026)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2396)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2398)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2399)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2400)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2401)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2402)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2403)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2404)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2405)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2018)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57572 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1998)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2340)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1999)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2001)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2003)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2004)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2005)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2360)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2365)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2368)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2371)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 353)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 354)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2391)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 355)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2008)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2009)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2011)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2015)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2016)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2019)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2022)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2024)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2025)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2027)
I20260709 13:03:40.910915  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:40.924484  1715 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2357) took 2843 ms (client timeout 3595 ms). Trace:
I20260709 13:03:40.939476  1685 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:40.943463  1715 rpcz_store.cc:269] 0709 13:03:38.081425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.719485 (+2638060us) service_pool.cc:224] Handling call
0709 13:03:40.911227 (+191742us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540002e4d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:40.924396 (+ 13169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257885952}
I20260709 13:03:40.932777  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:40.943222  1714 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:40.934621  5551 consensus_queue.cc:1059] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:03:40.900388  5587 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:40.953318  5597 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:40.950994  5473 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:40.860743  5498 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:40.954712  5473 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.955269  5473 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:40.955296  2345 raft_consensus.cc:2410] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:03:40.953452  5547 raft_consensus.cc:3055] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:40.958212  5547 raft_consensus.cc:740] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:40.958461  5592 consensus_queue.cc:260] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:40.959923  5592 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:40.939636  5413 raft_consensus.cc:2764] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:03:40.949877  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 66945.7us
I20260709 13:03:40.961546  1612 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:40.962525  5512 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:40.959580  5547 consensus_queue.cc:260] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:40.963096  5512 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.963671  5547 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:40.963755  5512 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:40.942438  5596 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:40.962671  1612 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:40.911677  5528 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:40.937708  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:40.969715  1569 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
W20260709 13:03:40.970671  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.971035  1685 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:40.974040  1685 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:40.940934  1537 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:40.977083  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 86182.8us
W20260709 13:03:40.978268  1537 leader_election.cc:336] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.571s (SENT)
I20260709 13:03:40.982573  2312 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:40.938055  5502 raft_consensus.cc:3055] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:40.957386  5498 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:40.987036  5601 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:41.000371  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:40.999809  1537 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:40.995754  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49841.7us
I20260709 13:03:41.002033  1714 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:03:40.986197  5535 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.990833  1470 catalog_manager.cc:5697] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 5 to 6, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 6 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:41.003126  1611 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:41.004669  1537 leader_election.cc:336] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.574s (SENT)
I20260709 13:03:40.993780  5597 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:41.003994  1470 catalog_manager.cc:5697] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 2 to 4, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:40.992977  5186 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:40.995924  1685 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:41.001202  1569 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:03:40.993566  5413 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:41.002079  5465 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:40.998848  5502 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:41.000602  5586 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:41.005296  5624 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:41.007431  5535 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:41.032541  5586 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:41.008780  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:41.010761  5186 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.012609  5528 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:41.033829  5186 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:41.034073  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:41.034825  1721 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:41.035470  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.036388  1717 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 11: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:41.017935  1685 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:41.038319  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:41.038892  1715 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:41.039150  1685 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:41.040726  1685 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:41.041855  5597 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:41.042693  5597 consensus_queue.cc:1048] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:41.018230  5587 raft_consensus.cc:2764] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 13:03:41.044181  5587 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:41.044906  5597 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:41.018481  5502 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:41.045758  1685 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:41.046062  5502 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:41.046631  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:41.024372  5473 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:41.050782  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.051376  5473 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:41.035748  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.021030  1470 catalog_manager.cc:5697] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:41.054857  1470 catalog_manager.cc:5697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:41.063045  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.065129  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.066175  1719 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:41.067306  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.068017  1719 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:41.068951  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:41.069422  1685 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:41.011327  5353 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:41.070868  5606 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:41.016191  5465 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.071841  5465 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:41.073401  1685 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:41.077150  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 114568us
I20260709 13:03:41.028405  5481 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:41.037328  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.090179  5552 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:41.091224  5552 raft_consensus.cc:697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:41.093344  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:41.094151  1720 raft_consensus.cc:2376] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9. Re-sending same reply.
I20260709 13:03:41.094792  5590 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:41.096151  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.097139  5590 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:41.097353  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.098095  1722 raft_consensus.cc:2393] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 8: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:41.097806  5512 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:41.098766  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:41.098994  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.099468  1569 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:41.099759  1722 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:41.100724  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.101737  5597 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:41.092682  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:41.102551  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 90610.2us
I20260709 13:03:41.103720  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.104358  1721 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:41.110061  1719 raft_consensus.cc:2410] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:03:41.095160  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:41.115525  2348 raft_consensus.cc:2393] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:41.105419  1721 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:03:41.116701  1721 raft_consensus.cc:606] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:41.117079  1721 raft_consensus.cc:614] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:41.092658  5552 consensus_queue.cc:237] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:41.118938  2313 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:41.119657  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.120600  1720 raft_consensus.cc:2410] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
W20260709 13:03:41.121497  1687 leader_election.cc:336] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.580s (SENT)
I20260709 13:03:41.061230  1470 catalog_manager.cc:5697] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 7 to 8, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 8 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:41.124150  2313 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:41.125313  2313 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:41.125617  1569 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:03:41.126957  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57572 (request call id 22) took 2298 ms. Trace:
I20260709 13:03:41.127477  2312 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:41.127434  1721 rpcz_store.cc:276] 0709 13:03:38.819151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.105270 (+2286119us) service_pool.cc:224] Handling call
0709 13:03:41.117459 (+ 12189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.128998  1722 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:41.129751  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1998) took 4093 ms. Trace:
I20260709 13:03:41.130117  1722 rpcz_store.cc:276] 0709 13:03:37.035982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.128159 (+4092177us) service_pool.cc:224] Handling call
0709 13:03:41.128825 (+   666us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.129461 (+   636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.129696 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.131907  1722 raft_consensus.cc:1275] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:03:41.132499  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1999) took 3665 ms. Trace:
I20260709 13:03:41.132850  1722 rpcz_store.cc:276] 0709 13:03:37.467225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.131329 (+3664104us) service_pool.cc:224] Handling call
0709 13:03:41.131606 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.132275 (+   669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.132452 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.134459  1718 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:03:41.136739  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.005159  1537 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:41.137428  1717 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:03:41.122076  1687 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:41.138355  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 133708us
W20260709 13:03:41.141114  1537 leader_election.cc:336] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.580s (SENT)
W20260709 13:03:41.142407  1537 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.582s (SENT)
W20260709 13:03:41.144115  1687 leader_election.cc:336] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.931s (SENT)
I20260709 13:03:41.143137  5353 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:41.144701  1719 raft_consensus.cc:1217] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Deduplicated request from leader. Original: 7.4->[8.5-8.5]   Dedup: 8.5->[]
I20260709 13:03:41.148340  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2001) took 3351 ms. Trace:
I20260709 13:03:41.148711  1719 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:03:41.148731  1722 rpcz_store.cc:276] 0709 13:03:37.796798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.143790 (+3346992us) service_pool.cc:224] Handling call
0709 13:03:41.144093 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.144268 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:41.144291 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:41.144327 (+    36us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:41.144356 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.147872 (+  3516us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.148012 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.150103  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2340) took 4113 ms. Trace:
I20260709 13:03:41.150619  1719 rpcz_store.cc:276] 0709 13:03:37.036659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.136177 (+4099518us) service_pool.cc:224] Handling call
0709 13:03:41.136620 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:41.149837 (+ 13217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.150047 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.152366  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2005) took 3349 ms. Trace:
I20260709 13:03:41.145725  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.156915  1713 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:41.158298  1713 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:41.158365  1719 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: [7.4-9.5]
I20260709 13:03:41.159123  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2371) took 2986 ms. Trace:
I20260709 13:03:41.159524  1713 rpcz_store.cc:276] 0709 13:03:38.172443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.157721 (+2985278us) service_pool.cc:224] Handling call
0709 13:03:41.158086 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.158860 (+   774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.159061 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42368}
I20260709 13:03:41.161566  1713 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:03:41.145951  1715 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:03:41.142982  1537 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:41.147223  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:40.856052  1688 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42278 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2017)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2021)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 357)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2029)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2026)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2396)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2398)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2399)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2400)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2401)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2402)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2403)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2404)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2405)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2018)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57572 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1998)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2340)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 1999)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2001)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2003)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2004)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2005)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2360)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2365)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2368)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2371)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 353)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 354)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2391)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 355)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2008)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2009)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2011)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2015)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2016)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2019)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2022)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2024)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2025)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2027)
I20260709 13:03:41.102450  5550 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:41.152807  1687 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:41.168831  1687 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.010s (SENT)
W20260709 13:03:41.169610  1687 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.126s (SENT)
I20260709 13:03:41.170043  1687 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:41.170774  1721 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: []
W20260709 13:03:41.171135  1687 leader_election.cc:336] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.363s (SENT)
I20260709 13:03:41.171679  1687 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:41.171756  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2365) took 3066 ms. Trace:
I20260709 13:03:41.172597  1721 rpcz_store.cc:276] 0709 13:03:38.105097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.170042 (+3064945us) service_pool.cc:224] Handling call
0709 13:03:41.170428 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.171487 (+  1059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.171694 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.169016  1722 rpcz_store.cc:276] 0709 13:03:37.802871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.151374 (+3348503us) service_pool.cc:224] Handling call
0709 13:03:41.151759 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.151963 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:41.151986 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:41.152024 (+    38us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:41.152060 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.152108 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.152214 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.144835  5502 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:41.115753  5586 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.167482  1716 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
W20260709 13:03:41.176712  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 72338.4us
I20260709 13:03:41.200896  1611 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:41.178148  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.202440  1714 raft_consensus.cc:2410] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 13 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:03:41.203537  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:41.204175  1714 raft_consensus.cc:2376] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9. Re-sending same reply.
I20260709 13:03:41.205124  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.205811  1714 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:41.182045  5413 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:41.206878  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.207525  1714 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:03:41.208976  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:41.209880  5502 raft_consensus.cc:697] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:41.210228  1714 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:41.210553  1470 catalog_manager.cc:5697] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 4 to 9, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:41.211033  5502 consensus_queue.cc:237] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:41.212848  1687 leader_election.cc:336] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.665s (SENT)
I20260709 13:03:41.214231  1720 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:03:41.214901  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2368) took 3062 ms. Trace:
I20260709 13:03:41.181484  5570 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:41.215255  1720 rpcz_store.cc:276] 0709 13:03:38.152255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.165029 (+3012774us) service_pool.cc:224] Handling call
0709 13:03:41.214005 (+ 48976us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:41.214639 (+   634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.214849 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.216426  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.217011  1720 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:41.217585  5597 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:41.178772  2313 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:41.219843  2313 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:41.153456  1718 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:03:41.180231  1717 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:41.181133  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2360) took 3056 ms. Trace:
I20260709 13:03:41.222645  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2003) took 3423 ms. Trace:
I20260709 13:03:41.223104  5497 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:41.223256  1718 rpcz_store.cc:276] 0709 13:03:37.799951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.152847 (+3352896us) service_pool.cc:224] Handling call
0709 13:03:41.153137 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.222378 (+ 69241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.222586 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.223702  5497 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:41.224363  5497 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:41.224740  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.224870  1611 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:41.225853  5527 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:41.226316  5527 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.226670  5527 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:41.222572  1719 rpcz_store.cc:276] 0709 13:03:38.103070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.157507 (+3054437us) service_pool.cc:224] Handling call
0709 13:03:41.158086 (+   579us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:41.158953 (+   867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.159140 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79104}
I20260709 13:03:41.227708  5637 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:41.228417  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.228905  5565 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:41.229215  5637 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:41.192590  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.231355  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.173911  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:41.179365  1537 leader_election.cc:336] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.698s (SENT)
I20260709 13:03:41.143479  5481 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:41.143671  5512 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:41.143334  5590 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:41.143600  5606 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:41.143456  5592 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:41.154633  5353 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.221942  1687 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:41.222833  5535 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:41.229337  5565 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.229245  1719 raft_consensus.cc:2410] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:41.412385  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:41.413153  1719 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:41.230219  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:41.414093  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.230297  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2004) took 3429 ms. Trace:
I20260709 13:03:41.415086  1717 rpcz_store.cc:276] 0709 13:03:37.801427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.147758 (+3346331us) service_pool.cc:224] Handling call
0709 13:03:41.180017 (+ 32259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.230001 (+ 49984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.230239 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.416381  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.416755  1714 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:03:41.417275  1717 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 9: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:41.194236  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:41.419324  1715 raft_consensus.cc:2410] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:41.233986  1721 raft_consensus.cc:2410] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 13:03:41.144871  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 66894.4us
I20260709 13:03:41.234099  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.118006  5547 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:03:41.234364  1537 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:41.237089  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.235545  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.236806  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.237344  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.422844  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:41.423381  1648 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:41.237284  5498 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.181013  5587 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:41.238219  5586 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:41.198679  5551 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:03:41.231998  5637 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.432729  5535 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:41.443091  5637 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:41.443612  5535 raft_consensus.cc:697] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:41.309693  5481 raft_consensus.cc:697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:41.446594  5481 consensus_queue.cc:237] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:41.447299  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.449050  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.374608  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.451058  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.451730  1641 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:41.451951  5586 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:41.452677  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.245440  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.375321  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.247289  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:41.376195  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.377252  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.456205  1720 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:41.375067  1646 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:03:41.263697  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:41.277482  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:41.388947  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:41.459075  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.458847  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:41.459853  1719 raft_consensus.cc:2410] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:03:41.212809  5186 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:41.394411  5413 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.181334  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 348) took 2901 ms. Trace:
I20260709 13:03:41.420962  5565 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:41.422861  1639 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:41.479697  1722 raft_consensus.cc:2410] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:03:41.425966  1721 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:41.425161  1612 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:41.429775  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.480407  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:41.480839  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 55748.1us
W20260709 13:03:41.434046  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 254709us
I20260709 13:03:41.481527  1612 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:41.481832  1537 leader_election.cc:336] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.893s (SENT)
I20260709 13:03:41.482347  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.480427  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 353) took 3208 ms. Trace:
I20260709 13:03:41.483740  1721 rpcz_store.cc:276] 0709 13:03:38.271649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.425373 (+3153724us) service_pool.cc:224] Handling call
0709 13:03:41.425735 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.480194 (+ 54459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.480375 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.485399  1721 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:41.428601  5527 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:41.374082  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:41.494621  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.464725  1644 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:41.442325  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.469235  1713 rpcz_store.cc:276] 0709 13:03:38.264457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.160773 (+2896316us) service_pool.cc:224] Handling call
0709 13:03:41.161291 (+   518us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:41.165558 (+  4267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.165843 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.497828  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 355) took 2206 ms. Trace:
I20260709 13:03:41.498322  1713 rpcz_store.cc:276] 0709 13:03:39.291582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.496674 (+2205092us) service_pool.cc:224] Handling call
0709 13:03:41.497089 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.497349 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:41.497386 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:41.497442 (+    56us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:41.497490 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.497560 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.497754 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.500247  1713 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:41.500900  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2008) took 1169 ms. Trace:
I20260709 13:03:41.412393  5353 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:41.469381  1642 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:41.422521  5552 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:41.482987  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:41.504712  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:41.504741  1643 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 5. Current term is 6. Ops: [4.6-5.7]
I20260709 13:03:41.481880  1641 raft_consensus.cc:2368] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10.
I20260709 13:03:41.442078  5592 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:41.487216  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2391) took 2302 ms. Trace:
I20260709 13:03:41.515642  1721 rpcz_store.cc:276] 0709 13:03:39.185213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.484737 (+2299524us) service_pool.cc:224] Handling call
0709 13:03:41.485076 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.485964 (+   888us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.487148 (+  1184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.440878  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:41.517191  1717 raft_consensus.cc:2410] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:03:41.482554  1612 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:41.503074  1644 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:03:41.501292  1713 rpcz_store.cc:276] 0709 13:03:40.331346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.499450 (+1168104us) service_pool.cc:224] Handling call
0709 13:03:41.499825 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.500618 (+   793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.500838 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.519057  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.516191  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 354) took 3244 ms. Trace:
I20260709 13:03:41.519878  1720 rpcz_store.cc:276] 0709 13:03:38.271966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.470732 (+3198766us) service_pool.cc:224] Handling call
0709 13:03:41.515593 (+ 44861us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.515840 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:41.515874 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:41.515920 (+    46us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:41.515961 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.516017 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.516129 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.505451  2348 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:03:41.522567  1647 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:03:41.526003  1612 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:41.528059  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.528330  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.528098  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.529235  1646 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
W20260709 13:03:41.531136  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 48230.1us
I20260709 13:03:41.445020  5535 consensus_queue.cc:237] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:41.537178  1718 raft_consensus.cc:2410] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:03:41.539120  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:41.551967  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.573917  1720 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:41.574870  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2011) took 1242 ms. Trace:
I20260709 13:03:41.575368  1720 rpcz_store.cc:276] 0709 13:03:40.332290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.521092 (+1188802us) service_pool.cc:224] Handling call
0709 13:03:41.573715 (+ 52623us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.574538 (+   823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.574796 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.543807  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.577673  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:03:41.579286  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 46509.7us
I20260709 13:03:41.495988  5606 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.580173  1717 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:41.584894  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2016) took 1247 ms. Trace:
I20260709 13:03:41.585496  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2015) took 1248 ms. Trace:
I20260709 13:03:41.586855  1719 rpcz_store.cc:276] 0709 13:03:40.336947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.526586 (+1189639us) service_pool.cc:224] Handling call
0709 13:03:41.584835 (+ 58249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.585057 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:41.585100 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:41.585160 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:41.585209 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.585273 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.585416 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.585366  1717 rpcz_store.cc:276] 0709 13:03:40.337204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.579609 (+1242405us) service_pool.cc:224] Handling call
0709 13:03:41.579925 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.584541 (+  4616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.584831 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:41.589056  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.474753  5590 raft_consensus.cc:697] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:41.581323  1612 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:41.518455  5547 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:41.604172  1612 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:41.504384  5586 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.534377  5461 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:41.604773  5606 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:41.440353  5498 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:41.504418  5551 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:03:41.607849  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.602226  1537 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:41.602455  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 389677us
I20260709 13:03:41.613775  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.540378  5601 raft_consensus.cc:2764] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:03:41.540062  5636 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:41.637033  1721 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:03:41.637605  1721 pending_rounds.cc:85] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 5
I20260709 13:03:41.637971  1721 pending_rounds.cc:107] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:03:41.638311  1721 raft_consensus.cc:2889] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:41.592796  1717 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:41.621770  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:41.622816  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.503989  5413 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:41.638058  1687 leader_election.cc:336] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 3.002s (SENT)
I20260709 13:03:41.640857  1687 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:41.594973  5528 consensus_queue.cc:1059] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:03:41.641901  5587 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:41.642741  5570 raft_consensus.cc:3055] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:41.643131  5570 raft_consensus.cc:740] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:41.643477  5527 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:41.645031  1687 leader_election.cc:336] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.971s (SENT)
I20260709 13:03:41.647383  1646 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:41.647979  1687 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:41.607976  5473 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:41.595156  5552 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:41.579186  5483 consensus_queue.cc:1048] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:03:41.663822  1717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:03:41.689388  1717 raft_consensus.cc:606] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:41.657068  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 54963.7us
I20260709 13:03:41.663874  5528 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:41.678267  5186 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:41.635082  5475 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:41.628269  5597 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:41.696997  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:41.710747  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:41.687744  5638 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:03:41.726709  1537 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:41.697852  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2022) took 1359 ms. Trace:
I20260709 13:03:41.649179  5636 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.722986  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:41.730506  1718 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:41.731314  1719 rpcz_store.cc:276] 0709 13:03:40.338806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.588661 (+1249855us) service_pool.cc:224] Handling call
0709 13:03:41.663094 (+ 74433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.697256 (+ 34162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:41.697294 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:41.697351 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:41.697384 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.697460 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.697758 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.734946  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.658026  5587 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:41.655812  5461 raft_consensus.cc:697] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:41.666294  5552 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.629084  5586 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:41.762146  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.763262  1644 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:41.644061  5570 consensus_queue.cc:260] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:41.689235  5551 raft_consensus.cc:697] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:41.770195  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.798950  1646 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
W20260709 13:03:41.724170  5465 consensus_meta.cc:287] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.652s	user 0.015s	sys 0.009s
I20260709 13:03:41.767668  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.780208  1716 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:41.723465  5475 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.812232  5475 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:41.703290  5473 raft_consensus.cc:697] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:41.741492  5636 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:41.818712  5473 consensus_queue.cc:237] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:41.739872  5592 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:41.766918  5552 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:41.810921  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:41.791128  5527 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:41.749284  5497 consensus_meta.cc:287] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.524s	user 0.020s	sys 0.002s
I20260709 13:03:41.822995  1717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:03:41.735174  5601 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:41.813208  5413 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:41.828169  1720 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:41.791826  5528 raft_consensus.cc:2764] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 13:03:41.838485  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 234464us
I20260709 13:03:41.839593  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.840072  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2025) took 1149 ms. Trace:
I20260709 13:03:41.840476  1722 rpcz_store.cc:276] 0709 13:03:40.690387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.610850 (+920463us) service_pool.cc:224] Handling call
0709 13:03:41.611228 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.611487 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:41.722293 (+110806us) log.cc:844] Serialized 14 byte log entry
0709 13:03:41.839484 (+117191us) log.cc:844] Serialized 14 byte log entry
0709 13:03:41.839668 (+   184us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:41.839718 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:41.839775 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.839844 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.840000 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1713,"raft.queue_time_us":1924,"thread_start_us":224841,"threads_started":2}
I20260709 13:03:41.840361  1646 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:41.825326  5592 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:41.855829  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2019) took 1516 ms. Trace:
W20260709 13:03:41.855882  1687 leader_election.cc:336] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.201s (SENT)
I20260709 13:03:41.860241  1687 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:41.860975  1720 rpcz_store.cc:276] 0709 13:03:40.338042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.584433 (+1246391us) service_pool.cc:224] Handling call
0709 13:03:41.584806 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.835374 (+250568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.844175 (+  8801us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b5400246b54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:41.854534 (+ 10359us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":559210240}
W20260709 13:03:41.856807  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.838160  5570 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:41.876977  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:41.874433  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 59d236b930214b1f91fb55ece6fecd68 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:03:41.879729  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:41.881742  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.826368  5465 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:41.893608  1718 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:03:41.897064  1717 raft_consensus.cc:606] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:41.896834  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.897787  1717 raft_consensus.cc:614] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:41.867736  5528 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.865090  5592 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:41.902415  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.836790  5527 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:41.843935  5512 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:03:41.836536  5497 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:41.905190  1647 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:41.859333  5483 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:03:41.924683  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.798445  5590 consensus_queue.cc:237] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:41.929520  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:41.910753  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.883164  5413 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:41.858291  5186 consensus_queue.cc:1059] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:03:41.892149  5547 raft_consensus.cc:697] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:41.936010  1640 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:41.934893  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:41.942334  1647 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:41.942977  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.942410  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:41.867376  5461 consensus_queue.cc:237] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:41.965215  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:41.965858  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.966472  1569 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 7.
I20260709 13:03:41.969514  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:41.970616  1564 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:41.973274  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:41.964229  5186 raft_consensus.cc:3055] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:41.910933  5528 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:41.938799  5457 consensus_queue.cc:1059] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:03:41.903930  5551 consensus_queue.cc:237] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:41.961472  5547 consensus_queue.cc:237] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:41.977389  2312 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.005s	sys 0.009s
I20260709 13:03:42.053066  1639 raft_consensus.cc:1240] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:42.090418  2312 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.005s	sys 0.009s
I20260709 13:03:42.036489  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:42.092188  1570 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 8: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:42.090759  1717 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:42.096052  1720 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:42.042960  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:42.099256  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:42.099983  1722 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:42.100155  1647 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
W20260709 13:03:42.101328  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.029589  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:42.102172  1565 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:42.015120  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:42.103217  1641 raft_consensus.cc:2376] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6. Re-sending same reply.
I20260709 13:03:42.048813  5527 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:42.057610  5353 consensus_meta.cc:287] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.003s
I20260709 13:03:42.014878  1646 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:03:42.012773  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:42.062402  1564 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:42.065727  5498 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:42.066365  5465 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:42.067662  5475 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:42.046345  5648 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:03:42.071585  5586 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:42.078109  5457 raft_consensus.cc:3055] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:03:42.080497  5565 consensus_meta.cc:287] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.562s	user 0.005s	sys 0.005s
I20260709 13:03:42.047219  5497 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:41.981506  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:42.063055  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2009) took 1731 ms. Trace:
W20260709 13:03:42.105752  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.106892  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:42.180629  1569 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:42.180962  1573 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:42.109867  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:41.978555  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 252144us
I20260709 13:03:42.185070  1537 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:42.109458  1640 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:03:42.127395  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:42.188244  1717 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:03:42.190903  1639 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:03:42.020697  5640 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.314s	user 0.002s	sys 0.000s
I20260709 13:03:42.133576  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:42.139007  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:42.139257  5653 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:42.139667  5652 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:42.140986  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:42.152055  1566 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:42.050303  5636 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:42.154737  1722 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16.
I20260709 13:03:42.159159  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:42.162667  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:42.210951  1645 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:03:42.179898  5457 raft_consensus.cc:740] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:42.173945  1714 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.018s	sys 0.016s
W20260709 13:03:42.223999  1714 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.018s	sys 0.016s
I20260709 13:03:41.997490  5657 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.052831  5186 raft_consensus.cc:740] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:41.979003  5606 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:03:42.181348  1713 rpcz_store.cc:276] 0709 13:03:40.331675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.519000 (+1187325us) service_pool.cc:224] Handling call
0709 13:03:41.519303 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.519577 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:41.974929 (+455352us) log.cc:844] Serialized 14 byte log entry
0709 13:03:42.061769 (+ 86840us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:42.061832 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:42.061903 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.062436 (+   533us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:42.062989 (+   553us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4409,"prepare.queue_time_us":413263,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":365,"spinlock_wait_cycles":62786816,"thread_start_us":535894,"threads_started":2,"wal-append.queue_time_us":61723}
I20260709 13:03:42.200301  2313 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:03:42.029940  5649 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.177s	user 0.002s	sys 0.000s
I20260709 13:03:42.209293  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:42.214524  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 73647.5us
I20260709 13:03:42.227308  1612 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:42.227461  1564 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 4.
I20260709 13:03:42.227589  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:42.228729  1568 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:42.229156  5668 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:42.229811  5668 raft_consensus.cc:697] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:42.162808  1565 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:03:42.221756  5497 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:42.230710  5668 consensus_queue.cc:237] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:42.221961  1567 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:42.209057  5601 raft_consensus.cc:697] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:42.152925  5527 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:03:42.224539  1722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:03:42.233022  5601 consensus_queue.cc:237] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:42.233600  1722 raft_consensus.cc:606] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:42.234079  1722 raft_consensus.cc:614] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:42.217276  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:42.234567  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:42.235281  1717 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:42.241086   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:42.241510  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:42.242544  1713 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:42.242897   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:42.244216  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:42.245443  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:42.246095  1720 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:42.248119  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:42.248952  1572 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:42.249547  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:42.250237  1718 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:42.250452  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.246115  1716 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:42.260223  1722 raft_consensus.cc:1275] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:03:42.263687  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:42.264763  1721 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:42.266119  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:42.266880  1719 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:42.268215  1719 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:42.160472  5642 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
W20260709 13:03:42.273068  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 52619.3us
I20260709 13:03:42.144675  5353 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:42.176477  5645 consensus_queue.cc:1059] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:03:42.207041  5413 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.209937  5586 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.290449  1573 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:03:42.293828  1687 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:42.197333  5565 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:42.204995  5552 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:42.382814  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.294878  1718 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
W20260709 13:03:42.323328  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 123144us
I20260709 13:03:42.389285  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:42.323011  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:42.337924  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.438562  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:42.322458  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:42.414865  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 352) took 4143 ms. Trace:
I20260709 13:03:42.384563  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:42.325502  1570 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:03:42.322933  1645 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: []
W20260709 13:03:42.394856  1538 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.504s (SENT)
I20260709 13:03:42.272327  5661 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:42.293869  1720 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:42.322707  1612 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:42.301982  1721 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:42.413048  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 351) took 4142 ms. Trace:
I20260709 13:03:42.440588  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:42.322060  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:42.440083  1714 rpcz_store.cc:276] 0709 13:03:38.271391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.425205 (+3153814us) service_pool.cc:224] Handling call
0709 13:03:41.427453 (+  2248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.428447 (+   994us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:42.224424 (+795977us) log.cc:844] Serialized 14 byte log entry
0709 13:03:42.414366 (+189942us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:42.414451 (+    85us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:42.414522 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.414620 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:42.414789 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":153732,"prepare.queue_time_us":430154,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":62,"raft.queue_time_us":155233,"spinlock_wait_cycles":52835584,"thread_start_us":748619,"threads_started":2,"wal-append.queue_time_us":27996}
I20260709 13:03:42.387202  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:42.450058  1565 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:42.450858  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:42.451170  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:42.438447  2313 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:42.452067  1565 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:42.452240  1717 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:42.452919  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:42.444912  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 260069us
I20260709 13:03:42.282049  5547 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.457049  5547 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:42.460157  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:42.461076  1564 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:42.320210  5353 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:42.466257  2348 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:42.443922  1538 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:42.472813  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 78028.2us
I20260709 13:03:42.440718  1568 raft_consensus.cc:2393] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:42.465057  1573 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:42.406054  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:42.475729  1566 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:42.335212  5637 consensus_meta.cc:287] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.891s	user 0.011s	sys 0.003s
I20260709 13:03:42.479887  1564 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:42.483515  1565 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:42.486095  1716 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:42.486768  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:42.487614  1572 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:42.296139  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 66002.7us
I20260709 13:03:42.489157  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:42.490645  1569 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:42.281658  5473 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:42.459946  5413 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:42.330526  5475 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:42.281234  5657 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:42.487459  1538 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:42.494999  1568 raft_consensus.cc:1275] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:03:42.504930  1568 pending_rounds.cc:85] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 4
I20260709 13:03:42.505259  1568 pending_rounds.cc:107] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:03:42.505606  1568 raft_consensus.cc:2889] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:42.502337  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 63973.6us
I20260709 13:03:42.327852  5528 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:42.332930  5636 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:42.325251  5646 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.324249  5498 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:42.511106  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 54596.4us
I20260709 13:03:42.533039  1566 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
W20260709 13:03:42.542824  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 55463us
I20260709 13:03:42.557786  1566 raft_consensus.cc:1275] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:03:42.548233  5551 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:42.557921  1538 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:42.562945  1537 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:42.574891  1538 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:42.493340  5654 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.331620  5592 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:42.588300  1537 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:42.588093  5692 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:42.596894  1571 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:42.534874  5606 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:42.493202  5670 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:42.461412  5586 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:42.555985  5498 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.326792  5186 consensus_queue.cc:260] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:42.603111  1537 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:42.608903  5686 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:42.568548  5672 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:42.614135  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 111144us
I20260709 13:03:42.640265  1611 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:42.613401  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 290785us
I20260709 13:03:42.565559  5551 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.617609  1569 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:03:42.555173  5353 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:42.326215  5457 consensus_queue.cc:260] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:42.493238  5637 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:42.623147  5498 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:42.582350  2314 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:42.581717  5668 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:42.642215  1612 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:42.649848  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.654394  1572 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:42.469906  5661 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:42.447163  1715 rpcz_store.cc:276] 0709 13:03:38.271109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.419966 (+3148857us) service_pool.cc:224] Handling call
0709 13:03:41.420295 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.420523 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:42.151035 (+730512us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b1800445540. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f824c87d3c7 00007f8248e8f9a7 00007f8248e9f0bf
0709 13:03:42.159742 (+  8707us) log.cc:844] Serialized 14 byte log entry
0709 13:03:42.412190 (+252448us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:42.412409 (+   219us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:42.412602 (+   193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.412740 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:42.412956 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":409682,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":2710,"raft.queue_time_us":187679,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":21195,"spinlock_wait_cycles":32326912,"thread_start_us":483597,"threads_started":2,"wal-append.queue_time_us":119640}
I20260709 13:03:42.601735  5692 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:42.600505  5590 consensus_queue.cc:1059] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:03:42.637895  5686 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:42.654888  5671 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.385s	user 0.004s	sys 0.005s
W20260709 13:03:42.669430  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 87236.5us
I20260709 13:03:42.654111  5653 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:42.682622  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:42.643496  5547 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:42.647006  5551 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:42.620602  5570 consensus_queue.cc:1059] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:03:42.562747  5646 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:42.681943  5481 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:42.661545  5353 raft_consensus.cc:697] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:42.646266  5550 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:03:42.611889  5654 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:42.703644  5565 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:42.704067  5646 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:42.704344  5654 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:42.690680  5689 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:42.705133  5689 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.706001  5535 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:42.706727  5465 raft_consensus.cc:3055] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:42.707712  5465 raft_consensus.cc:740] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:42.708598  5465 consensus_queue.cc:260] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:42.701481  5641 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:42.710413  5641 raft_consensus.cc:697] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:42.711771  5641 consensus_queue.cc:237] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:42.713382  5413 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:42.681455  5686 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:42.714988  5637 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:42.715754  5552 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:42.700271  5687 consensus_queue.cc:1059] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:03:42.717341  5592 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:42.677977  5692 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:42.720654  5661 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:42.704098  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 257598us
W20260709 13:03:42.724083  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.676633  5636 consensus_queue.cc:1059] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:03:42.683954  5653 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:42.727264  5653 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:42.705487  5689 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:42.729892  5481 raft_consensus.cc:697] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:42.730615  5657 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:42.646782  5672 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:42.730893  5481 consensus_queue.cc:237] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:42.733106  5465 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:42.733572  2312 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:42.736162  5646 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:42.736543  5646 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.736897  5646 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:42.738046  5678 raft_consensus.cc:3055] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:42.738581  5678 raft_consensus.cc:740] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:42.739168  2312 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:42.739494  5678 consensus_queue.cc:260] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:42.741165  5678 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:42.746183  5646 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:42.747512  5457 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:42.748459  5646 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:42.749606  5643 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:42.717387  5519 raft_consensus.cc:3055] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:42.750690  5643 raft_consensus.cc:697] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:42.750793  5519 raft_consensus.cc:740] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:42.728952  5586 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:42.745297  5186 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:42.721796  5353 consensus_queue.cc:237] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:42.752617  5643 consensus_queue.cc:237] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:42.757982  5590 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:42.762969  5672 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:42.764576  5519 consensus_queue.cc:260] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:42.770200  5519 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:42.770262  5692 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:42.779480  5692 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:42.782660  5551 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:42.785338  5551 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:42.785599  5686 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:42.787959  5686 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:42.756181  5653 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:42.834939  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 259558us
I20260709 13:03:42.935597  1538 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:42.655645  1573 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:03:42.841845  1642 raft_consensus.cc:1240] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: [7.3-9.4]
I20260709 13:03:42.843420  1643 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:42.659694  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 1958) took 1421 ms. Trace:
W20260709 13:03:42.846351  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 175347us
I20260709 13:03:42.938161  1648 rpcz_store.cc:276] 0709 13:03:41.326109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.483324 (+157215us) service_pool.cc:224] Handling call
0709 13:03:42.659380 (+1176056us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54000a5ad4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:42.659603 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2302846976}
I20260709 13:03:42.939392  1612 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:42.849251  1572 raft_consensus.cc:1240] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:42.941123  5657 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:42.942340  5535 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:42.938215  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:42.942930  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:42.943370  5535 raft_consensus.cc:697] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:42.943703  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 340532us
I20260709 13:03:42.944334  1572 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:42.897253  5592 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:42.943841  1573 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
W20260709 13:03:42.872048  1685 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.665s (SENT) [suppressed 52 similar messages]
W20260709 13:03:42.947163  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 75477.1us
W20260709 13:03:42.932034  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.785336  5689 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:42.954078  5678 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.956135  5678 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:42.960845  5653 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:42.956738  1722 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.006s	sys 0.022s
I20260709 13:03:42.785732  5498 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:42.941476  1612 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:42.976049  1722 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.007s	sys 0.022s
I20260709 13:03:42.993357  2312 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:42.998198  5481 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:42.999032  5481 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.999668  5481 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:43.004870  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:43.006150  2348 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:42.997895  5551 consensus_queue.cc:1048] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
W20260709 13:03:43.008776  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.015864  1572 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:03:43.021953  5597 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.023087  5597 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:43.024013  5498 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:43.025904  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.976528  5535 consensus_queue.cc:237] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:43.029628  5528 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:43.029954  5586 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.031272  5586 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:43.033341  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 1974) took 1334 ms. Trace:
I20260709 13:03:43.033814  5353 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:43.034495  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 99416.5us
I20260709 13:03:43.034601  5353 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:43.035768  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 90235.8us
I20260709 13:03:43.036919  5661 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:43.037746  5592 consensus_queue.cc:1059] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:43.039079  5592 raft_consensus.cc:3055] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:43.039527  5592 raft_consensus.cc:740] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:43.042061  5657 raft_consensus.cc:697] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:43.044041  5657 consensus_queue.cc:237] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:43.037200  1538 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:43.055892  5653 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:43.033826  1642 rpcz_store.cc:276] 0709 13:03:41.699317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.699546 (+   229us) service_pool.cc:224] Handling call
0709 13:03:41.700065 (+   519us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:42.937124 (+1237059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.032972 (+ 95848us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54000a73d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:43.033228 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2623370112}
I20260709 13:03:43.061069  1564 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:43.061832  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:43.052548  5592 consensus_queue.cc:260] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:43.060654  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet eca30b43c26f420c965ee25f475581a0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:43.063898  5592 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:43.039628  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 383159us
I20260709 13:03:43.037204  1537 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:43.084081  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 142941us
I20260709 13:03:43.105052  2348 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:03:43.047969  5692 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:43.111104  5498 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:43.057394  5653 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:43.119069  5689 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:43.114271  1687 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.000s	sys 0.016s
I20260709 13:03:43.125049  1572 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 15. Current term is 16. Ops: []
W20260709 13:03:43.138770  1687 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.000s	sys 0.017s
W20260709 13:03:43.154633  1719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.022s	sys 0.025s
W20260709 13:03:43.155165  1719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.022s	sys 0.025s
W20260709 13:03:43.162739  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.164427  5413 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:43.165319  5413 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:43.167084  5353 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:43.182292  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 54214.4us
I20260709 13:03:43.202783  5570 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.076s
I20260709 13:03:43.209065  5497 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:43.210249  5606 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.211364  5606 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:43.214965  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:43.218742  5586 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:43.237103  5661 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:43.242156  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 205198us
I20260709 13:03:43.244112  5597 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:43.262660  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 47906.5us
W20260709 13:03:43.271617  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 70004.3us
W20260709 13:03:43.245991  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 952339us
I20260709 13:03:43.276528   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:43.235179  5413 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:43.291203  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 72580.9us
I20260709 13:03:43.292208  1612 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:43.283210  5497 raft_consensus.cc:697] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:43.205128  5528 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 13:03:43.314918  5497 consensus_queue.cc:237] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:43.319576  5713 log.cc:927] Time spent T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:03:43.308355  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:43.309798  5186 consensus_meta.cc:287] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.535s	user 0.010s	sys 0.009s
I20260709 13:03:43.300354  5678 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:43.328604  5606 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:43.315277  5653 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:43.322357  5551 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:03:43.306316  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 43550us
W20260709 13:03:43.323185  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 68696.2us
I20260709 13:03:43.336604  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2047) took 1009 ms. Trace:
I20260709 13:03:43.303004  5597 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:43.341883  5551 raft_consensus.cc:697] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:43.346107  1611 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:43.348188  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.372923  5597 raft_consensus.cc:697] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:43.379133  5551 consensus_queue.cc:237] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:43.379851  5597 consensus_queue.cc:237] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:43.337213  1569 rpcz_store.cc:276] 0709 13:03:42.321201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.644348 (+323147us) service_pool.cc:224] Handling call
0709 13:03:42.874726 (+230378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:43.013975 (+139249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.014007 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:43.014603 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 13:03:43.099930 (+ 85327us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:43.100275 (+   345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.100563 (+   288us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.330598 (+230035us) raft_consensus.cc:1685] finished
0709 13:03:43.330618 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.330785 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":138154,"raft.queue_time_us":151187,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":263,"thread_start_us":84783,"threads_started":1,"wal-append.queue_time_us":122997}
I20260709 13:03:43.384769  1537 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:43.386884  5481 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:43.389899  5570 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.390707  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2043) took 1070 ms. Trace:
W20260709 13:03:43.396384  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 50733.5us
W20260709 13:03:43.407271  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 66600.6us
I20260709 13:03:43.382512  5689 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.418370  5707 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.411947  1612 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:43.422652  5606 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:43.423595  5186 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:43.423816  5606 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:43.424707  5457 consensus_queue.cc:1059] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:43.425598  5457 consensus_queue.cc:1243] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:03:43.426220  5653 raft_consensus.cc:3055] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:43.426831  5653 raft_consensus.cc:740] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:43.429051  5601 raft_consensus.cc:3055] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:43.428501  5653 consensus_queue.cc:260] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:43.410755  5535 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:43.424417  5186 raft_consensus.cc:697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:43.419425  5707 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:43.431663  5570 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:43.431892  5186 consensus_queue.cc:237] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:43.433037  5707 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:43.435207  5686 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:43.429572  5601 raft_consensus.cc:740] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:43.434587  5653 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:43.437058  1611 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:43.443701  5716 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.098s	user 0.002s	sys 0.000s
I20260709 13:03:43.445390  5481 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:43.446890  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2043) took 1183 ms. Trace:
I20260709 13:03:43.447289  1719 rpcz_store.cc:276] 0709 13:03:42.263767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.268943 (+  5176us) service_pool.cc:224] Handling call
0709 13:03:42.269228 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:43.155853 (+886625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.155908 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:43.156751 (+   843us) log.cc:844] Serialized 26 byte log entry
0709 13:03:43.261456 (+104705us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:43.261521 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.284612 (+ 23091us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.442871 (+158259us) spinlock_profiling.cc:243] Waited 38.9 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:43.446578 (+  3707us) raft_consensus.cc:1685] finished
0709 13:03:43.446602 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.446807 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":150843,"raft.queue_time_us":148270,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":17245,"spinlock_wait_cycles":89483648,"thread_start_us":845939,"threads_started":2,"wal-append.queue_time_us":187672}
I20260709 13:03:43.438755  5601 consensus_queue.cc:260] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:43.454530  5519 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.683s	user 0.020s	sys 0.009s
I20260709 13:03:43.457561  5601 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:43.399168  1568 rpcz_store.cc:276] 0709 13:03:42.320019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.506842 (+186823us) service_pool.cc:224] Handling call
0709 13:03:42.563360 (+ 56518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.563624 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:42.885231 (+321607us) log.cc:844] Serialized 14 byte log entry
0709 13:03:43.333022 (+447791us) log.cc:844] Serialized 14 byte log entry
0709 13:03:43.338142 (+  5120us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:43.368073 (+ 29931us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:43.390230 (+ 22157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.390346 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.390492 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5094,"prepare.queue_time_us":157944,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":1384,"raft.queue_time_us":67556,"raft.run_cpu_time_us":421,"raft.run_wall_time_us":22439,"spinlock_wait_cycles":12672,"thread_start_us":514349,"threads_started":2,"wal-append.queue_time_us":482773}
W20260709 13:03:43.475435  1685 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.010s (SENT)
I20260709 13:03:43.477149  5707 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:43.485601  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:43.481734  1685 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:43.436340  5570 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:43.497303  1567 raft_consensus.cc:3055] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:43.438937  5689 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:43.500384  1570 raft_consensus.cc:3055] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:43.488057  5606 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:43.502648  5715 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.156s	user 0.004s	sys 0.000s
I20260709 13:03:43.503125  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2047) took 1129 ms. Trace:
I20260709 13:03:43.503258  5413 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:43.503675  1571 consensus_meta.cc:287] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.873s	user 0.014s	sys 0.007s
W20260709 13:03:43.508879  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 55527.8us
W20260709 13:03:43.510613  5708 log.cc:927] Time spent T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.161s	user 0.003s	sys 0.000s
I20260709 13:03:43.512326  1564 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:03:43.514364  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:43.497838  1567 raft_consensus.cc:740] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:03:43.521901  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 149603us
I20260709 13:03:43.483019  5678 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.529886  1572 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:03:43.502358  5661 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:43.531435  1570 raft_consensus.cc:740] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:43.541210  5707 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:43.545441  5689 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:43.546941  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:43.549942  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.558919  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 611) took 1112 ms. Trace:
I20260709 13:03:43.548703  1571 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:43.537694  1718 rpcz_store.cc:276] 0709 13:03:42.264825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.436211 (+171386us) service_pool.cc:224] Handling call
0709 13:03:42.436643 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:42.449787 (+ 13144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.449823 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:42.464478 (+ 14655us) log.cc:844] Serialized 26 byte log entry
0709 13:03:42.998091 (+533613us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:42.998197 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.998336 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.372244 (+373908us) raft_consensus.cc:1685] finished
0709 13:03:43.372326 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.394522 (+ 22196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":48543,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":4058,"thread_start_us":368433,"threads_started":1,"wal-append.queue_time_us":552171}
I20260709 13:03:43.557578  5707 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:43.563211  1573 rpcz_store.cc:276] 0709 13:03:42.446117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.992621 (+546504us) service_pool.cc:224] Handling call
0709 13:03:43.007819 (+ 15198us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:43.046821 (+ 39002us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.046862 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:43.047521 (+   659us) log.cc:844] Serialized 26 byte log entry
0709 13:03:43.202319 (+154798us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:43.202387 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.202511 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.477569 (+275058us) spinlock_profiling.cc:243] Waited 37.6 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:43.549285 (+ 71716us) raft_consensus.cc:1685] finished
0709 13:03:43.549315 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.558828 (+  9513us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86568704,"thread_start_us":150414,"threads_started":1,"wal-append.queue_time_us":298646}
I20260709 13:03:43.577402  5689 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:43.572095  5481 consensus_queue.cc:1059] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:03:43.583643  5678 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:43.619069  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:43.626574  1718 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:43.606254  5535 raft_consensus.cc:697] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:43.611853  1567 consensus_queue.cc:260] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:43.617967  5497 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.620602  1570 consensus_queue.cc:260] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:43.644948  1570 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:43.641815  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 216382us
I20260709 13:03:43.648561  1716 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:43.648574  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:43.644850  1571 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2022) took 1631 ms (client timeout 1542 ms). Trace:
W20260709 13:03:43.657533  1571 rpcz_store.cc:269] 0709 13:03:42.012005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.012165 (+   160us) service_pool.cc:224] Handling call
0709 13:03:43.632578 (+1620413us) spinlock_profiling.cc:243] Waited 468 ms on lock 0x7b54000f73d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:43.641970 (+  9392us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1075404160}
I20260709 13:03:43.646312  5497 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:43.668454  5678 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:43.672646  5590 raft_consensus.cc:3055] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:43.674357  5535 consensus_queue.cc:237] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:43.678404  1537 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:43.680668  1537 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:43.681700  5528 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:03:43.681825  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 245187us
I20260709 13:03:43.682642  5528 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:43.683593  1716 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:43.689982  1567 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:43.673480  5590 raft_consensus.cc:740] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:43.699523  5590 consensus_queue.cc:260] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:43.701308  1611 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:43.704668  5689 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:43.705253  5689 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:43.705796  5689 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:43.708545  1720 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:03:43.713203  5718 log.cc:927] Time spent T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:03:43.714519  5551 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:43.719323  1611 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:43.721405  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:43.723981  1715 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:43.683450  1687 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:43.724553  1568 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:43.722357  2312 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:03:43.725131  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.724965  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:43.735353  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 52320.6us
I20260709 13:03:43.736549  5481 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:43.736688  1687 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:43.737177  5481 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.737619  5481 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:43.738380  5707 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:43.738950  5707 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:43.739349  5707 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:43.740783  5678 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:43.741255  5678 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:43.741712  5678 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:43.764303  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.724684  5590 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:43.771973  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 45459.8us
W20260709 13:03:43.784025  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 82491.2us
I20260709 13:03:43.785065  1537 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:43.786942  1537 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:43.790364  5657 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:43.791394  5657 raft_consensus.cc:697] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:43.791813  1687 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:43.793695  5413 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:43.794253  5413 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.794840  5413 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:43.796572  5186 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.797278  5186 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:43.746676  5498 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:03:43.766309  5497 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:43.744204  5653 consensus_queue.cc:1048] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
W20260709 13:03:43.821621  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 102492us
I20260709 13:03:43.835078  1715 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:03:43.828805  1570 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
W20260709 13:03:43.838218  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 48618.5us
I20260709 13:03:43.841049  5413 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:43.847599  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:43.780920  5661 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:43.855878  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.781325  5592 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:43.790391  5528 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:03:43.872577  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 44641us
I20260709 13:03:43.857951  1570 pending_rounds.cc:85] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 12
W20260709 13:03:43.874650  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 139371us
I20260709 13:03:43.882493  1570 pending_rounds.cc:107] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:03:43.886312  5601 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:43.887936  5601 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:43.888425  5601 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:43.890684  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 82912.6us
I20260709 13:03:43.903558  1570 raft_consensus.cc:2889] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:43.904940  5186 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:43.914001  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:43.915091  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.893378  1687 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:43.851655  5413 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:43.851492  5722 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:43.793299  5657 consensus_queue.cc:237] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:43.937794  5728 consensus_queue.cc:1059] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:43.940064  1568 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:03:43.929948  5606 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.942452  5661 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:43.883626  5528 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:43.962107  5186 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:43.962774  5725 log.cc:927] Time spent T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:03:43.963361  5353 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:43.971063  5606 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:43.973860  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 127073us
I20260709 13:03:43.948166  5413 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:43.975172  1537 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:43.977583  1537 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:43.933820  5597 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:43.993165  5570 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:43.989415  5686 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:43.959211  5528 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:43.977875  5592 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:44.010231  1718 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:03:44.016077  2312 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:44.028052  1468 catalog_manager.cc:5697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:44.030436  5686 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:44.041252  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 148024us
I20260709 13:03:43.995287  5481 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:44.042814  5498 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:43.962993  5186 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:44.066893  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 89775.8us
I20260709 13:03:44.045961  5592 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.074797  1537 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:44.075215  5592 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:44.008494  5597 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:44.076681  1537 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:44.078581  1537 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:44.079180  5597 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:44.081882  5570 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:44.083003  5570 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:44.083806  5570 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:44.084457  5186 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:44.086504  5570 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:03:44.108621  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 59d236b930214b1f91fb55ece6fecd68 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:03:44.119155  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 52125us
I20260709 13:03:44.144687  1567 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
W20260709 13:03:44.151808  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 158205us
I20260709 13:03:44.145233  1567 pending_rounds.cc:85] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 7
I20260709 13:03:44.163858  1567 pending_rounds.cc:107] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:03:44.164182  1567 raft_consensus.cc:2889] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:44.111012  5606 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:44.144491  5498 raft_consensus.cc:697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 14 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:44.088773  5707 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:44.197535  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:44.220737  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 68933us
I20260709 13:03:44.203158  5570 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:44.221683  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 287882us
I20260709 13:03:44.210484  5597 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:03:44.233856  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.218335  5413 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.243142  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:44.244305  1569 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:44.244755  5597 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:44.247231  5597 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:44.249224  5707 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:44.249832  5498 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:44.234992  5413 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:44.260578  5413 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:44.268524  5729 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
W20260709 13:03:44.270316  5731 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
W20260709 13:03:44.272063  5742 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:03:44.272647  5481 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:44.274700  5737 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.275462  5737 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:44.276898  1642 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:03:44.278301  5737 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:44.278904  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.279784  1643 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:44.252668  5601 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:44.289393  1537 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:44.289896  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 94670.1us
I20260709 13:03:44.240485  5661 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:44.314471  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.314077  5747 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:44.288089  5678 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:44.318739  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:44.323135  1643 raft_consensus.cc:2410] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:44.323724  1642 raft_consensus.cc:2410] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
W20260709 13:03:44.324116  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 223422us
W20260709 13:03:44.336045  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.349110  1713 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:44.305336  5570 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:44.355666  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.356513  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.356653  1639 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:44.357443  1645 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
W20260709 13:03:44.358877  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.352108  5686 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:44.365098  2313 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:44.366325  5498 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:44.366986  5733 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:44.367882  5498 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:44.370227  5678 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:44.373481  5353 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:44.374274  5353 raft_consensus.cc:697] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:44.375465  5353 consensus_queue.cc:237] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:44.375733  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:44.376662  1713 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:44.312706  5528 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:44.374301  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:44.385016  1719 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:44.379294  5601 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:44.383577  1722 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:03:44.392357  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 103074us
I20260709 13:03:44.344748  5606 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:44.349310  5737 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.416168  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.417011  1645 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:44.418072  1537 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:44.420981  5601 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:44.422883  2313 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:44.417408  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:44.426235  1718 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:44.431279  1713 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:44.438843  1719 raft_consensus.cc:2410] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:03:44.440066  1569 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
I20260709 13:03:44.384155  5728 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.446851  1537 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:44.458956  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.460198  1570 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:44.463124  1645 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 6.
W20260709 13:03:44.481048  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:44.480507  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 57186.8us
I20260709 13:03:44.486176  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.438599  5498 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
W20260709 13:03:44.490553  5750 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:03:44.442806  5601 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:03:44.508996  1718 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:03:44.440670  5722 raft_consensus.cc:697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:44.425384  5606 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:44.521580  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.519618  2312 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:03:44.526875  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.464985  5737 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:44.511917  5657 raft_consensus.cc:2764] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:03:44.554631  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.528782  5601 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:44.484903  5678 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:44.561751  1715 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:44.582098  1685 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.569s (SENT)
I20260709 13:03:44.474089  5728 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:44.521250  5498 consensus_queue.cc:1059] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
W20260709 13:03:44.559162  5689 consensus_meta.cc:287] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.853s	user 0.017s	sys 0.002s
I20260709 13:03:44.583153  1645 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
W20260709 13:03:44.593576  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 124397us
I20260709 13:03:44.590939  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.581034  5761 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:44.623526  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:44.624675  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 185096us
I20260709 13:03:44.600497  5528 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:44.607342  5481 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:44.638366  5551 raft_consensus.cc:3055] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:44.638871  5551 raft_consensus.cc:740] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:44.639963  5551 consensus_queue.cc:260] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:44.641148  5551 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:44.642174  1645 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:44.642903  5737 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:44.643754  5756 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:03:44.644181  5759 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:03:44.647032  5606 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:44.649847  5724 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.650779  5724 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:44.654737  5592 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:44.590821  5601 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:44.664635  5761 raft_consensus.cc:697] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:44.667268  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:44.668735  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.668447  5761 consensus_queue.cc:237] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:44.657459  5751 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.210s	user 0.001s	sys 0.001s
I20260709 13:03:44.599831  5689 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:44.653410  5724 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:44.690707  1538 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.555s (SENT)
W20260709 13:03:44.781994  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 91349.2us
W20260709 13:03:44.783176  1538 leader_election.cc:336] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.556s (SENT)
I20260709 13:03:44.783640  1538 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:44.695744  1719 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 13:03:44.784674  5592 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:44.785264  1719 pending_rounds.cc:85] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 4
I20260709 13:03:44.785748  1719 pending_rounds.cc:107] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:03:44.786026  1719 raft_consensus.cc:2889] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:44.787432  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:44.788141  1719 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:44.696130  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:44.790804  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:44.792337  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:44.792603  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.793002  1713 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:44.697924  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:44.795537  1715 raft_consensus.cc:2410] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:44.796926  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:44.797787  2341 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:44.799226  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:44.698280  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:44.699074  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:44.800516  1720 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:44.800985  1722 raft_consensus.cc:2393] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:44.700309  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:44.671728  5606 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.802145  5606 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:44.803197  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.804402  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:44.702740  1717 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:44.805055  1713 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:44.805986  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:44.705265  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:44.814544  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.716948  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:44.715312  2348 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:44.814670  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:44.717842  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 144048us
I20260709 13:03:44.721007  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:44.727228  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.728102  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.728168  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:44.729432  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.729661  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:44.730088  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.730214  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.730964  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.731177  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.629566  5763 consensus_queue.cc:1059] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.112s
I20260709 13:03:44.626463  5678 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:44.736124  5570 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:44.823482  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.740583  5747 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.738052  5689 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:44.634913  5722 consensus_queue.cc:237] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:44.751834  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:44.766038  5481 raft_consensus.cc:697] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:03:44.766350  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49574.8us
I20260709 13:03:44.772811  1646 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:44.843390  1646 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:44.806186  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:44.848969  1717 raft_consensus.cc:1275] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:03:44.703804  5757 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:03:44.852398  1717 raft_consensus.cc:1275] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:44.811199  1567 raft_consensus.cc:1275] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:03:44.811643  1571 raft_consensus.cc:1275] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:03:44.789445  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:44.857026  1719 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 9: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:44.860214  1719 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:44.812215  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:44.814525  1713 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:44.720243  5728 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:44.823024  1640 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:44.862475  1565 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:44.823482  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:44.864132  1713 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:44.865284  1713 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:44.823933  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.866376  1647 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:44.824299  1641 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:44.823896  1639 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:44.825393  1568 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 10: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:44.743851  5601 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:44.834645  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 233969us
W20260709 13:03:44.839425  1538 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.645s (SENT)
I20260709 13:03:44.805006  5606 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:44.853785  5481 consensus_queue.cc:237] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:44.870190  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.892552  1538 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:44.891048  1644 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:44.904601  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:44.908819  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:44.909019  2313 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:44.909845  1714 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:44.867648  5737 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.906615  5601 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:44.907073  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:44.926229  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:44.931231  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 106193us
I20260709 13:03:44.918772  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:44.914443  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:44.911500  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:44.918059  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:44.917769  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.899528  5606 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:44.914090  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:44.944828  1648 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:44.945803  1722 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 9: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:44.951141  1565 raft_consensus.cc:2368] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10.
W20260709 13:03:44.953564  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.919258  5769 raft_consensus.cc:3055] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:44.958868  1566 raft_consensus.cc:2368] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10.
I20260709 13:03:44.961432  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:44.872079  5747 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:44.965075  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:44.957751  5606 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:44.966270  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:44.981315  1720 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:44.966761  5592 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.986564  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:44.987288  1569 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:44.967597  5766 raft_consensus.cc:3055] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:44.970155  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:44.970026  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:44.927325  5764 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:03:44.911221  5689 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:44.949589  5737 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:44.975925  1641 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:44.973603  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:44.992241  1715 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
W20260709 13:03:44.986588  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 77673.5us
I20260709 13:03:45.000259  1721 raft_consensus.cc:2410] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:03:45.000519  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:44.999137  5763 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:45.003328  1715 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:45.003458  1641 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:45.004650  1642 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:45.004813  1715 raft_consensus.cc:1275] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:45.006105  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:45.007195  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:45.007673  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:45.008383  1640 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 5.
I20260709 13:03:44.994709  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:45.009037  1646 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:45.009438  1716 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:45.010107  5689 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:45.011157  5606 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:45.011284  1644 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 12.
I20260709 13:03:45.012316  5606 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.012814  5606 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:45.014218  5737 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:45.014894  5592 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:45.017083  5592 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:44.960417  5769 raft_consensus.cc:740] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:45.006234  1642 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:03:44.996569  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:44.993718  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:44.992221  1718 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:45.034989  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:45.039171  1565 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 16: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:44.991796  5766 raft_consensus.cc:740] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:44.995275  5678 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:45.052934  1566 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:45.054950  1648 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:45.055054  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.076875  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.078027  1568 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:45.083340  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:45.012316  5689 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:45.084512  1643 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:03:45.085302  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:45.096118  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.098291  2345 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:03:45.102449  1687 leader_election.cc:336] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.820s (SENT)
I20260709 13:03:45.098702  1648 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:03:45.103260  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:45.104269  1642 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:45.095394  1643 pending_rounds.cc:85] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 6
I20260709 13:03:45.109107  1643 pending_rounds.cc:107] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:03:45.109408  1643 raft_consensus.cc:2889] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:45.112968  2343 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:03:45.116848  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:45.117780  2349 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:03:45.120146  1568 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:03:45.120909  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.123162  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:45.110602  1685 leader_election.cc:336] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.665s (SENT)
W20260709 13:03:45.135723  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.140772  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.144562  1568 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:03:45.186162  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 55589.4us
I20260709 13:03:45.188762  1565 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
W20260709 13:03:45.195012  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.095676  5678 consensus_queue.cc:1059] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:03:45.191606  1687 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:45.228850  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:45.231487  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 120914us
I20260709 13:03:45.231870  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:45.185936  5689 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:45.235934  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 45831.7us
I20260709 13:03:45.241562  1612 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:45.242015  1685 leader_election.cc:336] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.723s (SENT)
I20260709 13:03:45.136997  5737 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 13:03:45.242554  1685 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:45.241763  1687 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:45.268071  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 302096us
I20260709 13:03:45.269724  5606 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:45.269882  5771 log.cc:927] Time spent T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:03:45.272790  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.273859  1564 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:03:45.274158  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.275272  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.276643  1564 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:45.268288  5724 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:45.279862  1564 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:45.283093  1648 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:03:45.286070  1566 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:45.287053  1565 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:03:45.290408  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:45.298579  5747 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:45.299510  5776 raft_consensus.cc:3055] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:45.300046  5776 raft_consensus.cc:740] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:03:45.301344  5772 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:03:45.302047  5678 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:45.302541  5678 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:45.302974  5678 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:45.226914  5766 consensus_queue.cc:260] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:45.241397  5689 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:45.310621  5776 consensus_queue.cc:260] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:45.311718  5776 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:45.310621  5777 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:45.312503  5689 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:45.264220  1687 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:45.320019  1714 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
W20260709 13:03:45.323535  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.330665  1648 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:45.325194  1642 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6.
W20260709 13:03:45.326956  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 85533.4us
I20260709 13:03:45.336845  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:45.354362  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 90212.7us
I20260709 13:03:45.355567  1687 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:45.356740  5779 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:45.319535  5606 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:45.363253  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 130786us
I20260709 13:03:45.357357  5779 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.369027  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.367699  5779 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:45.418658  1569 raft_consensus.cc:1240] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:45.420905  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:45.421747  1569 raft_consensus.cc:2368] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 5. Current term is 6.
W20260709 13:03:45.422151  5780 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.099s	user 0.000s	sys 0.003s
I20260709 13:03:45.422600  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.423811  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.424620  1569 raft_consensus.cc:2368] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 13. Current term is 14.
W20260709 13:03:45.427325  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 62235.1us
I20260709 13:03:45.361141  5606 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:45.436774  1685 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.787s (SENT)
I20260709 13:03:45.356758  5781 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:45.437414  1685 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:45.523684  1569 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:45.533716  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:45.558262  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 121533us
I20260709 13:03:45.425836  5592 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.561094  1569 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:45.521476  5781 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.531857  5606 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:45.595484  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:45.597131  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.635102  1565 raft_consensus.cc:2393] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:45.599767  1573 raft_consensus.cc:1240] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: [7.3-9.4]
I20260709 13:03:45.600620  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:45.605786  5592 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:45.610100  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.620839  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:45.622057  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:45.642935  1642 raft_consensus.cc:2393] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 8: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:45.622973  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:45.623667  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:45.643316  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:45.624315  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:45.624926  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:45.649964  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.532084  5737 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:03:45.634449  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.638392  1639 raft_consensus.cc:2410] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:03:45.641088  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.664781  1565 raft_consensus.cc:2368] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10.
I20260709 13:03:45.626713  5776 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:45.657158  1569 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:03:45.672199  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.676127  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:45.676923  1646 raft_consensus.cc:2368] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14.
I20260709 13:03:45.676923  1644 raft_consensus.cc:2393] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:45.684408  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.700260  1640 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:45.700618  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.704685  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.634459  5781 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:45.706796  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.708840  1648 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:45.709468  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.715742  1565 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:45.718727  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.721506  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.723184  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:45.726650  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.730192  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.743825  1567 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
W20260709 13:03:45.746709  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.754585  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.676759  5737 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:45.757520  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.758544  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.761006  2349 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:03:45.763026  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:45.763746  1640 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:45.769026  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:45.759744  1565 raft_consensus.cc:2410] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:03:45.770138  1640 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:45.771554  2345 raft_consensus.cc:1240] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:03:45.773460  1647 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:45.775583  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.776551  1573 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:45.781832  1639 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:03:45.782244  2345 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:45.781471  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.784214  1565 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:45.786404  2349 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:45.778015  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 420961us
I20260709 13:03:45.785354  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.745357  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.799535  1567 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:03:45.802448  1569 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:03:45.770005  5776 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.804009  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:45.817363  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:45.818550  1567 raft_consensus.cc:2376] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8. Re-sending same reply.
W20260709 13:03:45.819418  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 156144us
W20260709 13:03:45.820081  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:45.820756  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.821544  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:45.822525  1647 raft_consensus.cc:2410] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:03:45.789420  1687 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:45.804764  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.805745  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.810212  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.803215  1565 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 16: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:45.823218  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:45.834192  1639 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:45.836373  1569 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:45.837895  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:45.849916  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.851106  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.852320  1573 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:45.845441  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.853960  1567 raft_consensus.cc:2393] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 7: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:45.859586  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:45.862720  5592 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:45.863306  5769 consensus_queue.cc:260] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:45.864565  5787 log.cc:927] Time spent T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:03:45.865130  5769 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:45.865732  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 306651us
W20260709 13:03:45.872361  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 59d236b930214b1f91fb55ece6fecd68 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:45.876791  5766 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:45.822435   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:45.818066  5776 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:45.923481  1647 raft_consensus.cc:1275] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:45.930330  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:45.940392  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:45.944154  1685 leader_election.cc:336] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.789s (SENT)
I20260709 13:03:45.946774  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:45.981057  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:45.996177  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:46.011858  1611 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:46.020694  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:46.022848  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.033356  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:45.986238  5728 consensus_queue.cc:1048] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:03:46.054445  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:45.999501  5689 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:03:46.069977  1647 raft_consensus.cc:2393] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:46.055132  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:46.055883  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:46.039258  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 52092.2us
W20260709 13:03:46.084815  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 295650us
I20260709 13:03:46.096889  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:46.096974  1687 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:46.099994  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:46.101366  1611 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:46.103017  1569 raft_consensus.cc:2393] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:46.039395  5724 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:46.114488  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:46.059233  5795 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:46.034904  5590 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:03:46.055094  5777 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.120364  1611 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:46.047554  5737 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:46.129716  1572 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:46.129962  1567 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:46.130091  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:46.127707  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.141150  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:46.142877  1566 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:46.113257  5724 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.154003  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:46.127471  5795 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.163241  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:46.129781  5722 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:46.170464  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:46.171397  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:46.169929  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2078) took 1252 ms. Trace:
I20260709 13:03:46.157323  5724 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:46.198163  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.125461  5806 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:03:46.182796  1720 rpcz_store.cc:276] 0709 13:03:44.917308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.298242 (+380934us) service_pool.cc:224] Handling call
0709 13:03:45.298611 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:45.298864 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:45.645216 (+346352us) log.cc:844] Serialized 14 byte log entry
0709 13:03:46.149754 (+504538us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:46.149818 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:46.149907 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.149987 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.169851 (+ 19864us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10676,"prepare.queue_time_us":308900,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":235734,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":7224,"thread_start_us":839129,"threads_started":3,"wal-append.queue_time_us":406094}
W20260709 13:03:46.200724  1645 consensus_meta.cc:287] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.526s	user 0.014s	sys 0.002s
I20260709 13:03:46.205031  1645 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:46.180651  5722 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:46.207389  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.169157  5795 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:46.183394  5737 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:46.220283  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:46.143661  5777 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:46.232218  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:46.152614  5770 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:03:46.193922  5811 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:45.998757  1467 catalog_manager.cc:5697] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 3 to 14, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 14 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:46.233498  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:46.236061  1644 raft_consensus.cc:2410] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:03:46.217610  5776 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:46.216851  5722 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:46.212810  5689 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:46.232837  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:46.247510  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 114586us
W20260709 13:03:46.248502  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:46.251677  1468 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47518 (request call id 96) took 32359 ms (client timeout 15000 ms). Trace:
I20260709 13:03:46.257771  1687 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:46.252650  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:46.261174  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 349316us
I20260709 13:03:46.234113  5808 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:03:46.261688  5817 consensus_queue.cc:1059] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:03:46.262509  1685 leader_election.cc:336] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.816s (SENT)
I20260709 13:03:46.262780  5817 raft_consensus.cc:3055] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:46.263048  1685 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:46.263506  5817 raft_consensus.cc:740] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:46.264163  5808 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:46.264379  1685 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.848s (SENT)
W20260709 13:03:46.269125  5799 log.cc:927] Time spent T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
W20260709 13:03:46.271520  5800 log.cc:927] Time spent T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.184s	user 0.000s	sys 0.002s
W20260709 13:03:46.271850  2314 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.519s (SENT)
W20260709 13:03:46.274737  2314 leader_election.cc:336] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.536s (SENT)
W20260709 13:03:46.275346  5805 log.cc:927] Time spent T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:03:46.276446  5802 log.cc:927] Time spent T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
W20260709 13:03:46.277534  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.229668  5767 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
W20260709 13:03:46.256121  1468 rpcz_store.cc:269] 0709 13:03:13.872244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.872781 (+   537us) service_pool.cc:224] Handling call
0709 13:03:46.230923 (+32358142us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:44.232371 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:44.233287 (+   916us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:44.233351 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:03:44.233401 (+    50us) tablet.cc:662] Decoding operations
0709 13:03:44.233966 (+   565us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:44.234052 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:44.234082 (+    30us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:03:44.234397 (+   315us) tablet.cc:701] Row locks acquired
0709 13:03:44.234446 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:03:44.234640 (+   194us) write_op.cc:270] Start()
0709 13:03:44.234813 (+   173us) write_op.cc:276] Timestamp: P: 1783602224234569 usec, L: 0
0709 13:03:44.234835 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:03:44.281605 (+ 46770us) log.cc:844] Serialized 536 byte log entry
0709 13:03:44.436432 (+154827us) op_driver.cc:464] REPLICATION: finished
0709 13:03:44.524087 (+ 87655us) write_op.cc:301] APPLY: starting
0709 13:03:44.550855 (+ 26768us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:44.722497 (+171642us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:44.722669 (+   172us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:44.723620 (+   951us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:44.723892 (+   272us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:03:44.723959 (+    67us) write_op.cc:312] APPLY: finished
0709 13:03:44.724204 (+   245us) log.cc:844] Serialized 25 byte log entry
0709 13:03:44.724373 (+   169us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:44.724634 (+   261us) write_op.cc:454] Released schema lock
0709 13:03:44.724675 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:44.725112 (+   437us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:44.725153 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1051376,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":38211,"apply.run_cpu_time_us":5137,"apply.run_wall_time_us":211306,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8309,"num_ops":1,"prepare.queue_time_us":156162,"prepare.run_cpu_time_us":4034,"prepare.run_wall_time_us":65774,"replication_time_us":201387,"thread_start_us":141334,"threads_started":2}]]}
I20260709 13:03:46.282400  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:46.285482  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 165182us
I20260709 13:03:46.286608  1611 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:46.288213  1567 raft_consensus.cc:2393] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:46.246685  5797 log.cc:927] Time spent T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.290s	user 0.001s	sys 0.001s
W20260709 13:03:46.290326  1685 leader_election.cc:336] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.945s (SENT)
I20260709 13:03:46.291999  1685 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:46.243791  5811 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.255136  5689 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.302987  1687 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:46.311321  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:46.312346  1567 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:46.315351  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:46.316358  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.316814  1720 raft_consensus.cc:2410] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:03:46.326771  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:46.327821  1721 raft_consensus.cc:1275] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:03:46.328392  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:46.328856  5777 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:46.330181  1715 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:46.329980  5764 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:03:46.331832  5606 consensus_meta.cc:287] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.670s	user 0.009s	sys 0.004s
W20260709 13:03:46.332509  5804 log.cc:927] Time spent T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.212s	user 0.001s	sys 0.001s
I20260709 13:03:46.332304  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:46.333513  5806 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:46.333990  5769 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:46.335034  5806 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:46.335582  5806 raft_consensus.cc:697] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:46.336448  5806 consensus_queue.cc:237] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:46.301870  5737 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:46.341921  5737 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.342306  5737 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:46.349722  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:46.351768  1687 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:46.354605  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:46.355666  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:46.352708  1687 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:46.300390  5811 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:46.350421  1722 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:46.364243  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:46.313468  5689 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:46.370046  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2438) took 1577 ms. Trace:
W20260709 13:03:46.373570  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 83277us
W20260709 13:03:46.351106  5678 consensus_meta.cc:287] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.048s	user 0.011s	sys 0.004s
I20260709 13:03:46.304632  5776 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:46.397678  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:46.398670  1714 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:46.376101  1567 raft_consensus.cc:2410] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:03:46.377203  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:46.378638  5798 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:46.379521  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:46.380384  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:46.380780  5722 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:46.364765  5777 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:46.383659  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:46.420455  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:46.385273  1470 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38662 (request call id 209) took 20533 ms (client timeout 15000 ms). Trace:
W20260709 13:03:46.386173  5791 log.cc:927] Time spent T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.501s	user 0.000s	sys 0.004s
I20260709 13:03:46.388863  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:46.389822  5678 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:46.369904  5779 consensus_meta.cc:287] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.956s	user 0.013s	sys 0.001s
I20260709 13:03:46.465619  5779 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:46.467101  5678 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:46.467231  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2071) took 1359 ms. Trace:
I20260709 13:03:46.467831  5678 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.390467  1467 catalog_manager.cc:5697] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:46.468814  5818 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:46.469483  5818 raft_consensus.cc:697] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:46.391945  5822 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:46.470616  5818 consensus_queue.cc:237] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:46.471002  5822 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:46.472513  1722 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:03:46.407545  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.294849  5813 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 13:03:46.394027  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:46.451399  1470 rpcz_store.cc:269] 0709 13:03:25.754746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.792531 (+ 37785us) service_pool.cc:224] Handling call
0709 13:03:46.253397 (+20460866us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:41.962846 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:42.043362 (+ 80516us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:42.043400 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:03:42.043420 (+    20us) tablet.cc:662] Decoding operations
0709 13:03:42.044811 (+  1391us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:42.044895 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:42.044921 (+    26us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:03:42.225464 (+180543us) tablet.cc:701] Row locks acquired
0709 13:03:42.272243 (+ 46779us) write_op.cc:260] PREPARE: finished
0709 13:03:42.333551 (+ 61308us) write_op.cc:270] Start()
0709 13:03:42.335389 (+  1838us) write_op.cc:276] Timestamp: P: 1783602222333494 usec, L: 0
0709 13:03:42.335415 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:03:42.374406 (+ 38991us) log.cc:844] Serialized 5246 byte log entry
0709 13:03:42.530871 (+156465us) op_driver.cc:464] REPLICATION: finished
0709 13:03:42.742181 (+211310us) write_op.cc:301] APPLY: starting
0709 13:03:42.742285 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:42.792211 (+ 49926us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:42.792513 (+   302us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:43.495438 (+702925us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:43.501332 (+  5894us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 13:03:43.501380 (+    48us) write_op.cc:312] APPLY: finished
0709 13:03:43.502069 (+   689us) log.cc:844] Serialized 121 byte log entry
0709 13:03:43.502462 (+   393us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:43.502919 (+   457us) write_op.cc:454] Released schema lock
0709 13:03:43.503017 (+    98us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:43.503789 (+   772us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:43.503829 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1119712,"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":210899,"apply.run_cpu_time_us":34587,"apply.run_wall_time_us":761545,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58493,"num_ops":13,"prepare.queue_time_us":415835,"prepare.run_cpu_time_us":47749,"prepare.run_wall_time_us":591884,"raft.queue_time_us":154431,"raft.run_cpu_time_us":27068,"raft.run_wall_time_us":165063,"replication_time_us":195098,"spinlock_wait_cycles":29568,"thread_start_us":720134,"threads_started":5,"wal-append.queue_time_us":151104,"wal-append.run_cpu_time_us":51046,"wal-append.run_wall_time_us":3207084}]]}
W20260709 13:03:46.473917  1538 leader_election.cc:336] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.568s (SENT)
I20260709 13:03:46.474356  1720 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
W20260709 13:03:46.474718  1538 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.975s (SENT)
I20260709 13:03:46.382397  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:46.475317  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.476189  2341 raft_consensus.cc:2393] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 5: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
W20260709 13:03:46.372123  5789 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.482s	user 0.004s	sys 0.000s
W20260709 13:03:46.389683  5796 log.cc:927] Time spent T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.251s	user 0.001s	sys 0.001s
I20260709 13:03:46.477217  5747 consensus_queue.cc:1059] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:03:46.421228  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 46938.2us
I20260709 13:03:46.478242  5760 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:03:46.420778  1721 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:46.478643  5592 consensus_queue.cc:1059] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:46.373692  1717 rpcz_store.cc:276] 0709 13:03:44.792484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.865012 (+ 72528us) service_pool.cc:224] Handling call
0709 13:03:44.865307 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:45.118242 (+252935us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:45.360064 (+241822us) log.cc:844] Serialized 14 byte log entry
0709 13:03:45.822465 (+462401us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:45.823148 (+   683us) log.cc:844] Serialized 26 byte log entry
0709 13:03:45.826493 (+  3345us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:45.838348 (+ 11855us) log.cc:844] Serialized 14 byte log entry
0709 13:03:46.120726 (+282378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.120800 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:46.369800 (+249000us) raft_consensus.cc:1685] finished
0709 13:03:46.369847 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.369989 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11563,"prepare.queue_time_us":499324,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":74,"raft.queue_time_us":362508,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":241,"thread_start_us":1009263,"threads_started":4,"wal-append.queue_time_us":636036}
W20260709 13:03:46.415941  5788 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.505s	user 0.005s	sys 0.000s
I20260709 13:03:46.464308  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:46.466297  5481 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:46.467718  1571 rpcz_store.cc:276] 0709 13:03:45.108204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.228881 (+120677us) service_pool.cc:224] Handling call
0709 13:03:45.229296 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:45.234925 (+  5629us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:45.234956 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:45.243536 (+  8580us) log.cc:844] Serialized 26 byte log entry
0709 13:03:45.312789 (+ 69253us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b480036ffd8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ddaeaa1 00007f824dda5b37 00007f824dda59ca 00007f824de9d26f 00007f824de9cf37 00007f824deb155c 00007f824deb1508 00007f824deb1488 00007f824deb1443 00007f824deb06ec 00007f824cddaa96
0709 13:03:45.663064 (+350275us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:45.663109 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.663192 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:46.194515 (+531323us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:46.466944 (+272429us) raft_consensus.cc:1685] finished
0709 13:03:46.466968 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.467133 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":360,"raft.run_cpu_time_us":2284,"raft.run_wall_time_us":76025,"spinlock_wait_cycles":33189504,"thread_start_us":364920,"threads_started":1,"wal-append.queue_time_us":552122}
I20260709 13:03:46.488186  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 1985) took 1299 ms. Trace:
I20260709 13:03:46.418689  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:46.411067  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:46.371898  5816 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:46.456386  1565 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:46.332278  5606 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:46.481756  1568 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:03:46.463740  5781 consensus_meta.cc:287] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.738s	user 0.012s	sys 0.001s
I20260709 13:03:46.504814  1572 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
W20260709 13:03:46.485401  1685 leader_election.cc:336] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.969s (SENT)
I20260709 13:03:46.488659  1570 rpcz_store.cc:276] 0709 13:03:45.189101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.275438 (+ 86337us) service_pool.cc:224] Handling call
0709 13:03:45.275897 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:45.276913 (+  1016us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:45.276950 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:45.277585 (+   635us) log.cc:844] Serialized 26 byte log entry
0709 13:03:45.635143 (+357558us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:45.635204 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.635300 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:46.181056 (+545756us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:46.479839 (+298783us) raft_consensus.cc:1685] finished
0709 13:03:46.479863 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.488028 (+  8165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":186,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":139,"spinlock_wait_cycles":41776384,"thread_start_us":354656,"threads_started":1,"wal-append.queue_time_us":536770}
I20260709 13:03:46.498041  1567 raft_consensus.cc:2410] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:03:46.508601  1685 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:46.473942  5813 consensus_queue.cc:1243] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:03:46.511793  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.485584  5777 raft_consensus.cc:697] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:03:46.523267  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 232635us
I20260709 13:03:46.499119  5816 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.497555  5822 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:46.468444  5678 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:46.534978  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49629.5us
I20260709 13:03:46.534994  1714 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 13:03:46.536913  1685 leader_election.cc:336] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.976s (SENT)
W20260709 13:03:46.538149  1685 leader_election.cc:336] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.143s (SENT)
I20260709 13:03:46.538512  1685 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:46.517964  5776 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:46.318521  5817 consensus_queue.cc:260] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:46.521400  5779 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:46.532097  5816 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:46.569780  1715 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:46.576751  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:46.517052  5781 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:46.560753  5822 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:46.555989  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:46.587880  5747 raft_consensus.cc:3055] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:46.470638  1467 catalog_manager.cc:5697] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:46.617781  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 93472.9us
W20260709 13:03:46.627966  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 144022us
I20260709 13:03:46.597774  5747 raft_consensus.cc:740] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:03:46.655354  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 70963.4us
I20260709 13:03:46.676090  1715 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:46.679752  1716 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:46.680721  5833 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:03:46.630324  5766 consensus_meta.cc:287] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.751s	user 0.013s	sys 0.002s
I20260709 13:03:46.692324  5689 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:46.727142  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.726575  1722 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:46.684717  5795 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:46.739833  2349 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:03:46.744125  1645 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:46.745230  5689 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:46.745384  5795 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:46.745712  5722 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:46.667984  5822 consensus_queue.cc:1059] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 13:03:46.746547  5836 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.746871  5678 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:46.747188  5836 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:46.749353  5747 consensus_queue.cc:260] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:46.677350  1715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18bdd841c8294b08857684489689676d"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:03:46.752282  1715 raft_consensus.cc:606] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:46.752691  1715 raft_consensus.cc:614] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:46.753295  1713 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 12: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:46.754521  1685 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:46.754518  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 88626.2us
I20260709 13:03:46.757460  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2061) took 2009 ms. Trace:
W20260709 13:03:46.800595  5826 log.cc:927] Time spent T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
W20260709 13:03:46.801882  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.745280  5737 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:46.757982  1719 rpcz_store.cc:276] 0709 13:03:44.747838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.861426 (+113588us) service_pool.cc:224] Handling call
0709 13:03:44.864368 (+  2942us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:45.128902 (+264534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:45.128953 (+    51us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:45.140807 (+ 11854us) log.cc:844] Serialized 78 byte log entry
0709 13:03:45.711195 (+570388us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:45.733088 (+ 21893us) log.cc:844] Serialized 14 byte log entry
0709 13:03:46.110220 (+377132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.110318 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:46.196865 (+ 86547us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:46.739614 (+542749us) raft_consensus.cc:1685] finished
0709 13:03:46.757207 (+ 17593us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.757398 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22473,"prepare.queue_time_us":21682,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":392878,"raft.run_cpu_time_us":356,"raft.run_wall_time_us":15984,"spinlock_wait_cycles":41158400,"thread_start_us":1084893,"threads_started":3,"wal-append.queue_time_us":632385}
I20260709 13:03:46.806043  5606 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:46.805809  5795 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:46.722779  5798 consensus_queue.cc:1059] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:03:46.749392  5836 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:46.993459  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 256083us
W20260709 13:03:47.032675  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 278244us
I20260709 13:03:47.094182  5815 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.133316  5815 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:47.136317  5815 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:47.095060  5767 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.137310  5767 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:47.139930  5767 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:47.096554  5847 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.629422  5777 consensus_queue.cc:237] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:47.102249  5811 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.707s	user 0.010s	sys 0.005s
I20260709 13:03:47.103742  5781 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:47.104327  5678 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.107074  5737 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:47.108525  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.110159  5689 consensus_queue.cc:1059] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:47.116484  5817 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:46.723572  5764 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:03:47.183976  1715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:03:47.185016  1715 raft_consensus.cc:606] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:47.185470  1715 raft_consensus.cc:614] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:47.116585  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 359882us
I20260709 13:03:46.805332  5834 consensus_queue.cc:1048] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:03:47.121526  5850 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:47.124800  5816 consensus_meta.cc:287] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.533s	user 0.005s	sys 0.009s
I20260709 13:03:47.034374  5769 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:47.105527  1686 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:47.128075  5766 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.131788  5481 raft_consensus.cc:3055] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:47.155014  1611 leader_election.cc:400] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 5. Current term is 6.
I20260709 13:03:47.201881  1611 leader_election.cc:403] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:03:47.126256  5835 raft_consensus.cc:3055] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:47.131850  5836 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:47.168838  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:47.146323  5747 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:47.174925  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:47.198923  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 103323us
W20260709 13:03:47.200194  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 94748.1us
I20260709 13:03:47.178527  5678 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:47.211306  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 62051.1us
I20260709 13:03:47.215155  1611 leader_election.cc:400] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 13. Current term is 14.
I20260709 13:03:47.215828  1611 leader_election.cc:403] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:03:47.203104  5832 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.397s	user 0.002s	sys 0.009s
W20260709 13:03:47.217737  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.177413  5811 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:47.225392  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:47.228639  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 59908us
I20260709 13:03:47.229600  1685 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:47.207470  5781 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:47.170178  5847 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:47.201795  5481 raft_consensus.cc:740] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:47.209444  5737 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:47.108572  1467 catalog_manager.cc:5697] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 3 to 4, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 4 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:47.263228  1467 catalog_manager.cc:5697] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 6 to 8, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 8 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:47.204921  5816 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:47.268069  5816 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:47.276480  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:47.233873  1570 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:03:47.204722  5835 raft_consensus.cc:740] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:47.179082  5839 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:03:47.241773  1685 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:47.240732  1612 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:47.280210  1685 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:47.215207  5840 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 5. Current term is 6.
W20260709 13:03:47.247021  5828 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.533s	user 0.006s	sys 0.006s
I20260709 13:03:47.285828  5824 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:47.249758  5807 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.286798  5807 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:47.287993  5592 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:47.288672  5592 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:47.289368  5807 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:47.290122  5816 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:47.290401  5854 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:47.290932  5854 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:47.291594  1685 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:47.292563  5795 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:47.293251  1685 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:47.293253  5795 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:47.294137  5779 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:47.251995  5678 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:47.259409  5811 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:47.234704  5781 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:47.297142  1685 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:47.208794  5766 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:47.236536  5606 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:47.224370  5838 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 13. Current term is 14.
W20260709 13:03:47.336714  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 96122.1us
I20260709 13:03:47.223086  5836 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:47.342945  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45867.8us
W20260709 13:03:47.299705  5724 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.090s	user 0.015s	sys 0.006s
I20260709 13:03:47.300158  5834 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.350458  5836 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:47.350881  5840 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:47.351725  1685 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:47.352324  5724 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:47.354113  1612 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:47.354758  5824 consensus_queue.cc:1048] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:47.275969  5827 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 13:03:47.214640  5850 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:47.341538  5781 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:47.342592  5678 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:47.359053  1537 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:47.363029  2314 leader_election.cc:336] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.734s (SENT)
I20260709 13:03:47.378275  1639 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:47.356762  5855 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:03:47.388268  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 158740us
I20260709 13:03:47.391122  1612 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:47.361147  5836 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:47.364861  5838 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:03:47.396631  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 175667us
I20260709 13:03:47.367496  5606 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:47.377460  5811 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.398909  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:47.399570  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:47.399416  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:47.404245  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:47.407531  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:47.359848  5834 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:47.411319  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:47.411814  1537 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:47.412171  1648 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:47.413060  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:47.419070  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:47.398465  5722 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:47.421530  1645 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:47.422863  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:47.424160  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:47.356842  5835 consensus_queue.cc:260] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:47.452718  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 51008.3us
I20260709 13:03:47.464515  1648 raft_consensus.cc:1217] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Deduplicated request from leader. Original: 7.7->[9.8-13.10]   Dedup: 9.8->[11.9-13.10]
I20260709 13:03:47.466650  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:47.467048  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:47.467445  1643 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:47.467779  1642 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:47.475672  5766 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:47.425479  5838 consensus_queue.cc:1048] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:03:47.477880  5592 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:47.403187  5836 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:47.394577  5855 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:47.484319  1643 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
W20260709 13:03:47.487039  5830 log.cc:927] Time spent T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.732s	user 0.005s	sys 0.011s
I20260709 13:03:47.490240  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 653) took 1144 ms. Trace:
I20260709 13:03:47.490749  1566 rpcz_store.cc:276] 0709 13:03:46.345654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.348059 (+  2405us) service_pool.cc:224] Handling call
0709 13:03:46.373461 (+ 25402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:46.376996 (+  3535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:46.377032 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:46.377788 (+   756us) log.cc:844] Serialized 26 byte log entry
0709 13:03:46.539571 (+161783us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:46.539634 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.539736 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:47.486649 (+946913us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b2400166a18. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245c1e9c4 00007f8245c1cacb 00007f8245dd7dc9 00007f8245dd7a55 00007f8245dd709d 00007f8248a0910e 00007f8248a2088c 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221
0709 13:03:47.489760 (+  3111us) raft_consensus.cc:1685] finished
0709 13:03:47.489791 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:47.490117 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13601,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":136,"spinlock_wait_cycles":4927744,"thread_start_us":161260,"threads_started":1,"wal-append.queue_time_us":246011}
W20260709 13:03:47.498126  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 88342.1us
I20260709 13:03:47.465132  1648 pending_rounds.cc:85] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 8
I20260709 13:03:47.499754  1648 pending_rounds.cc:107] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:03:47.499703  1612 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:47.500420  1648 raft_consensus.cc:2889] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:47.467088  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:47.501157  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:47.501503  1644 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:03:47.506825  1645 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
W20260709 13:03:47.507973  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 51808.9us
I20260709 13:03:47.508745  2312 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:47.418834  5728 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:47.511159  5866 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:47.407449  1467 catalog_manager.cc:5697] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:47.519557  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:47.543865  1645 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:47.520751  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 109025us
I20260709 13:03:47.408785  5811 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:47.519869  5592 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:47.417687  5841 consensus_queue.cc:1059] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.099s
W20260709 13:03:47.550631  2314 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.915s (SENT)
I20260709 13:03:47.517670  5728 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.552852  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:47.557174  1643 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:47.557811  1537 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:47.559288  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 59701.1us
I20260709 13:03:47.540706  5862 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:47.516136  5866 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:47.519611  5806 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.559358  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:47.563019  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:47.563539  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:47.566023  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:47.566538  1570 raft_consensus.cc:1275] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:47.567312  1564 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:03:47.572501  1564 pending_rounds.cc:85] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 5
I20260709 13:03:47.572851  1564 pending_rounds.cc:107] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:03:47.573158  1564 raft_consensus.cc:2889] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:47.574757  2314 leader_election.cc:336] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.165s (SENT)
I20260709 13:03:47.591006  1568 raft_consensus.cc:1275] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:47.575291  2314 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:47.550495  5859 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:47.593693  1687 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:47.593194  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:47.594954  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.593593  1573 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:47.558938  5728 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:47.603696  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:47.542248  5781 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:47.605604  1564 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:47.555975  5861 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:47.562702  5862 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.606911  1568 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:47.610554  1565 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:47.591521  5866 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:47.591077  5847 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:47.624961  1565 raft_consensus.cc:1275] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:47.603102  5606 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:47.626067  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:47.633241  1687 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:47.643605  1568 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:47.635604  1643 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:47.640897  1642 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 7.
I20260709 13:03:47.605952  5807 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:47.644193  1687 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:47.594004  5806 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:47.621191  5862 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:47.657012  1687 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:47.643163  5873 raft_consensus.cc:3055] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:47.664332  1645 raft_consensus.cc:2410] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 12 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:03:47.665517  5795 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:47.649196  5777 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:03:47.681456  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 77652.7us
I20260709 13:03:47.649955  5747 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:47.666241  5795 raft_consensus.cc:697] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:47.658842  5678 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:47.600781  5859 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:47.667101  5873 raft_consensus.cc:740] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:47.697211  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 144796us
I20260709 13:03:47.647501  5871 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:03:47.713564  1537 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:47.691406  5777 raft_consensus.cc:697] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:47.693243  5678 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:47.728717  1568 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:03:47.695712  5807 raft_consensus.cc:697] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:47.732949  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:47.733588  1568 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:47.742673  1538 leader_election.cc:336] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.561s (SENT)
I20260709 13:03:47.750294  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:47.756006  5592 raft_consensus.cc:697] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:47.756610  5837 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:47.760176  5844 raft_consensus.cc:3055] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:47.760627  5844 raft_consensus.cc:740] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:47.761504  5844 consensus_queue.cc:260] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:47.730850  5863 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:47.739768  5678 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:47.780700  1568 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:47.783311  1568 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:03:47.784057  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 70613.5us
W20260709 13:03:47.785676  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.785676  1537 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:47.787971  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 77340.4us
I20260709 13:03:47.757982  5592 consensus_queue.cc:237] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:47.763438  5811 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:47.659397  1467 catalog_manager.cc:5697] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 11 to 15, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 15 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:47.825500  5859 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:47.802395  5844 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:47.831190  5481 consensus_queue.cc:260] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:47.866701  5481 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:47.838747  5806 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:47.839241  5777 consensus_queue.cc:237] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:47.870509  5868 log.cc:927] Time spent T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.192s	user 0.000s	sys 0.002s
I20260709 13:03:47.840675  5807 consensus_queue.cc:237] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:47.842094  5874 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:03:47.806622  5885 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:47.818872  5781 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:47.847223  5834 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:47.855957  5724 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:47.829386  1564 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
W20260709 13:03:47.833139  5817 consensus_meta.cc:287] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.637s	user 0.008s	sys 0.005s
W20260709 13:03:47.843703  1639 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2533) took 1538 ms (client timeout 1561 ms). Trace:
W20260709 13:03:47.877377  1538 leader_election.cc:336] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.833s (SENT)
W20260709 13:03:47.902483  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 116935us
W20260709 13:03:47.907182  1639 rpcz_store.cc:269] 0709 13:03:46.305014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.305125 (+   111us) service_pool.cc:224] Handling call
0709 13:03:47.843351 (+1538226us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54001fdbd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:47.843575 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2340347648}
I20260709 13:03:47.914477  1537 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:47.899617  5843 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.702s	user 0.006s	sys 0.000s
I20260709 13:03:47.801314  5795 consensus_queue.cc:237] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:47.919699  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 344999us
W20260709 13:03:47.955246  2314 leader_election.cc:336] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.971s (SENT)
W20260709 13:03:47.961163  1538 leader_election.cc:336] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.552s (SENT)
I20260709 13:03:47.978439  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:47.979856  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:47.980129  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:47.921622  5779 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.981957  1647 raft_consensus.cc:2376] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 6. Re-sending same reply.
I20260709 13:03:47.874436  5891 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:03:48.004220  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:48.006068  1640 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:03:48.008509  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 94203.2us
I20260709 13:03:48.013012  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:48.013198  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:47.930104  5859 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:47.911001  5806 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:03:48.009405  1537 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:48.014500  1647 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:03:48.022792  1645 raft_consensus.cc:2376] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10. Re-sending same reply.
I20260709 13:03:48.023891  1641 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:48.026319  1537 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:48.029044  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:47.908413  5885 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:48.036193  1687 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:48.037194  1643 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:03:48.037283  1687 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:48.020494  5689 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.046625  1537 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:48.052484  5871 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:48.040879  5811 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:48.054224  5811 raft_consensus.cc:697] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:48.056186  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:48.058194  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 392) took 1524 ms. Trace:
I20260709 13:03:48.058601  1717 rpcz_store.cc:276] 0709 13:03:46.534263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.534541 (+   278us) service_pool.cc:224] Handling call
0709 13:03:46.556080 (+ 21539us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:46.562148 (+  6068us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:46.562201 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:46.573338 (+ 11137us) log.cc:844] Serialized 26 byte log entry
0709 13:03:46.769819 (+196481us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:46.770083 (+   264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.770268 (+   185us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:47.711116 (+940848us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:48.045848 (+334732us) raft_consensus.cc:1685] finished
0709 13:03:48.045873 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.058125 (+ 12252us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18625,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":8219,"spinlock_wait_cycles":6559488,"thread_start_us":160403,"threads_started":1,"wal-append.queue_time_us":539394}
W20260709 13:03:48.042732  5890 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:03:48.058368  1568 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:03:48.012689  5779 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:48.078094  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 51966.2us
I20260709 13:03:48.062348  1537 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:48.071226  5893 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.071671  1568 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:48.072443  5885 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:48.060557  5689 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:48.075776  1648 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7. Ops: [4.6-6.8]
I20260709 13:03:48.045483  5817 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:48.040896  5880 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:48.081913  5859 raft_consensus.cc:2764] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:03:48.091274  5860 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:48.042994  5877 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:48.092218  5892 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:48.154424  1647 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:03:48.131507  5879 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:48.095430  5875 consensus_queue.cc:1059] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:03:48.137720  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:48.138278  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 59312.4us
W20260709 13:03:48.139268  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:48.143536  1538 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.735s (SENT)
I20260709 13:03:48.043115  5878 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.176s
I20260709 13:03:48.146564  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:48.162195  1537 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:48.134537  5893 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:48.166113  5885 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:03:48.178423  5903 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
W20260709 13:03:48.180729  5728 consensus_meta.cc:287] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.561s	user 0.003s	sys 0.007s
I20260709 13:03:48.181560  5728 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:48.154070  5877 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:48.183785  5877 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:48.185459  5728 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:48.155488  5678 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:48.185983  1644 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:48.156999  5860 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:48.182542  2314 leader_election.cc:336] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.171s (SENT)
I20260709 13:03:48.188752  1537 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:48.186870  5689 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:48.159277  5888 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:03:48.203552  1537 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:48.149876  5817 raft_consensus.cc:697] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:48.208238  1537 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:48.204684  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:48.206776  5866 consensus_meta.cc:287] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.555s	user 0.010s	sys 0.002s
I20260709 13:03:48.177177  5811 consensus_queue.cc:237] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:48.175796  5747 raft_consensus.cc:2764] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:03:48.215651  1611 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:48.145749  5872 consensus_queue.cc:1059] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
W20260709 13:03:48.227516  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.231252  1537 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:48.236591  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:48.238749  1612 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:48.217747  5907 log.cc:927] Time spent T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:03:48.212946  5885 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:48.209923  5866 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:48.246717  5833 consensus_queue.cc:1048] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:03:48.214927  5879 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:48.305074  5777 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:48.312310  5781 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:48.312573  5777 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:48.312994  5781 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:48.315023  5777 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:48.315632  5781 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:48.316044  5728 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:48.247011  5896 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:48.321707  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:48.324205  1720 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:48.322438  1569 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:48.245808  5689 consensus_queue.cc:1048] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:03:48.302989  5747 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:48.245219  5724 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 13:03:48.316569  5728 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:48.334002  5895 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:03:48.319340  5878 consensus_queue.cc:1059] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
W20260709 13:03:48.356395  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 105168us
W20260709 13:03:48.376327  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 145201us
W20260709 13:03:48.368073  5862 consensus_meta.cc:287] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.685s	user 0.009s	sys 0.004s
I20260709 13:03:48.391965  1537 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:48.372113  5905 log.cc:927] Time spent T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.228s	user 0.002s	sys 0.000s
I20260709 13:03:48.400594  1537 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:48.351830  5728 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:48.384330  5861 consensus_meta.cc:287] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.759s	user 0.013s	sys 0.000s
W20260709 13:03:48.407227  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 154491us
I20260709 13:03:48.326200  5817 consensus_queue.cc:237] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:48.417441  1538 leader_election.cc:336] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.927s (SENT)
I20260709 13:03:48.418342  1538 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:48.423043  5678 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:48.407548  5899 log.cc:927] Time spent T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.220s	user 0.000s	sys 0.002s
I20260709 13:03:48.418923  5878 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.443557  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:48.444368  2348 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:48.445564  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:48.446784  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:48.429286  5861 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:48.432113  5869 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:48.466305  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:48.471014  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 53625.3us
I20260709 13:03:48.407331  5862 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:48.448246  5860 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:48.474941  5901 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
W20260709 13:03:48.507037  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 110456us
I20260709 13:03:48.464955  5827 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:48.494254  2343 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:03:48.437961  5833 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:03:48.514690  2349 raft_consensus.cc:2393] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 11: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:48.535876  1611 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:48.542070  1611 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:48.542212  5807 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:48.542807  5807 raft_consensus.cc:697] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:48.543080  5891 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:48.543593  5872 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:48.543823  5891 raft_consensus.cc:697] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:48.543691  1611 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:48.547168  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 146661us
I20260709 13:03:48.548303  1537 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:48.549626  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:48.555743  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:48.538337  2346 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 16: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:48.461022  5878 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:48.532958  5592 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:48.533406  5861 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:48.506783  5855 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:48.574447  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 64366.1us
I20260709 13:03:48.512178  5918 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:48.554476  5872 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:48.549516  5866 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:48.550294  5850 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:48.582433  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:48.539681  5770 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:48.589071  5844 consensus_meta.cc:287] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.721s	user 0.009s	sys 0.005s
I20260709 13:03:48.545359  5860 consensus_queue.cc:1048] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:03:48.577937  5592 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:48.607257  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:48.606045  1642 raft_consensus.cc:2410] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:03:48.599071  5764 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.009s	sys 0.045s
I20260709 13:03:48.591131  5918 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.610524  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2092) took 1019 ms. Trace:
I20260709 13:03:48.600382  5689 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.543841  5807 consensus_queue.cc:237] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:48.637167  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:48.623273  5911 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 13:03:48.625283  5927 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.626121  5872 raft_consensus.cc:697] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:48.620076  5836 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:48.629601  5914 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
W20260709 13:03:48.630255  5915 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 13:03:48.630043  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:48.666260  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:48.631009  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:48.618652  5861 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:48.545053  5891 consensus_queue.cc:237] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:48.633975  5840 raft_consensus.cc:3055] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:48.635160  5878 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:48.616457  1567 rpcz_store.cc:276] 0709 13:03:47.591206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.598242 (+  7036us) service_pool.cc:224] Handling call
0709 13:03:47.602006 (+  3764us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:47.659931 (+ 57925us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:47.659968 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:47.683169 (+ 23201us) log.cc:844] Serialized 52 byte log entry
0709 13:03:47.912049 (+228880us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:47.912296 (+   247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.913353 (+  1057us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:48.610181 (+696828us) raft_consensus.cc:1685] finished
0709 13:03:48.610202 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.610443 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24713,"raft.queue_time_us":93235,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":14354,"thread_start_us":250223,"threads_started":2,"wal-append.queue_time_us":429701}
I20260709 13:03:48.639837  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:48.645812  5912 log.cc:927] Time spent T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.224s	user 0.000s	sys 0.002s
I20260709 13:03:48.604302  5770 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:48.646435  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:48.647449  5862 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:48.622011  5689 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:48.653098  1719 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:03:48.655462  1720 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:48.713146  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:48.713457  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:48.714140  1719 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 10: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:48.711800  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:48.714730  1720 raft_consensus.cc:2393] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 9: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:48.611065  5592 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:48.616359  5764 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.009s	sys 0.046s
I20260709 13:03:48.670246  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:48.668099  5861 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:48.663118  5781 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:48.675765  5840 raft_consensus.cc:740] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:48.618342  5918 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:48.725980  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.698233  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:48.812492  1716 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 10: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:48.715131  1713 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:03:48.812891  5861 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:48.857739  1719 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: [9.8-9.8]
W20260709 13:03:48.825470  5877 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.641s	user 0.017s	sys 0.000s
I20260709 13:03:48.716667  5879 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:48.867885  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 319675us
I20260709 13:03:48.836015  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:48.885375  1537 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:48.888988  1713 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:48.877481  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 220066us
I20260709 13:03:48.892081  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:48.892333  1686 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:48.899933  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:48.900390  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:48.901399  2345 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:48.903321  2341 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:48.910461  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:48.916554  2313 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:48.918720  5931 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:48.919150  5931 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.919497  5931 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:48.668593  5927 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:48.927482  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:48.928458  1714 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:48.929935  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:48.930639  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:48.931106  1714 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:03:48.746524  5795 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.166s
I20260709 13:03:48.932577  5852 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:48.932631  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:48.935400  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:48.936187  1714 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:03:48.891405  5877 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:48.937614  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:48.938539  1714 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
W20260709 13:03:48.939143  5922 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.289s	user 0.000s	sys 0.001s
I20260709 13:03:48.939397  5877 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:48.939711  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:48.940253  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:48.940657  1714 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:03:48.941735  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:48.943168  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:48.943861  1646 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:48.944862  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:48.945087  5931 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:48.946352  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 54124.3us
I20260709 13:03:48.946462  5777 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:48.947364  5777 raft_consensus.cc:697] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:48.948446  5777 consensus_queue.cc:237] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:48.952430  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:48.952829  1717 raft_consensus.cc:1275] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:48.953917  5927 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:48.956795  5913 log.cc:927] Time spent T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.638s	user 0.000s	sys 0.004s
I20260709 13:03:48.954505  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:48.957762  1573 raft_consensus.cc:2368] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11.
W20260709 13:03:48.958328  1685 connection.cc:256] RPC call timeout handler was delayed by -1.0114691979999861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.163s for a response
W20260709 13:03:48.958984  1685 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.566s (SENT)
I20260709 13:03:48.959023  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:48.959517  1685 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:48.960201  1573 raft_consensus.cc:2410] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:48.960566  1715 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:03:48.956563  5925 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:48.961162  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:48.941040  5928 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:03:48.962033  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:48.972167  1715 raft_consensus.cc:2410] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:03:48.906663  5920 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:48.959416  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.956075  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:48.977766  1714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:03:48.978451  1714 raft_consensus.cc:606] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:48.978884  1714 raft_consensus.cc:614] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:48.966284  1639 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:48.965505  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:48.955889  5689 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:48.969297  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:48.990074  1717 raft_consensus.cc:2410] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 11 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:03:48.970353  1612 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:48.960722  1685 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.717s (SENT) [suppressed 51 similar messages]
I20260709 13:03:48.994210  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:49.004024  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:49.004024  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:48.851392  5781 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:49.014427  1537 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:49.014745  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:49.015323  1648 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:49.015553  1537 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:49.013242  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:49.017484  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.019829  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.021519  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:49.026500  1573 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:48.995702  5838 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:49.007472  5689 consensus_queue.cc:1059] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:03:49.031225  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:49.036787  1642 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:49.023742  5840 consensus_queue.cc:260] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:49.040099  5840 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:49.040585  5938 log.cc:927] Time spent T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:03:49.039242  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:49.042546  1642 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:49.058895  1537 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:49.068001  1537 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:49.070344  1642 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:03:49.007697  5939 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:49.075861  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:48.885659  5872 consensus_queue.cc:237] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:49.038159  5838 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.077340  1642 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:49.103349  5940 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:03:49.104818  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 144181us
I20260709 13:03:49.104045  5940 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:49.072430  1648 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:48.993820  5920 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:49.079366  5926 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:49.079828  5927 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:49.080092  5934 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:49.086357  5838 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:49.180961  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:49.139207  1685 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:49.109673  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:49.183846  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 44818.8us
I20260709 13:03:49.187718  1685 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:49.189584  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.199409  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:49.201401  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:49.206245  1685 leader_election.cc:400] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11.
I20260709 13:03:49.208504  1685 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:49.211433  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:49.213629  1685 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:49.079717  5936 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:49.105618  5931 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:49.216531  5921 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:49.180294  5926 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:49.217677  5926 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:49.181032  5934 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:49.218403  5934 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:49.104583  5806 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.086357  5925 consensus_queue.cc:1048] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.124s
I20260709 13:03:49.223853  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.224923  1568 raft_consensus.cc:2410] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:49.223992  5806 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:49.226480  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.227780  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.229007  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:49.215504  1685 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:49.229554  1568 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:49.177929  5927 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.110057  5678 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:49.244053  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:49.184994  5939 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:49.256162  5728 consensus_meta.cc:287] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.827s	user 0.010s	sys 0.001s
I20260709 13:03:49.256873  5678 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:49.203823  5777 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:49.259773  5678 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:49.260069  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 670) took 1517 ms. Trace:
I20260709 13:03:49.260390  5678 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:49.239565  5936 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:49.262113  5936 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:49.208706  1565 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:49.246585  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 109974us
I20260709 13:03:49.248996  1713 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:49.253036  5934 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:49.267395  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.256860  5728 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:49.260439  1564 rpcz_store.cc:276] 0709 13:03:47.742210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.911406 (+169196us) service_pool.cc:224] Handling call
0709 13:03:47.911755 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:47.911987 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:47.923029 (+ 11042us) log.cc:844] Serialized 14 byte log entry
0709 13:03:49.259473 (+1336444us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b18003b1ac0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a2523a
0709 13:03:49.259707 (+   234us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:49.259761 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:49.259830 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.259898 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.260015 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13130,"prepare.queue_time_us":19026,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":952,"raft.queue_time_us":1946,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":130,"spinlock_wait_cycles":34267520,"thread_start_us":88913,"threads_started":1,"wal-append.queue_time_us":153915}
I20260709 13:03:49.256289  5892 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:49.284049  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:49.248039  5927 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:49.283664  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.097182  5689 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.172s
I20260709 13:03:49.194489  5859 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
I20260709 13:03:49.294979  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.269781  5931 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.264899  5926 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:49.318976  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 88591.5us
I20260709 13:03:49.325738  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:49.326646  1713 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 17
W20260709 13:03:49.329255  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 113842us
I20260709 13:03:49.331844  1685 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:49.291802  5892 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:49.334077  1568 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:49.344333  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.348310  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.350113  1685 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:49.330472  5940 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:49.402871  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 52868.3us
I20260709 13:03:49.407354  1685 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:49.422166  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.421463  1573 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:49.431241  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:49.399919  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:49.447065  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.319386  5931 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:49.333221  5918 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:49.421541  5592 consensus_meta.cc:287] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.608s	user 0.009s	sys 0.006s
I20260709 13:03:49.331632  5861 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:49.419571  5947 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:49.408629  5916 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:49.418776  5921 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:49.490173  1713 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17.
I20260709 13:03:49.397027  5952 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:49.457927  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.497246  1573 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:49.497732  1567 raft_consensus.cc:1240] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 13:03:49.464270  1685 leader_election.cc:336] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.956s (SENT)
I20260709 13:03:49.451555  5940 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:49.487830  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.492241  2313 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:49.501267  1564 raft_consensus.cc:2393] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:49.498710  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.503813  1573 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:49.486192  5837 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.477969  5956 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:49.516103  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:49.511440  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:49.522644  1713 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:49.510466  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.524718  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.525234  1573 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:49.509213  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 404) took 1133 ms. Trace:
W20260709 13:03:49.509503  5943 log.cc:927] Time spent T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.270s	user 0.002s	sys 0.000s
I20260709 13:03:49.530145  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.531711  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:49.479260  5592 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:49.533241  1564 raft_consensus.cc:2393] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:03:49.521628  5946 log.cc:927] Time spent T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.181s	user 0.001s	sys 0.001s
I20260709 13:03:49.509172  5934 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:49.535080  1573 raft_consensus.cc:2393] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 7: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:49.499509  5916 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:49.538782  5916 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:49.527611  5956 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:49.540860  5931 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:49.542217  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2097) took 1219 ms. Trace:
I20260709 13:03:49.542758  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2096) took 1220 ms. Trace:
I20260709 13:03:49.543015  5918 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:49.543183  1720 rpcz_store.cc:276] 0709 13:03:48.322601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.827810 (+505209us) service_pool.cc:224] Handling call
0709 13:03:48.828298 (+   488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:48.855495 (+ 27197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.855547 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:48.856466 (+   919us) log.cc:844] Serialized 26 byte log entry
0709 13:03:49.063993 (+207527us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:49.064058 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.064610 (+   552us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:49.542561 (+477951us) raft_consensus.cc:1685] finished
0709 13:03:49.542578 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.542699 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41571,"raft.queue_time_us":28983,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":36060,"thread_start_us":131307,"threads_started":1,"wal-append.queue_time_us":216012}
I20260709 13:03:49.545038  1720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:03:49.545542  1720 raft_consensus.cc:606] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
W20260709 13:03:49.545856  5840 consensus_meta.cc:287] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.505s	user 0.011s	sys 0.002s
I20260709 13:03:49.541148  5838 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:49.546650  5728 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:49.531731  5873 consensus_queue.cc:260] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:49.548125  5873 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:49.548327  5838 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:49.529461  5781 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:49.554600  5592 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:49.545926  1720 raft_consensus.cc:614] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:49.558975  5926 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:49.564754  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 408) took 1587 ms. Trace:
I20260709 13:03:49.566509  1571 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:49.546228  5920 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:49.565274  1639 rpcz_store.cc:276] 0709 13:03:47.977098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.977308 (+   210us) service_pool.cc:224] Handling call
0709 13:03:48.965796 (+988488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.979521 (+ 13725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.564489 (+584968us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b5400097750. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:49.564673 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2175828352}
I20260709 13:03:49.583480  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.584635  1573 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:03:49.585953  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:49.586934  1573 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:49.600809  1538 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:49.587939  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.542634  1716 rpcz_store.cc:276] 0709 13:03:48.323041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.849292 (+526251us) service_pool.cc:224] Handling call
0709 13:03:48.885250 (+ 35958us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:49.265446 (+380196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:49.265466 (+    20us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:49.287999 (+ 22533us) log.cc:844] Serialized 52 byte log entry
0709 13:03:49.402056 (+114057us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:49.402120 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.404999 (+  2879us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:49.541988 (+136989us) raft_consensus.cc:1685] finished
0709 13:03:49.542012 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.542150 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58451,"raft.queue_time_us":97305,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":13159,"thread_start_us":375467,"threads_started":2,"wal-append.queue_time_us":221540}
W20260709 13:03:49.553391  5948 log.cc:927] Time spent T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
W20260709 13:03:49.604949  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 112808us
I20260709 13:03:49.608654  1538 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:49.575739  5931 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:49.610788  5934 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:49.605327  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.612231  5934 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:49.612547  1564 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:49.528278  5837 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:49.614318  1713 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:49.615967  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:49.615967  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:49.616109  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:49.617487  1713 raft_consensus.cc:2410] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:03:49.617890  1538 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:49.624780  1716 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:49.625744  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2104) took 1240 ms. Trace:
I20260709 13:03:49.626212  1716 rpcz_store.cc:276] 0709 13:03:48.385061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.611130 (+1226069us) service_pool.cc:224] Handling call
0709 13:03:49.621492 (+ 10362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:49.625360 (+  3868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.625660 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:49.628001  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 50642.2us
I20260709 13:03:49.630185  1639 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:49.631320  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.630687  2314 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:49.632019  1564 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:49.615406  5934 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:49.641594  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 415) took 1656 ms. Trace:
W20260709 13:03:49.642792  1647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 424) took 1629 ms (client timeout 2114 ms). Trace:
W20260709 13:03:49.643138  1647 rpcz_store.cc:269] 0709 13:03:48.013243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.028781 (+ 15538us) service_pool.cc:224] Handling call
0709 13:03:49.642658 (+1613877us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54002dc154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:49.642748 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":286770048}
I20260709 13:03:49.621122  5592 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:49.622974  5926 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:49.653801  1716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:03:49.654452  1716 raft_consensus.cc:606] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:49.654793  1716 raft_consensus.cc:614] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:49.655815  5770 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:49.656085  1716 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:03:49.656702  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 408) took 1253 ms. Trace:
I20260709 13:03:49.656978  1716 rpcz_store.cc:276] 0709 13:03:48.403306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.655517 (+1252211us) service_pool.cc:224] Handling call
0709 13:03:49.655839 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:49.656467 (+   628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.656653 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:49.657609  5770 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.651014  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:49.642060  1641 rpcz_store.cc:276] 0709 13:03:47.985526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.985840 (+   314us) service_pool.cc:224] Handling call
0709 13:03:48.023556 (+ 37716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.024303 (+   747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.641352 (+1617049us) spinlock_profiling.cc:243] Waited 7.27 ms on lock 0x7b5400183650. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:49.641528 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16717184}
I20260709 13:03:49.659493  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 406) took 1266 ms. Trace:
I20260709 13:03:49.659960  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.628495  5917 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:49.659874  1713 rpcz_store.cc:276] 0709 13:03:48.392852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.631502 (+1238650us) service_pool.cc:224] Handling call
0709 13:03:49.653384 (+ 21882us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:49.653577 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:49.653605 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:49.659218 (+  5613us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:49.659274 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.659333 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.659444 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:49.663137  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:49.663982  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:49.658133  5770 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:49.676272  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:49.677876  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:49.683786  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:49.683832  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:49.684774  1641 raft_consensus.cc:2368] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11.
I20260709 13:03:49.687093  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:49.695075  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:49.697189  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2120) took 1068 ms. Trace:
I20260709 13:03:49.698462  5779 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:49.699258  5861 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:49.701685  5806 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:49.701823  5837 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:49.689384  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:49.697561  1564 rpcz_store.cc:276] 0709 13:03:48.629073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.696377 (+1067304us) service_pool.cc:224] Handling call
0709 13:03:49.696709 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:49.696879 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:49.696905 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:49.696949 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:49.696982 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.697032 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.697136 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:49.703827  1641 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:49.708635  5959 log.cc:927] Time spent T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 13:03:49.713585  1685 leader_election.cc:336] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.821s (SENT)
I20260709 13:03:49.716678  1685 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:49.721557  2341 raft_consensus.cc:1240] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:03:49.723268  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 405) took 1340 ms. Trace:
I20260709 13:03:49.723706  1720 rpcz_store.cc:276] 0709 13:03:48.382803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.558283 (+1175480us) service_pool.cc:224] Handling call
0709 13:03:49.558806 (+   523us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:49.559721 (+   915us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:49.559779 (+    58us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:49.560629 (+   850us) log.cc:844] Serialized 52 byte log entry
0709 13:03:49.622249 (+ 61620us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:49.622289 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.622386 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:49.723035 (+100649us) raft_consensus.cc:1685] finished
0709 13:03:49.723059 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.723210 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":186,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":115,"thread_start_us":60346,"threads_started":1,"wal-append.queue_time_us":77336}
W20260709 13:03:49.711998  5852 consensus_meta.cc:287] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.735s	user 0.008s	sys 0.013s
I20260709 13:03:49.726114  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 409) took 1314 ms. Trace:
I20260709 13:03:49.726248  5852 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:49.726249  5962 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:49.726514  1720 rpcz_store.cc:276] 0709 13:03:48.412095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.725241 (+1313146us) service_pool.cc:224] Handling call
0709 13:03:49.725558 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:49.725781 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:49.725814 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:49.725861 (+    47us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:49.725902 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.725956 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.726053 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:49.727845  5962 consensus_queue.cc:1243] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:03:49.679668  5931 raft_consensus.cc:697] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:03:49.733273  1685 leader_election.cc:336] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.918s (SENT)
I20260709 13:03:49.734031  1685 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:49.740231  1646 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
W20260709 13:03:49.740571  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:49.745543  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 76451.4us
I20260709 13:03:49.726433  5967 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:03:49.670898  5917 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:49.755633  1573 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
W20260709 13:03:49.795557  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 62332us
I20260709 13:03:49.797326  1641 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:49.802845  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:49.805145  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:49.704663  5936 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:49.797137  1685 leader_election.cc:336] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.084s (SENT)
I20260709 13:03:49.808465  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 425) took 1227 ms. Trace:
I20260709 13:03:49.809026  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:49.815744  1641 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:03:49.816929  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:49.741115  5724 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.752199  2341 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:03:49.824471  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:49.781574  5940 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:49.752405  5817 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:49.788408  5847 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:49.828845  5847 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:49.832116  5724 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:49.832657  5847 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:49.791255  5892 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:49.739288  5969 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:03:49.857769  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 60680.8us
I20260709 13:03:49.798941  5781 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.864336  5781 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:49.864508  5931 consensus_queue.cc:237] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:49.868530  5781 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:49.868739  5936 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:49.806319  5592 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.837550  5770 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:49.841216  5777 consensus_meta.cc:287] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.583s	user 0.009s	sys 0.005s
I20260709 13:03:49.869954  5592 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:49.871196  5777 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:49.860222  1685 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.246s (SENT)
I20260709 13:03:49.878602  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:49.887578  1685 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:49.879591  5927 consensus_meta.cc:287] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.560s	user 0.006s	sys 0.010s
I20260709 13:03:49.914676  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:49.915782  5811 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.345s	user 0.006s	sys 0.016s
I20260709 13:03:49.840060  5892 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:49.881697  2341 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:49.880055  5917 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:49.881228  2346 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:03:49.914119  5817 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:49.881624  5969 raft_consensus.cc:697] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:49.944152  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2106) took 1534 ms. Trace:
I20260709 13:03:49.944650  1713 rpcz_store.cc:276] 0709 13:03:48.409619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.667030 (+1257411us) service_pool.cc:224] Handling call
0709 13:03:49.678113 (+ 11083us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:49.688453 (+ 10340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:49.688502 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:49.694267 (+  5765us) log.cc:844] Serialized 52 byte log entry
0709 13:03:49.743813 (+ 49546us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:49.743861 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.743985 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:49.943885 (+199900us) raft_consensus.cc:1685] finished
0709 13:03:49.943910 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.944083 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12315,"raft.queue_time_us":20022,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":9650,"thread_start_us":49099,"threads_started":1,"wal-append.queue_time_us":234767}
I20260709 13:03:49.946664  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:49.947458  1713 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:49.933516  2346 pending_rounds.cc:85] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Aborting all ops after (but not including) 7
I20260709 13:03:49.927968  5592 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:49.932579  5811 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.365s	user 0.007s	sys 0.016s
I20260709 13:03:49.932336  5927 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:50.007375  5927 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:50.082876  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2109) took 1621 ms. Trace:
I20260709 13:03:49.976619  2346 pending_rounds.cc:107] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:03:50.083482  1720 rpcz_store.cc:276] 0709 13:03:48.460986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.728331 (+1267345us) service_pool.cc:224] Handling call
0709 13:03:49.788930 (+ 60599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:49.789224 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:49.789579 (+   355us) log.cc:844] Serialized 14 byte log entry
0709 13:03:49.934273 (+144694us) log.cc:844] Serialized 14 byte log entry
0709 13:03:49.935650 (+  1377us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:49.935717 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:50.082453 (+146736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.082582 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:50.082788 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1365,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":80,"raft.queue_time_us":363,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":1947,"spinlock_wait_cycles":4705024,"thread_start_us":134673,"threads_started":1,"wal-append.queue_time_us":123014}
I20260709 13:03:50.172329  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:50.181120  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:50.184536  1720 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:50.160046  1685 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.609s (SENT)
I20260709 13:03:50.170552  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:50.171701  2346 raft_consensus.cc:2889] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:50.245796  1721 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:50.239519  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:50.247058  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2090) took 1991 ms. Trace:
I20260709 13:03:50.253994  1721 rpcz_store.cc:276] 0709 13:03:48.255993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.256225 (+   232us) service_pool.cc:224] Handling call
0709 13:03:50.168239 (+1912014us) spinlock_profiling.cc:243] Waited 9.49 ms on lock 0x7b580009db98. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:03:50.168472 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.246668 (+ 78196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.246974 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21819264}
I20260709 13:03:50.448539  5969 consensus_queue.cc:237] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:50.618422  5934 consensus_meta.cc:287] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.967s	user 0.014s	sys 0.000s
W20260709 13:03:50.537426  5919 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.004s	sys 0.029s
W20260709 13:03:50.722735  5981 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 13:03:50.686219  5934 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:50.771683  5934 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:50.863292  1720 consensus_meta.cc:287] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.678s	user 0.011s	sys 0.000s
I20260709 13:03:50.863662  1720 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:03:50.865325  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2111) took 2310 ms. Trace:
I20260709 13:03:50.867455  5896 raft_consensus.cc:3055] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:50.870127  5986 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:50.874773  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:50.880225  1720 rpcz_store.cc:276] 0709 13:03:48.554689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.864468 (+2309779us) service_pool.cc:224] Handling call
0709 13:03:50.864795 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.864988 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:50.865015 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:50.865069 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:50.865115 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.865169 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:50.865270 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:50.883435  1718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2089) took 2627 ms (client timeout 1841 ms). Trace:
W20260709 13:03:50.884074  1718 rpcz_store.cc:269] 0709 13:03:48.254705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.255371 (+   666us) service_pool.cc:224] Handling call
0709 13:03:50.874528 (+2619157us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b580009db98. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:03:50.882469 (+  7941us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6659328}
I20260709 13:03:50.886102  5986 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:50.889262  5986 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:50.881717  5896 raft_consensus.cc:740] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:50.891213  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2112) took 2308 ms. Trace:
I20260709 13:03:50.891216  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2113) took 2290 ms. Trace:
I20260709 13:03:50.892009  1720 rpcz_store.cc:276] 0709 13:03:48.582994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.884199 (+2301205us) service_pool.cc:224] Handling call
0709 13:03:50.890426 (+  6227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.890723 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:50.890763 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:50.890829 (+    66us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:50.890891 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.890950 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:50.891120 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:50.892760  1718 rpcz_store.cc:276] 0709 13:03:48.600489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.885267 (+2284778us) service_pool.cc:224] Handling call
0709 13:03:50.890508 (+  5241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.890761 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:50.890801 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:50.890858 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:50.890902 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.890986 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:50.891139 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:50.900036  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2116) took 2277 ms. Trace:
I20260709 13:03:50.899828  5896 consensus_queue.cc:260] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:50.900843  1718 rpcz_store.cc:276] 0709 13:03:48.622417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.898881 (+2276464us) service_pool.cc:224] Handling call
0709 13:03:50.899356 (+   475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.899603 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:50.899638 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:50.899692 (+    54us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:50.899743 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.899810 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:50.899968 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:50.901674  5896 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:50.902974  1718 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:50.903957  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2450) took 2008 ms. Trace:
I20260709 13:03:50.904382  1718 rpcz_store.cc:276] 0709 13:03:48.895375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.902344 (+2006969us) service_pool.cc:224] Handling call
0709 13:03:50.902752 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.903603 (+   851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.903874 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:50.905223  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2110) took 2366 ms. Trace:
I20260709 13:03:50.905846  1718 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:50.905674  1721 rpcz_store.cc:276] 0709 13:03:48.539154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.515468 (+1976314us) service_pool.cc:224] Handling call
0709 13:03:50.515940 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:50.621138 (+105198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:50.621193 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:50.629100 (+  7907us) log.cc:844] Serialized 26 byte log entry
0709 13:03:50.879949 (+250849us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:50.879991 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.880064 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:50.905020 (+ 24956us) raft_consensus.cc:1685] finished
0709 13:03:50.905043 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:50.905167 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":133,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":10509,"thread_start_us":193082,"threads_started":1,"wal-append.queue_time_us":240808}
I20260709 13:03:50.910028  1721 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 13. Ops: [11.5-11.5]
I20260709 13:03:50.910796  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2455) took 2013 ms. Trace:
I20260709 13:03:50.911151  1721 rpcz_store.cc:276] 0709 13:03:48.897028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.908174 (+2011146us) service_pool.cc:224] Handling call
0709 13:03:50.909439 (+  1265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:50.910478 (+  1039us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.910730 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:50.907071  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2451) took 2011 ms. Trace:
I20260709 13:03:50.912801  1718 rpcz_store.cc:276] 0709 13:03:48.895714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.905228 (+2009514us) service_pool.cc:224] Handling call
0709 13:03:50.905667 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.906479 (+   812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.907009 (+   530us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:50.914381  1721 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:50.915323  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 410) took 1998 ms. Trace:
I20260709 13:03:50.915670  1721 rpcz_store.cc:276] 0709 13:03:48.970796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.913180 (+1942384us) service_pool.cc:224] Handling call
0709 13:03:50.914132 (+   952us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.915009 (+   877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.915252 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:50.918098  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2463) took 1901 ms. Trace:
I20260709 13:03:50.918529  1721 rpcz_store.cc:276] 0709 13:03:49.016238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.916532 (+1900294us) service_pool.cc:224] Handling call
0709 13:03:50.916902 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.917307 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:50.917335 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:50.917383 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:50.917425 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.917517 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:50.917690 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:50.920014  1721 raft_consensus.cc:1240] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:50.921170  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 413) took 1741 ms. Trace:
I20260709 13:03:50.921708  1721 rpcz_store.cc:276] 0709 13:03:49.180217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.919484 (+1739267us) service_pool.cc:224] Handling call
0709 13:03:50.919825 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.920877 (+  1052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.921076 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:50.924036  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:35480 (request call id 22) took 1480 ms. Trace:
I20260709 13:03:50.924417  1721 rpcz_store.cc:276] 0709 13:03:49.443693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.922591 (+1478898us) service_pool.cc:224] Handling call
0709 13:03:50.923943 (+  1352us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:50.925740  5934 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.926594  5934 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:50.929404  5936 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:50.930259  5855 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:50.930106  5936 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:50.931052  5855 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:50.933200  5936 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:50.956141  5936 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.929009  5934 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:50.933811  5592 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:50.957370  5592 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:50.933602  5855 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:50.930053  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:50.956805  5936 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:50.971742  5936 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:50.978677  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2559) took 1377 ms. Trace:
I20260709 13:03:50.979574  1641 rpcz_store.cc:276] 0709 13:03:49.600860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.929548 (+1328688us) service_pool.cc:224] Handling call
0709 13:03:50.978570 (+ 49022us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:50.981487  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:50.982486  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2561) took 1381 ms. Trace:
I20260709 13:03:50.982890  1641 rpcz_store.cc:276] 0709 13:03:49.605491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.981140 (+1375649us) service_pool.cc:224] Handling call
0709 13:03:50.982424 (+  1284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:50.984086  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:50.986686  5944 log.cc:927] Time spent T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Append to log took a long time: real 1.742s	user 0.010s	sys 0.005s
I20260709 13:03:50.994514  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 429) took 1398 ms. Trace:
I20260709 13:03:50.995057  1641 rpcz_store.cc:276] 0709 13:03:49.596268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.983645 (+1387377us) service_pool.cc:224] Handling call
0709 13:03:50.994427 (+ 10782us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:50.998611  1641 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:51.005560  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 426) took 2423 ms. Trace:
I20260709 13:03:51.006592  1641 rpcz_store.cc:276] 0709 13:03:48.581896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.996950 (+2415054us) service_pool.cc:224] Handling call
0709 13:03:50.998356 (+  1406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.001186 (+  2830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.005422 (+  4236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:51.009196  1641 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:51.009943  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2035) took 2099 ms. Trace:
I20260709 13:03:51.010357  1641 rpcz_store.cc:276] 0709 13:03:48.910425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.008187 (+2097762us) service_pool.cc:224] Handling call
0709 13:03:51.008610 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.009554 (+   944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.009860 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:51.013537  1641 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:51.014683  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2039) took 2102 ms. Trace:
I20260709 13:03:51.015700  1641 rpcz_store.cc:276] 0709 13:03:48.911691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.012437 (+2100746us) service_pool.cc:224] Handling call
0709 13:03:51.013266 (+   829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.014200 (+   934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.014480 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:51.021875  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2041) took 2109 ms. Trace:
I20260709 13:03:51.022336  1641 rpcz_store.cc:276] 0709 13:03:48.912282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.020863 (+2108581us) service_pool.cc:224] Handling call
0709 13:03:51.021244 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.021450 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:51.021475 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:51.021531 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:51.021562 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.021621 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.021804 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:51.024976  1641 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:51.026263  1537 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:51.027349  5993 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:51.027833  5993 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:51.028337  5993 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:51.030211  5986 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.030893  5986 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:51.033780  5986 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:51.035706  5986 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.025619  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2044) took 2105 ms. Trace:
I20260709 13:03:51.038647  1641 rpcz_store.cc:276] 0709 13:03:48.920197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.024275 (+2104078us) service_pool.cc:224] Handling call
0709 13:03:51.024598 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.025356 (+   758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.025564 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:51.039997  1641 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:03:51.038012  5986 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:51.040570  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2045) took 2120 ms. Trace:
I20260709 13:03:51.041040  5993 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:51.041683  1641 rpcz_store.cc:276] 0709 13:03:48.920460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.039293 (+2118833us) service_pool.cc:224] Handling call
0709 13:03:51.039646 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.040338 (+   692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.040519 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:51.043955  5993 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:51.044690  5993 consensus_queue.cc:1059] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:51.045624  5993 consensus_queue.cc:1059] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:51.047235  5993 raft_consensus.cc:3055] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:51.047485  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2046) took 2126 ms. Trace:
I20260709 13:03:51.047676  5993 raft_consensus.cc:740] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:51.049096  5993 consensus_queue.cc:260] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:51.048213  1641 rpcz_store.cc:276] 0709 13:03:48.920729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.042687 (+2121958us) service_pool.cc:224] Handling call
0709 13:03:51.043099 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.043336 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:51.043663 (+   327us) log.cc:844] Serialized 14 byte log entry
0709 13:03:51.046696 (+  3033us) log.cc:844] Serialized 14 byte log entry
0709 13:03:51.046962 (+   266us) log.cc:844] Serialized 14 byte log entry
0709 13:03:51.047124 (+   162us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:51.047164 (+    40us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:51.047220 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.047283 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.047415 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5595,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":86,"raft.queue_time_us":1682,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":161,"thread_start_us":2437,"threads_started":1,"wal-append.queue_time_us":2763}
I20260709 13:03:51.050716  5993 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:03:51.051403  1641 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:51.052534  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2047) took 2131 ms. Trace:
I20260709 13:03:51.052944  1641 rpcz_store.cc:276] 0709 13:03:48.920968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.050630 (+2129662us) service_pool.cc:224] Handling call
0709 13:03:51.050990 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.052271 (+  1281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.052483 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:51.054422  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 428) took 2087 ms. Trace:
I20260709 13:03:51.054797  1641 rpcz_store.cc:276] 0709 13:03:48.967546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.053821 (+2086275us) service_pool.cc:224] Handling call
0709 13:03:51.054071 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.054211 (+   140us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:51.054226 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:51.054253 (+    27us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:51.054271 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.054303 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.054383 (+    80us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:51.056586  1641 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 13. Ops: []
I20260709 13:03:51.057370  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2549) took 1460 ms. Trace:
I20260709 13:03:51.057827  5986 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:51.057945  1641 rpcz_store.cc:276] 0709 13:03:49.596959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.055914 (+1458955us) service_pool.cc:224] Handling call
0709 13:03:51.056398 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.057097 (+   699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.057313 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:51.057976  5855 raft_consensus.cc:3055] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:51.060717  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2552) took 1462 ms. Trace:
I20260709 13:03:51.061159  5855 raft_consensus.cc:740] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:51.061200  1641 rpcz_store.cc:276] 0709 13:03:49.598527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.059437 (+1460910us) service_pool.cc:224] Handling call
0709 13:03:51.059930 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.060298 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:51.060324 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:51.060394 (+    70us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:51.060443 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.060511 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.060664 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:51.062224  5855 consensus_queue.cc:260] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:51.063553  5855 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:51.153918  5986 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.154768  5986 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:51.157720  5986 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:51.208245  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2115) took 2586 ms. Trace:
I20260709 13:03:51.208756  1720 rpcz_store.cc:276] 0709 13:03:48.622047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.916533 (+2294486us) service_pool.cc:224] Handling call
0709 13:03:50.917045 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:50.937493 (+ 20448us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:50.937534 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:50.943280 (+  5746us) log.cc:844] Serialized 26 byte log entry
0709 13:03:51.207454 (+264174us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:51.207500 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.207617 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:51.207671 (+    54us) raft_consensus.cc:1685] finished
0709 13:03:51.207686 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.208172 (+   486us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19800,"raft.queue_time_us":21132,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":121,"thread_start_us":263663,"threads_started":1,"wal-append.queue_time_us":20207}
I20260709 13:03:51.211330  5990 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:51.211971  5990 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:51.212980  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2118) took 2584 ms. Trace:
I20260709 13:03:51.213416  1567 rpcz_store.cc:276] 0709 13:03:48.628421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.676428 (+ 48007us) service_pool.cc:224] Handling call
0709 13:03:48.677200 (+   772us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:49.502883 (+825683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.212447 (+1709564us) spinlock_profiling.cc:243] Waited 815 ms on lock 0x7b54002b9654. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:51.212837 (+   390us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1874356608}
I20260709 13:03:51.215081  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:51.215185  5990 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:51.215965  1567 raft_consensus.cc:2376] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9. Re-sending same reply.
W20260709 13:03:51.216490  1567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2079) took 1502 ms (client timeout 1692 ms). Trace:
W20260709 13:03:51.216904  1567 rpcz_store.cc:269] 0709 13:03:49.714095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.214741 (+1500646us) service_pool.cc:224] Handling call
0709 13:03:51.216432 (+  1691us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:51.217857  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:51.218505  1567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2082) took 1502 ms (client timeout 1946 ms). Trace:
W20260709 13:03:51.218725  1567 rpcz_store.cc:269] 0709 13:03:49.716160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.217582 (+1501422us) service_pool.cc:224] Handling call
0709 13:03:51.218464 (+   882us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:51.219494  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:51.220052  1567 raft_consensus.cc:2393] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:51.220371  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2081) took 1504 ms. Trace:
I20260709 13:03:51.220573  1567 rpcz_store.cc:276] 0709 13:03:49.715867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.219303 (+1503436us) service_pool.cc:224] Handling call
0709 13:03:51.220332 (+  1029us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:51.221312  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:51.222182  1567 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:51.223104  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:51.223879  1567 raft_consensus.cc:2393] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:51.224627  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:51.225201  1567 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
W20260709 13:03:51.225334  5923 log.cc:927] Time spent T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 2.628s	user 0.006s	sys 0.009s
I20260709 13:03:51.226198  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:51.226734  1567 raft_consensus.cc:2376] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9. Re-sending same reply.
I20260709 13:03:51.226958  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2093) took 2908 ms. Trace:
I20260709 13:03:51.227624  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:51.229945  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2120) took 1422 ms. Trace:
I20260709 13:03:51.227358  1722 rpcz_store.cc:276] 0709 13:03:48.319102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.319306 (+   204us) service_pool.cc:224] Handling call
0709 13:03:48.328455 (+  9149us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:48.328865 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.328894 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:48.336043 (+  7149us) log.cc:844] Serialized 26 byte log entry
0709 13:03:48.451484 (+115441us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:48.451542 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.451624 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:51.224737 (+2773113us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:51.226625 (+  1888us) raft_consensus.cc:1685] finished
0709 13:03:51.226645 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.226878 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5690,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":7814,"spinlock_wait_cycles":5081856,"thread_start_us":76167,"threads_started":1,"wal-append.queue_time_us":162732}
I20260709 13:03:51.233022  1720 rpcz_store.cc:276] 0709 13:03:49.807718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.210074 (+1402356us) service_pool.cc:224] Handling call
0709 13:03:51.210319 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.210480 (+   161us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:51.225953 (+ 15473us) log.cc:844] Serialized 14 byte log entry
0709 13:03:51.228880 (+  2927us) log.cc:844] Serialized 14 byte log entry
0709 13:03:51.229293 (+   413us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:51.229361 (+    68us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:51.229415 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.229494 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.229801 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2153,"prepare.run_cpu_time_us":137,"prepare.run_wall_time_us":144,"raft.queue_time_us":1396,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":189,"thread_start_us":17135,"threads_started":2,"wal-append.queue_time_us":3400}
I20260709 13:03:51.234987  1720 raft_consensus.cc:3055] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:51.235327  1720 raft_consensus.cc:740] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:51.235208  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2121) took 1427 ms. Trace:
I20260709 13:03:51.236177  1722 rpcz_store.cc:276] 0709 13:03:49.809577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.234323 (+1424746us) service_pool.cc:224] Handling call
0709 13:03:51.234700 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.234877 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:51.234902 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:51.234939 (+    37us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:51.234972 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.235040 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.235155 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:51.236423  1720 consensus_queue.cc:260] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:51.237475  1720 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:51.238250  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:51.239058  1565 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:51.239928  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:51.240463  1565 raft_consensus.cc:2393] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:51.241147  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:51.231204  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 59d236b930214b1f91fb55ece6fecd68 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:51.254768  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2123) took 1444 ms. Trace:
I20260709 13:03:51.256707  1565 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:51.255396  1722 rpcz_store.cc:276] 0709 13:03:49.810175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.237240 (+1427065us) service_pool.cc:224] Handling call
0709 13:03:51.237583 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:51.238246 (+   663us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:51.238272 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:51.238778 (+   506us) log.cc:844] Serialized 26 byte log entry
0709 13:03:51.254389 (+ 15611us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:51.254439 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.254513 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:51.254558 (+    45us) raft_consensus.cc:1685] finished
0709 13:03:51.254573 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.254700 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":119,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":93,"thread_start_us":15213,"threads_started":1,"wal-append.queue_time_us":12793}
I20260709 13:03:51.256783  1720 raft_consensus.cc:1275] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:03:51.264554  1720 pending_rounds.cc:85] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 6
I20260709 13:03:51.264994  1720 pending_rounds.cc:107] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:03:51.264299  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:51.265411  1720 raft_consensus.cc:2889] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:51.266165  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2122) took 1456 ms. Trace:
I20260709 13:03:51.266481  1720 rpcz_store.cc:276] 0709 13:03:49.809915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.234663 (+1424748us) service_pool.cc:224] Handling call
0709 13:03:51.234883 (+   220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.265948 (+ 31065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.266114 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:51.266602  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:51.267365  1722 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:51.267537  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:51.268056  1720 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:51.255724  5941 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 2.163s	user 0.005s	sys 0.013s
I20260709 13:03:49.838646  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 414) took 1854 ms. Trace:
I20260709 13:03:51.269869  1640 rpcz_store.cc:276] 0709 13:03:47.984126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.984292 (+   166us) service_pool.cc:224] Handling call
0709 13:03:48.002521 (+ 18229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.006515 (+  3994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.838428 (+1831913us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b54002db750. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:49.838584 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40801152}
W20260709 13:03:51.663582  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:51.670014  5978 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 1.810s	user 0.009s	sys 0.007s
W20260709 13:03:51.258543  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.682297  5967 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:51.683110  5967 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:51.683686  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:51.684823  1565 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:03:51.701172  1567 raft_consensus.cc:2410] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:03:51.702625  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:51.703723  1567 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:51.707315  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:51.720232  2346 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:03:51.721659  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:51.722225  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:51.727003  1564 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.017s	user 0.038s	sys 0.076s
W20260709 13:03:51.731927  1612 connection.cc:256] RPC call timeout handler was delayed by -1.802836794999962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1841s for a response
W20260709 13:03:51.737838  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 56571.5us
W20260709 13:03:51.741251  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.745760  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:51.764699  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:51.753203  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 80476.6us
I20260709 13:03:51.937595  5957 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:51.938017  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:51.938772  5957 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:51.939237  1565 raft_consensus.cc:2410] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:03:51.824213  5835 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:51.941037  5957 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:49.840912  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.941452  2345 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:03:51.890018  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:51.943173  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.877267  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:51.945395  1567 raft_consensus.cc:2410] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:51.946272  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:51.971903  5921 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:51.951911  1570 raft_consensus.cc:1240] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:52.030628  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2124) took 3383 ms. Trace:
I20260709 13:03:52.031152  1570 rpcz_store.cc:276] 0709 13:03:48.647189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.951273 (+3304084us) service_pool.cc:224] Handling call
0709 13:03:51.951688 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.030347 (+ 78659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.030569 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:51.971946  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:52.032507  1687 connection.cc:256] RPC call timeout handler was delayed by -1.9972971540000231s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3163s for a response
W20260709 13:03:52.033198  1687 connection.cc:256] RPC call timeout handler was delayed by -1.5881364620000227s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1632s for a response
W20260709 13:03:52.033900  1687 connection.cc:256] RPC call timeout handler was delayed by -1.4594364619999851s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
W20260709 13:03:52.034494  1687 connection.cc:256] RPC call timeout handler was delayed by -1.4324364619999983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1805s for a response
W20260709 13:03:52.034953  1687 connection.cc:256] RPC call timeout handler was delayed by -1.381136462000029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18620000000000003s for a response
W20260709 13:03:52.035485  1687 connection.cc:256] RPC call timeout handler was delayed by -1.1337191230000485s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
W20260709 13:03:52.036167  1687 connection.cc:256] RPC call timeout handler was delayed by -1.1300364620000209s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2141s for a response
W20260709 13:03:52.036609  1687 connection.cc:256] RPC call timeout handler was delayed by -1.1039364620000356s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.217s for a response
I20260709 13:03:51.956244  5882 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:51.760691  5967 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:51.762864  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2032) took 4213 ms (client timeout 3163 ms). Trace:
W20260709 13:03:51.969499  5939 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.713s	user 0.024s	sys 0.048s
I20260709 13:03:51.884614  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2090) took 4317 ms. Trace:
I20260709 13:03:52.040341  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:51.892767  2349 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 399) took 3449 ms (client timeout 3005 ms). Trace:
W20260709 13:03:51.857554  1722 consensus_meta.cc:287] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.590s	user 0.007s	sys 0.003s
I20260709 13:03:52.041941  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:52.042155  1722 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:03:52.042528  1570 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:51.894929  1639 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2551) took 2297 ms (client timeout 1533 ms). Trace:
I20260709 13:03:51.868952  5879 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:03:51.975979  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:52.041797  2349 rpcz_store.cc:269] 0709 13:03:48.442720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.442914 (+   194us) service_pool.cc:224] Handling call
0709 13:03:51.892147 (+3449233us) spinlock_profiling.cc:243] Waited 40.8 ms on lock 0x7b54005451d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:51.892379 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93776128}
W20260709 13:03:51.976879  1669 debug-util.cc:398] Leaking SignalData structure 0x7b08002b00e0 after lost signal to thread 431
I20260709 13:03:51.748284  1720 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:03:52.046933  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:51.968449  1564 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.258s	user 0.052s	sys 0.097s
W20260709 13:03:51.967557  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:52.048278  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 45382.1us
W20260709 13:03:52.051256  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:52.051664  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:51.862743  1573 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2059) took 2937 ms (client timeout 2609 ms). Trace:
W20260709 13:03:51.903194  5951 log.cc:927] Time spent T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 2.649s	user 0.011s	sys 0.015s
I20260709 13:03:52.061733  1687 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:52.062629  1570 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:03:51.907920  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2105) took 3503 ms. Trace:
W20260709 13:03:52.040066  5939 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.784s	user 0.024s	sys 0.048s
I20260709 13:03:52.040233  1572 rpcz_store.cc:276] 0709 13:03:47.566695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.566832 (+   137us) service_pool.cc:224] Handling call
0709 13:03:47.567170 (+   338us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:47.669516 (+102346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:47.669558 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:47.683012 (+ 13454us) log.cc:844] Serialized 52 byte log entry
0709 13:03:47.742153 (+ 59141us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:47.902854 (+160701us) log.cc:844] Serialized 14 byte log entry
0709 13:03:47.904201 (+  1347us) log.cc:844] Serialized 14 byte log entry
0709 13:03:47.912393 (+  8192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.912479 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:51.015061 (+3102582us) spinlock_profiling.cc:243] Waited 7.3 ms on lock 0x7b18003320e0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d662a6 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1
0709 13:03:51.884196 (+869135us) raft_consensus.cc:1685] finished
0709 13:03:51.884232 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.884484 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101687,"prepare.queue_time_us":84441,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":114,"raft.queue_time_us":91688,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":15296,"spinlock_wait_cycles":27993088,"thread_start_us":274417,"threads_started":3,"wal-append.queue_time_us":217713,"wal-append.run_cpu_time_us":2900,"wal-append.run_wall_time_us":1003448}
I20260709 13:03:52.066298  1648 pending_rounds.cc:85] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 7
I20260709 13:03:52.066668  1648 pending_rounds.cc:107] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:03:52.066716  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:52.067305  1648 raft_consensus.cc:2889] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:52.067682  1572 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
W20260709 13:03:51.900406  5919 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.978s	user 0.008s	sys 0.028s
I20260709 13:03:51.763370  1642 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:03:51.980109  5807 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.994405  5877 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:52.038887  5811 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:52.039319  5989 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:52.049566  5879 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:51.989601  5956 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:52.015262  1569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2105) took 4356 ms (client timeout 3005 ms). Trace:
W20260709 13:03:51.989938  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 47008.4us
I20260709 13:03:51.998622  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:51.814360  1721 raft_consensus.cc:1275] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:03:51.947576  1685 leader_election.cc:336] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.196s (SENT)
W20260709 13:03:52.038041  1644 rpcz_store.cc:269] 0709 13:03:47.549739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.550097 (+   358us) service_pool.cc:224] Handling call
0709 13:03:51.762344 (+4212247us) spinlock_profiling.cc:243] Waited 617 ms on lock 0x7b54003566d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:51.762736 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1418996480}
W20260709 13:03:52.042951  1639 rpcz_store.cc:269] 0709 13:03:49.597580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.611387 (+ 13807us) service_pool.cc:224] Handling call
0709 13:03:51.894276 (+2282889us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b54001c0554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:51.894455 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30037760}
W20260709 13:03:51.962143  1571 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2125) took 3314 ms (client timeout 1780 ms). Trace:
I20260709 13:03:52.097759  1685 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:52.098331  1571 rpcz_store.cc:269] 0709 13:03:48.647497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.665985 (+ 18488us) service_pool.cc:224] Handling call
0709 13:03:51.961627 (+3295642us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b54001b3d54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:51.962056 (+   429us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2068548992}
I20260709 13:03:52.100014  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:52.053067  1573 rpcz_store.cc:269] 0709 13:03:48.924952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.587605 (+662653us) service_pool.cc:224] Handling call
0709 13:03:51.862351 (+2274746us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54004065d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:51.862565 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":351368064}
I20260709 13:03:52.038918  5834 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.056691  5861 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.048897  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:52.095149  1569 rpcz_store.cc:269] 0709 13:03:47.658682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.749869 (+ 91187us) service_pool.cc:224] Handling call
0709 13:03:52.014985 (+4265116us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b5400014354. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:52.015129 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1313672576}
I20260709 13:03:52.069594  1687 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:52.063633  1716 rpcz_store.cc:276] 0709 13:03:48.404713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.657694 (+1252981us) service_pool.cc:224] Handling call
0709 13:03:49.658034 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:49.661537 (+  3503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:49.661567 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:49.665142 (+  3575us) log.cc:844] Serialized 26 byte log entry
0709 13:03:49.800918 (+135776us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:49.800974 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.801059 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:51.668354 (+1867295us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:51.897247 (+228893us) raft_consensus.cc:1685] finished
0709 13:03:51.897285 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.907782 (+ 10497us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3826,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":209,"spinlock_wait_cycles":39187840,"thread_start_us":135327,"threads_started":1,"wal-append.queue_time_us":135187}
I20260709 13:03:52.072451  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:52.102646  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:52.103427  1570 raft_consensus.cc:2410] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:03:52.075956  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:52.106482  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2118) took 2299 ms. Trace:
I20260709 13:03:52.079519  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2126) took 3153 ms. Trace:
W20260709 13:03:52.107209  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 159826us
I20260709 13:03:51.995245  5852 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:52.081409  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:52.112180  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:52.082504  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.112848  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:52.101696  1720 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:52.113090  1573 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:52.046494  1669 debug-util.cc:398] Leaking SignalData structure 0x7b08000b1220 after lost signal to thread 434
I20260709 13:03:52.114004  1570 raft_consensus.cc:2410] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:52.107723  1571 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 10: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:03:52.115940  1685 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.692s (SENT)
I20260709 13:03:52.107036  1721 rpcz_store.cc:276] 0709 13:03:49.807101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.925311 (+1118210us) service_pool.cc:224] Handling call
0709 13:03:51.813594 (+888283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.097278 (+283684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.106388 (+  9110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.117241  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:52.117861  1721 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:52.107573  1572 rpcz_store.cc:276] 0709 13:03:48.926445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.078451 (+3152006us) service_pool.cc:224] Handling call
0709 13:03:52.078762 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.078943 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:52.078969 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:52.079182 (+   213us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:52.079228 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.079286 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.079394 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.118736  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:52.119886  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 677) took 3156 ms. Trace:
W20260709 13:03:52.119954  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 121423us
I20260709 13:03:52.120223  1571 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:52.043685  5921 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:52.072779  5957 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:51.996359  5850 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:52.070115  5833 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:52.076749  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 88344.1us
I20260709 13:03:52.120997  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 679) took 3157 ms. Trace:
I20260709 13:03:52.104372  1640 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:52.069000  5967 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:03:52.079874  5930 consensus_queue.cc:1059] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:03:52.120373  1572 rpcz_store.cc:276] 0709 13:03:48.963067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.118941 (+3155874us) service_pool.cc:224] Handling call
0709 13:03:52.119299 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.119513 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:52.119547 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:52.119600 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:52.119649 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.119708 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.119833 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:52.117538  1685 leader_election.cc:336] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.805s (SENT)
W20260709 13:03:52.120841  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 51340.4us
W20260709 13:03:52.091707  5919 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.169s	user 0.011s	sys 0.035s
I20260709 13:03:52.112047  5969 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:52.145730  1566 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 669) took 4593 ms (client timeout 1581 ms). Trace:
I20260709 13:03:52.095660  5989 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:52.158619  1687 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:52.111577  5861 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:52.159372  1572 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:52.159870  5957 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:52.160059  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2072) took 3149 ms. Trace:
I20260709 13:03:52.140882  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:52.142211  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:52.161048  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 43535.6us
W20260709 13:03:52.158000  1566 rpcz_store.cc:269] 0709 13:03:47.552071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.552205 (+   134us) service_pool.cc:224] Handling call
0709 13:03:52.145430 (+4593225us) spinlock_profiling.cc:243] Waited 901 ms on lock 0x7b54001b6054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:52.145622 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2071256320}
I20260709 13:03:52.138525  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:52.129259  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:52.157928  5969 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:52.181126  1722 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:52.166071  1571 rpcz_store.cc:276] 0709 13:03:48.963829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.119788 (+3155959us) service_pool.cc:224] Handling call
0709 13:03:52.120068 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.120777 (+   709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.120948 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.166706  1572 rpcz_store.cc:276] 0709 13:03:49.010493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.158677 (+3148184us) service_pool.cc:224] Handling call
0709 13:03:52.159046 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.159797 (+   751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.160003 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.046516  5882 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:52.181722  1685 leader_election.cc:336] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.862s (SENT)
I20260709 13:03:52.111943  5866 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:52.195143  1685 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:52.124603  5835 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:52.202678  1572 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:52.119760  5934 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.124876  5939 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:52.201161  1571 raft_consensus.cc:1275] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:52.237586  1722 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:52.241863  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:52.242457  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2074) took 2796 ms. Trace:
I20260709 13:03:52.242753  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:52.243052  1571 rpcz_store.cc:276] 0709 13:03:49.445570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.200303 (+2754733us) service_pool.cc:224] Handling call
0709 13:03:52.200708 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.242101 (+ 41393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.242382 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.092670  5877 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:52.157810  5850 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:52.244125  1685 leader_election.cc:336] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.065s (SENT)
I20260709 13:03:52.166481  5957 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.254766  1685 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:52.256827  1685 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:52.119315  5956 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:52.231511  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2080) took 2516 ms. Trace:
I20260709 13:03:52.257900  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 417) took 4265 ms. Trace:
I20260709 13:03:52.241336  1566 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:52.259325  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2073) took 2814 ms. Trace:
I20260709 13:03:52.260326  1566 rpcz_store.cc:276] 0709 13:03:49.444761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.197419 (+2752658us) service_pool.cc:224] Handling call
0709 13:03:52.241069 (+ 43650us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.258956 (+ 17887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.259246 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.242462  1717 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:52.216480  5835 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.096077  5811 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:52.265906  1569 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:03:52.085949  5807 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:52.213964  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.118273  5834 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:52.258050  1572 rpcz_store.cc:276] 0709 13:03:49.715598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.195085 (+2479487us) service_pool.cc:224] Handling call
0709 13:03:52.195481 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.231187 (+ 35706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.231442 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.166401  5833 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:52.266757  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 192375us
I20260709 13:03:52.281030  1572 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:52.247221  1573 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:52.282066  1570 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: [7.6-9.7]
I20260709 13:03:52.261940  5957 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:52.284850  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 684) took 1016 ms. Trace:
I20260709 13:03:52.271358  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:52.272663  5835 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:52.226029  5866 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:52.276788  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 154440us
I20260709 13:03:52.125128  5852 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:52.291865  6011 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:03:52.293138  5852 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:52.294878  5807 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:52.296976  5882 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:52.298020  1717 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:52.299099  5927 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:52.299738  5927 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:52.302116  5927 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:52.302661  5927 consensus_queue.cc:1059] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:52.303267  5850 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:52.304127  5850 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.304653  5850 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:52.240206  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2123) took 3593 ms. Trace:
I20260709 13:03:52.250732  5960 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:52.284512  1719 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:52.264786  5779 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:52.254877  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 88930.6us
I20260709 13:03:52.242825  5795 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:52.287266  1569 rpcz_store.cc:276] 0709 13:03:51.267980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.283189 (+1015209us) service_pool.cc:224] Handling call
0709 13:03:52.283623 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.283838 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:52.283885 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:52.283939 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:52.283986 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.284667 (+   681us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.284790 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.275341  5939 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:52.303393  5927 consensus_queue.cc:1243] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer cf2f71592be8465fb18e36e15d0b2f89 log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:03:52.299732  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:52.239666  6025 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.285167  1566 raft_consensus.cc:3055] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:52.338677  1566 raft_consensus.cc:740] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:52.249317  5934 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:52.258610  1643 rpcz_store.cc:276] 0709 13:03:47.986404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.038007 (+ 51603us) service_pool.cc:224] Handling call
0709 13:03:48.038259 (+   252us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:48.079148 (+ 40889us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.079187 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:52.016911 (+3937724us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b18001320a0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d662a6 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824777ea11 00007f824778e372 00007f8248ec0a01 00007f8248e8543a
0709 13:03:52.018367 (+  1456us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b2c000120e8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824777ea11 00007f824778e372 00007f8248ec0a01 00007f8248e8543a 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2
0709 13:03:52.019090 (+   723us) log.cc:844] Serialized 26 byte log entry
0709 13:03:52.185853 (+166763us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:52.239524 (+ 53671us) log.cc:844] Serialized 14 byte log entry
0709 13:03:52.250505 (+ 10981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.250664 (+   159us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:52.251102 (+   438us) raft_consensus.cc:1685] finished
0709 13:03:52.251150 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.251420 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84168,"raft.queue_time_us":41331,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":8290,"spinlock_wait_cycles":7342208,"thread_start_us":162674,"threads_started":3,"wal-append.queue_time_us":181445}
I20260709 13:03:52.305821  1567 rpcz_store.cc:276] 0709 13:03:48.646854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.948245 (+3301391us) service_pool.cc:224] Handling call
0709 13:03:51.948708 (+   463us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:51.998824 (+ 50116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:51.998855 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:51.999506 (+   651us) log.cc:844] Serialized 52 byte log entry
0709 13:03:52.046216 (+ 46710us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:52.227970 (+181754us) log.cc:844] Serialized 14 byte log entry
0709 13:03:52.239844 (+ 11874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.239922 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:52.239980 (+    58us) raft_consensus.cc:1685] finished
0709 13:03:52.239993 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.240141 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43932,"prepare.queue_time_us":51559,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":99,"raft.queue_time_us":44576,"raft.run_cpu_time_us":392,"raft.run_wall_time_us":391,"thread_start_us":209826,"threads_started":3,"wal-append.queue_time_us":55012}
I20260709 13:03:52.345995  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:52.346686  1567 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:03:52.347605  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:52.348181  1567 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:52.314630  5921 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:52.312948  1537 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:52.324713  5779 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:52.334846  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:52.356374  1716 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:03:52.329326  2312 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:03:52.336028  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 79297us
I20260709 13:03:52.329355  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:52.354359  1537 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:52.349589  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:52.359318  1685 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:52.350519  1719 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:03:52.339526  1566 consensus_queue.cc:260] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:52.286008  5854 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:52.360888  1566 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:52.360487  1569 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:52.362525  2312 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:52.364089  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2125) took 2553 ms. Trace:
I20260709 13:03:52.364547  1717 rpcz_store.cc:276] 0709 13:03:49.810700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.344335 (+2533635us) service_pool.cc:224] Handling call
0709 13:03:52.363489 (+ 19154us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.363697 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:52.363723 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:52.363767 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:52.363835 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.363907 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.364028 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.359926  1567 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:03:52.367308  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2121) took 3737 ms. Trace:
I20260709 13:03:52.369263  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:52.368186  1564 rpcz_store.cc:276] 0709 13:03:48.629394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.704258 (+1074864us) service_pool.cc:224] Handling call
0709 13:03:49.708615 (+  4357us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:49.709096 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:49.709127 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:49.709839 (+   712us) log.cc:844] Serialized 52 byte log entry
0709 13:03:52.050167 (+2340328us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:52.057337 (+  7170us) log.cc:844] Serialized 14 byte log entry
0709 13:03:52.366475 (+309138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.366620 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:52.366683 (+    63us) raft_consensus.cc:1685] finished
0709 13:03:52.366702 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.367228 (+   526us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7398,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":5654,"raft.queue_time_us":68266,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":5727,"thread_start_us":2648016,"threads_started":3,"wal-append.queue_time_us":496646,"wal-append.run_cpu_time_us":1905,"wal-append.run_wall_time_us":1001928}
I20260709 13:03:52.356449  5852 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:52.351361  5927 raft_consensus.cc:3055] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:52.375583  1685 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:52.384016  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2127) took 2558 ms. Trace:
I20260709 13:03:52.398451  1717 rpcz_store.cc:276] 0709 13:03:49.826174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.365513 (+2539339us) service_pool.cc:224] Handling call
0709 13:03:52.383460 (+ 17947us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.383673 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:52.383699 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:52.383743 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:52.383785 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.383839 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.383950 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.382297  1716 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:03:52.399930  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2126) took 2589 ms. Trace:
I20260709 13:03:52.400287  1716 rpcz_store.cc:276] 0709 13:03:49.810931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.357332 (+2546401us) service_pool.cc:224] Handling call
0709 13:03:52.381912 (+ 24580us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:52.399637 (+ 17725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.399874 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.382121  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:52.382390  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 101015us
I20260709 13:03:52.372361  5747 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:52.357589  5779 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:52.402874  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:52.409480  1569 raft_consensus.cc:2410] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:52.328136  5960 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:52.413210  1538 connection.cc:256] RPC call timeout handler was delayed by -1.0170287149999808s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1533s for a response
W20260709 13:03:52.413914  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.413875  1569 raft_consensus.cc:1275] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:03:52.376174  5854 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:52.414769  1569 pending_rounds.cc:85] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 5
I20260709 13:03:52.435245  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:52.377799  6024 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:52.408679  1685 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:52.438848  1717 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:52.379870  6028 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:52.436770  1569 pending_rounds.cc:107] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:03:52.354033  6025 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:52.437954  1564 raft_consensus.cc:2393] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 12: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:03:52.422109  1538 leader_election.cc:336] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.564s (SENT)
I20260709 13:03:52.439517  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2128) took 2546 ms. Trace:
I20260709 13:03:49.814518  1645 rpcz_store.cc:276] 0709 13:03:48.581133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.551525 (+970392us) service_pool.cc:224] Handling call
0709 13:03:49.552115 (+   590us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:49.552371 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:49.651417 (+ 99046us) log.cc:844] Serialized 14 byte log entry
0709 13:03:49.808075 (+156658us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:49.808139 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:49.808197 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.808269 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.808403 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":99674,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":94,"raft.queue_time_us":6750,"thread_start_us":253522,"threads_started":2,"wal-append.queue_time_us":147827}
I20260709 13:03:52.440546  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2129) took 2509 ms. Trace:
I20260709 13:03:52.413111  5921 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:52.441942  1566 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:03:52.480423  1569 raft_consensus.cc:2889] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:52.482327  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 95034us
I20260709 13:03:52.412277  1721 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:03:52.443041  1612 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:52.469451  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.407897  5879 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:52.480342  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 58306.1us
I20260709 13:03:52.397987  5927 raft_consensus.cc:740] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:52.415127  5795 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.094s
I20260709 13:03:52.415272  5939 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 13:03:52.414980  5956 consensus_queue.cc:1059] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.113s
I20260709 13:03:52.479835  1717 rpcz_store.cc:276] 0709 13:03:49.893916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.399404 (+2505488us) service_pool.cc:224] Handling call
0709 13:03:52.438668 (+ 39264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.439254 (+   586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.439455 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:52.593585  1611 connection.cc:256] RPC call timeout handler was delayed by -2.0258809160000624s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22080000000000002s for a response
W20260709 13:03:52.601720  1611 connection.cc:256] RPC call timeout handler was delayed by -1.9099561859999881s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1767s for a response
W20260709 13:03:52.590945  1646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2545) took 4147 ms (client timeout 2091 ms). Trace:
I20260709 13:03:52.537791  1567 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:03:52.566246  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 403) took 4191 ms. Trace:
I20260709 13:03:52.505268  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2085) took 1523 ms. Trace:
I20260709 13:03:52.415441  5747 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:52.479933  1716 rpcz_store.cc:276] 0709 13:03:49.931511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.400995 (+2469484us) service_pool.cc:224] Handling call
0709 13:03:52.440057 (+ 39062us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.440254 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:52.440278 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:52.440320 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:52.440357 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.440404 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.440492 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.628093  5861 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:52.506973  6028 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:52.602444  1611 connection.cc:256] RPC call timeout handler was delayed by -1.8226561860000174s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1864s for a response
W20260709 13:03:52.507980  6020 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:03:52.629163  5861 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:52.602384  1646 rpcz_store.cc:269] 0709 13:03:48.443320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.944581 (+501261us) service_pool.cc:224] Handling call
0709 13:03:52.590692 (+3646111us) spinlock_profiling.cc:243] Waited 795 ms on lock 0x7b5400098654. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:52.590853 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1827592960}
I20260709 13:03:52.631083  1717 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:03:52.461230  5833 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:52.606571  5844 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:52.527241  5877 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:52.450591  5854 raft_consensus.cc:697] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:52.647770  5893 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:52.655506  5854 consensus_queue.cc:237] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:52.585443  5722 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:52.656805  5722 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:52.630798  5861 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:52.659372  5861 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:52.659736  5722 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:52.416229  5852 raft_consensus.cc:697] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:52.660313  5844 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:52.407224  5930 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:52.629453  6028 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:52.415028  6016 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:03:52.638126  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 195176us
I20260709 13:03:52.662143  6028 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:52.664242  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:52.662413  6016 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:52.666563  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:52.668246  6016 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:52.668463  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:52.669260  1646 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:52.670626  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:52.638582  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 230037us
W20260709 13:03:52.602165  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.490566  1721 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:52.573693   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:52.623179  1566 rpcz_store.cc:276] 0709 13:03:50.982319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.270140 (+1287821us) service_pool.cc:224] Handling call
0709 13:03:52.285056 (+ 14916us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.482438 (+197382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.505186 (+ 22748us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":771328}
I20260709 13:03:52.622820  1714 rpcz_store.cc:276] 0709 13:03:48.375053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.004994 (+629941us) service_pool.cc:224] Handling call
0709 13:03:49.005285 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:49.005595 (+   310us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:49.005692 (+    97us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:49.023914 (+ 18222us) log.cc:844] Serialized 26 byte log entry
0709 13:03:49.082603 (+ 58689us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:49.082670 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.082771 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:51.902365 (+2819594us) spinlock_profiling.cc:243] Waited 27.6 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:52.565751 (+663386us) raft_consensus.cc:1685] finished
0709 13:03:52.565786 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.566147 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":760,"raft.queue_time_us":18001,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":23914,"spinlock_wait_cycles":63396096,"thread_start_us":43274,"threads_started":1,"wal-append.queue_time_us":221114}
I20260709 13:03:52.673799  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2132) took 2434 ms. Trace:
I20260709 13:03:52.446614  5969 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:52.637883  6013 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:52.437801  6021 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.232s
I20260709 13:03:52.629226  1716 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:52.461066  5811 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:52.645954  1538 leader_election.cc:336] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.815s (SENT)
I20260709 13:03:52.685555  1538 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:52.481127  6017 log.cc:927] Time spent T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:03:52.622797  5875 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.602000  6032 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.559990  5834 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:52.438495  6024 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.468281  6034 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:52.698127  6024 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:52.698478  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:52.698326  6034 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:52.701500  6034 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:52.675621  1721 rpcz_store.cc:276] 0709 13:03:50.239478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.484607 (+2245129us) service_pool.cc:224] Handling call
0709 13:03:52.490356 (+  5749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.673369 (+183013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.673718 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.678664  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:52.679416  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:52.671804  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:52.703382  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:52.703917  1640 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:52.704075  1646 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:52.489704  5921 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:52.447170  5882 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.189s
I20260709 13:03:52.680403  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:52.708258  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:52.708703  1641 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:52.646240  1611 connection.cc:256] RPC call timeout handler was delayed by -1.8088604650000661s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17800000000000002s for a response
W20260709 13:03:52.709571  1611 connection.cc:256] RPC call timeout handler was delayed by -1.7790440910000029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3005s for a response
W20260709 13:03:52.709926  1611 connection.cc:256] RPC call timeout handler was delayed by -1.7542561860000205s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.194s for a response
I20260709 13:03:52.681759  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:52.682924  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2466) took 1436 ms. Trace:
I20260709 13:03:52.710870  1639 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:03:52.710770  1716 rpcz_store.cc:276] 0709 13:03:51.246045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.628609 (+1382564us) service_pool.cc:224] Handling call
0709 13:03:52.629030 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.682503 (+ 53473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.682838 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:52.687392  1538 leader_election.cc:336] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.953s (SENT)
I20260709 13:03:52.712349  1538 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:49.528036  1715 rpcz_store.cc:276] 0709 13:03:48.375326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.010266 (+634940us) service_pool.cc:224] Handling call
0709 13:03:49.013492 (+  3226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:49.013739 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:49.323468 (+309729us) log.cc:844] Serialized 14 byte log entry
0709 13:03:49.508794 (+185326us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:49.508872 (+    78us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:49.508940 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.509040 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.509162 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14708,"prepare.queue_time_us":51999,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":62,"raft.queue_time_us":13327,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":9718,"thread_start_us":446125,"threads_started":2}
I20260709 13:03:52.689599  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:52.649515  6027 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:52.686970   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:52.661365  5852 consensus_queue.cc:237] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:52.693171  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:52.694048  5875 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:52.685499  6013 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:52.731658  6013 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:52.696945  6032 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:52.732539  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:52.715838  1643 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
I20260709 13:03:52.733348  2342 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 5.
I20260709 13:03:52.734395  6032 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:52.690307  6016 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:52.734733  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:52.736447  5875 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:52.718766  2314 connection.cc:256] RPC call timeout handler was delayed by -1.0019215519999989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2114s for a response
I20260709 13:03:52.713691  5811 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:52.732354  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:52.733877  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:52.750548  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:52.751593  2342 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 5.
I20260709 13:03:52.720355  5834 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:52.754410  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:52.755774  2341 raft_consensus.cc:2410] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:03:52.756880  2349 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:03:52.757340  5592 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:52.716645  1611 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:52.760761  5833 raft_consensus.cc:2764] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:03:52.761606  5875 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:52.712536  5866 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:52.662505  5927 consensus_queue.cc:260] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:52.704546  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:52.740526  1573 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:52.738847  1644 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:52.760864  1611 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:52.745237  1641 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:52.728263  1538 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:52.746012  2314 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:52.713378  5921 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:52.720665  6027 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.802479  1640 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:52.805508  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:52.806821  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.808775  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:52.818379  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 90201.6us
I20260709 13:03:52.748257  5811 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:52.819658  1538 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:52.826458  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 65677.9us
I20260709 13:03:52.827481  1611 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:52.802141  6027 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:52.826284  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:52.807319  5904 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:52.820461  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:52.832187  5875 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:52.821197  5930 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:52.825117  5866 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:52.828342  5811 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:52.809831  6021 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:03:52.836091  1538 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:52.839768  1537 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:52.832989  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:52.840974  5854 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:52.864356  5934 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:52.866734  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2063) took 3892 ms. Trace:
I20260709 13:03:52.885346  1570 rpcz_store.cc:276] 0709 13:03:48.973964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.122148 (+3148184us) service_pool.cc:224] Handling call
0709 13:03:52.281806 (+159658us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:52.282490 (+   684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.866472 (+583982us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54004056d0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:52.866657 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":366502144}
I20260709 13:03:52.831766  5904 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:52.897929  5904 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:52.898808  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 194334us
W20260709 13:03:52.900696  1669 debug-util.cc:398] Leaking SignalData structure 0x7b08000a2a20 after lost signal to thread 1505
W20260709 13:03:52.901875  1686 connection.cc:256] RPC call timeout handler was delayed by -1.6782518180000352s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
W20260709 13:03:52.902423  1686 connection.cc:256] RPC call timeout handler was delayed by -1.4055518179999922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18080000000000002s for a response
I20260709 13:03:52.905102  5967 raft_consensus.cc:3055] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:52.830844  6039 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:03:52.850045  6025 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:52.905498  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:52.841756  6041 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:52.906363  6041 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.906541  1715 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 10: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:52.906747  6041 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:52.907657  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:52.908279  1716 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
W20260709 13:03:52.908715  5957 consensus_meta.cc:287] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.601s	user 0.005s	sys 0.008s
I20260709 13:03:52.889400  5854 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:52.909245  5957 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:52.909946  5854 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:52.910441  5854 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:52.911716  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:52.912343  1713 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:52.912670  5724 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:52.846097  5844 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:52.849298  5866 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:52.930485  5866 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:52.896462  5835 consensus_meta.cc:287] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.572s	user 0.011s	sys 0.001s
I20260709 13:03:52.905608  5967 raft_consensus.cc:740] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:52.932878  5835 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:52.933707  5967 consensus_queue.cc:260] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:52.935395  5967 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:52.935190  1686 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:52.936450  5835 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:52.876909  5850 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:52.913230  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:52.846302  5930 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.913888  5795 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:52.918265  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2472) took 1222 ms. Trace:
I20260709 13:03:52.975986  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:52.976141  1719 rpcz_store.cc:276] 0709 13:03:51.696356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.046670 (+350314us) service_pool.cc:224] Handling call
0709 13:03:52.918104 (+871434us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b54002990d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:52.918206 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":544777472}
I20260709 13:03:52.923230  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:52.979888  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 250837us
I20260709 13:03:52.984503  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2122) took 4354 ms. Trace:
I20260709 13:03:52.929180  5960 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:52.937204  1571 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:52.747305  5779 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:52.948068  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 459278us
W20260709 13:03:52.884383  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 48367us
W20260709 13:03:52.957985  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 120595us
I20260709 13:03:52.988368  1565 rpcz_store.cc:276] 0709 13:03:48.646447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.940024 (+3293577us) service_pool.cc:224] Handling call
0709 13:03:51.940399 (+   375us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:51.989486 (+ 49087us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:52.007882 (+ 18396us) log.cc:844] Serialized 14 byte log entry
0709 13:03:52.079137 (+ 71255us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:52.094132 (+ 14995us) log.cc:844] Serialized 52 byte log entry
0709 13:03:52.094305 (+   173us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:52.695298 (+600993us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b180033e3e0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f824c87d3c7 00007f8248e8f9a7 00007f8248e9f0bf 00007f8248e9f038
0709 13:03:52.981868 (+286570us) log.cc:844] Serialized 14 byte log entry
0709 13:03:52.983974 (+  2106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.984074 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:52.984229 (+   155us) raft_consensus.cc:1685] finished
0709 13:03:52.984244 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.984433 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22847,"prepare.queue_time_us":117673,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":109,"raft.queue_time_us":49334,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":9456,"spinlock_wait_cycles":5550464,"thread_start_us":372949,"threads_started":3,"wal-append.queue_time_us":146432}
W20260709 13:03:53.012058  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.012984  1537 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:52.842509  5919 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:52.978240  2349 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:03:52.928601  6028 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:52.982971  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 158942us
I20260709 13:03:53.016602  5850 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:53.017557  5850 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:03:52.991461  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 151693us
I20260709 13:03:52.997167  1612 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:52.842648  5927 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:53.005079  1713 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:53.019223  1537 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:53.020917  1537 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:53.018137  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:53.016616  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:53.018530  5850 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:52.974918  5930 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:53.022424  1537 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:53.032135  1537 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:53.038846  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:53.044929  1573 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:03:53.066355  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 69293.9us
I20260709 13:03:53.068076  1611 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:53.071964  1537 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:52.950881  5940 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.085191  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:53.086324  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:53.087328  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:52.975152  5939 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:52.985033  6025 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:52.985321  6005 raft_consensus.cc:3055] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:52.933028  5854 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:53.095889  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:53.002283  6029 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.096647  1648 raft_consensus.cc:2368] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11.
I20260709 13:03:53.097527  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:53.097723  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:53.098867  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:53.099308  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:53.099742  1640 raft_consensus.cc:2376] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9. Re-sending same reply.
I20260709 13:03:53.100178  1648 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:53.100741  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:53.101328  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:53.104542  1537 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:53.105713  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:52.936641  6041 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:53.111929  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 43977.4us
I20260709 13:03:53.120512  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:53.120568  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:53.121399  1611 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:52.905119  6036 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.187s
W20260709 13:03:53.132572  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.118811  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:53.134233  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:53.083616  5835 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:53.125264  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:53.028463  5934 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:53.118531  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:53.167312  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:53.119572  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:53.169866  1573 raft_consensus.cc:2410] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:03:53.124356  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:53.171767  1646 raft_consensus.cc:2376] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10. Re-sending same reply.
I20260709 13:03:53.139362  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:53.143235  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:53.173898  1719 raft_consensus.cc:2393] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 12: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:53.090231  5834 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:53.147778  1713 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:53.148066  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:53.151643  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.155916  1611 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:53.164849  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:53.101935  5876 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:53.187564  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:53.188539  1715 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:53.003454  5795 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:53.103504  5592 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:53.167635  1565 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:53.095755  6025 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.096141  6005 raft_consensus.cc:740] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:53.194448  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:53.196048  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:53.198976  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:53.200276  1715 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:53.202579  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:53.203894  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:53.120718  6030 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:53.123909  5724 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:53.207376  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:53.165410  6015 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:53.222821  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:53.230072  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:53.235318  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:53.095068  5844 consensus_queue.cc:1059] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.098s
I20260709 13:03:53.236487  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:53.237934  1719 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:53.180384  5957 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:53.226907  6030 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:53.239732  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.240913  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:53.214104  1538 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:53.458755  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:53.463745  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 249645us
I20260709 13:03:53.444468  1570 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
W20260709 13:03:53.443486  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:53.464238  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.469887  1570 pending_rounds.cc:85] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 11
W20260709 13:03:53.446702  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 290751us
I20260709 13:03:53.474179  1570 pending_rounds.cc:107] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:03:53.475761  1570 raft_consensus.cc:2889] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:53.478689  1611 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:53.479661  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.479763  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2083) took 3760 ms. Trace:
I20260709 13:03:53.480408  1571 rpcz_store.cc:276] 0709 13:03:49.719684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.244303 (+2524619us) service_pool.cc:224] Handling call
0709 13:03:52.244675 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.002324 (+757649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.479060 (+476736us) spinlock_profiling.cc:243] Waited 692 ms on lock 0x7b54002251d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:53.479515 (+   455us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1592358272}
I20260709 13:03:53.482372  1538 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:53.487087  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2538) took 5092 ms. Trace:
I20260709 13:03:53.488181  1642 rpcz_store.cc:276] 0709 13:03:48.394168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.293665 (+899497us) service_pool.cc:224] Handling call
0709 13:03:51.762792 (+2469127us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:52.074253 (+311461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.486201 (+1411948us) spinlock_profiling.cc:243] Waited 2.43 s on lock 0x7b54003566d0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:53.486571 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5600179968}
W20260709 13:03:53.504177  1669 debug-util.cc:398] Leaking SignalData structure 0x7b08003a1980 after lost signal to thread 1597
W20260709 13:03:53.504932  1669 debug-util.cc:398] Leaking SignalData structure 0x7b08001077e0 after lost signal to thread 1598
W20260709 13:03:53.508240  1669 debug-util.cc:398] Leaking SignalData structure 0x7b0800394900 after lost signal to thread 1672
W20260709 13:03:53.510995  1669 debug-util.cc:398] Leaking SignalData structure 0x7b080032b880 after lost signal to thread 1673
I20260709 13:03:53.515388  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:53.516484  1564 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:53.519191  1669 debug-util.cc:398] Leaking SignalData structure 0x7b080030aaa0 after lost signal to thread 1746
I20260709 13:03:53.291692  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:53.311962  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:53.566171  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:53.566934  2341 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:03:53.300535  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 411) took 4327 ms. Trace:
W20260709 13:03:53.567169  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 45820.6us
I20260709 13:03:53.304912  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:53.570946  1719 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:53.571314  6074 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:53.573098  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:53.196775  1713 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:53.575055  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:53.526015  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:53.568099  1718 rpcz_store.cc:276] 0709 13:03:48.973081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.913767 (+1940686us) service_pool.cc:224] Handling call
0709 13:03:50.915560 (+  1793us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.916182 (+   622us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:52.837979 (+1921797us) log.cc:844] Serialized 14 byte log entry
0709 13:03:53.251630 (+413651us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:53.251715 (+    85us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:53.251785 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.300137 (+ 48352us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.300455 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":295886,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":273207,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":9220,"thread_start_us":494234,"threads_started":3,"wal-append.queue_time_us":385679}
I20260709 13:03:53.531015  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:53.538411  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:53.582230  1722 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:53.583510  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:53.584219  1722 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:03:53.586848  1537 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:53.587875  1719 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:53.522720  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:53.546567  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 459252us
I20260709 13:03:53.300438  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:53.620608  1570 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:53.622169  1467 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47518 (request call id 97) took 21792 ms (client timeout 15000 ms). Trace:
I20260709 13:03:53.522789  1687 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:53.627375  6016 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:03:53.565426  1669 debug-util.cc:398] Leaking SignalData structure 0x7b08001fa640 after lost signal to thread 1747
I20260709 13:03:53.567392  2345 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:53.250005  5592 raft_consensus.cc:697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:53.647490  5592 consensus_queue.cc:237] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:53.649749  6015 raft_consensus.cc:697] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:53.652755  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:53.652328  6015 consensus_queue.cc:237] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:53.181375  5722 raft_consensus.cc:2764] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:03:53.237263  5724 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.678948  5724 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:53.677111  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 184720us
I20260709 13:03:53.182308  5834 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.191708  5835 raft_consensus.cc:697] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:03:53.682031  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 448723us
I20260709 13:03:53.188393  5879 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:53.353350  5904 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:53.683702  5835 consensus_queue.cc:237] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:53.374027  5930 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:53.687674  5904 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:53.313944  5921 consensus_meta.cc:287] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.504s	user 0.011s	sys 0.000s
W20260709 13:03:53.678331  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 441945us
I20260709 13:03:53.677994  1537 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:53.096809  5940 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:53.302657  5956 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:53.300686  5960 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:53.186239  5939 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:53.814230  5939 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:53.815038  2313 leader_election.cc:336] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.564s (SENT)
I20260709 13:03:53.815659  1717 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:53.344316  5957 consensus_queue.cc:1059] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:03:53.818630  5957 raft_consensus.cc:3055] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:53.817320  5939 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:53.332374  6005 consensus_queue.cc:260] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:53.300818  6024 consensus_meta.cc:287] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.592s	user 0.009s	sys 0.003s
I20260709 13:03:53.225308  6025 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:53.388818  6027 consensus_meta.cc:287] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.550s	user 0.015s	sys 0.001s
I20260709 13:03:53.120918  6029 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:53.291078  6030 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:53.327349  6036 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:03:53.840725  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 93244.5us
I20260709 13:03:53.359475  6041 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:53.316992  6042 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.472s	user 0.000s	sys 0.002s
I20260709 13:03:53.301692  6061 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.846125  1721 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:03:53.846561  6061 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:53.820010  5722 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:03:53.632543  1564 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:03:53.849325  1716 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:03:53.627916  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 1.46832e+06us
I20260709 13:03:53.833535  6029 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:53.636240  2349 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:03:53.636734  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:53.852501  1687 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:53.853138  2348 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:03:53.580093  6074 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:03:53.652362  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.854197  6059 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:53.388263  5876 raft_consensus.cc:697] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:03:53.668254  1686 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.010s	sys 0.008s
W20260709 13:03:53.856560  1686 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.011s	sys 0.008s
W20260709 13:03:53.857216  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 920503us
I20260709 13:03:53.854081  6074 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:53.857896  5876 consensus_queue.cc:237] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:53.676882  1722 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:53.682132  5834 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:53.861006  6063 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:53.822697  6031 consensus_queue.cc:1059] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:03:53.863023  1568 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:53.356066  5854 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:53.679447  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 200941us
I20260709 13:03:53.866575  5956 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:53.867301  5956 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:53.867599  1611 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:53.693846  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 78355.7us
I20260709 13:03:53.819155  5957 raft_consensus.cc:740] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:03:53.656458  5866 consensus_meta.cc:287] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.725s	user 0.013s	sys 0.001s
W20260709 13:03:53.747924  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 53719.9us
I20260709 13:03:53.870784  5866 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:53.863561  6063 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:03:53.872786  6021 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:53.748952  1570 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:03:53.875145  6073 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:53.878614  5866 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:53.879899  5882 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:53.880167  1611 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:53.880462  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:53.881542  1642 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:03:53.881671  1570 raft_consensus.cc:1240] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:53.882040  6066 consensus_queue.cc:1059] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:53.882289  6064 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:53.883546  6066 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:53.884166  6066 raft_consensus.cc:697] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:53.884958  6067 raft_consensus.cc:3055] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:53.885097  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:53.887482  1642 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:53.695549  5921 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:53.767318  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.693854  5919 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:53.689597  5930 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:53.694340  5779 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:53.690114  5795 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:53.698721  6069 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:53.709406  6016 raft_consensus.cc:697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:53.808894  5678 raft_consensus.cc:3055] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:53.813067  5747 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:53.851848  5989 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:53.635380  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.637122  5833 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:53.823312  6075 consensus_queue.cc:1059] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:03:53.869570  1538 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:53.696769  1669 debug-util.cc:398] Leaking SignalData structure 0x7b0800349640 after lost signal to thread 2373
W20260709 13:03:53.698205  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 55590.3us
I20260709 13:03:53.694626  6079 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:53.886092  1720 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:53.819139  5841 consensus_queue.cc:1048] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:03:53.911490  6024 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:53.912248  6027 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:53.915709  1564 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: []
W20260709 13:03:53.915882  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.841355  6036 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:53.918426  5854 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:53.885404  6066 consensus_queue.cc:237] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:53.919220  5850 consensus_meta.cc:287] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.808s	user 0.017s	sys 0.000s
W20260709 13:03:53.622774  1467 rpcz_store.cc:269] 0709 13:03:31.829803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.873886 (+ 44083us) service_pool.cc:224] Handling call
0709 13:03:53.621783 (+21747897us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:48.354812 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:48.509173 (+154361us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:48.511720 (+  2547us) write_op.cc:435] Acquired schema lock
0709 13:03:48.535237 (+ 23517us) tablet.cc:662] Decoding operations
0709 13:03:48.648294 (+113057us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:48.648358 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:48.648398 (+    40us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:03:48.649196 (+   798us) tablet.cc:701] Row locks acquired
0709 13:03:48.649216 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:03:48.649388 (+   172us) write_op.cc:270] Start()
0709 13:03:48.649510 (+   122us) write_op.cc:276] Timestamp: P: 1783602228649333 usec, L: 0
0709 13:03:48.649531 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:03:48.718865 (+ 69334us) log.cc:844] Serialized 2898 byte log entry
0709 13:03:50.984098 (+2265233us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:51.011486 (+ 27388us) op_driver.cc:464] REPLICATION: finished
0709 13:03:51.021327 (+  9841us) write_op.cc:301] APPLY: starting
0709 13:03:51.021420 (+    93us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:51.025622 (+  4202us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:51.025683 (+    61us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:51.031474 (+  5791us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:51.031736 (+   262us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 13:03:51.031783 (+    47us) write_op.cc:312] APPLY: finished
0709 13:03:51.032095 (+   312us) log.cc:844] Serialized 73 byte log entry
0709 13:03:51.033131 (+  1036us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:51.033288 (+   157us) write_op.cc:454] Released schema lock
0709 13:03:51.033353 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:51.034088 (+   735us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:51.034125 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1092362,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":9498,"apply.run_cpu_time_us":11521,"apply.run_wall_time_us":12735,"cfile_cache_hit":30,"cfile_cache_hit_bytes":33401,"num_ops":7,"prepare.queue_time_us":173873,"prepare.run_cpu_time_us":76909,"prepare.run_wall_time_us":737118,"raft.queue_time_us":20128,"raft.run_cpu_time_us":7578,"raft.run_wall_time_us":12860,"replication_time_us":2361729,"spinlock_wait_cycles":39772032,"thread_start_us":274396,"threads_started":4,"wal-append.queue_time_us":295697,"wal-append.run_cpu_time_us":21046,"wal-append.run_wall_time_us":2861217}]]}
I20260709 13:03:53.926765  6013 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:03:53.923715  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 71544.8us
I20260709 13:03:53.928864   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:53.931283  5940 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:53.999843  1646 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:53.880704  5882 raft_consensus.cc:697] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 13 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:54.000823  6005 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:54.001628  5882 consensus_queue.cc:237] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:54.003656  6054 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.751s	user 0.003s	sys 0.000s
I20260709 13:03:53.842155  1685 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:53.946602  5678 raft_consensus.cc:740] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:53.885537  6067 raft_consensus.cc:740] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:53.953826  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 84492.7us
I20260709 13:03:53.957728  1713 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:03:53.946856  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:53.959106  6068 consensus_queue.cc:1059] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:03:53.973801  1719 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:53.984445  5921 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:53.988104  5919 raft_consensus.cc:3055] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:53.988407  5957 consensus_queue.cc:260] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:54.043015  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.966336  6079 raft_consensus.cc:697] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:03:53.989908  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.990240  5850 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:53.993060  1687 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:53.993976  6036 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:03:53.995914  6027 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:54.005678  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 45271.1us
I20260709 13:03:53.908638  6069 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.094290  6069 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:53.967363  6061 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:54.051345  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:54.105121  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:54.110806  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 117881us
I20260709 13:03:54.111385  1537 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:54.111779  5850 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:54.111954  6016 consensus_queue.cc:237] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:54.113900  5777 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:54.114732  6028 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:54.114985  5777 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:54.115290  6024 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:54.115510  6082 log.cc:927] Time spent T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:03:54.115723  6028 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:54.116171  6061 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:03:54.116809  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 274925us
I20260709 13:03:54.118016  1685 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:54.119279  6079 consensus_queue.cc:237] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:54.124320  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:54.124927  1722 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:03:54.126060  6067 consensus_queue.cc:260] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:54.126870  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:54.127807  1722 raft_consensus.cc:2376] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13. Re-sending same reply.
I20260709 13:03:54.128357  1721 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:54.128636  1538 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:54.048781  2313 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 2.034s (SENT) [suppressed 39 similar messages]
I20260709 13:03:54.130604  6069 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:54.131415  1714 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:03:54.132133  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2109) took 1012 ms. Trace:
I20260709 13:03:54.123441  1537 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:54.134033  2347 raft_consensus.cc:2393] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 12: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:03:54.134449  6073 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:54.134255  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:54.135375  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55890 (request call id 427) took 1119 ms. Trace:
I20260709 13:03:54.135454  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:54.136816  1564 raft_consensus.cc:2393] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 12: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:54.137012  2349 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 13:03:54.137980  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 161308us
I20260709 13:03:54.135962  2347 rpcz_store.cc:276] 0709 13:03:53.016079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.016223 (+   144us) service_pool.cc:224] Handling call
0709 13:03:54.135099 (+1118876us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540035f2d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:54.135285 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2321050752}
I20260709 13:03:54.142611  1537 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:54.143816  1537 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:54.127837  6067 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:54.144891  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:54.145991  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:54.146063  5592 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:54.146943  1643 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:54.146991  5592 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:54.147740  5592 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:54.138022  6047 raft_consensus.cc:3055] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:54.149094  6047 raft_consensus.cc:740] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:54.152256  5957 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:54.150985  6047 consensus_queue.cc:260] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:54.154243  5876 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:54.155067  5876 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.155591  5876 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:54.156253  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:54.157281  1644 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:54.159608  1641 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:54.161049  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:54.162444  1647 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:54.100780  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:54.130623  1538 leader_election.cc:400] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11.
I20260709 13:03:54.181002  1538 leader_election.cc:403] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:03:54.167081  1718 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:03:54.042577  5919 raft_consensus.cc:740] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:54.190618  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 72681.1us
I20260709 13:03:54.194096  5678 consensus_queue.cc:260] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:54.195055  6047 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:54.196659  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:54.133379  6069 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:54.194133  1713 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
W20260709 13:03:54.196152  5777 consensus_queue.cc:1175] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f5f4b2707a9647e5bb99e065068dea2b" responder_term: 8 status { last_received { term: 8 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 8 index: 6 } }
I20260709 13:03:54.206945  1717 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 12: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:54.113263  5850 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:54.190274  5854 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:54.233279  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 54202.3us
I20260709 13:03:54.233383  6030 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:54.251201  6030 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:54.254323  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:54.257800  6036 consensus_queue.cc:1059] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:54.259673  1713 raft_consensus.cc:1275] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:54.259549  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:54.260660  1645 raft_consensus.cc:2393] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 6: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:54.135411  6073 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:54.246727  1537 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:54.269480  1718 raft_consensus.cc:1240] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:54.133175  1565 rpcz_store.cc:276] 0709 13:03:53.119378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.191283 (+ 71905us) service_pool.cc:224] Handling call
0709 13:03:53.191618 (+   335us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:53.571235 (+379617us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.571302 (+    67us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:53.638453 (+ 67151us) log.cc:844] Serialized 52 byte log entry
0709 13:03:53.765439 (+126986us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:53.765496 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.794781 (+ 29285us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:54.131068 (+336287us) raft_consensus.cc:1685] finished
0709 13:03:54.131105 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.131432 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11013,"raft.queue_time_us":68831,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":7955,"thread_start_us":453104,"threads_started":2,"wal-append.queue_time_us":308697}
I20260709 13:03:54.195806  5678 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:54.230326  5940 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:54.283748  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.261428  1717 raft_consensus.cc:2410] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:03:54.275868  2314 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:54.259763  6036 raft_consensus.cc:3055] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:54.287451  6036 raft_consensus.cc:740] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:54.288038  6035 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:54.288602  6035 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:54.289247  6035 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:54.289027  6036 consensus_queue.cc:260] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:54.288864  6008 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:03:54.290884  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 250399us
W20260709 13:03:54.257867  5811 consensus_meta.cc:287] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.408s	user 0.008s	sys 0.014s
W20260709 13:03:54.299192  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 102360us
I20260709 13:03:54.301997  1647 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:54.303041  1644 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:03:54.304823  5866 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:54.305207  5834 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:54.308622  5834 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:54.293592  5835 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11.
I20260709 13:03:54.310293  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:54.316642  1713 raft_consensus.cc:1240] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:54.322429  1537 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:03:54.323088  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 192728us
I20260709 13:03:54.323638  1537 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:54.324556  5835 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:54.324970  1537 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:03:54.332312  5724 consensus_meta.cc:287] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.653s	user 0.019s	sys 0.000s
W20260709 13:03:54.333793  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.334457  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:54.335898  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:54.336643  1648 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:03:54.337687  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:54.338100  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:54.339025  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:54.340137  1641 raft_consensus.cc:1275] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:54.340636  1648 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:54.349437  6036 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:54.294951  5989 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:54.321080  1718 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:54.341370  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:54.325192  1538 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:54.283413  5854 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:54.305328  5866 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.270287  6061 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:03:54.341246  5940 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:54.360591  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:54.378093  1648 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:03:54.277599  6025 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:54.316576  6074 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:54.362696  5852 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:54.357326  5989 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:54.363369  6073 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:54.333101  5724 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:54.364921  5876 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:54.388316  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:54.371086  5866 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:54.393177  1566 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2108) took 1273 ms (client timeout 1501 ms). Trace:
I20260709 13:03:54.393895  6073 raft_consensus.cc:697] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:54.394253  5893 raft_consensus.cc:3055] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:54.397683  5989 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:54.399410  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:54.400801  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2141) took 1466 ms. Trace:
I20260709 13:03:54.404259  1722 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:54.381599  5834 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:54.314851  5811 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:54.411841  1566 rpcz_store.cc:269] 0709 13:03:53.119142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.149111 (+ 29969us) service_pool.cc:224] Handling call
0709 13:03:54.392732 (+1243621us) spinlock_profiling.cc:243] Waited 860 ms on lock 0x7b5400012554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:54.392969 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1978124544}
I20260709 13:03:54.413030  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:54.397365  5724 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:54.432025  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:54.418658  1716 rpcz_store.cc:276] 0709 13:03:52.934223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.975342 (+ 41119us) service_pool.cc:224] Handling call
0709 13:03:53.032297 (+ 56955us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.850525 (+818228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.400454 (+549929us) spinlock_profiling.cc:243] Waited 816 ms on lock 0x7b5400407c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:54.400713 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1876738432}
I20260709 13:03:54.386989  5852 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:54.370213  5904 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:03:54.393502  6074 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:54.435696  1720 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:54.444154  1648 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:54.447606  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 123713us
I20260709 13:03:54.409916  5835 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:54.417816  5893 raft_consensus.cc:740] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:54.470681  1715 raft_consensus.cc:2393] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:54.474766  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:54.487260  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:54.489264  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:54.490701  6025 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:54.490610  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:54.435397  5834 raft_consensus.cc:697] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:54.440784  5852 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:54.479498  1685 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:54.492652  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:54.446632  5921 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:03:54.505163  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 180024us
I20260709 13:03:54.506989  1565 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:54.514153  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:54.508399  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 196818us
I20260709 13:03:54.511603  6079 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.517719  6079 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:54.518460  1572 raft_consensus.cc:2393] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 6: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:54.520955  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:54.521888  1641 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:54.522172  1685 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:54.522886  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:54.524262  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:54.525473  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.525719  1716 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:54.525902  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.522068  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:54.494058  5835 raft_consensus.cc:697] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:54.538319  1640 raft_consensus.cc:2410] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:54.537196  1648 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:03:54.415058  5919 consensus_queue.cc:260] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:54.467237  5592 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:54.525280  6028 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:54.524371  5779 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:54.565515  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:54.573714  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:54.574746  1647 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:03:54.585367  5940 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:54.585523  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:54.586802  1571 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:54.513521  5921 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:54.589699  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:54.590360  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:54.577080  5919 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:54.594700  5835 consensus_queue.cc:237] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:54.596772  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.598625  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:54.535980  2346 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:54.610143  1719 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:54.611327  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:54.598645  5893 consensus_queue.cc:260] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:54.612790  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:54.614125  1538 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:54.618886  5921 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:54.546510  6073 consensus_queue.cc:237] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:54.627137  5724 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:03:54.628199  6091 log.cc:927] Time spent T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:03:54.628808  5834 consensus_queue.cc:237] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:54.650959  5921 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.652079  5893 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:54.651827  5921 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:54.635123  1648 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:03:54.640444  1565 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:54.641343  1722 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:03:54.644876  1564 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
W20260709 13:03:54.657859  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 45206.4us
I20260709 13:03:54.658865  1612 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:54.659075  5921 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:54.657898  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:54.649801  5724 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:54.660543  1648 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:54.662158  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:54.668826  1571 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 8: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:54.668934  1569 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:54.672335  5841 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:54.675602  5841 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:54.676112  5841 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:54.676719  5811 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:54.678736  5882 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:54.679356  5882 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:54.679844  5882 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:54.687209  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:54.666244  5724 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:54.685328  5592 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:54.698527  1566 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:54.637594  5852 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:54.705955  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 310064us
I20260709 13:03:54.736194  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:54.705299  5956 consensus_meta.cc:287] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.837s	user 0.012s	sys 0.011s
W20260709 13:03:54.745424  6096 log.cc:927] Time spent T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 13:03:54.747452  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:54.750402  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:54.749442  1641 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:03:54.760563  1570 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:54.763427  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:54.760303  5957 consensus_meta.cc:287] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.607s	user 0.010s	sys 0.006s
I20260709 13:03:54.758533  5956 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:54.767135  2347 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 13:03:54.769104  2313 leader_election.cc:336] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 2.143s (SENT)
I20260709 13:03:54.768899  5956 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:54.780139  1569 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:03:54.782205  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:54.783429  1573 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:54.784157  5866 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:54.823465  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:54.831914  1716 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:54.826582  5876 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:54.889748  6036 consensus_meta.cc:287] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.534s	user 0.012s	sys 0.000s
I20260709 13:03:54.904364  1648 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
W20260709 13:03:54.897588  6035 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.607s	user 0.002s	sys 0.012s
W20260709 13:03:54.931547  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.862923  5841 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:54.899428  6028 raft_consensus.cc:697] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:03:54.936551  1639 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.065s	sys 0.122s
I20260709 13:03:54.896983  5779 raft_consensus.cc:697] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:03:54.939245  1639 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.066s	sys 0.122s
W20260709 13:03:54.935184  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 411391us
I20260709 13:03:54.937505  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:54.950810  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:03:54.961719  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.965823  5779 consensus_queue.cc:237] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:54.968441  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 701) took 1054 ms. Trace:
I20260709 13:03:54.971053  5678 consensus_queue.cc:1059] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:03:54.972936  6101 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:03:54.985248  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2495) took 1458 ms. Trace:
I20260709 13:03:54.991616  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:54.934569  6035 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:54.986042  1715 rpcz_store.cc:276] 0709 13:03:53.523820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.525159 (+  1339us) service_pool.cc:224] Handling call
0709 13:03:54.981186 (+1456027us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b5400212d54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:54.981335 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2047753088}
W20260709 13:03:55.031973  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.033752  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:54.980654  5747 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:55.039939  1721 raft_consensus.cc:1240] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:55.042233  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:55.039726  6106 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:03:55.038893  6028 consensus_queue.cc:237] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:54.973050  5678 raft_consensus.cc:3055] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:54.985206  5811 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:54.983402  6069 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:55.024322  6016 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:55.051857  1721 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:55.051512  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 385865us
I20260709 13:03:55.017238  6111 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:55.043023  1565 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:55.043103  6025 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:03:55.081478  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.057863  5678 raft_consensus.cc:740] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:55.070142  6081 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:03:55.094117  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 480090us
I20260709 13:03:55.095036  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:55.113039  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:55.117038  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:55.117684  1565 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:55.094610  5876 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:55.095731  5811 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.119426  6016 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:55.173343  1538 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:55.114863  1685 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:55.085842  1720 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:55.119556  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:55.178457  2347 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:55.096810  5678 consensus_queue.cc:260] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:55.179448  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:55.180183  5678 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:55.180220  1565 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
W20260709 13:03:55.128235  5989 consensus_meta.cc:287] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.678s	user 0.001s	sys 0.011s
I20260709 13:03:55.095837  6069 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:55.182267  5989 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.119207  1647 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:03:55.119145  5882 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.183604  6016 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:55.178629  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 63895.3us
I20260709 13:03:55.184759  5989 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:55.185132  5882 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:55.185946  1685 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:55.186465  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 439) took 1230 ms. Trace:
I20260709 13:03:55.182660  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:55.187754  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:55.186947  1642 rpcz_store.cc:276] 0709 13:03:53.955597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.960542 (+  4945us) service_pool.cc:224] Handling call
0709 13:03:54.032190 (+ 71648us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:54.196196 (+164006us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:54.370718 (+174522us) log.cc:844] Serialized 14 byte log entry
0709 13:03:54.708843 (+338125us) log.cc:844] Serialized 14 byte log entry
0709 13:03:55.185748 (+476905us) spinlock_profiling.cc:243] Waited 25.8 ms on lock 0x7b1800463480. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d12c0a 00007f824f452219 00007f8245d9aa40 00007f8245d954e4 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287 00007f8248ea0208
0709 13:03:55.185930 (+   182us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:55.185978 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:55.186050 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.186125 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.186373 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1015,"prepare.queue_time_us":267355,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":3696,"raft.queue_time_us":7164,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":7543,"spinlock_wait_cycles":59350784,"thread_start_us":450147,"threads_started":2,"wal-append.queue_time_us":477274}
I20260709 13:03:55.191545  5866 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:55.178339  5811 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:55.185858  1713 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:55.193801  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 498031us
I20260709 13:03:55.196086  6030 consensus_queue.cc:1059] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:55.131577  5841 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:55.197906  5841 raft_consensus.cc:3055] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:55.198361  5841 raft_consensus.cc:740] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:55.198428  5875 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:55.192953  1685 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:55.199509  5841 consensus_queue.cc:260] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.201213  6117 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:55.130163  6035 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:55.202471  5811 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:55.202499  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:55.202971  5811 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:55.203776  5811 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:55.204422  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.205042  6069 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:55.205350  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.189365  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:55.207433  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 441) took 1053 ms. Trace:
I20260709 13:03:55.185485  5989 raft_consensus.cc:2764] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 13:03:55.205276  6089 log.cc:927] Time spent T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.803s	user 0.001s	sys 0.003s
I20260709 13:03:55.207919  1643 rpcz_store.cc:276] 0709 13:03:54.153970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.179903 (+ 25933us) service_pool.cc:224] Handling call
0709 13:03:54.262012 (+ 82109us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:54.262383 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:54.537766 (+275383us) log.cc:844] Serialized 14 byte log entry
0709 13:03:55.206602 (+668836us) log.cc:844] Serialized 14 byte log entry
0709 13:03:55.206952 (+   350us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:55.207015 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:55.207075 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.207144 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.207276 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":267924,"prepare.queue_time_us":275736,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":86,"thread_start_us":630557,"threads_started":2}
I20260709 13:03:55.210856  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 440) took 1177 ms. Trace:
I20260709 13:03:55.211357  1639 rpcz_store.cc:276] 0709 13:03:54.036317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.040840 (+  4523us) service_pool.cc:224] Handling call
0709 13:03:54.189871 (+149031us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:54.233099 (+ 43228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:54.233143 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:54.233755 (+   612us) log.cc:844] Serialized 26 byte log entry
0709 13:03:54.382603 (+148848us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:54.943178 (+560575us) log.cc:844] Serialized 14 byte log entry
0709 13:03:55.210466 (+267288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.210563 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:55.210612 (+    49us) raft_consensus.cc:1685] finished
0709 13:03:55.210627 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.210790 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":266731,"prepare.queue_time_us":255699,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":266990,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":184,"thread_start_us":670976,"threads_started":2,"wal-append.queue_time_us":247622}
I20260709 13:03:55.171419  5876 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.214845  1564 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:03:55.216609  5876 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:55.216892  1564 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:55.127090  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:55.192374  5866 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:55.096901  6111 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.203779  1719 raft_consensus.cc:2410] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:03:54.968959  1567 rpcz_store.cc:276] 0709 13:03:53.914296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.921808 (+  7512us) service_pool.cc:224] Handling call
0709 13:03:53.922211 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.922583 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:54.042196 (+119613us) log.cc:844] Serialized 14 byte log entry
0709 13:03:54.482408 (+440212us) log.cc:844] Serialized 14 byte log entry
0709 13:03:54.688408 (+206000us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b180033e2c0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d98522 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:03:54.967567 (+279159us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b180033e2c0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d12c0a 00007f824f452219 00007f8245d9aa40 00007f8245d954e4 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287 00007f8248ea0208
0709 13:03:54.967864 (+   297us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:54.967928 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:54.967989 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.968082 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.968240 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":473297,"prepare.queue_time_us":235123,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":103,"raft.queue_time_us":495587,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":11106,"spinlock_wait_cycles":33494144,"thread_start_us":326628,"threads_started":2,"wal-append.queue_time_us":462833}
I20260709 13:03:55.209785  1538 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:55.137297  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:55.201912  1685 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:55.125499  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:55.233141  1715 raft_consensus.cc:2410] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:03:55.084141  5852 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:55.238795  1685 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:55.234532  5841 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:55.201998  6117 raft_consensus.cc:697] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:55.242192  1568 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
W20260709 13:03:55.250393  6124 log.cc:927] Time spent T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:03:55.248263  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:55.249784  6117 consensus_queue.cc:237] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.251919  1564 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
W20260709 13:03:55.244046  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.249784  1537 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:55.257513  1564 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:55.258085  1567 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:55.264554  1568 raft_consensus.cc:1240] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:55.266070  1568 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:55.271447  1722 raft_consensus.cc:2410] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:03:55.154170  5724 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:55.260236  1612 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:55.243393  5866 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:55.252236  6028 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:55.283329  1537 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:55.259647  6110 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:55.296965  1564 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:55.269472  6111 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:55.301258  1565 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:55.301769  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 53655us
I20260709 13:03:55.310334  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:55.311945  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:55.312878  1570 raft_consensus.cc:2410] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 14 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:03:55.291447  5882 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:55.314098  5882 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.314543  5882 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:55.327313  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:55.300369  5893 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.330876  5893 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.332877  6067 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:55.289039  6028 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.334030  6028 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:55.333562  6067 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:55.336880  6067 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:55.352190  1568 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:55.338455  6081 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:55.289173  5852 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:55.294023  6016 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:03:55.362039  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 72126.2us
I20260709 13:03:55.301425  5876 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:55.373759  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:55.333505  5893 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:55.378679  1718 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:55.369380  5852 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.357347  6110 raft_consensus.cc:697] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:55.366341  5866 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.385842  5852 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:55.385385  1538 connection.cc:256] RPC call timeout handler was delayed by -1.1319605049999382s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16200000000000003s for a response
I20260709 13:03:55.374388  5876 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:55.360824  6035 consensus_queue.cc:1059] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:03:55.406931  2347 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 6.
W20260709 13:03:55.408936  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:55.417990  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 44781.3us
I20260709 13:03:55.418915  1611 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:03:55.421471  6132 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:03:55.391366  6025 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:55.423243  6035 raft_consensus.cc:3055] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:55.423836  6035 raft_consensus.cc:740] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:55.423545  6025 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.424819  6035 consensus_queue.cc:260] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:55.426066  6025 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:55.426141  6110 consensus_queue.cc:237] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.430199  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 689) took 2392 ms. Trace:
I20260709 13:03:55.430827  1572 rpcz_store.cc:276] 0709 13:03:53.038160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.038396 (+   236us) service_pool.cc:224] Handling call
0709 13:03:55.418647 (+2380251us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b5400307854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:55.430078 (+ 11431us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3163136256}
I20260709 13:03:55.432345  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:55.432978  1572 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:55.434983  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2159) took 1115 ms. Trace:
I20260709 13:03:55.435406  1713 rpcz_store.cc:276] 0709 13:03:54.319590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.320513 (+   923us) service_pool.cc:224] Handling call
0709 13:03:54.366444 (+ 45931us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:55.192872 (+826428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.434713 (+241841us) spinlock_profiling.cc:243] Waited 819 ms on lock 0x7b54003002d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:55.434919 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1884153216}
I20260709 13:03:55.438547  1471 catalog_manager.cc:5697] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:55.440528  1471 catalog_manager.cc:5697] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:55.442406  1645 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2583) took 2358 ms (client timeout 1620 ms). Trace:
W20260709 13:03:55.443090  1645 rpcz_store.cc:269] 0709 13:03:53.083637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.083836 (+   199us) service_pool.cc:224] Handling call
0709 13:03:55.419217 (+2335381us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54001a9854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:55.442314 (+ 23097us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2544478592}
I20260709 13:03:55.447067  5724 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:55.453836  6067 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:55.456408  6067 consensus_queue.cc:1059] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:03:55.465941  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 98023.2us
I20260709 13:03:55.398386  5876 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:55.462159  5919 raft_consensus.cc:3055] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:55.395741  5866 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:55.502692  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:55.505308  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.499521  1471 catalog_manager.cc:5697] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 5 to 7, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 7 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:55.510450  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:55.493134  5724 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:55.517966  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 99131.5us
I20260709 13:03:55.410532  5893 consensus_queue.cc:1059] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.094s
I20260709 13:03:55.391098  6081 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.501104  5919 raft_consensus.cc:740] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:55.528553  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2051) took 1228 ms. Trace:
W20260709 13:03:55.528429  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 74861us
I20260709 13:03:55.510753  6025 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:55.534408  1640 rpcz_store.cc:276] 0709 13:03:54.296506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.309755 (+ 13249us) service_pool.cc:224] Handling call
0709 13:03:55.522091 (+1212336us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b54001f3454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:55.524024 (+  1933us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":519138560}
I20260709 13:03:55.550985  5852 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.556035  6067 consensus_queue.cc:1059] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:55.557058  6067 consensus_queue.cc:1059] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:55.561456  5852 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:55.566753  1572 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:03:55.568306  6028 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.569236  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:55.571522  5807 raft_consensus.cc:3055] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:55.574309  5807 raft_consensus.cc:740] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:55.576999  5807 consensus_queue.cc:260] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.578131  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:55.579241  5807 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:55.579046  6076 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:55.581905  5779 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:55.582700  5779 raft_consensus.cc:697] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:55.584149  5779 consensus_queue.cc:237] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:55.590969  2346 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 410) took 2373 ms (client timeout 1584 ms). Trace:
W20260709 13:03:55.591566  2346 rpcz_store.cc:269] 0709 13:03:53.217043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.221272 (+  4229us) service_pool.cc:224] Handling call
0709 13:03:53.932121 (+710849us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b580003a298. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:03:55.590451 (+1658330us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b54003f84d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:55.590643 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":938332800}
W20260709 13:03:55.601096  6129 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.275s	user 0.002s	sys 0.000s
I20260709 13:03:55.611752  6081 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:55.612673  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:55.614930  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:55.615921  1571 raft_consensus.cc:2393] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 6: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:55.617085  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:55.617964  6025 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:55.619306  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2599) took 1075 ms. Trace:
I20260709 13:03:55.546661  6035 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:55.620623  6025 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:55.517303  5811 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.622112  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:55.609449  6138 log.cc:927] Time spent T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:03:55.623420  5811 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:55.623839  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:55.623992  5866 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:55.611414  1568 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:55.602435  5919 consensus_queue.cc:260] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.625732  5919 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:55.625869  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:55.604002  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 78781.5us
I20260709 13:03:55.628676  1611 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:55.519989  5724 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:55.630681  5724 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:55.632170  5882 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:55.634038  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:55.634935  5893 raft_consensus.cc:3055] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:55.635638  5893 raft_consensus.cc:740] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:55.619851  1647 rpcz_store.cc:276] 0709 13:03:54.544106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.672470 (+128364us) service_pool.cc:224] Handling call
0709 13:03:54.675683 (+  3213us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:55.183625 (+507942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.619032 (+435407us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b54000a5ad4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:55.619241 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1019300224}
I20260709 13:03:55.641512  1612 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:55.644017  5841 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:55.645350  6047 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:55.613467  1565 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:55.624701  1569 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:55.649537  6105 log.cc:927] Time spent T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.617s	user 0.003s	sys 0.003s
I20260709 13:03:55.669318  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:55.650745  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:55.672791  2347 raft_consensus.cc:2393] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 6: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:55.656298  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2115) took 1167 ms. Trace:
I20260709 13:03:55.665963  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:55.669754  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2161) took 1305 ms. Trace:
I20260709 13:03:55.674381  1720 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:55.656280  1564 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:55.663841  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:55.674067  1566 rpcz_store.cc:276] 0709 13:03:54.502319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.506903 (+  4584us) service_pool.cc:224] Handling call
0709 13:03:54.517557 (+ 10654us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:54.706666 (+189109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.656031 (+949365us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54002ba554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:55.656235 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":415615360}
I20260709 13:03:55.680867  1569 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:55.684953  1713 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:55.663463  1611 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:55.657163  1687 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:55.636906  5893 consensus_queue.cc:260] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.679250  1537 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:55.694288  5834 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:55.694706  5834 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:55.695050  5834 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:55.696038  1568 raft_consensus.cc:2410] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:55.703387  5834 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.704802  1537 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:55.663404  2312 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:55.674067  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:55.708736  1569 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:03:55.709221  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:55.709290  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:55.710394  1722 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:55.713884  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 50567.8us
I20260709 13:03:55.675220  1717 rpcz_store.cc:276] 0709 13:03:54.363890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.365773 (+  1883us) service_pool.cc:224] Handling call
0709 13:03:54.380114 (+ 14341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:54.402432 (+ 22318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:54.402471 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:54.403080 (+   609us) log.cc:844] Serialized 26 byte log entry
0709 13:03:54.710408 (+307328us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:54.710468 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.710575 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:55.649109 (+938534us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b24000d4bb8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245c1e9c4 00007f8245c1cacb 00007f8245dd7dc9 00007f8245dd7a55 00007f8245dd709d 00007f8248a0910e 00007f8248a2088c 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221
0709 13:03:55.669033 (+ 19924us) raft_consensus.cc:1685] finished
0709 13:03:55.669057 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.669359 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11292,"raft.queue_time_us":18317,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":15659,"spinlock_wait_cycles":335016704,"thread_start_us":306809,"threads_started":1,"wal-append.queue_time_us":536501}
I20260709 13:03:55.717841  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:55.718626  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:55.655107  6079 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:55.656873  5722 raft_consensus.cc:3055] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:55.655782  6047 raft_consensus.cc:697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:55.725785  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.706000  5811 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.665555  6067 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:55.730785  1721 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:55.732980  6008 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:55.736768  2312 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:55.721169  1685 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:55.662638  6036 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:03:55.742228  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:55.751541  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 88156us
W20260709 13:03:55.752429  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.733675  5722 raft_consensus.cc:740] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:55.746218  1564 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:55.748756  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 44043.7us
I20260709 13:03:55.686962  1572 raft_consensus.cc:1240] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:55.753345  1611 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:55.747941  1471 catalog_manager.cc:5697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:55.717046  6030 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:55.763839  1716 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:03:55.764326  1717 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:55.766925  1721 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:03:55.745685  6125 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.594s	user 0.000s	sys 0.005s
I20260709 13:03:55.770799  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:55.743570  5893 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:55.783583  1564 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:55.774410  1571 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:03:55.794287  1722 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:03:55.797683  1570 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:55.773496  6116 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:55.808425  5969 raft_consensus.cc:3055] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:55.808990  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:55.810571  2349 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 14: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
W20260709 13:03:55.810418  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 73803.1us
W20260709 13:03:55.800794  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 44269.8us
I20260709 13:03:55.812383  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:55.809222  1471 catalog_manager.cc:5697] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:55.765450  5866 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:03:55.813419  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 156367us
I20260709 13:03:55.821539  1687 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:55.823582  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 70324.6us
I20260709 13:03:55.818760  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:55.765147  6061 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:55.827149  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.827265  1722 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:55.827955  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:55.829190  1612 connection.cc:256] RPC call timeout handler was delayed by -1.0107567919999951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1584s for a response
I20260709 13:03:55.766616  6076 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:55.782325  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:55.784040  6008 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:55.835568  1721 raft_consensus.cc:1275] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:55.754635  5811 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.766248  6036 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.808964  6116 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:55.842429  1471 catalog_manager.cc:5697] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 8 to 10, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 10 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:55.884126  2348 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:55.838994  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:55.892308  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:55.808974  5969 raft_consensus.cc:740] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:55.815302  6013 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:55.908169  1646 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:55.897655  6076 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:55.909751  6069 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:55.911531  6061 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.912093  6076 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:55.913792  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:55.914145  6036 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:55.914667  1642 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:55.914819  6036 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:55.915017  1568 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:55.915549  6016 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:55.917210  1568 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:55.913245  1537 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:55.917870  1571 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:55.920329  1571 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:55.920539  6016 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:55.894006  5835 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.819456  5678 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:55.922125  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:55.922266  6047 consensus_queue.cc:237] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.922901  6028 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:55.922186  5835 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.923163  1646 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:55.898679  2349 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
W20260709 13:03:55.897078  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 52959.1us
I20260709 13:03:55.899170  6116 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:55.899740  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:55.899924  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:55.902114  5834 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:55.902535  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:55.903573  6122 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.863s	user 0.000s	sys 0.011s
I20260709 13:03:55.944197  1645 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 14: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:55.904549  5852 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:55.946965  1569 raft_consensus.cc:1275] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:03:55.904752  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:55.905987  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:55.906226  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:55.906734  1564 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:55.906572  1471 catalog_manager.cc:5697] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 7 to 9, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 9 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:55.820679  6079 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:55.930226  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 446020us
I20260709 13:03:55.953985  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:55.954960  1642 raft_consensus.cc:2393] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 6: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:55.956072  1566 raft_consensus.cc:1275] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:03:55.930706  5678 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:55.947011  6068 consensus_queue.cc:1059] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:03:55.960378  6049 raft_consensus.cc:3055] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:55.959988  1471 catalog_manager.cc:5697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 9 to 11, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 11 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:55.960824  6049 raft_consensus.cc:740] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:55.960933  1641 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:55.962040  5678 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:55.962157  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:55.944165  5841 consensus_queue.cc:1059] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:03:55.962839  1471 catalog_manager.cc:5697] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 6 to 9, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 9 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:55.963317  5834 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:55.964424  1471 catalog_manager.cc:5697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 7 to 8, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 8 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:55.956524  5835 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:55.963382  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:55.966987  1571 raft_consensus.cc:1240] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:55.966749  1471 catalog_manager.cc:5697] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:55.940953  2347 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:03:55.940263  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:55.942315  1648 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:55.969962  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:55.969964  5989 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:55.970677  1644 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:55.972757  6028 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:55.903270  5747 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:55.974507  6115 consensus_queue.cc:1059] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:55.975436  5989 raft_consensus.cc:697] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:55.975585  6115 consensus_queue.cc:1048] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:55.976413  5989 consensus_queue.cc:237] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:55.977802  6036 raft_consensus.cc:3055] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:03:55.973400  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.978080  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:55.978238  6036 raft_consensus.cc:740] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:55.965559  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:55.979312  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:55.980333  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:55.980542  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:55.981086  1639 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:55.981957  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:55.982451  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:55.983373  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:55.983494  5811 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:55.984346  1642 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:55.942247  1721 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:03:55.946583  1718 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:03:55.813060  5722 consensus_queue.cc:260] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.979205  6036 consensus_queue.cc:260] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.988082  6036 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:55.967154  5882 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:55.990549  5876 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.993588  1573 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:55.995441  5882 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.996204  5882 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:55.997015  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.931232  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:55.986104  1641 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:03:55.998181  1722 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:55.998740  1641 pending_rounds.cc:85] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 7
I20260709 13:03:55.999089  1641 pending_rounds.cc:107] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:03:55.999419  1641 raft_consensus.cc:2889] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:55.995960  5969 consensus_queue.cc:260] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:55.997243  6067 consensus_queue.cc:1059] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.004s
W20260709 13:03:55.932327  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 75335.9us
I20260709 13:03:55.837687  5866 raft_consensus.cc:697] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:56.005430  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:56.004669  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 317731us
I20260709 13:03:56.012348  1537 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:55.984973  1643 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:03:56.017164  1720 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:03:56.017402  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:56.018290  1569 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:56.019789  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:56.026024  1612 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:56.027235  1645 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 13: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:55.912120  6061 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:56.034659  1537 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:03:56.037545  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.008574  1648 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:56.015280  2314 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:56.046450  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:55.963655  5841 raft_consensus.cc:3055] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:56.046254  6024 consensus_queue.cc:1059] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:03:56.053591  1644 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:03:56.055526  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:56.067096  1717 raft_consensus.cc:2410] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:56.056324  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:56.058488  1640 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:55.876223  1713 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:56.023406  5835 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:56.060461  1471 catalog_manager.cc:5697] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 5 to 8, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 8 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:56.068727  5835 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.957664  6079 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:56.087054  1471 catalog_manager.cc:5697] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 7 to 9, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 9 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:56.024060  5834 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:03:56.097970  5876 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:56.098501  5834 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.098958  5876 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:56.099610  5876 raft_consensus.cc:697] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:56.099398  5834 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:55.961912  6049 consensus_queue.cc:260] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:56.102334  6049 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:56.103317  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 77419.6us
I20260709 13:03:56.105149  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:56.107357  5882 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:56.108053  6013 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:56.108718  6013 raft_consensus.cc:697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:56.101727  5866 consensus_queue.cc:237] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:56.106665  5876 consensus_queue.cc:237] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:56.109560  6079 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:56.109719  6013 consensus_queue.cc:237] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:56.020632  5811 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:03:56.079973  1569 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:56.070838  6076 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:56.057904  5841 raft_consensus.cc:740] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:56.126785  1646 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
I20260709 13:03:56.079093  1722 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:56.116834  5919 consensus_queue.cc:1059] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:03:56.138017  6116 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:56.140829  1471 catalog_manager.cc:5697] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:03:56.047127  6068 consensus_queue.cc:1175] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "066ee0109aba4465ba17fa65e14ef504" responder_term: 7 status { last_received { term: 6 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:56.143311  6068 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:56.143146  1471 catalog_manager.cc:5697] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 6 to 9, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 9 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:56.143952  6068 raft_consensus.cc:697] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:56.145028  6068 consensus_queue.cc:237] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:56.083616  5777 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.093333  5835 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:56.146628  5777 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:56.092444  5969 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:56.155068  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.156124  1714 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:03:56.131629  5724 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.500s	user 0.011s	sys 0.001s
W20260709 13:03:56.189446  2315 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.020s	sys 0.031s
I20260709 13:03:56.135843  6076 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:56.191313  6021 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.020s	sys 0.041s
I20260709 13:03:56.092247  5807 consensus_queue.cc:1059] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
W20260709 13:03:56.210539  2315 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.021s	sys 0.031s
I20260709 13:03:56.132104  5882 consensus_queue.cc:1059] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:03:56.228089  1640 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:03:56.210080  6076 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:56.211010  6021 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.021s	sys 0.041s
I20260709 13:03:56.144997  1471 catalog_manager.cc:5697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 8 to 9, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 9 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
W20260709 13:03:56.359375  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 537905us
W20260709 13:03:56.223801  6117 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.006s	sys 0.013s
I20260709 13:03:56.414211  2347 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:03:56.414777  1471 catalog_manager.cc:5697] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 7 to 8, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 8 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:03:56.297091  5678 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.302347  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:56.434898  1719 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:56.200388  5724 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:56.451247  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:56.415257  6117 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.007s	sys 0.013s
I20260709 13:03:56.455319  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:56.456540  1719 raft_consensus.cc:2410] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:56.298676  6016 raft_consensus.cc:697] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:56.413038  5834 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:56.448526  5893 consensus_meta.cc:287] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.635s	user 0.011s	sys 0.000s
I20260709 13:03:56.497671  1717 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:56.495693  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 480540us
W20260709 13:03:56.499420  1565 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.013s	sys 0.017s
W20260709 13:03:56.499830  1565 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.013s	sys 0.017s
W20260709 13:03:56.500967  6081 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.003s	sys 0.026s
I20260709 13:03:56.502439  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:56.518010  1471 catalog_manager.cc:5697] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 8 to 9, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 9 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
W20260709 13:03:56.519102  1567 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.003s	sys 0.027s
W20260709 13:03:56.519137  6081 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.003s	sys 0.027s
I20260709 13:03:56.521611  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:56.520642  1567 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.003s	sys 0.027s
I20260709 13:03:56.305929  5841 consensus_queue.cc:260] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:56.530998  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:56.446908  5678 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:56.454123  6036 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:03:56.541455  1471 catalog_manager.cc:5697] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:56.543074  1722 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:56.543576  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:56.551098  1687 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:56.551324  1471 catalog_manager.cc:5697] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:03:56.557783  6028 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:56.559587  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:56.561333  1716 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:56.562888  6140 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:03:56.564345  1716 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:56.566677  1718 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:56.572671  1718 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:56.565609  1716 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:56.530114  5893 raft_consensus.cc:3055] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:56.533062  6061 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:56.603698  1719 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 12: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:56.606400  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:56.608844  2313 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.008s	sys 0.043s
W20260709 13:03:56.609285  2313 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.008s	sys 0.043s
W20260709 13:03:56.623718  1538 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.018s	sys 0.013s
W20260709 13:03:56.626500  1538 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.018s	sys 0.013s
I20260709 13:03:56.671459  1538 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:56.676916  1715 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:03:56.603212  5893 raft_consensus.cc:740] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:56.677938  2314 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:56.691015  1538 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:56.693095  1538 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:56.695205  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 148810us
W20260709 13:03:56.701932  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.702133  1687 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:56.714114  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:56.729628  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:56.735631  2349 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:56.736550  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.740532  1714 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
W20260709 13:03:56.725888  1614 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.006s	sys 0.010s
W20260709 13:03:56.741904  1614 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.006s	sys 0.010s
I20260709 13:03:56.743978   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:03:56.753471  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 422805us
W20260709 13:03:56.754103  1611 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.008s	sys 0.011s
W20260709 13:03:56.759513  1611 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.008s	sys 0.011s
I20260709 13:03:56.694707  6150 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:03:56.762949  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 645996us
I20260709 13:03:56.724545  6024 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:03:56.741855  6013 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.022s	sys 0.038s
I20260709 13:03:56.739437  6061 raft_consensus.cc:2764] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:03:56.801298  1687 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:56.801556  1611 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:56.740350  5779 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:56.706910  6155 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
W20260709 13:03:56.792075  6013 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.023s	sys 0.039s
I20260709 13:03:56.680081  6016 consensus_queue.cc:237] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:56.812156  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 119149us
I20260709 13:03:56.815578  1538 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:56.813406  1715 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:03:56.804013  6036 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.810750  6028 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:56.858223  5678 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:56.853792  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:56.844471  6161 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:03:56.862197  1714 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:56.864517  6161 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.781498  6150 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:56.867316  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:56.868218  1716 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:56.812544  5807 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.871814  6150 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:56.846244  6155 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:56.890987  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 707) took 1223 ms. Trace:
I20260709 13:03:56.803128  6025 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:03:56.891700  1565 rpcz_store.cc:276] 0709 13:03:55.667832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.826680 (+158848us) service_pool.cc:224] Handling call
0709 13:03:55.895876 (+ 69196us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:56.500352 (+604476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:56.500409 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:56.525134 (+ 24725us) log.cc:844] Serialized 26 byte log entry
0709 13:03:56.689456 (+164322us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:56.689504 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.689597 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:56.890727 (+201130us) raft_consensus.cc:1685] finished
0709 13:03:56.890750 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:56.890924 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":38548,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"thread_start_us":737401,"threads_started":2,"wal-append.queue_time_us":322494}
I20260709 13:03:56.850003  6036 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:56.864001  6152 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:56.853929  5969 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.855036  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:56.903736  1719 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:56.856076  6061 consensus_queue.cc:1059] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:56.856626  6067 consensus_queue.cc:1059] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:03:56.918962  6141 log.cc:927] Time spent T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.296s	user 0.001s	sys 0.001s
I20260709 13:03:56.921229  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:56.921891  5835 consensus_meta.cc:287] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.775s	user 0.013s	sys 0.000s
I20260709 13:03:56.823660  6186 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:56.869431  5807 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:56.846058  5724 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:56.877012  6180 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:56.868077  6151 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:03:56.884111  6160 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.138s	user 0.001s	sys 0.002s
I20260709 13:03:56.883996  6149 consensus_queue.cc:1059] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:03:56.886087  6148 log.cc:927] Time spent T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.157s	user 0.002s	sys 0.008s
I20260709 13:03:56.889000  6116 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:56.865368  6161 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:56.891073  6049 raft_consensus.cc:3055] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:56.848948  6068 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.894714  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 447) took 1063 ms. Trace:
I20260709 13:03:56.952601  1647 rpcz_store.cc:276] 0709 13:03:55.831636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.831834 (+   198us) service_pool.cc:224] Handling call
0709 13:03:55.832263 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:55.884968 (+ 52705us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:55.885001 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:55.896869 (+ 11868us) log.cc:844] Serialized 26 byte log entry
0709 13:03:56.222782 (+325913us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:56.222823 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.222894 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:56.894448 (+671554us) raft_consensus.cc:1685] finished
0709 13:03:56.894467 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:56.894653 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":41728,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":11903,"thread_start_us":325437,"threads_started":1,"wal-append.queue_time_us":991237}
I20260709 13:03:56.807900  5893 consensus_queue.cc:260] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:56.884042  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:56.900692  1687 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:56.922075  5969 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:56.813187  5779 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:56.883023  5834 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:56.927855  6186 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:56.936934  6115 raft_consensus.cc:3055] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:56.937156  6110 consensus_queue.cc:1059] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:03:56.939106  5807 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:56.922413  5835 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:56.922075  2346 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:03:56.951213  6049 raft_consensus.cc:740] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:03:56.973119  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 157629us
I20260709 13:03:56.947618  5882 raft_consensus.cc:3055] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:03:56.979386  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 178140us
W20260709 13:03:56.979970  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 221514us
I20260709 13:03:56.986037  1538 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:56.970538  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 69846.2us
I20260709 13:03:56.930958  6172 consensus_queue.cc:1059] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:03:56.913398  6152 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.009861  6067 raft_consensus.cc:3055] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:57.010406  6067 raft_consensus.cc:740] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:57.011593  6152 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:56.989451  5807 consensus_queue.cc:1059] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:57.012807  5807 raft_consensus.cc:3055] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:57.013180  5807 raft_consensus.cc:740] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:57.014055  5807 consensus_queue.cc:260] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.015228  5807 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:56.975996  5989 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:56.975781  6115 raft_consensus.cc:740] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:57.019763  6115 consensus_queue.cc:260] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:57.020483  6047 raft_consensus.cc:3055] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:57.020910  6047 raft_consensus.cc:740] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:57.021274  6115 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:57.021767  6047 consensus_queue.cc:260] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.022866  6047 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:56.976223  6186 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:56.989750  1719 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:03:56.992933  6189 consensus_queue.cc:1059] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:56.996266  6180 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:56.996071  1612 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:56.999394  5724 consensus_queue.cc:1059] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:57.002915  6179 raft_consensus.cc:3055] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:57.003145  1538 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:57.003948  6188 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:56.987735  5882 raft_consensus.cc:740] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:57.005137  6182 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.071s	user 0.000s	sys 0.003s
I20260709 13:03:57.066742  6049 consensus_queue.cc:260] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:57.086230  6049 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:56.975754  5834 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:57.006454  6036 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:57.033563  2349 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:03:57.035581  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2131) took 1590 ms. Trace:
W20260709 13:03:57.029809  6076 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.564s	user 0.004s	sys 0.009s
I20260709 13:03:57.004606  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:57.031380  5893 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:56.968734  6068 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:57.095364  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:57.098523  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 95468.2us
I20260709 13:03:57.103474  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:57.107611  1719 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:57.091915  6178 log.cc:927] Time spent T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:03:57.096553  2343 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:03:57.091030  1572 rpcz_store.cc:276] 0709 13:03:55.445346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.800322 (+354976us) service_pool.cc:224] Handling call
0709 13:03:55.801453 (+  1131us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:55.801742 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:55.833829 (+ 32087us) log.cc:844] Serialized 14 byte log entry
0709 13:03:56.224339 (+390510us) log.cc:844] Serialized 14 byte log entry
0709 13:03:56.893130 (+668791us) spinlock_profiling.cc:243] Waited 3.3 ms on lock 0x7b1800196d20. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287 00007f8248ea0208
0709 13:03:57.035072 (+141942us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:57.035123 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:57.035171 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.035336 (+   165us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.035473 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17396,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":105,"raft.queue_time_us":668655,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":154,"spinlock_wait_cycles":7589632,"thread_start_us":531767,"threads_started":2,"wal-append.queue_time_us":780963}
I20260709 13:03:57.110072  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:57.103711  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.115442  1685 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:03:57.042893  6149 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.123137  1538 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:57.124569  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:57.124984  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:57.124050  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:57.130599  1714 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:03:57.133138  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 137304us
I20260709 13:03:57.134083  1719 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:03:57.135179  1612 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:57.134047  1715 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:57.138348  6188 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:57.138885  6188 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:57.123987  6016 consensus_queue.cc:1059] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:03:57.140027  6171 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:57.140542  6016 raft_consensus.cc:3055] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:57.140976  6016 raft_consensus.cc:740] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:57.119889  5866 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:57.138662  5852 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:57.139266  6152 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:57.142562  1538 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:57.146421  5834 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:57.147300  5834 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:57.147799  5834 raft_consensus.cc:697] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:57.148074  6190 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:57.120242  6028 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:57.129201  6111 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:57.150265  6111 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.152395  6111 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:57.152999  6066 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:57.105741  6036 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:57.139302  6188 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:57.154043  6036 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.156304  6181 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.156065  5882 consensus_queue.cc:260] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.153862  6116 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:57.156924  6181 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.157169  6116 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.157456  5882 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:57.159206  6181 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:57.159547  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 44262.9us
W20260709 13:03:57.160497  6183 log.cc:927] Time spent T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 13:03:57.142513  5866 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:57.146523  5779 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:57.165910  1718 raft_consensus.cc:1275] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:03:57.180438  1687 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.505s (SENT)
I20260709 13:03:57.187623  5835 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:57.191430  5841 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:57.190169  6016 consensus_queue.cc:260] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.159368  6116 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:57.197948  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.148845  5834 consensus_queue.cc:237] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:57.208309  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 84957.2us
W20260709 13:03:57.236142  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:57.203006  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 67959.9us
W20260709 13:03:57.232151  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:57.240723  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.103271  6179 raft_consensus.cc:740] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:57.192404  5852 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.243239  5852 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.244735  5835 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:57.192343  6028 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:57.148145  5969 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:57.246212  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:57.260329  1713 raft_consensus.cc:2410] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:03:57.252549  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.266216  1537 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:57.261968  5989 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:57.248736  5866 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:57.269513  5866 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.272670  5866 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:57.245442  5835 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.273979  6173 consensus_queue.cc:1059] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:57.275146  5835 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:57.275802  6142 log.cc:927] Time spent T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.828s	user 0.000s	sys 0.004s
I20260709 13:03:57.276675  5724 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:57.230557  6189 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:57.200757  6195 raft_consensus.cc:3055] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:57.105778  6076 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:57.067080  6166 consensus_queue.cc:1059] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.166s
I20260709 13:03:57.011605  6067 consensus_queue.cc:260] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.259196  6180 consensus_queue.cc:1059] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:03:57.245555  5852 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:03:57.292301  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 111916us
I20260709 13:03:57.249712  6151 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.204567  6149 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.247041  1611 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:57.298866  6190 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:03:57.310461  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 44424.2us
I20260709 13:03:57.312078  5866 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:57.281271  6195 raft_consensus.cc:740] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:57.278803  6016 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:57.281921  6028 raft_consensus.cc:697] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:03:57.299593  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 139269us
W20260709 13:03:57.317956  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 71039.8us
W20260709 13:03:57.321995  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.321120  6180 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.326550  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:57.327555  1572 raft_consensus.cc:2410] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:57.315639  6067 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:57.308079  5852 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:03:57.299474  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 186848us
I20260709 13:03:57.316728  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:57.333823  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:57.336892  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:57.321394  6166 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:57.339955  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:57.341260  1566 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:57.324934  6161 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:57.353178  1567 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:03:57.340945  5724 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:03:57.319777  6151 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.375216  6149 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:57.376334  6151 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:57.322966  6179 consensus_queue.cc:260] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.353605  6193 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:57.319978  6190 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.356617  6068 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:57.358597  6028 consensus_queue.cc:237] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.332959  6180 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.366099  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2534) took 1334 ms. Trace:
I20260709 13:03:57.366343  6195 consensus_queue.cc:260] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:57.379055  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 107176us
I20260709 13:03:57.380844  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:57.382460  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:57.382053  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:57.388087  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 48234.2us
I20260709 13:03:57.394337  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:57.393427  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 93008.9us
I20260709 13:03:57.397383  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:57.398734  1571 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:57.412515  1568 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:57.341748  5852 raft_consensus.cc:697] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:57.406149  1567 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:03:57.416576  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:57.417325  1573 raft_consensus.cc:1275] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:57.417529  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.416575  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:57.418215  1640 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:57.419730  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.420656  1639 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:57.422294  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.422640  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:57.423058  1645 raft_consensus.cc:2368] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10.
I20260709 13:03:57.423189  1569 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:03:57.423986  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:57.424571  1569 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:57.427896  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:57.428179  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.428579  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.407053  1566 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:03:57.400699  1721 rpcz_store.cc:276] 0709 13:03:56.031545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.031697 (+   152us) service_pool.cc:224] Handling call
0709 13:03:56.032173 (+   476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:56.032665 (+   492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:56.032706 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:57.002751 (+970045us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b1800409ba0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d662a6 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824777ea11 00007f824778e372 00007f8248ec0a01 00007f8248e8543a
0709 13:03:57.003896 (+  1145us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b2c00013bb8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824777ea11 00007f824778e372 00007f8248ec0a01 00007f8248e8543a 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2
0709 13:03:57.004494 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 13:03:57.280772 (+276278us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:57.280815 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.280892 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:57.365781 (+ 84889us) raft_consensus.cc:1685] finished
0709 13:03:57.365807 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.366008 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":500,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":12882,"spinlock_wait_cycles":77018240,"thread_start_us":275888,"threads_started":1,"wal-append.queue_time_us":330958}
I20260709 13:03:57.429894  1570 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 13. Current term is 14. Ops: [13.6-13.6]
I20260709 13:03:57.433303  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:57.434250  1569 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 12: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:57.435031  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:57.436147  5777 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:57.436622  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:57.429389  6186 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.438300  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.440683  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:57.441752  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.396301  5882 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:57.444263  1640 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:57.444543  1648 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:57.445544  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.445439  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:57.397629  6151 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:57.447115  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:57.444407  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:57.377351  6173 consensus_queue.cc:1059] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:03:57.451615  1643 raft_consensus.cc:2368] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11.
I20260709 13:03:57.437788  6201 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:57.455605  1644 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:57.457374  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:57.458652  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.459702  1568 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
W20260709 13:03:57.460284  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 150276us
I20260709 13:03:57.405979  6030 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.468037  1645 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:57.468241  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.424938  5919 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.469841  1639 raft_consensus.cc:2410] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:57.474908  1537 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:57.477114  5807 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:57.477427  6030 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.477945  6206 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:57.478471  6147 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:57.462684  5882 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:57.479368  6117 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:57.479703  5852 consensus_queue.cc:237] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:57.428840  5724 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.434178  6066 raft_consensus.cc:3055] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:57.489190  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.491232  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.491688  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.491935  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.494553  1537 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:03:57.497732  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.499498  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.500448  6173 raft_consensus.cc:3055] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:57.516829  1645 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 12: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:57.503965  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:57.508505  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:57.509552  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.513476  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.518086  1644 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:57.500289  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:57.518043  1639 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:57.503456  6161 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:57.454569  6191 consensus_queue.cc:1059] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:03:57.502169  5678 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:57.462812  6188 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:57.523598  1537 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:57.512221  6066 raft_consensus.cc:740] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:57.524983  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:57.526825  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.493628  5919 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.519767  5807 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:57.516829  6173 raft_consensus.cc:740] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:57.547205  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.546465  1640 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:57.492350  6201 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:57.527448  6193 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:57.535075  6151 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:57.516556  5724 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:57.534715  6161 raft_consensus.cc:697] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:57.578013  1686 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:57.513650  6147 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:57.573526  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.574162  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:57.570361  6211 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:03:57.514967  6117 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:57.581965  1648 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:57.593014  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:57.579236  1686 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:57.584172  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.585942  1721 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:57.532295  5678 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.586750  5882 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:57.587896  1639 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:03:57.589646  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 140391us
I20260709 13:03:57.549625  6190 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:57.595674  1644 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:03:57.597534  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:57.605408  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:57.606125  2349 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:57.609980  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.616050  6188 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:57.620162  6147 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:57.590934  5777 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:57.582674  6193 raft_consensus.cc:697] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:03:57.554241  5807 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.624099  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:57.588192  6066 consensus_queue.cc:260] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.631856  1718 raft_consensus.cc:2410] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:03:57.632953  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.633800  1644 raft_consensus.cc:1275] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:03:57.600013  6116 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:57.647688  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.648624  1639 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
W20260709 13:03:57.655210  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.612921  1612 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:57.611058  6186 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:57.660538  1642 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
W20260709 13:03:57.664989  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 52181.4us
I20260709 13:03:57.625623  5777 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.666198  1687 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:57.605073  6206 consensus_queue.cc:1059] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:03:57.669777  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:57.671540  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:57.672302  1644 raft_consensus.cc:2376] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10. Re-sending same reply.
I20260709 13:03:57.649613  6188 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.642134  6116 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.677059  1611 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:57.690230  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.664265  6158 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.694413  1565 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:57.669821  5777 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:57.664359  6180 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:57.671535  5882 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:57.676507  2346 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
W20260709 13:03:57.712947  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 99962.1us
W20260709 13:03:57.691968  6208 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.167s	user 0.000s	sys 0.002s
I20260709 13:03:57.676085  6147 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:57.716883  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:57.682909  6206 raft_consensus.cc:3055] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:03:57.688203  6204 log.cc:927] Time spent T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:03:57.719878  1687 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:57.719878  1686 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:57.720363  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:57.962863  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 243049us
I20260709 13:03:57.637998  6173 consensus_queue.cc:260] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.717801  5882 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.723845  6216 consensus_queue.cc:1059] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:03:57.724450  6193 consensus_queue.cc:237] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.683449  6161 consensus_queue.cc:237] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.725385  6117 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:57.725800  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:57.690408  6116 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:57.683271  6066 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:57.771356  6202 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:03:57.729108  6206 raft_consensus.cc:740] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:57.945278  5807 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:57.977061  6028 consensus_queue.cc:1059] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:03:57.991482  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:57.994882  6028 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:57.995064  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:57.729969  6147 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.947542  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:57.948161  5678 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:57.993280  6036 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:57.729369  5724 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:58.003412  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:57.953943  5841 consensus_meta.cc:287] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.713s	user 0.019s	sys 0.002s
W20260709 13:03:57.954527  6016 consensus_meta.cc:287] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.631s	user 0.008s	sys 0.012s
I20260709 13:03:57.691782  6191 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:57.716702  6158 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:57.958369  1565 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:57.959457  5722 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:57.960291  6201 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:57.694990  6188 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:57.732344  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:57.963004  6110 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:57.963863  5882 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:57.721508  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.964797  6216 raft_consensus.cc:3055] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:57.978204  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:57.978384  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:57.985939  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:58.021287  2345 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:57.988485  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:57.991477  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:58.022429  2342 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:57.948756  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:58.023411  1569 raft_consensus.cc:2410] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 12 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 2.
I20260709 13:03:57.942852  5893 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:57.975436  6207 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.001991  6147 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:57.978652  6151 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:57.993299  6212 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.012135  1641 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:57.986959  1720 raft_consensus.cc:1275] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:03:58.024075  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 304272us
I20260709 13:03:58.023952  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.029826  1687 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:58.020948  6110 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.021029  6216 raft_consensus.cc:740] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:58.036136  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.039139  1722 raft_consensus.cc:2410] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 13:03:58.040287  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.052824  1687 leader_election.cc:400] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11.
I20260709 13:03:58.053795  1687 leader_election.cc:403] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:03:58.056237  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.056298  1612 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:58.035792  5893 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:58.057778  6233 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:03:58.058205  6217 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.219s	user 0.001s	sys 0.009s
I20260709 13:03:58.058521  6233 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:58.059149  6233 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:58.059911  5893 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:58.060111  6216 consensus_queue.cc:260] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.063589  6188 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:58.031421  6212 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.066387  6195 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:58.068013  6212 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:58.035578  6110 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:58.035718  6191 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.090830  6191 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.088747  5893 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:58.084627  6216 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:58.027443  6207 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.041479  6181 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:58.101848  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:58.111074  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:58.115011  1569 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:03:58.116398  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.117221  1722 raft_consensus.cc:2410] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:03:58.040069  6201 consensus_queue.cc:1059] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 13:03:58.096585  6036 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11.
I20260709 13:03:58.128577  5893 raft_consensus.cc:697] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:58.138379  5893 consensus_queue.cc:237] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:58.134441  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.156778  6171 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:58.127247  6181 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:58.130290  6228 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.268s	user 0.004s	sys 0.016s
W20260709 13:03:58.134972  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 97660.5us
I20260709 13:03:58.078856  5777 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:58.175268  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:58.169339  6234 consensus_queue.cc:1048] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:58.174050  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:58.161166  6110 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:58.032698  6206 consensus_queue.cc:260] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.189440  1566 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:58.191778  1720 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:58.197181  1687 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:58.198720  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:58.199561  1571 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:03:58.198918  6158 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:58.181548  6171 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:58.205068  1719 raft_consensus.cc:1240] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:58.208004  1646 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:03:58.208132  1687 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:58.209736  1687 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:58.204562  6079 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:58.210235  6206 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:58.219580  1687 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:58.210386  6232 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.211894  6201 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:58.223168  1687 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:58.205886  5678 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:03:58.215119  6036 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:58.226842  5722 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:58.231647  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:58.211606  6233 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.235111  1643 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:58.234448  1687 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:58.224385  6181 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:58.225492  6240 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:58.225420  5882 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.240381  5722 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:58.241905  6233 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:03:58.244025  6235 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 13:03:58.261482  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.269196  1611 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:58.265955  5678 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:58.237473  6201 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:58.225356  6232 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.296953  1648 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:58.261322  6240 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:58.301767  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:03:58.303866  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 69419.7us
I20260709 13:03:58.304929  1642 raft_consensus.cc:2393] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 6: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:58.290622  6201 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:58.305401  1611 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:58.286394  6181 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:58.287176  6110 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:58.298307  5777 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:58.313171  1611 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:58.315295  6013 raft_consensus.cc:3055] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:58.315771  6013 raft_consensus.cc:740] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:03:58.317741  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.318321  5678 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:58.319767  6066 consensus_queue.cc:1048] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:58.320550  6173 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:58.321534  6191 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:58.322332  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:58.312579  6234 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:58.323894  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.312450  1687 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:58.324819  1642 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:03:58.325978  1687 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:58.328035  1611 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:58.305980  6240 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:58.297914  6172 consensus_queue.cc:1059] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:03:58.305029  6161 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:58.329804  1687 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:58.328241  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.331480  6036 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:58.328293  5678 consensus_queue.cc:1048] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:03:58.307750  6201 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:58.332118  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.331799  6110 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.336207  1719 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:58.336520  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:58.329309  5777 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:58.338230  6036 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:58.339262  5989 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:58.340426  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.343971  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.345031  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.345815  1642 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:58.345906  1719 raft_consensus.cc:2393] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 9: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:58.348289  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.348637  1611 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:58.333704  5893 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:58.335142  6181 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:58.350672  1611 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:58.344740  5777 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:58.345274  6036 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:58.353854  5777 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:58.354555  5724 consensus_queue.cc:1059] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:58.355932  6195 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:58.353382  1538 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:58.357452  1611 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:58.338774  6079 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:58.357127  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:03:58.361969  6241 log.cc:927] Time spent T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:03:58.352569  6066 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:58.367923  1717 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:58.368731  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:58.371176  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.372068  1719 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:58.349800  5989 raft_consensus.cc:697] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:58.338429  6161 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.377153  1538 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:58.375324  5724 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.378382  5989 consensus_queue.cc:237] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.344906  6110 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.341939  6191 consensus_queue.cc:1059] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:03:58.352870  6181 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:58.385550  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.361876  5841 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:58.386657  1642 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:58.395175  6110 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:58.386067  5724 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.374171  6193 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:58.400550  1719 raft_consensus.cc:1275] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:58.402184  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:58.357407  6147 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.384826  6186 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:58.385329  6181 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:58.402868  1642 raft_consensus.cc:2376] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8. Re-sending same reply.
I20260709 13:03:58.407179  5724 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:58.390224  5841 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.418063  5841 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:58.424577  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:58.430753  6161 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:58.432003  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:58.432591  2349 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:58.432948  1612 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:58.434780  1713 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:03:58.435205  5724 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:58.435876  5724 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:58.430648  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:58.436430  2348 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:58.436471  6047 raft_consensus.cc:3055] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:58.437247  6047 raft_consensus.cc:740] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:58.438521  6116 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:58.438463  6047 consensus_queue.cc:260] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.440506  6047 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:58.440886  6116 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:58.434655  1612 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:58.447566  1722 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:58.459780  6206 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.460498  6206 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.462320  1719 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:58.448880  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.448741  2347 consensus_queue.cc:237] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:58.465411  1717 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:03:58.465873  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.462174  2348 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:58.448381  2349 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:03:58.462981  6206 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:58.464087  1713 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:58.470252  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.466485  1642 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:03:58.472229  1721 raft_consensus.cc:2410] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:58.413661  6186 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.470600  1685 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:58.478969  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:58.479990  2349 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:58.485620  1685 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:58.497035  1685 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:58.502552  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.464962  6191 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:58.503600  1642 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:03:58.510398  1612 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:58.512992  1612 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:58.483007  6186 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:58.478504  5811 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:58.526165  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.495788  6116 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:58.485445  5989 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:58.529150  1642 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:58.520022  6110 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:58.534881  1611 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:39930 dropped due to backpressure: service queue is full with 50 items [suppressed 1 similar messages]
I20260709 13:03:58.508766  6079 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:58.541388  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.478768  6218 consensus_queue.cc:1059] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:03:58.504417  5893 consensus_queue.cc:1059] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:03:58.524171  6206 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:58.552028  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.531502  6147 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:58.530864  6212 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:58.537905  6116 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:58.561626  2349 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:03:58.544037  5779 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.536955  5811 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.570888  2349 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:58.554024  6206 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.537439  1611 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:39930 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2656)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2112)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2116)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 452)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2111)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2657)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2665)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2662)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2120)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2658)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2123)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2113)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2117)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2659)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2118)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2650)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2645)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2114)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2661)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2652)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2115)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2631)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2629)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2633)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2637)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2638)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2639)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2641)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2642)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2643)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2647)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2648)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2649)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2655)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 450)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 451)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 453)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 455)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 456)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 458)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2660)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2663)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2664)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2110)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2119)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2121)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2122)
I20260709 13:03:58.564051  6116 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:58.540691  5989 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.586989  6201 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:58.588325  6186 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.590932  6171 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.592051  6161 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:58.592778  6161 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:58.594820  6240 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.596282  6161 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:58.598356  6066 raft_consensus.cc:3055] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:58.598841  6066 raft_consensus.cc:740] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:58.599771  6066 consensus_queue.cc:260] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.600965  6066 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:58.601174  6240 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:58.601990  6201 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:58.607968  1611 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:39914 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2116)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 452)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2111)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2657)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2665)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2662)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2120)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2658)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2127)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2123)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2113)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2668)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2117)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2659)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2118)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2650)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2645)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2667)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2114)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2661)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2126)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2652)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2115)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2666)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2125)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2629)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2633)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2637)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2638)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2639)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2641)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2642)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2643)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2647)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2648)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2649)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2655)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 450)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 451)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 453)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 455)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 456)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 458)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2660)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2663)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2664)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2110)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2119)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2121)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2122)
I20260709 13:03:58.574642  6206 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:58.597361  5722 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:58.625133  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:58.625787  1569 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
W20260709 13:03:58.627331  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.626523  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2156) took 1152 ms. Trace:
I20260709 13:03:58.630024  1572 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:58.628867  2312 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:03:58.631184  1687 leader_election.cc:336] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.566s (SENT)
I20260709 13:03:58.630501  1573 rpcz_store.cc:276] 0709 13:03:57.474087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.488029 (+ 13942us) service_pool.cc:224] Handling call
0709 13:03:58.613077 (+1125048us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b5400014d54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:58.626449 (+ 13372us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1333786368}
I20260709 13:03:58.569999  5779 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:58.642937  5811 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:58.621182  5777 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.637774  6049 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:58.646615  6049 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.647058  6049 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:58.648360  6207 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:58.650437  6189 raft_consensus.cc:3055] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:58.650849  6189 raft_consensus.cc:740] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:58.648203  1612 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:39914 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2116)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 452)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2111)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2657)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2665)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2662)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2120)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2658)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2127)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2123)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2113)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2668)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2117)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2659)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2118)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2650)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2645)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2667)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2114)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2661)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2126)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2652)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2115)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2666)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2125)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2629)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2633)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2637)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2638)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2639)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2641)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2642)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2643)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2647)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2648)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2649)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2655)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 450)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 451)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 453)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 455)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 456)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 458)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2660)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2663)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2664)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2110)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2119)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2121)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2122)
I20260709 13:03:58.651721  6189 consensus_queue.cc:260] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.653160  6189 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:58.650871  5777 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:58.637369  5882 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:58.663079  6164 consensus_queue.cc:1059] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:58.655817  1611 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:39930 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2116)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 452)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2111)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2657)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2665)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2662)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2120)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2658)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2127)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2123)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2113)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2668)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2117)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2659)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2118)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2650)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2645)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2667)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2114)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2661)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2126)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2652)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2115)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2666)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2125)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2629)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2633)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2637)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2638)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2639)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2641)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2642)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2643)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2647)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2648)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2649)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2655)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 450)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 451)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 453)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 455)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 456)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 458)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2660)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2663)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2664)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2110)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2119)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2121)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2122)
I20260709 13:03:58.602065  6186 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:58.589507  6036 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.665853  1611 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:39930 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2116)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 452)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2111)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2657)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2665)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2662)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2120)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2658)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2127)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2129)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2123)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2113)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2668)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2117)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2659)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2118)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2650)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2645)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2667)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2114)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2661)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2652)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2115)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2666)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2125)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2629)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2633)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2637)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2638)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2639)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2641)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2642)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2643)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2647)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2648)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2649)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2655)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 450)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 451)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 453)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 455)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 456)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 458)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2660)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2663)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2664)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2110)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2119)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2121)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2122)
I20260709 13:03:58.675004  2312 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:58.634311  6171 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:58.673377  1611 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:39914 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2116)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 452)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2111)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2657)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2665)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2130)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2662)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2120)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2658)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2127)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2129)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2123)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2113)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2117)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2659)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2118)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2650)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2645)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2667)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2114)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2661)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2652)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2115)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2666)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2125)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2629)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2633)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2637)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2638)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2639)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2641)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2642)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2643)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2647)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2648)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2649)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2655)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 450)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 451)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 453)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 455)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 456)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 458)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2660)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2663)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2664)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2110)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2119)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2121)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2122)
I20260709 13:03:58.664839  5779 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:58.656729  6049 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:58.681622  5777 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.680883  6224 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:58.689450  6230 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.726s	user 0.003s	sys 0.000s
W20260709 13:03:58.696481  2314 leader_election.cc:336] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.975s (SENT)
I20260709 13:03:58.686035  6165 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:03:58.694489  5777 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.712872  6216 raft_consensus.cc:3055] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:58.713084  5777 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:58.713923  6061 raft_consensus.cc:3055] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:58.714645  6061 raft_consensus.cc:740] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:58.715718  6061 consensus_queue.cc:260] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:58.717334  6061 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:58.705680  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2090) took 1245 ms. Trace:
I20260709 13:03:58.718739  1643 rpcz_store.cc:276] 0709 13:03:57.460086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.483284 (+ 23198us) service_pool.cc:224] Handling call
0709 13:03:58.705483 (+1222199us) spinlock_profiling.cc:243] Waited 680 ms on lock 0x7b54001f2f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:58.705601 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1564261248}
I20260709 13:03:58.719396  6116 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.719933  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.719655  6013 consensus_queue.cc:260] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.720552  1643 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:03:58.720996  6013 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:58.721943  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:58.723122  1643 raft_consensus.cc:2376] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8. Re-sending same reply.
I20260709 13:03:58.722465  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:58.731448  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.731751  1639 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:03:58.732259  1640 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:58.724066  5989 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:58.723932  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:58.722088  6206 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.738291  6226 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:58.739297  6206 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:58.739552  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:03:58.743850  6252 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:03:58.744056  6232 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:58.739584  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:58.745766  1612 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:39914 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2665)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2130)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2662)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2120)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2658)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2127)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2129)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2123)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2113)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2117)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2659)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2118)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2650)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2645)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2667)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2114)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2661)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2652)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2115)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2666)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2125)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2131)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2629)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2633)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2637)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2638)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2639)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2641)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2642)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2643)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2647)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2648)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2649)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2655)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 450)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 451)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 453)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 455)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 456)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 458)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2660)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2663)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2664)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2110)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2119)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2121)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2122)
I20260709 13:03:58.749792  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:58.750401  1640 raft_consensus.cc:2376] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11. Re-sending same reply.
I20260709 13:03:58.751415  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.751945  1640 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
W20260709 13:03:58.753479  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.679824  5841 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.713408  6216 raft_consensus.cc:740] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:58.697006  6224 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:58.702981  6214 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 1.089s	user 0.004s	sys 0.000s
I20260709 13:03:58.758925  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:58.706394  6158 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.701412  6049 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:58.763105  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.760066  6224 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:58.768496  6036 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:58.772008  5919 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:58.774981  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.690665  6181 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:58.781307  1687 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:58.735936  6116 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:58.785192  1641 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
W20260709 13:03:58.794791  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 56115.4us
I20260709 13:03:58.795595  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:58.796857  2349 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:58.799678  5841 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:58.803189  1640 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:03:58.806317  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.807389  1640 raft_consensus.cc:2393] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 9: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:58.811106  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:03:58.810452  6184 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.196s	user 0.001s	sys 0.007s
W20260709 13:03:58.811900  6184 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.197s	user 0.001s	sys 0.008s
I20260709 13:03:58.811947  1640 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:58.813339  6215 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.204s	user 0.000s	sys 0.003s
I20260709 13:03:58.796342  6115 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:58.816897  1687 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:58.820108  1640 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:58.822640  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.823321  1640 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:03:58.796810  5919 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.830250  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2073) took 1413 ms. Trace:
I20260709 13:03:58.830910  6030 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:58.830812  1647 rpcz_store.cc:276] 0709 13:03:57.416780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.416947 (+   167us) service_pool.cc:224] Handling call
0709 13:03:57.417255 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:57.417581 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:57.417611 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:57.429112 (+ 11501us) log.cc:844] Serialized 26 byte log entry
0709 13:03:57.517419 (+ 88307us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:57.517459 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.517570 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:58.812681 (+1295111us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:03:58.815831 (+  3150us) raft_consensus.cc:1685] finished
0709 13:03:58.815853 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.830153 (+ 14300us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12389,"raft.queue_time_us":26644,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":30840,"spinlock_wait_cycles":3721472,"thread_start_us":79321,"threads_started":1,"wal-append.queue_time_us":145198}
I20260709 13:03:58.833760  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 435) took 1702 ms. Trace:
I20260709 13:03:58.833590  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.835031  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.784965  6158 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:58.836285  1714 rpcz_store.cc:276] 0709 13:03:57.131254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.494327 (+363073us) service_pool.cc:224] Handling call
0709 13:03:57.509229 (+ 14902us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:57.600203 (+ 90974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:57.600240 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:57.600283 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:57.600332 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.833462 (+1233130us) spinlock_profiling.cc:243] Waited 90.7 ms on lock 0x7b5400053f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:58.833511 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.833696 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208701312}
W20260709 13:03:58.841259  1718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 438) took 1261 ms (client timeout 1555 ms). Trace:
I20260709 13:03:58.840197  1647 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:58.841889  1718 rpcz_store.cc:269] 0709 13:03:57.580277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.580443 (+   166us) service_pool.cc:224] Handling call
0709 13:03:58.838379 (+1257936us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b540038fed4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:58.841183 (+  2804us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56694528}
I20260709 13:03:58.842439  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:58.843252  1640 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:58.824046  6115 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:03:58.847307  6115 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:58.850675  6076 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:58.852720  6250 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.275s	user 0.002s	sys 0.000s
I20260709 13:03:58.855261  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:58.856437  1538 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:58.857687  1538 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:58.858034  1643 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:58.859489  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:58.859786  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:58.860890  1713 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:58.861289  2342 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:58.857620  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.862942  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.862258  1718 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:03:58.865191  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.866021  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:58.866429  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.841449  6036 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:58.869731  1715 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:58.866662  1713 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:58.870240  1687 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:58.867153  1721 raft_consensus.cc:2393] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 10: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:03:58.863593  1719 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:03:58.868643  1718 raft_consensus.cc:2410] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:03:58.868927  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:58.873432  1714 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:58.863554  6171 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:58.863364  6186 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:58.874692  6171 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:03:58.857579  6249 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.255s	user 0.002s	sys 0.000s
I20260709 13:03:58.875576  6171 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:58.807003  6216 consensus_queue.cc:260] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:58.878759  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:58.881749  6216 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:58.869899  6036 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:58.861370  6035 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.876886  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.888015  6035 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:58.910553  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:58.890048  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2107) took 1257 ms (client timeout 1609 ms). Trace:
I20260709 13:03:58.892824  6203 raft_consensus.cc:3055] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:58.874769  6186 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:03:58.880892  6251 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.303s	user 0.000s	sys 0.002s
W20260709 13:03:58.927510  1646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2086) took 1469 ms (client timeout 1566 ms). Trace:
I20260709 13:03:58.897895  6161 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.898269  6179 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:58.898643  6258 log.cc:927] Time spent T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:03:58.899183  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:58.899961  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:58.901549  6116 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.902347  6181 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:58.929800  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 59663.6us
I20260709 13:03:58.930158  1571 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 10: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:58.905218  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:58.844995  5919 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:58.907094  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.936321  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:58.908023  1717 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:58.910400  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:58.937703  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.914299  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:58.913061  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:58.916498  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:58.915555  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:58.918906  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:58.930840  1642 raft_consensus.cc:2376] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7. Re-sending same reply.
I20260709 13:03:58.919749  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:58.939760  1714 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:03:58.940346  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:58.921286  1718 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:03:58.911631  1648 rpcz_store.cc:269] 0709 13:03:57.632251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.632572 (+   321us) service_pool.cc:224] Handling call
0709 13:03:58.889744 (+1257172us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b54000a69d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:58.889891 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1508837760}
I20260709 13:03:58.940882  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:58.941738  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.919252  1685 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:58.897159  5852 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.922675  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:03:58.926367  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:58.885380  6036 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:58.927917  1646 rpcz_store.cc:269] 0709 13:03:57.457886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.458114 (+   228us) service_pool.cc:224] Handling call
0709 13:03:58.927284 (+1469170us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b54001ff4d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:58.927442 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1594408704}
I20260709 13:03:58.954303  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:58.954738  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.955608  1713 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:03:58.955901  1646 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
W20260709 13:03:58.957484  6259 log.cc:927] Time spent T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
I20260709 13:03:58.957990  2346 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:58.927345  6186 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:58.906508  1715 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:58.929623  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.974189  1716 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:03:58.971230  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:03:58.977038  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 90753.4us
W20260709 13:03:58.979012  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.880668  6158 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:58.935776  1645 raft_consensus.cc:1275] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:03:58.936513  1643 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:58.986378  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2650) took 1289 ms. Trace:
I20260709 13:03:58.986975  1643 rpcz_store.cc:276] 0709 13:03:57.696750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.906845 (+1210095us) service_pool.cc:224] Handling call
0709 13:03:58.986293 (+ 79448us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:58.919134  1721 raft_consensus.cc:1275] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:58.920027  1640 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:58.989718  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:58.990608  1640 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:58.942227  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:58.940403  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:58.995008  1568 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:58.940864  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:58.996155  1570 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:58.930783  1687 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:58.997443  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 66753.4us
I20260709 13:03:58.946959  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2630) took 1256 ms. Trace:
I20260709 13:03:58.999460  1639 rpcz_store.cc:276] 0709 13:03:57.690237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.690664 (+   427us) service_pool.cc:224] Handling call
0709 13:03:58.946723 (+1256059us) spinlock_profiling.cc:243] Waited 892 ms on lock 0x7b54001fe5d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:58.946879 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2050853760}
I20260709 13:03:58.948758  1648 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:03:59.001488  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.952103  1612 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:58.928987  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:03:58.930379  1644 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:59.003151  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 51190.2us
I20260709 13:03:58.957366  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.923198  6203 raft_consensus.cc:740] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:03:58.935032  1686 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:59.003922  1687 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:58.984123  1647 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:59.001226  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.966624  6076 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.961057  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:59.004419  1612 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:59.004977  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2652) took 1287 ms. Trace:
I20260709 13:03:59.005446  1639 rpcz_store.cc:276] 0709 13:03:57.717534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.000723 (+1283189us) service_pool.cc:224] Handling call
0709 13:03:59.004914 (+  4191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.004469  1646 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:59.006824  1538 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:58.953138  6061 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:59.008749  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:59.015018  1642 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:59.010753  1612 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:59.015936  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:58.896102  6115 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:03:59.010807  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 75884.8us
I20260709 13:03:59.019668  1686 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:59.019959  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:59.011245  1687 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:59.014201  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:59.020810  1643 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:59.021091  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:59.021862  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:59.038443  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:58.967136  6180 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.031704  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2645) took 1336 ms. Trace:
I20260709 13:03:58.962702  6181 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:59.065990  1648 rpcz_store.cc:276] 0709 13:03:57.695415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.941505 (+1246090us) service_pool.cc:224] Handling call
0709 13:03:59.031634 (+ 90129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.067715  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:59.068451  1648 raft_consensus.cc:2393] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:59.009464  5989 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:58.936328  6161 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:59.020823  1686 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:03:59.085431  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 64687.3us
I20260709 13:03:59.086649  1714 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:59.041061  1640 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:03:59.015269  6206 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:59.037292  1687 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:59.031486  6047 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:59.016729  6028 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:03:59.036710  6236 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:03:59.053947  6111 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:59.038292  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:59.111159  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 72954.5us
I20260709 13:03:59.112640  1612 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:59.036547  6233 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:59.055222  6061 raft_consensus.cc:697] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:59.019029  6067 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:03:58.961788  5852 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:59.013298  6076 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:59.012509  6030 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:58.945132  6116 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:59.065119  6035 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:59.028810  1568 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:03:59.069303  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:59.072229  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:59.074677  1643 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:59.071767  1570 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:03:59.017590  6149 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:59.158710  1641 raft_consensus.cc:2410] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:03:59.133181  6206 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:59.039600  1642 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:03:59.066068  6180 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:59.096547  1714 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:03:59.103734  1647 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
W20260709 13:03:59.107559  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 70357.1us
I20260709 13:03:59.116580  1572 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:03:59.117272  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 110520us
I20260709 13:03:59.118558  1639 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:59.123394  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:03:59.173542  1719 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:59.101442  6117 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:03:59.138346  2313 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.555s (SENT) [suppressed 31 similar messages]
I20260709 13:03:59.156495  1612 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:59.167946  1646 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 12: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:03:59.078063  1644 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:03:59.118563  5777 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:59.118748  6240 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:59.173259  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:59.180199  1612 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:59.181147  6047 consensus_queue.cc:1059] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:59.181933  1640 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:59.182250  6216 raft_consensus.cc:3055] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:03:59.166128  6189 consensus_meta.cc:287] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.509s	user 0.015s	sys 0.000s
I20260709 13:03:59.194691  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2629) took 1505 ms. Trace:
I20260709 13:03:59.195451  1612 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:59.198248  1646 rpcz_store.cc:276] 0709 13:03:57.689866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.071914 (+1382048us) service_pool.cc:224] Handling call
0709 13:03:59.194585 (+122671us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.130986  6200 consensus_queue.cc:1059] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:03:59.172086  6206 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.166734  6173 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:59.187868  6030 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:59.226523  1644 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:03:59.204521  1687 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:59.228096  1687 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:59.229017  1687 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:59.176608  6266 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.230027  6266 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:59.231925  6186 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:59.205305  2314 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:59.234164  2314 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:59.234975  6035 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:59.235414  6186 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:59.235750  6035 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:59.236258  6035 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.236466  6186 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.183430  6216 raft_consensus.cc:740] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:59.183327  6240 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.237793  6240 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:59.238330  6216 consensus_queue.cc:260] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:59.240159  6216 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:59.242300  1646 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:59.208388  6271 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:59.243461  6271 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:03:59.243880  6271 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:59.224402  6013 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.246115  6013 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:59.247825  6116 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:59.248335  6013 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:59.248773  6116 raft_consensus.cc:3055] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:59.236603  6035 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:59.249277  6116 raft_consensus.cc:740] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:59.250723  6161 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:59.252557  6076 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:59.253198  6061 consensus_queue.cc:237] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:59.201402  6047 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:59.232692  6266 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:59.191020  6115 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:59.154220  6171 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:59.254168  1687 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:59.302451  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 48375.7us
I20260709 13:03:59.274703  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:59.304021  1721 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:03:59.207284  6066 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:59.135905  6203 consensus_queue.cc:260] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:59.183254  6224 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:59.217752  6206 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:59.206035  6232 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 13:03:59.258665  6067 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.276369  5779 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:59.289095  6161 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.313000  6115 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:59.320132  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2633) took 1626 ms. Trace:
I20260709 13:03:59.326073  1640 rpcz_store.cc:276] 0709 13:03:57.691194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.096614 (+1405420us) service_pool.cc:224] Handling call
0709 13:03:59.181747 (+ 85133us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.182396 (+   649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.317420 (+135024us) spinlock_profiling.cc:243] Waited 84.7 ms on lock 0x7b54001c0550. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:59.317653 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194920832}
I20260709 13:03:59.327586  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:59.328429  1640 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:03:59.322193  6171 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:59.305954  6047 raft_consensus.cc:697] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:59.307094  5919 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:59.323441  6224 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:59.337731  1687 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:59.260134  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:59.318954  6066 raft_consensus.cc:697] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:03:59.321233  6201 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:59.360718  1641 raft_consensus.cc:1275] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:59.361682  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2643) took 1666 ms. Trace:
I20260709 13:03:59.363193  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2099) took 1792 ms. Trace:
I20260709 13:03:59.342990  1719 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:03:59.363641  1645 rpcz_store.cc:276] 0709 13:03:57.571366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.571606 (+   240us) service_pool.cc:224] Handling call
0709 13:03:58.928501 (+1356895us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.985806 (+ 57305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.362991 (+377185us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54001ff4d0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:59.363133 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3119937792}
I20260709 13:03:59.362026  1641 rpcz_store.cc:276] 0709 13:03:57.694848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.346102 (+1651254us) service_pool.cc:224] Handling call
0709 13:03:59.360317 (+ 14215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.361200 (+   883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.361398 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.368561  6066 consensus_queue.cc:237] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:59.372818  6047 consensus_queue.cc:237] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:59.327462  6067 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:59.351934  2314 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:59.370304  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:59.339095  6266 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:59.403311  1645 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:59.348064  1687 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:03:59.426497  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 74764.7us
W20260709 13:03:59.438067  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 90193.2us
W20260709 13:03:59.441161  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 267983us
I20260709 13:03:59.353737  6201 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:59.396397  6269 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:03:59.396986  6270 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
I20260709 13:03:59.330704  6161 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:59.486519  6161 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:59.477736  6171 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:03:59.489359  6171 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:59.478262  6201 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:59.502218  6171 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:03:59.450719  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.440259  6049 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:59.505237  6049 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.505851  6049 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:59.366336  6028 raft_consensus.cc:697] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:59.511126  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2659) took 1205 ms. Trace:
I20260709 13:03:59.511874  1647 rpcz_store.cc:276] 0709 13:03:58.305417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.859400 (+553983us) service_pool.cc:224] Handling call
0709 13:03:59.510794 (+651394us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400096d54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:59.511022 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284291840}
I20260709 13:03:59.514921  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2115) took 1034 ms. Trace:
I20260709 13:03:59.426297  5919 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.395215  5777 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:59.510690  6028 consensus_queue.cc:237] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:59.395541  6067 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:59.515609  1639 rpcz_store.cc:276] 0709 13:03:58.480891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.006256 (+525365us) service_pool.cc:224] Handling call
0709 13:03:59.514497 (+508241us) spinlock_profiling.cc:243] Waited 79.5 ms on lock 0x7b54000cf3d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:03:59.514834 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182878976}
I20260709 13:03:59.425772  6266 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.345422  6186 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:59.503391  1642 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:03:59.548405  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2638) took 1855 ms. Trace:
I20260709 13:03:59.548900  1642 rpcz_store.cc:276] 0709 13:03:57.692951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.502547 (+1809596us) service_pool.cc:224] Handling call
0709 13:03:59.503015 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:59.548083 (+ 45068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.548343 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.521436  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:59.521944  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:59.551719  1568 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:03:59.525508  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:59.525990  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:59.445981  1538 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:59.535723  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:03:59.482255  6264 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 13:03:59.540668  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:59.556669  1564 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:59.544790  1611 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:59.323992  6116 consensus_queue.cc:260] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:59.563431  1716 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:59.566089  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.557896  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:59.559767  1572 raft_consensus.cc:2393] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 10: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:59.572841  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:59.573715  1687 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:59.577172  1722 raft_consensus.cc:1240] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:59.578379  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:59.579358  1565 raft_consensus.cc:2410] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:59.570538  1611 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:59.582806  1713 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:03:59.516557  5852 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:03:59.586058  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:59.532505  5841 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.602844  1611 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:59.606487  1611 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:59.550896  6266 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:59.568111  6180 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:59.621767  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:59.599207  6111 raft_consensus.cc:697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:59.628754  1647 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
W20260709 13:03:59.601011  6270 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:03:59.575831  6067 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:59.645606  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:59.608183  6171 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.597354  6079 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:59.559010  5777 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:59.546599  5919 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:59.646304  1647 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:59.675997  1565 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:59.597957  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 152065us
I20260709 13:03:59.546234  1719 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:03:59.617905  6013 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:59.611341  5882 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:59.626452  6076 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:59.668910  1571 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:59.669148  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2641) took 1975 ms. Trace:
I20260709 13:03:59.609285  5852 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.680795  1719 raft_consensus.cc:606] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:59.704936  1719 raft_consensus.cc:614] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:59.577292  6117 raft_consensus.cc:697] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:59.682783  1646 rpcz_store.cc:276] 0709 13:03:57.694100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.199239 (+1505139us) service_pool.cc:224] Handling call
0709 13:03:59.199630 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.242758 (+ 43128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.668804 (+426046us) spinlock_profiling.cc:243] Waited 42.2 ms on lock 0x7b5400184554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:59.669036 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97145600}
I20260709 13:03:59.680078  1538 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:59.683142  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:59.684408  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:03:59.719801  1612 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:59.633661  6233 raft_consensus.cc:697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:59.634258  6287 raft_consensus.cc:2764] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:03:59.622318  6116 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:59.717206  1645 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:59.707787  2312 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:03:59.738282  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:59.732127  1646 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:03:59.718783  1538 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:59.702615  1647 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:03:59.665905  6079 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:03:59.614500  5841 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:59.625262  6181 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:03:59.745869  1639 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:03:59.660511  1564 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:03:59.676134  6200 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:03:59.772275  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 53568.3us
I20260709 13:03:59.782194  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:59.782507  1538 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:59.783118  1565 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:59.740041  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2648) took 2043 ms. Trace:
I20260709 13:03:59.784250  1538 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:03:59.784698  1646 rpcz_store.cc:276] 0709 13:03:57.696213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.731464 (+2035251us) service_pool.cc:224] Handling call
0709 13:03:59.731882 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:59.739774 (+  7892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.739986 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.741225  1642 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:03:59.749310  1647 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:59.787534  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:59.792621  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2649) took 2096 ms. Trace:
I20260709 13:03:59.793041  1642 rpcz_store.cc:276] 0709 13:03:57.696478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.740630 (+2044152us) service_pool.cc:224] Handling call
0709 13:03:59.740995 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:59.792339 (+ 51344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.792564 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.798559  1642 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:59.794083  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:59.736658  6036 consensus_meta.cc:287] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.779s	user 0.015s	sys 0.001s
I20260709 13:03:59.791479  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2655) took 1836 ms. Trace:
I20260709 13:03:59.810708  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:59.810498  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:03:59.818240  1713 raft_consensus.cc:2410] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:59.820780  1647 rpcz_store.cc:276] 0709 13:03:57.955568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.746055 (+1790487us) service_pool.cc:224] Handling call
0709 13:03:59.748550 (+  2495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.791068 (+ 42518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.791390 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.822793  1647 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:59.823611  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 453) took 1693 ms. Trace:
I20260709 13:03:59.824069  1647 rpcz_store.cc:276] 0709 13:03:58.130018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.822035 (+1692017us) service_pool.cc:224] Handling call
0709 13:03:59.822541 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.823327 (+   786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.823548 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:59.825872  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 245898us
I20260709 13:03:59.826268  1639 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:59.827011  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 450) took 1744 ms. Trace:
I20260709 13:03:59.827420  1639 rpcz_store.cc:276] 0709 13:03:58.082868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.787530 (+1704662us) service_pool.cc:224] Handling call
0709 13:03:59.826099 (+ 38569us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.826698 (+   599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.826944 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.829221  1647 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:59.829538  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 451) took 1720 ms. Trace:
I20260709 13:03:59.830322  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 455) took 1697 ms. Trace:
W20260709 13:03:59.791752  6270 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:03:59.673349  6171 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:59.831307  1647 rpcz_store.cc:276] 0709 13:03:58.133353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.825023 (+1691670us) service_pool.cc:224] Handling call
0709 13:03:59.825394 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.829981 (+  4587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.830260 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.831161  1642 rpcz_store.cc:276] 0709 13:03:58.083196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.797784 (+1714588us) service_pool.cc:224] Handling call
0709 13:03:59.798284 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.803229 (+  4945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.803785 (+   556us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.670050  6278 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.146s
W20260709 13:03:59.788631  6149 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.007s	sys 0.017s
I20260709 13:03:59.836122  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:59.833182  1687 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:59.859795  1687 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:03:59.776789  6079 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:59.740828  6279 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.861253  1687 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:03:59.738797  6216 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:59.845834  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 458) took 1578 ms. Trace:
I20260709 13:03:59.790807  6272 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:59.788568  6275 raft_consensus.cc:2764] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
W20260709 13:03:59.862104  6149 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.008s	sys 0.017s
I20260709 13:03:59.873344  6283 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:59.882400  6149 raft_consensus.cc:697] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:59.889998  1714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:03:59.890684  1714 raft_consensus.cc:606] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:03:59.891227  6149 consensus_queue.cc:237] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:59.893700  6233 consensus_queue.cc:237] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:59.895167  6035 consensus_meta.cc:287] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.646s	user 0.014s	sys 0.000s
I20260709 13:03:59.895737  6035 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:59.882742  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:59.899264  6035 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:59.899564  1564 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 12: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:03:59.900432  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:59.901186  1564 raft_consensus.cc:2368] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10.
I20260709 13:03:59.901996  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:59.911582  1565 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:03:59.854523  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:59.946610  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:59.868499  1647 rpcz_store.cc:276] 0709 13:03:58.267012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.844706 (+1577694us) service_pool.cc:224] Handling call
0709 13:03:59.845148 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.845398 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.845435 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.845498 (+    63us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:59.845545 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.845609 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.845763 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:59.948624  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.866122  6224 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:59.832703  6200 raft_consensus.cc:697] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:03:59.848563  6287 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:59.920835  5882 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.931774  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:03:59.874310  6240 consensus_meta.cc:287] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.636s	user 0.013s	sys 0.000s
I20260709 13:03:59.981559  1568 raft_consensus.cc:1240] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:59.982120  6240 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:03:59.983592  1718 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:03:59.984221  6283 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:59.984949  6240 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:03:59.986825  1611 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:59.987867  6207 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:59.988473  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:59.942247  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:59.989439  1571 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
W20260709 13:03:59.882835  6284 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:03:59.947963  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:03:59.991403  1571 raft_consensus.cc:1240] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:03:59.991878  1647 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:59.992309  1570 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:59.994693  1567 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:59.738624  6281 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:59.955485  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:03:59.868335  6216 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.833611  6036 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:59.833889  6117 consensus_queue.cc:237] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:03:59.961294  2313 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.021s	sys 0.025s
I20260709 13:03:59.733094  5852 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:03:59.976295  6287 raft_consensus.cc:697] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:03:59.976840  6282 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:04:00.022256  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:00.022050  6287 consensus_queue.cc:237] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:00.023315  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2637) took 2330 ms. Trace:
I20260709 13:04:00.023769  1643 rpcz_store.cc:276] 0709 13:03:57.692667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.175387 (+1482720us) service_pool.cc:224] Handling call
0709 13:03:59.175841 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:59.176080 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.176112 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:59.192830 (+ 16718us) log.cc:844] Serialized 26 byte log entry
0709 13:03:59.546726 (+353896us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:59.546769 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.547111 (+   342us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:00.023126 (+476015us) raft_consensus.cc:1685] finished
0709 13:04:00.023146 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:00.023261 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":328299,"threads_started":1,"wal-append.queue_time_us":577758}
I20260709 13:03:59.978209  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2639) took 2285 ms. Trace:
I20260709 13:04:00.024921  1644 rpcz_store.cc:276] 0709 13:03:57.693226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.178922 (+1485696us) service_pool.cc:224] Handling call
0709 13:03:59.179286 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:59.227013 (+ 47727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.977936 (+750923us) spinlock_profiling.cc:243] Waited 47 ms on lock 0x7b54002ab7d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:03:59.978144 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108042112}
I20260709 13:03:59.979096  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:03:59.979612  6079 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:00.026458  1572 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:00.027783  6079 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:03:59.990666  5841 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:00.028224  5852 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:00.028489  6203 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:00.030392  6179 consensus_meta.cc:287] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.077s	user 0.012s	sys 0.012s
I20260709 13:04:00.031734  1648 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:00.034096  6234 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.034916  6234 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:00.036075  1645 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:04:00.037220  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2110) took 1561 ms. Trace:
I20260709 13:04:00.002739  1611 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:00.039841  1611 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:03:59.800803  6111 consensus_queue.cc:237] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:00.004385  1568 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:04:00.003188  1565 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:00.042620  2314 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:00.007865  1647 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:04:00.044139  1643 raft_consensus.cc:1240] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:59.883119  5811 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:03:59.951818  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:00.013171  2313 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.021s	sys 0.025s
I20260709 13:03:59.969156  6186 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:00.004755  6281 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:00.017755  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:00.005013  5882 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:00.012319  6286 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:00.019351  6291 raft_consensus.cc:2764] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
W20260709 13:04:00.019799  6289 log.cc:927] Time spent T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:04:00.019644  6021 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:00.020311  6036 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:00.009477  6216 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:00.043292  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2131) took 1316 ms. Trace:
I20260709 13:03:59.960162  6161 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:00.045083  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 597745us
I20260709 13:03:59.882861  6279 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:00.049494  1645 rpcz_store.cc:276] 0709 13:03:58.475591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.034942 (+1559351us) service_pool.cc:224] Handling call
0709 13:04:00.035509 (+   567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:00.036635 (+  1126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.037142 (+   507us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:00.074084  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 289909us
I20260709 13:04:00.011684  6240 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:00.063617  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 111865us
W20260709 13:04:00.067003  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 323591us
I20260709 13:04:00.041512  5841 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:00.095151  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:00.074304  6278 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:00.051636  6281 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:00.079324  1568 raft_consensus.cc:1275] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:04:00.079988  6180 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:00.051466  6181 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:00.084062  1648 rpcz_store.cc:276] 0709 13:03:58.727111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.069058 (+341947us) service_pool.cc:224] Handling call
0709 13:04:00.043123 (+974065us) spinlock_profiling.cc:243] Waited 877 ms on lock 0x7b54002add54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:00.043220 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2017112832}
I20260709 13:04:00.116883  1648 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:04:00.117388  1648 pending_rounds.cc:85] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 5
I20260709 13:04:00.117667  1648 pending_rounds.cc:107] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:04:00.117959  1648 raft_consensus.cc:2889] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:00.119071  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2122) took 1627 ms. Trace:
I20260709 13:04:00.119474  1648 rpcz_store.cc:276] 0709 13:03:58.491486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.116045 (+1624559us) service_pool.cc:224] Handling call
0709 13:04:00.116452 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:00.118720 (+  2268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.118999 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:00.075429  1644 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 13:04:00.088500  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:00.096187  1564 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
W20260709 13:04:00.102468  6293 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:04:00.102814  6186 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:00.127478  6186 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:00.128196  6186 raft_consensus.cc:697] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:00.103716  5841 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.130236  5841 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:00.134662  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2206) took 1223 ms. Trace:
I20260709 13:04:00.135329  1566 rpcz_store.cc:276] 0709 13:03:58.911134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.912882 (+  1748us) service_pool.cc:224] Handling call
0709 13:03:58.913186 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:59.359084 (+445898us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.359118 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:59.450583 (+ 91465us) log.cc:844] Serialized 26 byte log entry
0709 13:03:59.794738 (+344155us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:59.794783 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.009450 (+214667us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54001b3d54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:00.009502 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:00.133830 (+124328us) raft_consensus.cc:1685] finished
0709 13:04:00.133863 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:00.134548 (+   685us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4603,"raft.queue_time_us":182389,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":8997,"spinlock_wait_cycles":473062784,"thread_start_us":479417,"threads_started":2,"wal-append.queue_time_us":437439}
I20260709 13:04:00.110458  1537 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:00.123977  1538 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:00.123420  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2119) took 1633 ms. Trace:
I20260709 13:04:00.142709  1644 rpcz_store.cc:276] 0709 13:03:58.490487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.064447 (+1573960us) service_pool.cc:224] Handling call
0709 13:04:00.074908 (+ 10461us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:00.123103 (+ 48195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.123357 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:00.083740  6286 raft_consensus.cc:697] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:00.090766  6240 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.166309  1640 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:04:00.086300  6021 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:00.115216  6061 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.181326  6240 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:00.202993  6274 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.005s	sys 0.004s
W20260709 13:04:00.203609  6274 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.005s	sys 0.005s
I20260709 13:04:00.140388  6291 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:00.220808  1567 raft_consensus.cc:1275] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:04:00.116050  6200 consensus_queue.cc:237] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:00.234575  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 139923us
I20260709 13:04:00.163821  6179 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.115688  5852 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:04:00.256705  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 456) took 2096 ms. Trace:
W20260709 13:04:00.268630  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.259518  1611 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:00.115345  6283 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 13:04:00.270531  1639 rpcz_store.cc:276] 0709 13:03:58.162881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.828374 (+1665493us) service_pool.cc:224] Handling call
0709 13:03:59.868354 (+ 39980us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.869976 (+  1622us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:00.180248 (+310272us) log.cc:844] Serialized 14 byte log entry
0709 13:04:00.256228 (+ 75980us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:00.256300 (+    72us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:00.256373 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.256452 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:00.256612 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45395,"thread_start_us":258825,"threads_started":2}
I20260709 13:04:00.144871  6272 raft_consensus.cc:697] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:00.175506  6013 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.285915  2342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 431) took 1005 ms. Trace:
I20260709 13:04:00.139878  6180 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:00.200634  5919 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:00.129441  6186 consensus_queue.cc:237] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:00.225416  6291 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:00.286404  2342 rpcz_store.cc:276] 0709 13:03:59.280412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.280930 (+   518us) service_pool.cc:224] Handling call
0709 13:03:59.309276 (+ 28346us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:59.349004 (+ 39728us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.349049 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:59.365846 (+ 16797us) log.cc:844] Serialized 52 byte log entry
0709 13:03:59.648555 (+282709us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:59.854916 (+206361us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.863514 (+  8598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.863606 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:00.285634 (+422028us) raft_consensus.cc:1685] finished
0709 13:04:00.285654 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:00.285837 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17191,"prepare.queue_time_us":459037,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":190236,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":226,"thread_start_us":464754,"threads_started":2,"wal-append.queue_time_us":455053}
I20260709 13:04:00.293809  6290 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:00.270540  6061 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:00.283661  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet eca30b43c26f420c965ee25f475581a0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:04:00.278764  6179 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:00.293440  6171 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:00.323292  1456 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 162081us
I20260709 13:04:00.298460  6299 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:00.286854  6013 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:00.334108  5882 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:00.336122  6179 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:00.336651  6171 consensus_queue.cc:1059] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:00.341229  5852 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:00.342803  5777 consensus_meta.cc:287] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.637s	user 0.012s	sys 0.000s
W20260709 13:04:00.329936  5779 consensus_meta.cc:287] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.003s	user 0.013s	sys 0.000s
I20260709 13:04:00.374302  5779 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:00.374692  6318 consensus_queue.cc:1059] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:04:00.345224  6206 consensus_meta.cc:287] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.019s	user 0.013s	sys 0.001s
I20260709 13:04:00.322507  6224 raft_consensus.cc:697] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:00.306558  6066 consensus_queue.cc:1048] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.061s
W20260709 13:04:00.367828  6289 log.cc:927] Time spent T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 13:04:00.368450  6271 consensus_meta.cc:287] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.124s	user 0.021s	sys 0.000s
I20260709 13:04:00.343907  6290 raft_consensus.cc:697] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:00.328267  6319 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:00.372606  5882 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:00.489672  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 234791us
W20260709 13:04:00.467453  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 185437us
I20260709 13:04:00.537191  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:00.537935  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 46416.9us
I20260709 13:04:00.387909  6179 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.378846  5777 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:00.469477  6271 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:00.466583  6319 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.546239  6290 consensus_queue.cc:237] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:00.548987  6321 log.cc:927] Time spent T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
W20260709 13:04:00.550344  6316 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 13:04:00.540329  5882 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:00.469848  6115 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.139s	user 0.016s	sys 0.000s
I20260709 13:04:00.387282  6317 consensus_queue.cc:1059] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.179s
I20260709 13:04:00.396011  6206 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:00.602999  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 457751us
I20260709 13:04:00.312063  6286 consensus_queue.cc:237] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:00.610905  1573 raft_consensus.cc:1240] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:00.610870  1538 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:00.627043  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:00.627802  2349 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
W20260709 13:04:00.640553  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 389425us
W20260709 13:04:00.667270  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.596818  6315 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.576406  6319 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:00.589478  6279 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:00.681556  1642 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:00.582841  6061 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:00.609622  6115 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:00.687362  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:00.567219  6179 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:00.700822  2346 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 6.
I20260709 13:04:00.619444  6287 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:00.688076  1537 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.003s	sys 0.014s
W20260709 13:04:00.705772  1537 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.003s	sys 0.014s
I20260709 13:04:00.389599  6272 consensus_queue.cc:237] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:00.694370  5841 consensus_meta.cc:287] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.563s	user 0.009s	sys 0.004s
W20260709 13:04:00.706801  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 596486us
W20260709 13:04:00.682101  2314 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.002s	sys 0.014s
W20260709 13:04:00.715384  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 91486.7us
I20260709 13:04:00.687599  6021 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:00.662945  6275 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:04:00.707491  6278 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:00.709314  6180 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:00.721660  2314 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.002s	sys 0.015s
W20260709 13:04:00.781622  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 728627us
W20260709 13:04:00.725553  1614 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.001s	sys 0.023s
W20260709 13:04:00.782821  1614 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.001s	sys 0.023s
W20260709 13:04:00.802459  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 137489us
I20260709 13:04:00.705120  6279 consensus_queue.cc:1059] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:04:00.639339  6224 consensus_queue.cc:237] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:00.719527  6233 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.018s	sys 0.006s
I20260709 13:04:00.692986  6315 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:00.779028  6049 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 1.273s	user 0.011s	sys 0.004s
W20260709 13:04:00.823069  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 196415us
W20260709 13:04:00.747226  6201 consensus_meta.cc:287] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.249s	user 0.015s	sys 0.000s
I20260709 13:04:00.713812  6287 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:00.846956  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 124582us
I20260709 13:04:00.832068  2314 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:00.851344  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 130225us
W20260709 13:04:00.820143  6233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.018s	sys 0.006s
I20260709 13:04:00.823164  6232 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.730082  5841 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:00.833580  6291 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:00.833269  6021 raft_consensus.cc:3055] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:00.790761  6268 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 13:04:00.829350  6335 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.805382  6180 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:00.951151  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 116492us
W20260709 13:04:00.951421  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 141323us
I20260709 13:04:00.936061  6336 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:00.855587  6201 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:00.941504  6021 raft_consensus.cc:740] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:04:00.921619  6323 log.cc:927] Time spent T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.428s	user 0.000s	sys 0.002s
I20260709 13:04:00.853364  6049 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:00.942643  6203 consensus_meta.cc:287] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.914s	user 0.009s	sys 0.006s
W20260709 13:04:00.942811  6266 consensus_meta.cc:287] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.293s	user 0.012s	sys 0.004s
I20260709 13:04:00.888813  6013 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.011545  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2224) took 1010 ms. Trace:
I20260709 13:04:01.057078  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:00.951644  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 119672us
W20260709 13:04:01.004611  6308 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
I20260709 13:04:00.943174  6335 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:00.935217  6232 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:01.022737  6332 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
W20260709 13:04:01.010821  6330 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:04:01.010023  6206 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:00.988701  5882 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.025539  6336 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:01.028712  6305 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.354s	user 0.001s	sys 0.001s
I20260709 13:04:01.105751  5882 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.106611  6232 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:01.003324  6339 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:01.055840  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 67748us
I20260709 13:04:01.039054  6179 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.011827  6271 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.057559  1570 rpcz_store.cc:276] 0709 13:04:00.000964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.023338 (+ 22374us) service_pool.cc:224] Handling call
0709 13:04:00.023648 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:00.023980 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:00.024008 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:00.024415 (+   407us) log.cc:844] Serialized 26 byte log entry
0709 13:04:00.203179 (+178764us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:00.203243 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.203345 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.011292 (+807947us) raft_consensus.cc:1685] finished
0709 13:04:01.011314 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.011481 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2048,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":113,"thread_start_us":178281,"threads_started":1,"wal-append.queue_time_us":864866}
I20260709 13:04:01.058115  1567 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17.
I20260709 13:04:01.076267  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:01.086232  6021 consensus_queue.cc:260] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:01.089218  6334 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:04:01.048331  6278 raft_consensus.cc:697] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:01.089879  6340 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 13:04:01.090000  6313 raft_consensus.cc:2955] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Committing config change with OpId 10.9: config changed from index 6 to 9, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:04:01.092554  5841 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.093067  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2642) took 3398 ms. Trace:
I20260709 13:04:01.041215  6337 raft_consensus.cc:3055] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:01.068027  6352 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.094360  6357 consensus_queue.cc:1059] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:01.075371  6266 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.106384  5882 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.010563  6275 consensus_queue.cc:1059] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:04:01.106781  6336 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.059216  5777 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:01.148680  6333 consensus_queue.cc:1059] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:04:01.166635  6278 consensus_queue.cc:237] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.067641  6158 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:04:01.115343  6049 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.116243  6287 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:01.117172  6335 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.116907  6339 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:01.129807  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 281129us
I20260709 13:04:01.137338  1640 rpcz_store.cc:276] 0709 13:03:57.694389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.333487 (+1639098us) service_pool.cc:224] Handling call
0709 13:03:59.356806 (+ 23319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:00.178081 (+821275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.092732 (+914651us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b5400182c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:01.092989 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1860752640}
W20260709 13:04:01.077322  6331 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.284s	user 0.000s	sys 0.002s
I20260709 13:04:01.143602  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2647) took 3447 ms. Trace:
I20260709 13:04:01.108754  6076 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.167616  6266 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.096539  6013 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.172245  2312 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:01.141542  6337 raft_consensus.cc:740] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:01.157550  6344 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:04:01.185390  6216 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.087s	user 0.008s	sys 0.008s
I20260709 13:04:01.190716  1641 rpcz_store.cc:276] 0709 13:03:57.695921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.368500 (+1672579us) service_pool.cc:224] Handling call
0709 13:03:59.369020 (+   520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:59.371527 (+  2507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.371575 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:59.985935 (+614360us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b2c000120e8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824777ea11 00007f824778e372 00007f8248ec0a01 00007f8248e8543a 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2
0709 13:03:59.986309 (+   374us) log.cc:844] Serialized 26 byte log entry
0709 13:04:00.265864 (+279555us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:00.266671 (+   807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.266794 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.137194 (+870400us) raft_consensus.cc:1685] finished
0709 13:04:01.137217 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.143501 (+  6284us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1783,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"spinlock_wait_cycles":41305472,"thread_start_us":278937,"threads_started":1,"wal-append.queue_time_us":700526}
I20260709 13:04:01.205502  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2158) took 1561 ms. Trace:
I20260709 13:04:01.220659  1573 rpcz_store.cc:276] 0709 13:03:59.644530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.644690 (+   160us) service_pool.cc:224] Handling call
0709 13:03:59.676793 (+ 32103us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:00.621226 (+944433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.205200 (+583974us) spinlock_profiling.cc:243] Waited 934 ms on lock 0x7b5400224cd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:01.205429 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2147966848}
I20260709 13:04:01.222265  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:01.222369  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:01.218189  1611 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.006s	sys 0.013s
I20260709 13:04:01.222780  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:01.223208  1611 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.006s	sys 0.013s
I20260709 13:04:01.224372  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:01.224541  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:01.223564  1714 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:04:01.226856  2346 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:04:01.196892  6161 consensus_meta.cc:287] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.099s	user 0.015s	sys 0.001s
I20260709 13:04:01.223141  1718 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:04:01.235525  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:01.238446  6310 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.239071  6310 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.184633  6315 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.149973  6271 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.064s
W20260709 13:04:01.211880  6116 consensus_meta.cc:287] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.457s	user 0.010s	sys 0.007s
I20260709 13:04:01.216472  6021 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:01.251250  2315 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.016s	sys 0.024s
I20260709 13:04:01.211967  6347 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.252581  1722 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:04:01.254088  2315 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.016s	sys 0.024s
I20260709 13:04:01.257894  1720 raft_consensus.cc:2410] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:04:01.216208  6200 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:01.227819  6117 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.153517  6352 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.038076  6180 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.214494  6076 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.241127  6161 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:01.262816  6347 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:01.277101  6337 consensus_queue.cc:260] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.279086  5882 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.282424  6234 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.227409  6013 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.254276  6344 raft_consensus.cc:697] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:01.233000  6216 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.266268  6292 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:01.317615  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 145502us
I20260709 13:04:01.318238  1566 raft_consensus.cc:1240] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:01.284396  6201 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:01.318681  6281 consensus_meta.cc:287] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.171s	user 0.007s	sys 0.010s
I20260709 13:04:01.263571  6200 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.336540  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:01.336820  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:01.362407  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 66800.8us
I20260709 13:04:01.313148  6351 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.275712  6117 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.354425  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:01.331066  6290 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:01.376670  6290 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:01.377106  6352 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.378003  6342 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:04:01.377169  6290 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:01.382100  6076 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.353367  6365 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:01.386211  6365 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:01.377929  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:01.390290  1570 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:04:01.355576  6234 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:01.394307  6240 consensus_meta.cc:287] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.191s	user 0.018s	sys 0.000s
I20260709 13:04:01.394160  6234 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.394749  6240 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.396519  6234 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.332262  6337 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:01.396961  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:01.398267  1568 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:04:01.331195  6161 consensus_queue.cc:1059] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:04:01.330361  6111 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:01.365355  1687 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:01.371866  1646 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:01.360915  5777 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:04:01.370347  6116 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:04:01.373936  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:01.360497  6310 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:04:01.375464  6339 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.376032  6117 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:01.394497  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:01.443317  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:01.357350  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 48859us
I20260709 13:04:01.443991  1639 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:01.361389  1537 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.011s	sys 0.025s
I20260709 13:04:01.358680  5779 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.358466  6292 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:01.398578  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2232) took 1588 ms. Trace:
I20260709 13:04:01.403823  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:01.413273  1647 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:01.448375  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:01.449132  1640 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:01.422251  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 57103.4us
I20260709 13:04:01.369004  6353 raft_consensus.cc:3055] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:01.437955  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:01.453491  1641 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:01.453199  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:01.454579  1647 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:01.397084  6240 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:01.450971  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 596666us
I20260709 13:04:01.452570  1572 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:01.459892  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:01.462014  1647 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:01.440034  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 48204us
I20260709 13:04:01.463524  1537 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:01.446259  5779 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:01.447120  1715 rpcz_store.cc:276] 0709 13:03:59.810005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.811805 (+  1800us) service_pool.cc:224] Handling call
0709 13:03:59.856616 (+ 44811us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:59.870616 (+ 14000us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:00.235286 (+364670us) log.cc:844] Serialized 14 byte log entry
0709 13:04:00.563035 (+327749us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:00.599861 (+ 36826us) log.cc:844] Serialized 26 byte log entry
0709 13:04:00.600616 (+   755us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:00.600765 (+   149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.600980 (+   215us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.398318 (+797338us) raft_consensus.cc:1685] finished
0709 13:04:01.398345 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.398503 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16228,"prepare.queue_time_us":509200,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":13127,"raft.queue_time_us":113532,"raft.run_cpu_time_us":456,"raft.run_wall_time_us":56601,"thread_start_us":564943,"threads_started":2,"wal-append.queue_time_us":1069266}
I20260709 13:04:01.467188  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:01.476410  1572 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:01.421361  6234 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.465777  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:01.466082  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:01.421253  6066 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.444294  6286 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:01.482898  1719 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:01.471606  6028 consensus_queue.cc:1059] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:04:01.387475  6344 consensus_queue.cc:237] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:01.485430  1537 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:01.486253  1718 raft_consensus.cc:1240] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:04:01.508661  1722 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:01.479661  1570 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:04:01.510653  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:01.453227  6353 raft_consensus.cc:740] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:01.497138  1716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:01.366272  6281 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.496737  6271 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:01.513499  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:01.436864  6111 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:01.530632  6304 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:01.518234  1716 raft_consensus.cc:606] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:01.518422  1537 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:01.398358  6351 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:01.488325  1687 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:01.504462  1639 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:04:01.489984  6342 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.467203  6357 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.490446  6286 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.527231  1640 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:01.516072  1647 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:04:01.535537  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:01.535933  1537 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:01.589771  1564 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
W20260709 13:04:01.590131  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 54291.9us
I20260709 13:04:01.592797  1537 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:01.538614  1570 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:01.469487  6310 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:04:01.594519  6271 consensus_queue.cc:1059] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:01.537446  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2660) took 3230 ms. Trace:
I20260709 13:04:01.534035  6297 consensus_queue.cc:1059] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 13:04:01.595547  6271 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:01.595854  1642 rpcz_store.cc:276] 0709 13:03:58.307350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.848867 (+1541517us) service_pool.cc:224] Handling call
0709 13:03:59.849299 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:00.693004 (+843705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.525456 (+832452us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b54003566d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:01.537370 (+ 11914us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1913407872}
I20260709 13:04:01.596516  6297 raft_consensus.cc:3055] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:01.542353  5852 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:01.597250  5852 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:01.596938  6297 raft_consensus.cc:740] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:01.597777  5852 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:01.534474  6355 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:01.597051  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:01.501807  6201 consensus_queue.cc:1059] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 13:04:01.559588  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 71357.5us
I20260709 13:04:01.604465  1687 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:01.498811  6066 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:01.561612  6342 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.490784  6234 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.562233  1722 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:04:01.547505  1569 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:04:01.574884  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:01.608080  1639 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:01.576911  1572 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:04:01.544149  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:01.599367  1642 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:01.567505  5777 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:01.569031  6365 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.599984  6180 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.606141  6364 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.596933  6271 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:01.611654  6116 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.611894  6216 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.615032  6161 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:01.623590  6367 log.cc:927] Time spent T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.254s	user 0.002s	sys 0.000s
I20260709 13:04:01.647476  6116 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.614471  5777 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:01.620361  6201 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:04:01.664590  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 71848.9us
I20260709 13:04:01.636806  6013 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:01.659591  6341 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:04:01.598449  6297 consensus_queue.cc:260] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:01.679675  6297 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:01.638029  6364 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:01.605289  6353 consensus_queue.cc:260] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:01.662137  5777 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:01.660676  6200 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:01.693678  6353 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:01.614217  6355 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:01.568084  6357 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:01.698312  1537 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:01.670097  6349 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:01.674575  6299 raft_consensus.cc:3055] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:01.672720  6201 raft_consensus.cc:697] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:01.724398  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2664) took 3338 ms. Trace:
I20260709 13:04:01.711459  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:01.739959  1639 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:01.673676  6347 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:01.743131  2348 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 15: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:01.742143  1643 rpcz_store.cc:276] 0709 13:03:58.385627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.024989 (+1639362us) service_pool.cc:224] Handling call
0709 13:04:00.043977 (+ 18988us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:00.044472 (+   495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.724168 (+1679696us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b54001fe0d0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:01.724325 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42993408}
I20260709 13:04:01.743923  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:01.746119  1639 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:01.686661  6286 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:01.709010  6358 log.cc:927] Time spent T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
I20260709 13:04:01.749063  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:01.687549  6013 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.759899  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:01.694823  6272 consensus_queue.cc:1175] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "130bc5ff39b847b1b3204626f284f838" responder_term: 9 status { last_received { term: 9 index: 7 } last_committed_idx: 4 last_received_current_leader { term: 9 index: 7 } }
W20260709 13:04:01.712198  6374 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:04:01.683436  6158 consensus_queue.cc:1059] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:04:01.766216  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:01.766898  1644 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:04:01.726539  6234 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.788957  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:01.713414  6111 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:01.710892  6357 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.776837  1641 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:01.779528  1639 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 12: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:01.737650  6299 raft_consensus.cc:740] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:01.709203  6277 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.790087  1643 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 8.
I20260709 13:04:01.703279  6224 consensus_queue.cc:1059] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:04:01.752640  6383 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:01.717710  6371 consensus_queue.cc:1048] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:04:01.709061  6366 consensus_queue.cc:1048] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.257s
I20260709 13:04:01.796865  6278 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:01.822227  1643 raft_consensus.cc:1275] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:04:01.824190  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 125956us
I20260709 13:04:01.777935  6361 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:01.716491  6349 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:01.712850  6290 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:01.800593  1641 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:04:01.799919  6277 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:01.794288  6271 raft_consensus.cc:3055] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:01.848670  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:01.850812  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:01.846516  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 242149us
I20260709 13:04:01.833338  1639 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:01.838500  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:01.839931  1643 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:01.841676  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:01.843124  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:01.788170  6013 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:01.825961  6383 raft_consensus.cc:697] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:01.856923  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.864068  2349 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:01.869761  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:01.870096  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:01.876752  1715 raft_consensus.cc:2410] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:04:01.878500  1718 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:04:01.870061  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:04:01.855481  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 271892us
I20260709 13:04:01.879213  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:01.889712  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:01.881032  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:01.894560  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:01.884445  1716 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:04:01.895721  1639 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
W20260709 13:04:01.893910  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.893723  1714 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 14: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:01.891305  1643 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:01.903365  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:01.856978  6021 consensus_meta.cc:287] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.595s	user 0.011s	sys 0.004s
I20260709 13:04:01.852479  6234 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.845158  6271 raft_consensus.cc:740] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:01.914750  1642 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:01.833093  6278 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.919313  2347 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:04:01.923790  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:01.923751  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:01.912376  6319 consensus_meta.cc:287] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.208s	user 0.009s	sys 0.009s
I20260709 13:04:01.883536  6352 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.880061  6066 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:01.937806  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:01.837303  6361 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.942239  2341 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:01.942874  6342 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.844960  6201 consensus_queue.cc:237] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.943423  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:01.968735  1687 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:01.978322  1643 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:04:01.973217  1537 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:01.979840  2348 raft_consensus.cc:1240] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 13:04:01.934201  6337 consensus_meta.cc:287] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.005s
I20260709 13:04:02.017372  1611 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:01.985224  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.026163  1567 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 12: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:01.959067  6310 raft_consensus.cc:3055] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:04:01.979524  6341 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 13:04:02.033823  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 61860.2us
I20260709 13:04:02.034181  1640 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:04:01.965008  6013 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:01.944160  6278 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:01.958014  5779 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:01.962994  6290 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:01.948810  6378 log.cc:927] Time spent T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 13:04:01.949160  6364 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:01.923460  6234 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.978614  6035 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.967474  6117 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:02.041815  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:02.054334  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.080646  1564 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:04:01.881459  6299 consensus_queue.cc:260] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.085041  6299 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:02.036365  1611 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:02.073374  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.037019  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:02.093369  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:01.955610  6355 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:02.046805  6339 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:02.098340  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 62228us
I20260709 13:04:01.967419  6319 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:01.966202  6352 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.041373  6275 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.041887  6310 raft_consensus.cc:740] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:02.123574  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:02.047497  6385 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:04:02.126255  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 157659us
I20260709 13:04:02.075902  5779 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.128003  1611 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:02.075177  6349 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:02.142818  1687 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:02.144279  6240 raft_consensus.cc:3055] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:04:02.144734  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 448) took 2605 ms (client timeout 1839 ms). Trace:
I20260709 13:04:02.078212  6351 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:02.149456  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:02.151153  1611 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:02.153399  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:02.154117  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:02.061775  6161 consensus_queue.cc:1059] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
W20260709 13:04:02.119460  6341 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:04:02.045794  5777 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:02.149696  1720 rpcz_store.cc:269] 0709 13:03:59.535281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.535460 (+   179us) service_pool.cc:224] Handling call
0709 13:04:02.140889 (+2605429us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b5400212d54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:02.141041 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3913346432}
I20260709 13:04:02.171594  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:02.150305  1687 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:02.154363  1715 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:02.066931  6388 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:04:02.067140  6278 consensus_queue.cc:1059] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:04:02.142403  6385 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.124825  6364 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:02.124955  6393 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:02.200531  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:02.201870  1567 raft_consensus.cc:2376] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13. Re-sending same reply.
I20260709 13:04:02.131206  6339 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.184587  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:02.211431  1720 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:02.209317  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2135) took 2684 ms. Trace:
I20260709 13:04:02.165077  6384 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:02.096709  6035 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:02.195024  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:02.160775  6066 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:02.190321  6266 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:02.097136  6117 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.041177  6383 consensus_queue.cc:237] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.149483  6240 raft_consensus.cc:740] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:02.205566  1721 raft_consensus.cc:2410] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:04:02.185293  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.174537  6351 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:02.040297  6271 consensus_queue.cc:260] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.221164  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:02.216492  1648 rpcz_store.cc:276] 0709 13:03:59.524665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.120692 (+596027us) service_pool.cc:224] Handling call
0709 13:04:00.121683 (+   991us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:00.268021 (+146338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:00.268083 (+    62us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:04:00.376778 (+108695us) log.cc:844] Serialized 78 byte log entry
0709 13:04:00.732984 (+356206us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:00.779282 (+ 46298us) log.cc:844] Serialized 14 byte log entry
0709 13:04:00.977494 (+198212us) log.cc:844] Serialized 14 byte log entry
0709 13:04:00.994155 (+ 16661us) log.cc:844] Serialized 14 byte log entry
0709 13:04:01.337480 (+343325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.338300 (+   820us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.382732 (+ 44432us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b1800053140. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d98522 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:04:02.208944 (+826212us) raft_consensus.cc:1685] finished
0709 13:04:02.208979 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.209226 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":74268,"prepare.run_cpu_time_us":254,"prepare.run_wall_time_us":7343,"raft.queue_time_us":503208,"raft.run_cpu_time_us":689,"raft.run_wall_time_us":35984,"spinlock_wait_cycles":4360576,"thread_start_us":814376,"threads_started":4,"wal-append.queue_time_us":967451}
I20260709 13:04:02.236886  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:02.260303  1713 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:04:02.261219  1713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:02.212826  6278 consensus_queue.cc:1243] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer cf2f71592be8465fb18e36e15d0b2f89 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:02.146991  6275 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:02.194567  6364 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.239647  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:02.159086  5779 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.198479  6385 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:02.213299  6339 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:02.247192  1611 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:02.278703  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.239537  6320 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:02.252195  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.261896  1713 raft_consensus.cc:606] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:02.287202  1713 raft_consensus.cc:614] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:04:02.204059  6391 log.cc:927] Time spent T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:04:02.240103  6066 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.288895  6066 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:02.256191  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:02.265446  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 115237us
I20260709 13:04:02.290797  1722 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:04:02.256368  6028 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:02.257498  6013 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:02.284698  6352 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:02.271481  6351 raft_consensus.cc:697] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:02.268352  6181 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:02.272797  6364 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:02.294585  1720 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:04:02.291409  1687 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:02.312671  1687 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:02.298326  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:02.284853  6240 consensus_queue.cc:260] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.289455  6275 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:02.285308  6361 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:02.285145  6360 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:02.241753  6310 consensus_queue.cc:260] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.304492  6028 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.322418  6035 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:02.305330  6013 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:02.323626  6313 consensus_meta.cc:287] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 1.080s	user 0.014s	sys 0.005s
I20260709 13:04:02.311008  6318 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.324041  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.316929  1687 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:02.336206  6117 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:02.336776  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2663) took 4024 ms. Trace:
I20260709 13:04:02.341534  6234 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:02.338423  6277 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:02.337098  5777 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:02.333273  6275 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.332163  6360 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.309077  6395 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:02.356796  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.337980  1715 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:04:02.350667  6319 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:02.351534  6304 consensus_meta.cc:287] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.808s	user 0.016s	sys 0.000s
I20260709 13:04:02.352123  1646 rpcz_store.cc:276] 0709 13:03:58.312353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.896056 (+1583703us) service_pool.cc:224] Handling call
0709 13:03:59.896400 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.425788 (+1529388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.336437 (+910649us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54001fdbd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:02.336665 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3392991616}
I20260709 13:04:02.346179  6310 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:02.349434  6351 consensus_queue.cc:237] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:02.347378  5852 consensus_meta.cc:287] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.749s	user 0.014s	sys 0.003s
W20260709 13:04:02.367707  6390 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:04:02.368556  5779 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:02.304174  6320 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:02.370394  6277 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:02.353590  6275 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:02.373049  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 742) took 1337 ms. Trace:
I20260709 13:04:02.373769  6360 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:02.338203  6392 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:04:02.376204  6353 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.682s	user 0.013s	sys 0.002s
I20260709 13:04:02.376682  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.403450  6304 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:02.357290  6318 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:02.398180  6395 raft_consensus.cc:697] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:04:02.383693  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 66847.7us
I20260709 13:04:02.384680  1572 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:04:02.385183  6066 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.388372  6292 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:02.393815  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:02.419344  6277 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:02.385164  6386 log.cc:927] Time spent T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.149s	user 0.000s	sys 0.002s
I20260709 13:04:02.420244  1566 rpcz_store.cc:276] 0709 13:04:01.035985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.036637 (+   652us) service_pool.cc:224] Handling call
0709 13:04:01.037215 (+   578us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.359827 (+322612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.372444 (+1012617us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b5400013e54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:02.372714 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":645849344}
I20260709 13:04:02.456086  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.457834  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:02.377123  6400 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:02.462157  6400 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.462765  6400 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:02.402285  6287 raft_consensus.cc:2764] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:04:02.359555  6393 raft_consensus.cc:697] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:04:02.421306  6297 consensus_meta.cc:287] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.741s	user 0.017s	sys 0.000s
I20260709 13:04:02.444047  1687 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:02.315900  6181 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.467267  1570 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:04:02.465946  6317 raft_consensus.cc:3055] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:02.469460  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.412278  5852 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.466121  6354 raft_consensus.cc:3055] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:02.480281  1567 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:02.454725  6277 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:02.505060  1611 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:02.445022  6392 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.473685  6384 raft_consensus.cc:697] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:02.510957  1716 raft_consensus.cc:1275] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:04:02.478689  6317 raft_consensus.cc:740] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:04:02.497189  6286 consensus_meta.cc:287] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.719s	user 0.010s	sys 0.006s
I20260709 13:04:02.490242  6354 raft_consensus.cc:740] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:02.466487  6364 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.520853  6313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:54222 (request call id 0) took 5666 ms. Trace:
I20260709 13:04:02.521278  6335 raft_consensus.cc:3055] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:02.537045  6335 raft_consensus.cc:740] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:02.537458  6403 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:02.538273  6335 consensus_queue.cc:260] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:02.539654  6335 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:02.539829  6403 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:02.522503  6392 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:02.523819  6327 log.cc:927] Time spent T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 2.149s	user 0.006s	sys 0.003s
I20260709 13:04:02.524147  5779 consensus_queue.cc:1059] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:02.500017  6234 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.536788  1687 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:02.526836  6320 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:02.528293  6317 consensus_queue.cc:260] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:02.515880  6361 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:02.498821  6319 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.502388  6404 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:02.542347  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.543445  1687 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:02.509694  6287 consensus_queue.cc:1048] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:04:02.556418  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:02.552687  1566 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:02.506363  6292 consensus_queue.cc:1059] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:04:02.562645  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.570544  1566 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:04:02.547430  6392 raft_consensus.cc:2764] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:04:02.515622  5777 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:04:02.578271  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2121) took 4087 ms. Trace:
I20260709 13:04:02.572870  6404 raft_consensus.cc:697] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:02.596288  1645 rpcz_store.cc:276] 0709 13:03:58.491184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.112155 (+1620971us) service_pool.cc:224] Handling call
0709 13:04:00.162320 (+ 50165us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:00.177186 (+ 14866us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:00.177235 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:00.177908 (+   673us) log.cc:844] Serialized 26 byte log entry
0709 13:04:00.372392 (+194484us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:00.372444 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.372527 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:02.523116 (+2150589us) spinlock_profiling.cc:243] Waited 9.63 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:02.572820 (+ 49704us) raft_consensus.cc:1685] finished
0709 13:04:02.572846 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.578097 (+  5251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14272,"raft.queue_time_us":93018,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":8483,"spinlock_wait_cycles":22156032,"thread_start_us":87626,"threads_started":1,"wal-append.queue_time_us":174414}
I20260709 13:04:02.598419  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:02.580905  6364 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:02.590423  6384 consensus_queue.cc:237] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.547066  6361 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:02.531481  6117 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:04:02.560320  5779 raft_consensus.cc:3055] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:02.542043  6286 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.574096  6317 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:02.571389  6116 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:02.552928  6234 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.608114  6013 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:02.627316  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.630291  6286 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:02.546593  6339 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.559865  6393 consensus_queue.cc:237] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.650846  6339 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:02.612867  5779 raft_consensus.cc:740] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:02.660356  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.671873  6400 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:02.674119  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.660903  5779 consensus_queue.cc:260] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.675449  1611 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:02.676043  6349 raft_consensus.cc:3055] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:02.610167  6354 consensus_queue.cc:260] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.624571  6275 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:02.676453  6339 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:02.682363  6339 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.682791  6339 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:02.677348  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:02.678586  6234 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:02.684674  6380 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.600s	user 0.004s	sys 0.000s
I20260709 13:04:02.678872  6404 consensus_queue.cc:237] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:02.688153  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:02.627259  6277 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.679104  6342 consensus_queue.cc:1059] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:04:02.690351  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2234) took 1288 ms. Trace:
I20260709 13:04:02.676601  6349 raft_consensus.cc:740] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:02.678436  6354 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:02.613227  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.633905  1719 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 12: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:04:02.635758  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 92414.7us
I20260709 13:04:02.636759  1571 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:02.596115  6405 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:02.638350  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.608421  6361 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:02.608685  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:02.560191  6319 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:02.689322  1568 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:02.698700  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:02.700630  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.696832  1687 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:02.714152  1572 raft_consensus.cc:2410] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:04:02.691082  1565 rpcz_store.cc:276] 0709 13:04:01.402066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.402653 (+   587us) service_pool.cc:224] Handling call
0709 13:04:01.403088 (+   435us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:01.437979 (+ 34891us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:01.474595 (+ 36616us) log.cc:844] Serialized 14 byte log entry
0709 13:04:01.845583 (+370988us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:01.860324 (+ 14741us) log.cc:844] Serialized 52 byte log entry
0709 13:04:01.860488 (+   164us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:01.860520 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.860605 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:02.684220 (+823615us) spinlock_profiling.cc:243] Waited 7.39 ms on lock 0x7b24001668f8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245c1e9c4 00007f8245c1cacb 00007f8245dd7dc9 00007f8245dd7a55 00007f8245dd709d 00007f8248a0910e 00007f8248a2088c 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221
0709 13:04:02.689733 (+  5513us) raft_consensus.cc:1685] finished
0709 13:04:02.689854 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.690075 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42831,"prepare.queue_time_us":36393,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":5571,"raft.queue_time_us":40078,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":25061,"spinlock_wait_cycles":18203136,"thread_start_us":323083,"threads_started":1,"wal-append.queue_time_us":521542}
I20260709 13:04:02.709152  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:02.553467  6395 consensus_queue.cc:237] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.718899  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:02.715051  2348 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:02.715998  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:02.732750  1565 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:02.733780  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.717002  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:02.738444  2346 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:04:02.739331  2313 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:02.739807  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2160) took 1135 ms. Trace:
I20260709 13:04:02.739977  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:02.740685  1567 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:02.740302  1639 rpcz_store.cc:276] 0709 13:04:01.604455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.896885 (+292430us) service_pool.cc:224] Handling call
0709 13:04:01.921761 (+ 24876us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:01.922188 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:01.922219 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:01.922788 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 13:04:02.494923 (+572135us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b1800397e40. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a250d1 00007f8248e40a0f
0709 13:04:02.595080 (+100157us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:02.595151 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.595256 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:02.739502 (+144246us) raft_consensus.cc:1685] finished
0709 13:04:02.739523 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.739751 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":874,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":12933,"spinlock_wait_cycles":43760000,"thread_start_us":99634,"threads_started":1,"wal-append.queue_time_us":771311}
I20260709 13:04:02.743139  1565 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:02.705482  5779 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:02.745700  6333 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:02.746913  6234 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.747201  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.747584  6234 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.750321  6234 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:02.751231  6066 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:02.637491  6271 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:02.726123  6274 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:02.727077  6277 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:02.728114  6400 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:02.729036  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:02.718783  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.715119  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.731005  6396 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:02.758065  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:02.759316  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:02.792560  1639 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:04:02.793974  1644 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:02.795532  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:02.796620  1644 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 12: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:02.691095  6342 consensus_queue.cc:1243] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:04:02.759351  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:02.764353  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.765058  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:02.766557  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.768702  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.799333  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:02.799733  1571 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 8.
I20260709 13:04:02.800042  1641 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:02.800748  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.802042  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.747840  1565 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:04:02.782243  1568 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
I20260709 13:04:02.766098  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:02.788244  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2169) took 2990 ms. Trace:
W20260709 13:04:02.774366  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 257451us
I20260709 13:04:02.740398  2313 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:04:02.809333  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 69009.3us
I20260709 13:04:02.792877  1647 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:04:02.807885  1569 rpcz_store.cc:276] 0709 13:03:59.770240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.990392 (+220152us) service_pool.cc:224] Handling call
0709 13:03:59.990765 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:01.610817 (+1620052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.760203 (+1149386us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b54002247d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:02.760643 (+   440us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3573902208}
I20260709 13:04:02.706310  6405 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:02.817346  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.813445  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:02.810571  1646 raft_consensus.cc:2368] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12.
I20260709 13:04:02.811915  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:02.721271  6366 consensus_queue.cc:1059] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.097s
I20260709 13:04:02.813148  1720 raft_consensus.cc:1240] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:04:02.805860  1567 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:02.759878  6234 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.813742  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.832629  1565 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:04:02.833873  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.834867  1565 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:02.835438  1572 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 12: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:02.814322  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:02.834789  1713 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:02.837823  1571 raft_consensus.cc:2368] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12.
I20260709 13:04:02.717460  6320 consensus_queue.cc:1059] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 13:04:02.815308  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:02.840958  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.841580  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.842160  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.843485  1572 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:04:02.843875  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:02.844808  1565 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:02.844394  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.760430  6318 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:02.847034  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.820317  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:02.809895  6274 raft_consensus.cc:697] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:02.812543  1721 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:02.814412  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.838083  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:02.864109  1569 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:02.833086  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.811556  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:02.849179  1612 connection.cc:256] RPC call timeout handler was delayed by -1.175520175000031s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1613s for a response
W20260709 13:04:02.864899  1612 connection.cc:256] RPC call timeout handler was delayed by -1.1735331340000243s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2081s for a response
I20260709 13:04:02.865178  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:02.865249  1568 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:02.849815  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.818049  6333 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:02.815917  5777 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:02.824940  6405 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:02.851156  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 133723us
I20260709 13:04:02.859277  1571 raft_consensus.cc:2410] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:04:02.847298  1611 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:02.890285  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 57363.5us
W20260709 13:04:02.890807  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 43738.4us
I20260709 13:04:02.892298  1611 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:02.838682  1720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:02.866890  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.865392  1567 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:02.864931  1714 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:04:02.896437  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.536428  6313 rpcz_store.cc:276] 0709 13:03:56.847444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.849435 (+  1991us) service_pool.cc:224] Handling call
0709 13:03:57.088599 (+239164us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b5400467e20. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e02520 00007f8248e8bb42 00007f8248e8a800 00007f8248e839fa 00007f824de4b4c9 00007f82473c7c54 00007f82473c7be2 00007f82473c7b11 00007f82473c7a8c 00007f82473c6d42
0709 13:03:58.447696 (+1359097us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b5800516ed0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248de10aa 00007f8248e63447 00007f8248e8070a 00007f8248e8176a 00007f8248e8091b 00007f8248e8c03c 00007f8248e8b6b3 00007f8248e839fa 00007f824de4b4c9 00007f82473c7c54
0709 13:03:58.477032 (+ 29336us) log.cc:844] Serialized 501 byte log entry
0709 13:04:02.513701 (+4036669us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11064,"raft.run_cpu_time_us":1928,"raft.run_wall_time_us":36731,"spinlock_wait_cycles":1200674944,"thread_start_us":25506,"threads_started":1,"wal-append.queue_time_us":120387,"wal-append.run_cpu_time_us":2697,"wal-append.run_wall_time_us":1088991}
I20260709 13:04:02.852118  6234 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:02.901257  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.840137  6277 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:04:02.902181  1571 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:02.903069  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.874296  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:02.850756  1641 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
W20260709 13:04:02.876089  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.904321  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.905532  6388 consensus_queue.cc:1059] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:02.906514  6388 consensus_queue.cc:1243] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:02.906993  6388 raft_consensus.cc:3055] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:02.907368  6388 raft_consensus.cc:740] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:02.862294  6066 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:02.908258  6388 consensus_queue.cc:260] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.910555  6414 consensus_queue.cc:1059] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:02.912479  6339 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.894063  1720 raft_consensus.cc:606] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:02.923570  1720 raft_consensus.cc:614] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:02.903187  6234 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:02.913267  6319 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:02.913869  6301 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 2.539s	user 0.002s	sys 0.003s
I20260709 13:04:02.915112  1685 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:02.917353  6201 raft_consensus.cc:3055] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:02.918095  1456 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 59d236b930214b1f91fb55ece6fecd68 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 10)
I20260709 13:04:02.894573  1568 raft_consensus.cc:2393] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 10: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:02.919497  6335 raft_consensus.cc:3055] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:02.886025  6186 raft_consensus.cc:3055] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:02.985759  6384 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:02.885671  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 75312.2us
I20260709 13:04:02.861810  6013 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:02.868558  1565 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
W20260709 13:04:02.943245  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.930985  6201 raft_consensus.cc:740] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:02.899852  6315 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:02.889292  5777 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.883481  6333 raft_consensus.cc:697] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:02.971469  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:02.986603  1639 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:02.971486  1687 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:02.909200  6066 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:03.038266  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 114909us
I20260709 13:04:02.924275  6388 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:03.012933  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.923673  6393 consensus_queue.cc:1059] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.093s
I20260709 13:04:02.983508  6161 raft_consensus.cc:3055] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:04:03.042434  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 107353us
I20260709 13:04:03.044086  1687 leader_election.cc:400] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12.
I20260709 13:04:03.045418  1687 leader_election.cc:403] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:04:02.982638  1567 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:03.024427  6335 raft_consensus.cc:740] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:03.030211  6186 raft_consensus.cc:740] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:02.978044  6234 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:02.933550  6310 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.090s
W20260709 13:04:03.035243  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 168570us
W20260709 13:04:02.973357  1456 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 55283us
I20260709 13:04:03.039710  1571 raft_consensus.cc:2410] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:04:03.074383  1612 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:02.988543   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:04:02.986125  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.060470  1687 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:03.052613  6415 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.164s	user 0.002s	sys 0.000s
I20260709 13:04:03.062978  1611 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:02.974852  6339 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:03.036727  6421 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.973764  6274 consensus_queue.cc:237] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:02.977905  6392 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:03.101675  1564 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:04:03.027246  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:03.065579  1538 leader_election.cc:336] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.770s (SENT)
I20260709 13:04:03.041811  6161 raft_consensus.cc:740] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:03.109512  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 577) took 1390 ms. Trace:
I20260709 13:04:03.110090  2347 rpcz_store.cc:276] 0709 13:04:01.702552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.703091 (+   539us) service_pool.cc:224] Handling call
0709 13:04:01.703479 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.924757 (+221278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.092799 (+1168042us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54000cc954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:03.093095 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":495615360}
I20260709 13:04:03.060796  5779 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:03.080417  6287 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:03.082727  6066 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:03.084311  6186 consensus_queue.cc:260] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:03.080262  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.035535  6013 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:03.090909  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:03.098249  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.102488  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2569) took 1261 ms. Trace:
I20260709 13:04:03.078723   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:03.080246  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:03.039794  6313 consensus_queue.cc:1059] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:04:03.133155  1571 raft_consensus.cc:1813] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Handling vote request from an unknown peer f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:03.035635  6384 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:03.135212  1719 rpcz_store.cc:276] 0709 13:04:01.841254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.841384 (+   130us) service_pool.cc:224] Handling call
0709 13:04:03.102219 (+1260835us) spinlock_profiling.cc:243] Waited 772 ms on lock 0x7b5400212354. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:03.102414 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1775305600}
I20260709 13:04:03.138049  1571 raft_consensus.cc:2410] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:04:03.114390  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2255) took 1062 ms. Trace:
I20260709 13:04:03.067054  6414 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:03.147945  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.116881  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:03.067706  6234 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.118103  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:03.157159  1567 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:03.142907  1568 rpcz_store.cc:276] 0709 13:04:02.052557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.026117 (+973560us) service_pool.cc:224] Handling call
0709 13:04:03.114296 (+ 88179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:03.170145  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:03.113055  6116 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.175514  1568 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:03.176476  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 744) took 1142 ms. Trace:
I20260709 13:04:03.177791  1571 raft_consensus.cc:1275] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:04:03.178112  1568 rpcz_store.cc:276] 0709 13:04:02.034145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.174789 (+1140644us) service_pool.cc:224] Handling call
0709 13:04:03.175161 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.176153 (+   992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.176401 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:03.179044  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2205) took 1225 ms. Trace:
I20260709 13:04:03.181795  1571 rpcz_store.cc:276] 0709 13:04:01.953454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.176840 (+1223386us) service_pool.cc:224] Handling call
0709 13:04:03.177312 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.178663 (+  1351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.178958 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:03.115166  5779 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.113314  6404 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:03.117551  6339 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:03.190508  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 127095us
I20260709 13:04:03.184078  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 746) took 1149 ms. Trace:
W20260709 13:04:03.195431  6425 log.cc:927] Time spent T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 13:04:03.196904  6409 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.724s	user 0.004s	sys 0.000s
W20260709 13:04:03.200249  6385 consensus_meta.cc:287] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.914s	user 0.012s	sys 0.000s
I20260709 13:04:03.204246  6028 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:03.206049  6013 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:03.207007  6420 log.cc:927] Time spent T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.197s	user 0.001s	sys 0.001s
I20260709 13:04:03.145699  6333 consensus_queue.cc:237] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:03.207527  6281 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:03.207984  6161 consensus_queue.cc:260] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:03.208541  5852 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:03.208771  6400 raft_consensus.cc:3055] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:03.212423  6384 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:03.111624  6421 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:03.099462  6201 consensus_queue.cc:260] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:03.215018  6405 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:03.216063  6339 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:03.242738  1685 leader_election.cc:336] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.646s (SENT)
I20260709 13:04:03.284235  1685 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:03.219174  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.286592  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:03.243235  1568 rpcz_store.cc:276] 0709 13:04:02.034932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.182974 (+1148042us) service_pool.cc:224] Handling call
0709 13:04:03.183425 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.183677 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:03.183715 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:03.183779 (+    64us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:03.183820 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.183873 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.184004 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:03.245702  1719 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:03.287375  2342 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
W20260709 13:04:03.229813  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.273366  1718 raft_consensus.cc:1240] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:03.222419  6116 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:03.289448  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:03.254426  6385 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:03.255888  6066 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:03.303607  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:03.306665  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:03.252512  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:03.291006  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:03.176312  6234 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:03.205528  6360 consensus_meta.cc:287] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.761s	user 0.000s	sys 0.009s
I20260709 13:04:03.170709  6287 raft_consensus.cc:697] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:03.263386  2312 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:03.274103  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:03.189579  5779 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:03.192494  6186 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:03.287166  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:03.246295  1714 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:03.331640  1685 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:03.237107  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 747) took 1201 ms. Trace:
I20260709 13:04:03.348251  1571 rpcz_store.cc:276] 0709 13:04:02.035187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.183628 (+1148441us) service_pool.cc:224] Handling call
0709 13:04:03.236286 (+ 52658us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.236603 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:03.236640 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:03.236704 (+    64us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:03.236751 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.236825 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.237014 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:03.241271  1567 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
W20260709 13:04:03.324220  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.273790  6315 raft_consensus.cc:697] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:03.333312  1720 raft_consensus.cc:2410] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 13:04:03.334818  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 45446.9us
I20260709 13:04:03.332722  1568 raft_consensus.cc:2393] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 12: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:04:03.353421  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.335860  2343 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:03.332222  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:03.354452  1719 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:03.314839  6028 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:03.288261  6281 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:04:03.342195  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 78915.4us
I20260709 13:04:03.364928  2312 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:03.365270  1569 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:03.330029  6161 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:03.366348  2312 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:03.367470  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.217151  6404 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:03.279613  6400 raft_consensus.cc:740] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:03.353722  1537 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:03.260952  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 186639us
I20260709 13:04:03.371723  1612 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:03.357033  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:03.297871  6286 consensus_queue.cc:1175] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f5f4b2707a9647e5bb99e065068dea2b" responder_term: 11 status { last_received { term: 11 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 11 index: 5 } }
I20260709 13:04:03.360641  1685 leader_election.cc:400] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12.
I20260709 13:04:03.398303  1685 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:03.290408  5852 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.363003  1718 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:04:03.404017  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.369424  1568 raft_consensus.cc:1240] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:03.372094  1537 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:03.253280  6423 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:03.376237  1612 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:03.379052  6400 consensus_queue.cc:260] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:03.415805  1685 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:03.385056  1571 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:03.386092  6287 consensus_queue.cc:237] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:03.386587  1715 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:03.343343  6360 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:03.420364  1685 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:03.358507  6338 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:03.342833  6388 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:03.387364  6313 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:03.388852  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2582) took 1458 ms. Trace:
I20260709 13:04:03.288405  6224 consensus_queue.cc:1059] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 13:04:03.334263  6066 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.399956  6013 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:03.339438  6413 consensus_queue.cc:1059] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
W20260709 13:04:03.402118  6424 log.cc:927] Time spent T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:04:03.411917  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 749) took 1376 ms. Trace:
I20260709 13:04:03.402490  6271 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.328969  6384 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:04:03.417430  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 750) took 1381 ms. Trace:
W20260709 13:04:03.426821  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 50656us
W20260709 13:04:03.386116  6422 log.cc:927] Time spent T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 13:04:03.435065  1713 rpcz_store.cc:276] 0709 13:04:01.930389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.296834 (+366445us) service_pool.cc:224] Handling call
0709 13:04:02.297178 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:02.840063 (+542885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.388482 (+548419us) spinlock_profiling.cc:243] Waited 533 ms on lock 0x7b540002e9d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:03.388770 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1225155328}
I20260709 13:04:03.431169  6180 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:03.376688  6318 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:03.430939  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:03.526399  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 95555us
I20260709 13:04:03.527208  1646 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: [9.8-10.9]
I20260709 13:04:03.527801  1685 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:03.395766  6272 raft_consensus.cc:2764] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:04:03.419200  6171 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:03.498283  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:03.511161  1571 rpcz_store.cc:276] 0709 13:04:02.035932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.374518 (+1338586us) service_pool.cc:224] Handling call
0709 13:04:03.374911 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.417088 (+ 42177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.417349 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:03.541390  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 175125us
I20260709 13:04:03.511961  1568 rpcz_store.cc:276] 0709 13:04:02.035699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.366995 (+1331296us) service_pool.cc:224] Handling call
0709 13:04:03.369165 (+  2170us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.411554 (+ 42389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.411840 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:03.429502  6285 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:03.432436  6200 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:03.422015  6320 consensus_queue.cc:1048] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
W20260709 13:04:03.516870  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 105509us
I20260709 13:04:03.517546  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:03.518410  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:03.520387  1719 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:03.553902  1641 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:03.376340  6281 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:03.522023  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:03.484941  6423 raft_consensus.cc:697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:03.430404  6338 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:04:03.523554  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 268604us
I20260709 13:04:03.555744  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:03.528393  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:03.485185  6354 consensus_meta.cc:287] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.788s	user 0.010s	sys 0.005s
I20260709 13:04:03.529253  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:03.533134  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:03.435514  6313 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.483239  6028 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:03.542120  1685 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:03.542590  1714 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:04:03.447327  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.480329  6066 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:03.544250  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:03.419693  6359 consensus_queue.cc:1059] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 13:04:03.489006  6400 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:03.550563  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2256) took 1483 ms. Trace:
I20260709 13:04:03.419361  5852 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:03.434172  6388 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:03.560891  1716 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:03.556663  1644 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:03.567564  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:03.568338  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:03.556725  6361 consensus_meta.cc:287] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.831s	user 0.009s	sys 0.004s
W20260709 13:04:03.551246  6426 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.381s	user 0.002s	sys 0.000s
I20260709 13:04:03.561177  6413 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:03.583637  1568 rpcz_store.cc:276] 0709 13:04:02.066936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.548470 (+1481534us) service_pool.cc:224] Handling call
0709 13:04:03.548861 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.549202 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:03.549235 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:03.549500 (+   265us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:03.549561 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.549624 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.550490 (+   866us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:03.604102  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:03.582664  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:03.587899  1641 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:04:03.487362  6275 consensus_queue.cc:1059] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
W20260709 13:04:03.589831  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 47785.7us
I20260709 13:04:03.581213  1611 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:03.584038  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:03.556377  6395 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:03.602221  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:03.615221  1716 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:04:03.560972  6392 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:03.560218  6338 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:03.575910  6448 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.520040  6271 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:03.605794  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:03.622583  1641 raft_consensus.cc:2376] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15. Re-sending same reply.
I20260709 13:04:03.623593  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:03.606796  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.621901  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:03.624941  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.576054  6313 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:03.612556  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.598212  6388 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:03.618883  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:03.574007  6028 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:03.622076  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:03.642386  1719 raft_consensus.cc:2410] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:04:03.604933  1568 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:03.631615  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:03.632673  1644 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:03.637845  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:03.643496  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:03.628408  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.522003  6315 consensus_queue.cc:237] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:03.644408  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:03.644761  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2261) took 1354 ms. Trace:
I20260709 13:04:03.660939  1568 rpcz_store.cc:276] 0709 13:04:02.290453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.604380 (+1313927us) service_pool.cc:224] Handling call
0709 13:04:03.604700 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.644408 (+ 39708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.644706 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:03.643093  1648 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
W20260709 13:04:03.638947  6428 log.cc:927] Time spent T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Append to log took a long time: real 0.321s	user 0.001s	sys 0.001s
I20260709 13:04:03.646441  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:03.669497  1644 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:03.621049  1537 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:03.674119  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 53178us
W20260709 13:04:03.647753  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 101839us
I20260709 13:04:03.619580  6180 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:03.610513  6439 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:03.524858  6429 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.119s
W20260709 13:04:03.673331  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.657457  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:03.694569  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.694941  1641 raft_consensus.cc:2376] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12. Re-sending same reply.
I20260709 13:04:03.619602  6392 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.662560  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:03.696029  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:03.696475  1648 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:04:03.696717  1641 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:04:03.697348  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:03.697479  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:03.698125  1648 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:03.646068  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:03.698721  1643 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:03.699059  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:03.699640  1648 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:04:03.700379  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:03.700839  1648 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:03.663655  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:03.664469  1646 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 12: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:03.702401  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:03.703377  6433 consensus_queue.cc:1059] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:03.644872  6028 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:03.603320  6361 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:03.706140  6361 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:03.706705  6361 raft_consensus.cc:3055] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:03.707041  6361 raft_consensus.cc:740] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:03.707840  6361 consensus_queue.cc:260] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:03.703500  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:03.709098  1647 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:04:03.640259  6405 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:03.654189  6013 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.676420  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:03.702401  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:03.712575  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:03.718988  1643 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:04:03.727458  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:03.727345  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:03.728463  1642 raft_consensus.cc:2410] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:04:03.728472  1644 raft_consensus.cc:2410] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:04:03.729537  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:03.698154  1641 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:03.732149  1644 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:03.665407  6339 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:03.744480  1642 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:03.677229  6421 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:03.691514  6318 raft_consensus.cc:697] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:03.651634  6384 consensus_queue.cc:1048] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:04:03.755323  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:03.691604  6452 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.711328  6392 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:03.690474  6171 raft_consensus.cc:697] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:03.742143  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:03.668440  6320 consensus_queue.cc:1059] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 13:04:03.716755  6116 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:03.718369  6180 raft_consensus.cc:697] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:03.761199  1643 raft_consensus.cc:2410] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:04:03.762554  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 70939.4us
I20260709 13:04:03.763562  1648 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:04:03.630446  6448 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:03.773397  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:03.777977  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:03.778409  1643 raft_consensus.cc:1240] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:04:03.779263  1641 raft_consensus.cc:1275] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:04:03.778882  1719 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:03.758706  6421 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:04:03.782385  5852 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:03.784070  1643 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:04:03.693357  6272 consensus_queue.cc:1059] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:04:03.786139  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:03.789299  1643 raft_consensus.cc:1217] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Deduplicated request from leader. Original: 7.5->[8.6-10.7]   Dedup: 10.7->[]
I20260709 13:04:03.789955  1643 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:04:03.760185  6360 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:03.726280  6013 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:03.775282  6317 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 1.141s	user 0.016s	sys 0.000s
I20260709 13:04:03.767042  6339 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.802398  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:03.766891  6275 consensus_queue.cc:1243] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:03.807209  2345 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:04:03.787667  6320 consensus_queue.cc:1243] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:03.787570  6447 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.817127  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:03.787932  6453 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:03.666163  6423 consensus_queue.cc:237] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:03.804159  6272 consensus_queue.cc:1243] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:03.771198  6452 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:03.827831  6440 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.224s	user 0.001s	sys 0.001s
I20260709 13:04:03.811162  6281 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:03.808570  6116 consensus_queue.cc:1059] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 13:04:03.810204  6339 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:03.828652  6454 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:04:03.830183  6447 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:03.875638  6445 log.cc:927] Time spent T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:04:03.890751  6447 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:03.868083  6453 raft_consensus.cc:697] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:03.885491  6436 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.225s	user 0.002s	sys 0.000s
I20260709 13:04:03.891160  6448 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:03.896631  6442 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.172s	user 0.000s	sys 0.002s
I20260709 13:04:03.905853  6452 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:03.902573  6453 consensus_queue.cc:237] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:03.917826  6414 consensus_meta.cc:287] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.741s	user 0.002s	sys 0.014s
I20260709 13:04:03.918484  6414 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12.
W20260709 13:04:03.962116  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:03.965721  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 301718us
I20260709 13:04:03.983124  1646 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 14: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:03.982852  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:03.984486  1719 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 12: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:03.987704  1641 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:04:03.992497  6013 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:03.996462  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:03.996021  6028 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:03.998018  1719 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 11: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:03.999766  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:04.001785  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:04.002700  1719 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 12: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:04.002426  1720 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:04.004084  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:03.981088  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:03.850251  6171 consensus_queue.cc:237] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:04.017215  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:04.019073  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:04.019234  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.836265  6180 consensus_queue.cc:237] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:03.820675  6318 consensus_queue.cc:237] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:04.012319  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:04.024436  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:03.891275  6339 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:03.917085  6413 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:03.867295  6271 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:04.037057  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:04.038971  1641 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:04.040625  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:04.041815  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.042187  1721 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:04.043952  1641 raft_consensus.cc:2376] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9. Re-sending same reply.
I20260709 13:04:04.046972  1641 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:04:04.057937  6274 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.059156  6274 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:04.061743  6405 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:04:04.072041  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.063330  6405 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:04.069012  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2182) took 1263 ms. Trace:
I20260709 13:04:04.084926  6274 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:04.097529  6405 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:04.082952  6028 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:04.109887  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:04.143782  6186 consensus_meta.cc:287] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.787s	user 0.014s	sys 0.013s
W20260709 13:04:04.148711  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 44905.8us
W20260709 13:04:04.089350  6400 consensus_meta.cc:287] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.006s
W20260709 13:04:04.195031  6161 consensus_meta.cc:287] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.817s	user 0.018s	sys 0.015s
W20260709 13:04:04.211025  6354 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.037s	sys 0.075s
W20260709 13:04:04.211248  6470 log.cc:927] Time spent T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.280s	user 0.002s	sys 0.000s
W20260709 13:04:04.214257  6354 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.038s	sys 0.076s
I20260709 13:04:04.090495  1647 rpcz_store.cc:276] 0709 13:04:02.805389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.810866 (+  5477us) service_pool.cc:224] Handling call
0709 13:04:02.830370 (+ 19504us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:03.709954 (+879584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.067956 (+358002us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b54002aeed4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:04.068687 (+   731us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2020495104}
I20260709 13:04:04.221817  5919 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:04.222658  5919 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:04.223268  5919 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:04.216778  6161 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.228422  6161 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:04.233970  6186 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.234885  6186 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:04.237115  6271 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.237895  5852 raft_consensus.cc:3055] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:04.237958  6271 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:04.238641  5852 raft_consensus.cc:740] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:04.243193  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.241972  5852 consensus_queue.cc:260] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:04.245442  5852 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:04.234160  5779 consensus_meta.cc:287] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.839s	user 0.023s	sys 0.007s
I20260709 13:04:04.248183  5779 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:04.251138  5779 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:04.252477  1713 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 10: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:04:04.254576  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 285151us
W20260709 13:04:04.257159  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 279521us
I20260709 13:04:04.258782  1612 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:04.237280  6186 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:04.262552  1641 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:04.267501  1568 raft_consensus.cc:2410] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:04:04.315604  6181 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:04.366771  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:04.393945  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 137917us
I20260709 13:04:04.387349  1647 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:04:04.321203  6392 consensus_meta.cc:287] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.544s	user 0.010s	sys 0.007s
I20260709 13:04:04.422415  6360 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:04.316874  6349 consensus_queue.cc:260] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:04.270517  6342 raft_consensus.cc:3055] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:04.264597  6310 raft_consensus.cc:3055] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:04:04.259308  6313 consensus_meta.cc:287] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.604s	user 0.007s	sys 0.013s
W20260709 13:04:04.437026  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 178542us
I20260709 13:04:04.341173  6339 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:04.449996  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 179636us
I20260709 13:04:04.453348  6385 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:04.448843  6310 raft_consensus.cc:740] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:04.448588  6342 raft_consensus.cc:740] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:04.462018  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:04.460319  6399 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 1.981s	user 0.000s	sys 0.006s
I20260709 13:04:04.450810  6392 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:04.453822  6028 consensus_queue.cc:1059] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:04:04.478564  6384 raft_consensus.cc:3055] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:04.519445  6384 raft_consensus.cc:740] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:04.486624  6449 raft_consensus.cc:3055] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:04.492523  6361 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:04.493382  6450 raft_consensus.cc:3055] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:04:04.495913  6200 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.948s	user 0.014s	sys 0.010s
W20260709 13:04:04.486692  6388 consensus_meta.cc:287] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.792s	user 0.009s	sys 0.012s
I20260709 13:04:04.486626  6404 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:04.498394  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:04.500696  6234 consensus_meta.cc:287] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.155s	user 0.022s	sys 0.001s
I20260709 13:04:04.501329  1647 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:04.510725  1612 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:04.512247  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2241) took 1015 ms. Trace:
W20260709 13:04:04.513954  6066 consensus_meta.cc:287] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.915s	user 0.008s	sys 0.016s
I20260709 13:04:04.514181  6433 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:04.514520  6478 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
W20260709 13:04:04.514902  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:04.501197  1564 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2194) took 2556 ms (client timeout 1646 ms). Trace:
I20260709 13:04:04.496372  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:04.518920  6028 raft_consensus.cc:3055] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:04:04.519706  6477 log.cc:927] Time spent T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
W20260709 13:04:04.534442  1564 rpcz_store.cc:269] 0709 13:04:01.944101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.093104 (+149003us) service_pool.cc:224] Handling call
0709 13:04:04.500713 (+2407609us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b54002ba554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:04.500938 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2152102784}
I20260709 13:04:04.524045  6449 raft_consensus.cc:740] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:04.538033  6433 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:04.538722  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:04.539746  1564 raft_consensus.cc:2376] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14. Re-sending same reply.
I20260709 13:04:04.539371  6449 consensus_queue.cc:260] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:04.541180  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:04.490934  6338 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.852s	user 0.016s	sys 0.004s
I20260709 13:04:04.537359  6200 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:04.532333  6388 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:04.532166  6268 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:04.535589  6028 raft_consensus.cc:740] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:04.541432  6433 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:04.547478  6028 consensus_queue.cc:260] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:04.530205  1720 rpcz_store.cc:276] 0709 13:04:03.497235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.497412 (+   177us) service_pool.cc:224] Handling call
0709 13:04:04.511970 (+1014558us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b54003903d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:04.512143 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":977548672}
I20260709 13:04:04.457654  6313 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:04.545245  1564 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:04.468310  6161 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:04.541193  6449 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:04.525532  6450 raft_consensus.cc:740] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:04.554504  6028 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:04.555009  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:04.545909  6240 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:04.562198  1567 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:04.585920  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.574582  1566 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:04.605202  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2576) took 2706 ms. Trace:
I20260709 13:04:04.608491  6161 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.605870  6393 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.610147  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:04.608631  1717 rpcz_store.cc:276] 0709 13:04:01.898824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.899120 (+   296us) service_pool.cc:224] Handling call
0709 13:04:01.900181 (+  1061us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:02.092983 (+192802us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:02.093017 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:02.093506 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 13:04:02.245152 (+151646us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:02.245193 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.245276 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:04.440626 (+2195350us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:04.580588 (+139962us) raft_consensus.cc:1685] finished
0709 13:04:04.580619 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:04.604984 (+ 24365us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":139192,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":13878,"spinlock_wait_cycles":7235328,"thread_start_us":292650,"threads_started":2,"wal-append.queue_time_us":274280}
I20260709 13:04:04.610221  6393 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:04.613935  6393 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:04.614866  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.609313  6161 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:04.615831  2314 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.208s (SENT) [suppressed 31 similar messages]
I20260709 13:04:04.616797  6360 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.617496  6360 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:04.617779  6161 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:04.623358  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2252) took 1016 ms. Trace:
I20260709 13:04:04.623782  1713 rpcz_store.cc:276] 0709 13:04:03.606984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.613414 (+  6430us) service_pool.cc:224] Handling call
0709 13:04:04.623113 (+1009699us) spinlock_profiling.cc:243] Waited 627 ms on lock 0x7b5400210f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:04.623306 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1441735936}
I20260709 13:04:04.558673  6433 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.612118  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:04.624986  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:04.551538  6271 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:04.559145  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:04.631240  1713 raft_consensus.cc:2410] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 13:04:04.633023  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.548988  5919 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:04.635263  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.622953  6450 consensus_queue.cc:260] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:04.633250  1717 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:04:04.559211  6388 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:04.641131  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.551915  6349 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:04.648648  2343 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:04.520352  6339 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:04.656704  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:04.658306  1720 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:04:04.553277  6234 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:04.669586  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2246) took 1117 ms. Trace:
I20260709 13:04:04.670581  1720 rpcz_store.cc:276] 0709 13:04:03.547358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.558628 (+1011270us) service_pool.cc:224] Handling call
0709 13:04:04.664865 (+106237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:04.674235  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:04.555908  6066 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:04.683058  1641 raft_consensus.cc:1240] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:04:04.683115  6180 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:04.684922  6180 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.685966  1717 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:04.685386  6180 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:04.544713  6384 consensus_queue.cc:260] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:04.692028  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:04.629227  6338 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:04.654884  6433 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:04.695082  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 251142us
I20260709 13:04:04.696094  6200 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.698374  1644 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:04.698561  6200 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:04.698267  1720 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
W20260709 13:04:04.706126  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.705981  1566 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:04.707561  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2260) took 1020 ms. Trace:
I20260709 13:04:04.708055  1720 rpcz_store.cc:276] 0709 13:04:03.684829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.679413 (+994584us) service_pool.cc:224] Handling call
0709 13:04:04.697849 (+ 18436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.704523 (+  6674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.704808 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:04.712772  1564 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:04.714395  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:04.715229  1566 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:04:04.716215  2346 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:04:04.716425  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2597) took 1415 ms. Trace:
I20260709 13:04:04.716598  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:04.716882  1718 rpcz_store.cc:276] 0709 13:04:03.301442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.303845 (+  2403us) service_pool.cc:224] Handling call
0709 13:04:03.304152 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.403322 (+ 99170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.715887 (+1312565us) spinlock_profiling.cc:243] Waited 58.3 ms on lock 0x7b5400213c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:04.716201 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":133982464}
I20260709 13:04:04.714571  1642 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:04.718027  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:04.719049  2346 pending_rounds.cc:85] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: Aborting all ops after (but not including) 8
I20260709 13:04:04.720651  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2229) took 1026 ms. Trace:
I20260709 13:04:04.720992  1564 rpcz_store.cc:276] 0709 13:04:03.694646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.625049 (+930403us) service_pool.cc:224] Handling call
0709 13:04:04.720605 (+ 95556us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:04.727406  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 111667us
W20260709 13:04:04.729326  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.717906  6450 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:04.737560  1566 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:04.719307  2346 pending_rounds.cc:107] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: Aborting uncommitted NO_OP operation due to leader change: 10.9
W20260709 13:04:04.736074  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 225566us
I20260709 13:04:04.714617  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:04.738492  2346 raft_consensus.cc:2889] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:04.722146  6388 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.688177  6310 consensus_queue.cc:260] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:04.742959  1713 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:04.681843  6342 consensus_queue.cc:260] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:04.746274  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.748649  1718 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 11. Current term is 12. Ops: []
W20260709 13:04:04.751890  6439 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.951s	user 0.075s	sys 0.046s
W20260709 13:04:04.752389  6439 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.996s	user 0.075s	sys 0.046s
I20260709 13:04:04.752496  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:04.752890  6224 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:04.753214  1566 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:04.753701  6224 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:04.752763  6439 raft_consensus.cc:697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:04.756733  1713 raft_consensus.cc:1240] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:04:04.756424  6439 consensus_queue.cc:237] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:04.752760  5919 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:04.757311  6224 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:04.758607  6268 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:04.759454  6268 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:04.735695  6200 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:04.772436  1717 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:04:04.775926  1566 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
W20260709 13:04:04.779733  1538 leader_election.cc:336] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 3.379s (SENT)
W20260709 13:04:04.759399  1715 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2595) took 1486 ms (client timeout 1726 ms). Trace:
I20260709 13:04:04.743719  6384 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:04.773332  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2265) took 1039 ms. Trace:
I20260709 13:04:04.782287  1717 rpcz_store.cc:276] 0709 13:04:03.734451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.648727 (+914276us) service_pool.cc:224] Handling call
0709 13:04:04.773267 (+124540us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:04.783053  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:04.784543  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:04.784911  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.785938  1611 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:04.781044  1715 rpcz_store.cc:269] 0709 13:04:03.272389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.273707 (+  1318us) service_pool.cc:224] Handling call
0709 13:04:04.758914 (+1485207us) spinlock_profiling.cc:243] Waited 41 ms on lock 0x7b5400212854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:04.759147 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94398336}
I20260709 13:04:04.767973  6271 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.796479  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:04.786537  1538 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:04.800225  5852 consensus_meta.cc:287] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.547s	user 0.009s	sys 0.005s
I20260709 13:04:04.748051  6388 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:04.790933  6392 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:04.823276  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 43658.1us
I20260709 13:04:04.824122  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:04.826524  6351 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:04.831284  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 475) took 1299 ms. Trace:
I20260709 13:04:04.834640  1566 raft_consensus.cc:2410] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:04:04.799700  6268 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:04.841811  2349 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:04:04.240082   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:04.844017  2349 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:04:04.845697  6435 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.334s	user 0.000s	sys 0.005s
I20260709 13:04:04.846462  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2683) took 1311 ms. Trace:
I20260709 13:04:04.847040  1646 rpcz_store.cc:276] 0709 13:04:03.534864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.703274 (+168410us) service_pool.cc:224] Handling call
0709 13:04:04.846128 (+1142854us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b5400097c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:04.846362 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":625825536}
I20260709 13:04:04.742065  6335 consensus_queue.cc:260] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:04.849466  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:04.819448  6313 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:04.836862  1644 rpcz_store.cc:276] 0709 13:04:03.531429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.744259 (+212830us) service_pool.cc:224] Handling call
0709 13:04:03.744479 (+   220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.699301 (+954822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.830973 (+131672us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b54002db754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:04.831193 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2176075520}
W20260709 13:04:04.858855  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.858949  6161 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:04.829486  6388 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:04.862178  1564 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:04:04.862921  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2223) took 1172 ms. Trace:
I20260709 13:04:04.863258  1564 rpcz_store.cc:276] 0709 13:04:03.690725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.738150 (+1047425us) service_pool.cc:224] Handling call
0709 13:04:04.862849 (+124699us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:04.871567  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2184) took 1842 ms. Trace:
I20260709 13:04:04.861337  6201 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:04.851917  6234 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:04.892342  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:04.894018  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:04.896092  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 117145us
I20260709 13:04:04.897769  1612 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:04.898116  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:04.899149  1646 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:04.901450  6364 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:04.903923  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2196) took 2959 ms. Trace:
I20260709 13:04:04.904356  1573 rpcz_store.cc:276] 0709 13:04:01.944801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.096782 (+1151981us) service_pool.cc:224] Handling call
0709 13:04:04.502500 (+1405718us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.517316 (+ 14816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:04.517361 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:04.517432 (+    71us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:04.517478 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.518310 (+   832us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:04.903503 (+385193us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54002ba550. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:04.903679 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14095,"raft.queue_time_us":15561,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":274,"spinlock_wait_cycles":3182791936}
I20260709 13:04:04.881976  6351 raft_consensus.cc:697] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 13 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:04.907150  6351 consensus_queue.cc:237] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:04.907720  1639 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2180) took 2131 ms (client timeout 2036 ms). Trace:
W20260709 13:04:04.908170  1639 rpcz_store.cc:269] 0709 13:04:02.776137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.795921 (+ 19784us) service_pool.cc:224] Handling call
0709 13:04:04.907275 (+2111354us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54000a64d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:04.907489 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":316188544}
I20260709 13:04:04.873247  6066 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:04.909368  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:04.919464  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:04.919842  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:04.922363  1570 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2249) took 2883 ms (client timeout 1614 ms). Trace:
I20260709 13:04:04.923027  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:04.923774  1565 raft_consensus.cc:1275] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:04:04.924149  1573 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 12: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:04.924837  2348 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:04.924664  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2228) took 1230 ms. Trace:
I20260709 13:04:04.925133  6342 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:04.925362  1573 rpcz_store.cc:276] 0709 13:04:03.694414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.922356 (+1227942us) service_pool.cc:224] Handling call
0709 13:04:04.924596 (+  2240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:04.926733  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:04.927323  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:04.932979  2344 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:04.934347  1573 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 12: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:04.877362  1640 rpcz_store.cc:276] 0709 13:04:03.011325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.012686 (+  1361us) service_pool.cc:224] Handling call
0709 13:04:03.019740 (+  7054us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:03.020068 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:03.020094 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:03.020588 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 13:04:03.262668 (+242080us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:03.263964 (+  1296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.264097 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:04.762554 (+1498457us) spinlock_profiling.cc:243] Waited 7.22 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:04.850166 (+ 87612us) raft_consensus.cc:1685] finished
0709 13:04:04.850193 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:04.853508 (+  3315us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99054,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":18654,"spinlock_wait_cycles":16596736,"thread_start_us":241695,"threads_started":1,"wal-append.queue_time_us":354789}
I20260709 13:04:04.936496  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2227) took 1242 ms. Trace:
W20260709 13:04:04.921464  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.937068  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:04.922762  1570 rpcz_store.cc:269] 0709 13:04:02.038514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.323712 (+285198us) service_pool.cc:224] Handling call
0709 13:04:04.915992 (+2592280us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54001b4254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:04.922147 (+  6155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274733952}
I20260709 13:04:04.935420  6339 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:04.936892  1573 rpcz_store.cc:276] 0709 13:04:03.694169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.926276 (+1232107us) service_pool.cc:224] Handling call
0709 13:04:04.936433 (+ 10157us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:04.939122  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 151792us
I20260709 13:04:04.946599  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:04.936800  6338 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:04.869041  6271 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:04.948189  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:04.951184  1570 raft_consensus.cc:2410] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 15 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:04:04.951234  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:04.955812  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:04.961045  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 188825us
I20260709 13:04:04.947371  6433 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:04.956808  1639 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:04:04.960757  6115 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.964475  6115 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:04.967556  6115 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:04:04.968991  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:04.971657  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:04.960551  1573 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:04:04.978438  1685 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.914s (SENT)
I20260709 13:04:04.982040  1640 raft_consensus.cc:2368] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 14. Current term is 15.
W20260709 13:04:04.982795  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.002948  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2230) took 1308 ms. Trace:
I20260709 13:04:05.007310  1570 rpcz_store.cc:276] 0709 13:04:03.694900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.968493 (+1273593us) service_pool.cc:224] Handling call
0709 13:04:05.002832 (+ 34339us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:04.952078  6360 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:05.011699  1643 raft_consensus.cc:2410] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:04:05.013343  1687 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:05.015638  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2263) took 2606 ms. Trace:
I20260709 13:04:05.016059  1573 rpcz_store.cc:276] 0709 13:04:02.403310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.984345 (+2581035us) service_pool.cc:224] Handling call
0709 13:04:05.004794 (+ 20449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.009360 (+  4566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:05.009415 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:05.009511 (+    96us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:05.009549 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.009613 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.009820 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:05.017577  6487 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:04:05.020386  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:05.022756  1573 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:04:05.025537  1538 leader_election.cc:336] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.519s (SENT)
I20260709 13:04:05.026854  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2187) took 1462 ms. Trace:
I20260709 13:04:05.013461  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:05.029062  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:05.030658  1566 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 12.
I20260709 13:04:05.026574  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2213) took 2570 ms. Trace:
I20260709 13:04:05.031786  1573 rpcz_store.cc:276] 0709 13:04:02.455809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.017106 (+2561297us) service_pool.cc:224] Handling call
0709 13:04:05.017526 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.023168 (+  5642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.026480 (+  3312us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:05.032677  1538 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:05.037163  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2709) took 1054 ms. Trace:
I20260709 13:04:05.037617  1643 rpcz_store.cc:276] 0709 13:04:03.975220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.016964 (+ 41744us) service_pool.cc:224] Handling call
0709 13:04:04.949230 (+932266us) spinlock_profiling.cc:243] Waited 7.22 ms on lock 0x7b5800003098. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:04:05.029545 (+ 80315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16605568}
I20260709 13:04:05.027266  1647 rpcz_store.cc:276] 0709 13:04:03.556869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.612531 (+1055662us) service_pool.cc:224] Handling call
0709 13:04:04.613141 (+   610us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:04.650623 (+ 37482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:04.650667 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:04.695667 (+ 45000us) log.cc:844] Serialized 52 byte log entry
0709 13:04:04.863270 (+167603us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:04.863328 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.863422 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:05.016604 (+153182us) spinlock_profiling.cc:243] Waited 8.96 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:05.019278 (+  2674us) raft_consensus.cc:1685] finished
0709 13:04:05.019302 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.019530 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3401,"raft.queue_time_us":25281,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":5653,"spinlock_wait_cycles":20611456,"thread_start_us":126172,"threads_started":1,"wal-append.queue_time_us":208154}
W20260709 13:04:05.039510  1569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2262) took 2725 ms (client timeout 2771 ms). Trace:
W20260709 13:04:05.048347  1569 rpcz_store.cc:269] 0709 13:04:02.306756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.873253 (+566497us) service_pool.cc:224] Handling call
0709 13:04:04.984478 (+2111225us) spinlock_profiling.cc:243] Waited 456 ms on lock 0x7b54002ba054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:05.031998 (+ 47520us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1048284544}
I20260709 13:04:05.051921  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:05.054121  6277 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:05.054692  6277 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.055234  6277 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:05.055456  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.031782  6234 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:05.080098  1716 raft_consensus.cc:2410] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:04:05.059238  6180 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:05.084681  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 748) took 3049 ms. Trace:
I20260709 13:04:05.086892  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:05.087939  1564 raft_consensus.cc:2410] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:04:05.088436  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2225) took 1394 ms. Trace:
I20260709 13:04:05.088857  1564 rpcz_store.cc:276] 0709 13:04:03.693636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.065082 (+1371446us) service_pool.cc:224] Handling call
0709 13:04:05.088366 (+ 23284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:05.072570  6388 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:05.093825  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:05.072921  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:05.085151  1567 rpcz_store.cc:276] 0709 13:04:02.035459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.365029 (+1329570us) service_pool.cc:224] Handling call
0709 13:04:03.413272 (+ 48243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.654047 (+1240775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.084361 (+430314us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54000f69d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:05.084607 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2641428864}
I20260709 13:04:05.102427  1687 leader_election.cc:400] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 14. Current term is 15.
I20260709 13:04:05.102906  1687 leader_election.cc:403] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:04:05.105621  1570 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:04:05.105077  1643 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
W20260709 13:04:05.109498  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 147180us
I20260709 13:04:05.110422  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:05.115540  1648 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:05.058471  6338 consensus_queue.cc:1059] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:04:05.113219  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:05.118141  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2211) took 2657 ms. Trace:
I20260709 13:04:05.120558  1570 rpcz_store.cc:276] 0709 13:04:02.454333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.011603 (+2557270us) service_pool.cc:224] Handling call
0709 13:04:05.071738 (+ 60135us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.106372 (+ 34634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.111782 (+  5410us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:05.122566  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:05.127097  1647 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:04:05.129371  1718 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:04:05.107719  6234 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:05.135629  6297 consensus_queue.cc:1059] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
W20260709 13:04:05.137971  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 94226.9us
I20260709 13:04:05.137373  1646 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:04:05.140110  1640 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:05.148821  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2264) took 2613 ms. Trace:
I20260709 13:04:05.151566  6271 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:05.149461  1566 rpcz_store.cc:276] 0709 13:04:02.535455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.031436 (+2495981us) service_pool.cc:224] Handling call
0709 13:04:05.106895 (+ 75459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.137412 (+ 30517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:05.137444 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:05.142833 (+  5389us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:05.148333 (+  5500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.148408 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.148752 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:05.157088  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 144322us
I20260709 13:04:05.146235  6388 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.131114  6028 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:04:05.167448  1685 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:05.188633  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.157186  6297 consensus_queue.cc:1243] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:04:05.192309  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.196707  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2186) took 1632 ms. Trace:
I20260709 13:04:05.197297  1641 rpcz_store.cc:276] 0709 13:04:03.556545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.446416 (+889871us) service_pool.cc:224] Handling call
0709 13:04:04.469751 (+ 23335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:04.683998 (+214247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.188591 (+504593us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54000a6ed4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:05.188846 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":456821504}
I20260709 13:04:05.198038  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:05.199251  1641 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:04:05.199594  1718 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 11: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:05.198614  6115 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:05.206643  1567 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:04:05.202839  1687 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:05.207314  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2701) took 1607 ms. Trace:
I20260709 13:04:05.209316  1641 rpcz_store.cc:276] 0709 13:04:03.593227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.198381 (+1605154us) service_pool.cc:224] Handling call
0709 13:04:05.198829 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.199646 (+   817us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.199875 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:05.211174  1647 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:05.211727  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2699) took 1644 ms. Trace:
I20260709 13:04:05.211992  1647 rpcz_store.cc:276] 0709 13:04:03.567004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.155600 (+1588596us) service_pool.cc:224] Handling call
0709 13:04:05.206993 (+ 51393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.211548 (+  4555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.211687 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:05.154793  6439 consensus_queue.cc:1059] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:04:05.218808  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2226) took 1524 ms. Trace:
I20260709 13:04:05.220101  1567 rpcz_store.cc:276] 0709 13:04:03.693924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.113612 (+1419688us) service_pool.cc:224] Handling call
0709 13:04:05.166159 (+ 52547us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:05.214291 (+ 48132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.218764 (+  4473us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:05.220980  1640 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:04:05.221705  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2192) took 1642 ms. Trace:
I20260709 13:04:05.222062  1640 rpcz_store.cc:276] 0709 13:04:03.579689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.158968 (+1579279us) service_pool.cc:224] Handling call
0709 13:04:05.205846 (+ 46878us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.221406 (+ 15560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.221625 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:05.223699  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2232) took 1528 ms. Trace:
I20260709 13:04:05.224133  1566 rpcz_store.cc:276] 0709 13:04:03.695381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.154036 (+1458655us) service_pool.cc:224] Handling call
0709 13:04:05.201282 (+ 47246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.223342 (+ 22060us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:05.223372 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:05.223419 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:05.223458 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.223524 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.223638 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:05.228137  6486 log.cc:927] Time spent T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.242s	user 0.002s	sys 0.000s
I20260709 13:04:05.229297  1647 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:04:05.205251  6396 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:05.182399  6388 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:05.240119  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.237625  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2191) took 1664 ms. Trace:
I20260709 13:04:05.239151  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:05.243527  1646 rpcz_store.cc:276] 0709 13:04:03.573009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.156263 (+1583254us) service_pool.cc:224] Handling call
0709 13:04:05.191402 (+ 35139us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.227685 (+ 36283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:05.227710 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:05.237305 (+  9595us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:05.237376 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.237453 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.237561 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:05.244820  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:05.246033  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 78696.1us
W20260709 13:04:05.248945  6484 log.cc:927] Time spent T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.406s	user 0.003s	sys 0.000s
I20260709 13:04:05.222332  6392 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:05.250816  6392 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.251030  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2672) took 1718 ms. Trace:
I20260709 13:04:05.251245  6392 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:05.251677  1642 rpcz_store.cc:276] 0709 13:04:03.532194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.745126 (+212932us) service_pool.cc:224] Handling call
0709 13:04:03.768709 (+ 23583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.740442 (+971733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.249190 (+508748us) spinlock_profiling.cc:243] Waited 934 ms on lock 0x7b5400099554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:05.250934 (+  1744us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2148713856}
I20260709 13:04:05.253144  6395 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:05.253674  6395 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:05.254212  6395 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:05.248353  1713 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:04:05.226090  6234 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:05.247524  1685 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:05.257658  1719 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:05.258697  6310 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:05.255438  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2233) took 1536 ms. Trace:
I20260709 13:04:05.263379  5777 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:05.261294  6200 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.267109  6200 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:05.265516  6277 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:05.217371  6360 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:05.274914  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2593) took 2029 ms. Trace:
I20260709 13:04:05.275386  1721 rpcz_store.cc:276] 0709 13:04:03.245841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.246405 (+   564us) service_pool.cc:224] Handling call
0709 13:04:03.246814 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.042875 (+796061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.274562 (+1231687us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b54002995d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:05.274843 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1785860352}
W20260709 13:04:05.281391  6485 log.cc:927] Time spent T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.277s	user 0.001s	sys 0.001s
W20260709 13:04:05.283187  6240 consensus_meta.cc:287] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.645s	user 0.014s	sys 0.002s
I20260709 13:04:05.284596  1722 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:04:05.286461  6287 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.245044  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2190) took 1687 ms. Trace:
W20260709 13:04:05.249845  6349 consensus_meta.cc:287] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.586s	user 0.004s	sys 0.008s
I20260709 13:04:05.287782  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2704) took 1552 ms. Trace:
I20260709 13:04:05.239413  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:05.292819  1647 rpcz_store.cc:276] 0709 13:04:03.684706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.223125 (+1538419us) service_pool.cc:224] Handling call
0709 13:04:05.228729 (+  5604us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:05.236919 (+  8190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.237176 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:05.260263  1567 rpcz_store.cc:276] 0709 13:04:03.718883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.220841 (+1501958us) service_pool.cc:224] Handling call
0709 13:04:05.254365 (+ 33524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.254636 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:05.254661 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:05.254711 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:05.255122 (+   411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.255197 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.255344 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:05.289811  1643 rpcz_store.cc:276] 0709 13:04:03.557939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.137300 (+1579361us) service_pool.cc:224] Handling call
0709 13:04:05.172621 (+ 35321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.230022 (+ 57401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:05.230053 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:05.244801 (+ 14748us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:05.244843 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.244891 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.244995 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:05.304421  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:05.306914  1714 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2248) took 1740 ms (client timeout 1665 ms). Trace:
W20260709 13:04:05.303273  2314 leader_election.cc:336] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.719s (SENT)
W20260709 13:04:05.307626  1714 rpcz_store.cc:269] 0709 13:04:03.566124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.582368 (+ 16244us) service_pool.cc:224] Handling call
0709 13:04:05.299676 (+1717308us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b5400172854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:05.306599 (+  6923us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3223439616}
W20260709 13:04:05.317667  6450 consensus_meta.cc:287] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.568s	user 0.012s	sys 0.001s
I20260709 13:04:05.322829  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:05.327283  1643 raft_consensus.cc:1275] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:04:05.325619  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:05.338382  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.289179  6287 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:05.344170  1719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2262) took 1632 ms (client timeout 2115 ms). Trace:
I20260709 13:04:05.347568  1647 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:04:05.344062  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:05.350144  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:05.351614  2343 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
W20260709 13:04:05.353965  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.334890  6201 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:04:05.346508  1719 rpcz_store.cc:269] 0709 13:04:03.711842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.018573 (+306731us) service_pool.cc:224] Handling call
0709 13:04:05.343972 (+1325399us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400247a54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:05.344101 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2846733184}
I20260709 13:04:05.360424  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:05.358816  1721 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:04:05.361003  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 113647us
I20260709 13:04:05.361467  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:05.360028  1643 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:05.367195  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:05.367511  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 64340.8us
W20260709 13:04:05.368391  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.368726  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:05.370517  6388 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:05.371284  1572 raft_consensus.cc:2410] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:04:05.371470  1720 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:04:05.373946  6453 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.374732  6453 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:05.376680  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:05.363471  6201 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.381111  6481 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:05.382318  6201 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:05.383037  6481 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:05.384953  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2190) took 3476 ms. Trace:
I20260709 13:04:05.385414  1565 rpcz_store.cc:276] 0709 13:04:01.908560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.035201 (+1126641us) service_pool.cc:224] Handling call
0709 13:04:04.923227 (+1888026us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.924233 (+  1006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.384686 (+460453us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b54001b4250. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:05.384850 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4341478272}
W20260709 13:04:05.389453  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.390059  1567 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:05.387296  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:05.405913  1565 raft_consensus.cc:2393] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:05.407380  1715 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:04:05.409245  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:05.413476  6480 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.596s	user 0.004s	sys 0.001s
I20260709 13:04:05.415021  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:05.415632  6494 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.416304  6494 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:05.419288  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:05.420234  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2276) took 1809 ms. Trace:
I20260709 13:04:05.420580  1717 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:04:05.414731  6392 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:05.422398  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2277) took 1810 ms. Trace:
I20260709 13:04:05.427578  1564 rpcz_store.cc:276] 0709 13:04:03.611597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.111234 (+1499637us) service_pool.cc:224] Handling call
0709 13:04:05.158513 (+ 47279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.422006 (+263493us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:05.422042 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:05.422091 (+    49us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:05.422137 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.422222 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.422341 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":242673}
W20260709 13:04:05.421774  2345 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 434) took 2746 ms (client timeout 2310 ms). Trace:
W20260709 13:04:05.431241  2345 rpcz_store.cc:269] 0709 13:04:02.675401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.676867 (+  1466us) service_pool.cc:224] Handling call
0709 13:04:05.421317 (+2744450us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400800054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:05.421474 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2582806912}
I20260709 13:04:05.422950  6325 raft_consensus.cc:3055] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:05.423655  6401 raft_consensus.cc:3055] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:05.464970  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:05.476368  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:05.462126  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 50046.4us
I20260709 13:04:05.470949  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:05.477933  1565 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:05.475422  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:05.477013  1718 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:05.478942  2343 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 13:04:05.420639  1569 rpcz_store.cc:276] 0709 13:04:03.611049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.049296 (+1438247us) service_pool.cc:224] Handling call
0709 13:04:05.051057 (+  1761us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.051321 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:05.052107 (+   786us) log.cc:844] Serialized 14 byte log entry
0709 13:04:05.156725 (+104618us) log.cc:844] Serialized 14 byte log entry
0709 13:04:05.252484 (+ 95759us) spinlock_profiling.cc:243] Waited 1.12 ms on lock 0x7b180021cb80. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d98522 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:04:05.382913 (+130429us) spinlock_profiling.cc:243] Waited 8.91 ms on lock 0x7b180021cb80. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d98522 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:04:05.419704 (+ 36791us) spinlock_profiling.cc:243] Waited 5.2 ms on lock 0x7b180021cb80. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d12c0a 00007f824f452219 00007f8245d9aa40 00007f8245d954e4 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287 00007f8248ea0208
0709 13:04:05.419881 (+   177us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:05.419937 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:05.419984 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.420048 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.420144 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5820,"prepare.queue_time_us":4351,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":86,"raft.queue_time_us":6196,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":7630,"spinlock_wait_cycles":42073344,"thread_start_us":103710,"threads_started":1,"wal-append.queue_time_us":175600}
I20260709 13:04:05.483424  1644 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:05.421029  6494 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:05.482409  1643 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:04:05.477327  6492 log.cc:927] Time spent T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.239s	user 0.002s	sys 0.001s
I20260709 13:04:05.465225  6268 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.493386  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:05.491084  1685 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:05.470412  6325 raft_consensus.cc:740] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:05.497844  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:05.464674  6481 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:05.460685  6338 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:04:05.498126  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:05.503983  1569 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:05.498567  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:05.476158  6401 raft_consensus.cc:740] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:05.501978  6497 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:05.508813  1643 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:05.499066  2312 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:04:05.503712  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:05.510804  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 46296.2us
I20260709 13:04:05.511092  6392 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:05.511469  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:05.511838  1686 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:05.511933  6203 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:05.512076  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:05.512477  6203 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.512876  6203 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:05.513093  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 66483.6us
I20260709 13:04:05.513612  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:05.514408  2340 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
W20260709 13:04:05.515463  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.505119  1714 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:04:05.525417  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2588) took 3462 ms. Trace:
I20260709 13:04:05.526348  1722 rpcz_store.cc:276] 0709 13:04:02.063442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.378293 (+314851us) service_pool.cc:224] Handling call
0709 13:04:02.378609 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.285045 (+2906436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.525006 (+239961us) spinlock_profiling.cc:243] Waited 2.91 s on lock 0x7b5400246154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:05.525333 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6681868032}
I20260709 13:04:05.529342  6401 consensus_queue.cc:260] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:05.530596  6401 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:05.516151  6277 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:05.532223  6201 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:05.502909  6271 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.535320  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:05.536453  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:05.525249  6325 consensus_queue.cc:260] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:05.540194  1717 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 11: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:05.547256  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:05.548779  1718 raft_consensus.cc:2410] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:04:05.554746  1569 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:05.502561  6268 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:05.537673  6392 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:05.568976  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:05.575095  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2253) took 1967 ms (client timeout 1656 ms). Trace:
I20260709 13:04:05.582438  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:05.578802  1716 rpcz_store.cc:269] 0709 13:04:03.607365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.615892 (+  8527us) service_pool.cc:224] Handling call
0709 13:04:04.498007 (+882115us) spinlock_profiling.cc:243] Waited 4.09 ms on lock 0x7b580009db98. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:04:05.574590 (+1076583us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b540002dfd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:05.574908 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1229198464}
I20260709 13:04:05.584414  1569 raft_consensus.cc:2410] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:04:05.607484  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.565933  6494 consensus_queue.cc:1059] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:04:05.630262  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:05.632128  1645 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:05.632229  6200 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:05.634743  1569 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
W20260709 13:04:05.634977  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.640772  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2691) took 2104 ms. Trace:
I20260709 13:04:05.649204  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:05.651678  1718 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:05.646680  1645 rpcz_store.cc:276] 0709 13:04:03.536752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.791807 (+255055us) service_pool.cc:224] Handling call
0709 13:04:05.631582 (+1839775us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b5800003098. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:04:05.631823 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.640468 (+  8645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.640706 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47190400}
I20260709 13:04:05.619210  6349 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.570052  6271 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:05.656702  1565 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
W20260709 13:04:05.658687  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 136825us
I20260709 13:04:05.662804  1722 raft_consensus.cc:1275] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:04:05.672554  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:05.651185  6435 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:04:05.672341  2312 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:05.673980  1538 leader_election.cc:336] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.671s (SENT)
I20260709 13:04:05.678062  1538 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:05.684991  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:05.689122  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:05.671505  5852 raft_consensus.cc:2764] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:04:05.699854  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:05.699887  1569 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:05.709285  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 84794.1us
I20260709 13:04:05.720363  1722 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:04:05.671162  6349 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:05.721729  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:05.737303  6268 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:05.738641  6506 log.cc:927] Time spent T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:04:05.716369  6338 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:04:05.722191  6503 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.193s	user 0.002s	sys 0.000s
I20260709 13:04:05.744835  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:05.747388  1722 raft_consensus.cc:2393] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 14: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:05.722564  6449 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:05.729583  1685 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:05.757849  1537 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:05.759954  6396 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 14. Current term is 15.
I20260709 13:04:05.761516  6203 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:05.775950  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 102018us
W20260709 13:04:05.764173  6395 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.504s	user 0.014s	sys 0.000s
I20260709 13:04:05.741969  6161 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:04:05.783754  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:05.787492  6203 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:05.788002  6453 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:05.819191  6271 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:05.808718  6310 consensus_meta.cc:287] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.549s	user 0.007s	sys 0.006s
W20260709 13:04:05.791522  1568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2278) took 2179 ms (client timeout 2870 ms). Trace:
I20260709 13:04:05.794550  6201 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:05.795835  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2705) took 2047 ms (client timeout 1765 ms). Trace:
I20260709 13:04:05.975523  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 473) took 1233 ms. Trace:
I20260709 13:04:05.796876  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2239) took 1830 ms. Trace:
W20260709 13:04:05.797690  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.977061  1566 rpcz_store.cc:276] 0709 13:04:03.966696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.224855 (+1258159us) service_pool.cc:224] Handling call
0709 13:04:05.241417 (+ 16562us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:05.465444 (+224027us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:05.465487 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:05.500849 (+ 35362us) log.cc:844] Serialized 26 byte log entry
0709 13:04:05.691241 (+190392us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:05.691288 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.691361 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:05.796560 (+105199us) raft_consensus.cc:1685] finished
0709 13:04:05.796590 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.796770 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":194490,"raft.queue_time_us":219125,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":11228,"thread_start_us":156749,"threads_started":1,"wal-append.queue_time_us":218860}
I20260709 13:04:05.797885  1573 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:04:05.979115  1573 pending_rounds.cc:85] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 7
I20260709 13:04:05.979527  1573 pending_rounds.cc:107] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:04:05.979938  1573 raft_consensus.cc:2889] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:05.798094  6180 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:05.802740  1715 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:04:05.803498  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:05.804472  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:05.804847  6507 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:04:05.805301  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:05.806151  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2231) took 2111 ms. Trace:
W20260709 13:04:05.806442  1538 leader_election.cc:336] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.677s (SENT)
I20260709 13:04:05.697570  6508 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 13:04:05.806715  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2205) took 1143 ms. Trace:
W20260709 13:04:05.807585  6512 log.cc:927] Time spent T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.067s	user 0.002s	sys 0.000s
I20260709 13:04:05.808010  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2198) took 1185 ms. Trace:
W20260709 13:04:05.808682  6509 log.cc:927] Time spent T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:04:05.808862  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2702) took 2201 ms. Trace:
I20260709 13:04:05.824028  6497 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:05.810708  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2199) took 1187 ms. Trace:
W20260709 13:04:05.810923  6502 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 13:04:05.817845  1567 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:05.816941  6395 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:05.983693  1570 rpcz_store.cc:276] 0709 13:04:03.695143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.121490 (+1426347us) service_pool.cc:224] Handling call
0709 13:04:05.169263 (+ 47773us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:05.422692 (+253429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:05.422733 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:05.423334 (+   601us) log.cc:844] Serialized 26 byte log entry
0709 13:04:05.619398 (+196064us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:05.619454 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.619540 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:05.805931 (+186391us) raft_consensus.cc:1685] finished
0709 13:04:05.805950 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.806098 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":209184,"raft.queue_time_us":234326,"spinlock_wait_cycles":38146944,"thread_start_us":195264,"threads_started":1,"wal-append.queue_time_us":292016}
I20260709 13:04:05.791057  6349 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:05.922358  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:05.951359  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:05.834048  1568 rpcz_store.cc:269] 0709 13:04:03.611903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.676085 (+ 64182us) service_pool.cc:224] Handling call
0709 13:04:05.791232 (+2115147us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b5400305f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:05.791451 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1096902400}
I20260709 13:04:06.000175  1714 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:05.981441  1716 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
W20260709 13:04:05.975975  1648 rpcz_store.cc:269] 0709 13:04:03.749019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.773136 (+ 24117us) service_pool.cc:224] Handling call
0709 13:04:05.795250 (+2022114us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400099054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:05.795734 (+   484us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2341548416}
I20260709 13:04:05.819021  6344 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:04:05.983927  1538 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:05.991751  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:05.809795  6505 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:04:05.991974  1646 rpcz_store.cc:276] 0709 13:04:04.623059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.244638 (+621579us) service_pool.cc:224] Handling call
0709 13:04:05.270392 (+ 25754us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.270894 (+   502us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:05.438653 (+167759us) log.cc:844] Serialized 14 byte log entry
0709 13:04:05.806912 (+368259us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:05.807695 (+   783us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:05.807741 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:05.807784 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.807841 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.807960 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":174101,"prepare.queue_time_us":224928,"prepare.run_cpu_time_us":124,"prepare.run_wall_time_us":10926,"raft.queue_time_us":120932,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"spinlock_wait_cycles":28303744,"thread_start_us":315472,"threads_started":2,"wal-append.queue_time_us":279341}
I20260709 13:04:05.996021  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:05.998296  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 278170us
I20260709 13:04:05.982698  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:05.975881  6268 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:06.002152  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:05.976060  1713 rpcz_store.cc:276] 0709 13:04:04.742338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.775538 (+ 33200us) service_pool.cc:224] Handling call
0709 13:04:04.787649 (+ 12111us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.256800 (+469151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.848373 (+591573us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b5400213254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:05.975439 (+127066us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1058455296}
I20260709 13:04:06.015262  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:05.984385  1647 rpcz_store.cc:276] 0709 13:04:04.663037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.360464 (+697427us) service_pool.cc:224] Handling call
0709 13:04:05.370169 (+  9705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.370486 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:05.429487 (+ 59001us) log.cc:844] Serialized 14 byte log entry
0709 13:04:05.569872 (+140385us) log.cc:844] Serialized 14 byte log entry
0709 13:04:05.802324 (+232452us) spinlock_profiling.cc:243] Waited 7.69 ms on lock 0x7b18003f8e00. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d12c0a 00007f824f452219 00007f824cd5958f 00007f8245d98571 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd
0709 13:04:05.806193 (+  3869us) log.cc:844] Serialized 14 byte log entry
0709 13:04:05.806345 (+   152us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:05.806386 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:05.806441 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.806502 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.806662 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":272904,"prepare.queue_time_us":189588,"prepare.run_cpu_time_us":183,"prepare.run_wall_time_us":7299,"raft.queue_time_us":241481,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":5264,"spinlock_wait_cycles":18303616,"thread_start_us":121370,"threads_started":2,"wal-append.queue_time_us":237923}
I20260709 13:04:06.017004  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:05.994352  1641 rpcz_store.cc:276] 0709 13:04:03.607953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.210098 (+1602145us) service_pool.cc:224] Handling call
0709 13:04:05.289808 (+ 79710us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.324074 (+ 34266us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:05.487071 (+162997us) log.cc:844] Serialized 14 byte log entry
0709 13:04:05.808514 (+321443us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:05.808567 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:05.808610 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.808705 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.808806 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":140006,"prepare.queue_time_us":177130,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":74,"raft.queue_time_us":115555,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"thread_start_us":340063,"threads_started":2,"wal-append.queue_time_us":249250}
I20260709 13:04:05.997454  1642 rpcz_store.cc:276] 0709 13:04:04.623422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.299705 (+676283us) service_pool.cc:224] Handling call
0709 13:04:05.319664 (+ 19959us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.334540 (+ 14876us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:05.480472 (+145932us) log.cc:844] Serialized 14 byte log entry
0709 13:04:05.810182 (+329710us) log.cc:844] Serialized 14 byte log entry
0709 13:04:05.810363 (+   181us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:05.810409 (+    46us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:05.810457 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.810528 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.810651 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":128807,"prepare.queue_time_us":191302,"prepare.run_cpu_time_us":184,"prepare.run_wall_time_us":11573,"raft.queue_time_us":87676,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":175,"thread_start_us":326032,"threads_started":2}
I20260709 13:04:06.030735  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:06.032295  1569 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:04:06.035890  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:06.037216  1719 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:04:06.037554  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:06.037808  1722 raft_consensus.cc:2410] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:04:06.039662  1537 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:06.040923  6395 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:06.035686  6161 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:06.042830  6161 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:06.043218  6161 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.043560  6161 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:06.044451  1566 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:04:06.045310  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2243) took 2076 ms. Trace:
I20260709 13:04:06.045773  1566 rpcz_store.cc:276] 0709 13:04:03.968613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.043756 (+2075143us) service_pool.cc:224] Handling call
0709 13:04:06.044192 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:06.044931 (+   739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.045179 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:06.045985  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:06.047519  1566 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:06.048064  1567 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:04:06.048195  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 755) took 1978 ms. Trace:
I20260709 13:04:06.048679  1566 rpcz_store.cc:276] 0709 13:04:04.070396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.047002 (+1976606us) service_pool.cc:224] Handling call
0709 13:04:06.047344 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.047945 (+   601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.048139 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:06.046248  1537 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:06.050280  1566 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:06.048852  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 756) took 1978 ms. Trace:
I20260709 13:04:06.051103  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2283) took 1301 ms. Trace:
I20260709 13:04:06.051779  1566 rpcz_store.cc:276] 0709 13:04:04.749932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.049490 (+1299558us) service_pool.cc:224] Handling call
0709 13:04:06.049975 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.050641 (+   666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.051037 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:06.045305  1564 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.527s	user 0.007s	sys 0.007s
I20260709 13:04:06.053098  1567 rpcz_store.cc:276] 0709 13:04:04.070972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.047448 (+1976476us) service_pool.cc:224] Handling call
0709 13:04:06.047817 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:06.048575 (+   758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.048793 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:06.054139  1564 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:04:06.042068  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:06.020584  6268 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:06.038187  1713 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:06.061774  1537 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:06.063023  1570 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:04:06.063422  1570 pending_rounds.cc:85] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 7
I20260709 13:04:06.063864  1570 pending_rounds.cc:107] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:04:06.063818  6449 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:06.064486  1570 raft_consensus.cc:2889] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:06.065357  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2242) took 2097 ms. Trace:
I20260709 13:04:06.050642  6349 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:06.062006  1565 raft_consensus.cc:1275] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:04:06.086246  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2241) took 2118 ms. Trace:
I20260709 13:04:06.086979  1565 rpcz_store.cc:276] 0709 13:04:03.967668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.036001 (+2068333us) service_pool.cc:224] Handling call
0709 13:04:06.061616 (+ 25615us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.085302 (+ 23686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.085990 (+   688us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:06.063308  6171 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:04:06.067698  6161 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:06.069999  6453 consensus_queue.cc:1059] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:04:06.070679  6335 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:06.075850  6325 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:06.077082  6224 raft_consensus.cc:3055] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:06.079030  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:06.093101  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 366503us
W20260709 13:04:06.093927  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:06.058008  6401 consensus_meta.cc:287] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.527s	user 0.011s	sys 0.000s
I20260709 13:04:06.086781  6349 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.020741  6271 consensus_queue.cc:1059] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
W20260709 13:04:06.096072  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 289678us
W20260709 13:04:06.101999  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.102213  1565 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:04:06.108476  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2286) took 1312 ms. Trace:
I20260709 13:04:06.108973  1565 rpcz_store.cc:276] 0709 13:04:04.795840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.088420 (+1292580us) service_pool.cc:224] Handling call
0709 13:04:06.101780 (+ 13360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.108086 (+  6306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.108399 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:06.082810  6268 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:06.085114  1570 rpcz_store.cc:276] 0709 13:04:03.968162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.038213 (+2070051us) service_pool.cc:224] Handling call
0709 13:04:06.062684 (+ 24471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.065078 (+  2394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.065299 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:06.113047  1611 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:06.123378  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:06.129835  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:06.130610  1570 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:06.134073  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:06.105082  6224 raft_consensus.cc:740] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:06.114363  6349 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:06.147332  6161 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:06.150369  2312 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:06.153578  6344 consensus_queue.cc:1059] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:06.155237  6344 raft_consensus.cc:3055] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:06.155803  6344 raft_consensus.cc:740] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:06.157459  6344 consensus_queue.cc:260] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:06.159179  6344 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:06.163646  1538 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.763s (SENT)
I20260709 13:04:06.164242  1538 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:06.165676  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:06.176499  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:06.195258  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:06.196091  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:06.196173  6520 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:04:06.172874  6401 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:04:06.197804  6421 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:06.202136  1685 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:06.130254  6494 consensus_queue.cc:1059] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:04:06.198652  6421 raft_consensus.cc:697] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:06.170702  6161 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.215258  6315 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:04:06.216866  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 47046.3us
I20260709 13:04:06.135040  6271 consensus_queue.cc:1243] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:04:06.223475  6287 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:06.236639  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 759) took 1371 ms. Trace:
I20260709 13:04:06.244064  1569 rpcz_store.cc:276] 0709 13:04:04.865372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.235403 (+1370031us) service_pool.cc:224] Handling call
0709 13:04:06.235842 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.236147 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:06.236206 (+    59us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:06.236267 (+    61us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:06.236323 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.236395 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:06.236557 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:06.241827  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 180132us
W20260709 13:04:06.241165  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.247504  1537 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:06.245849  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.255234  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:06.215915  6315 raft_consensus.cc:697] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 15 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:06.214133  6508 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:06.260345  1569 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: [11.7-12.8]
I20260709 13:04:06.261394  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2244) took 1042 ms. Trace:
I20260709 13:04:06.261699  1569 rpcz_store.cc:276] 0709 13:04:05.218452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.245152 (+1026700us) service_pool.cc:224] Handling call
0709 13:04:06.259848 (+ 14696us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:06.260943 (+  1095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.261228 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:06.262416  1612 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:06.266032  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 127183us
I20260709 13:04:06.267211  1687 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:06.272405  1687 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:06.246424  6286 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:06.248605  6395 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:06.271106  1572 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2221) took 3807 ms (client timeout 2216 ms). Trace:
I20260709 13:04:06.259088  6171 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:06.278759  1571 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:04:06.282307  1571 pending_rounds.cc:85] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 6
I20260709 13:04:06.282655  1571 pending_rounds.cc:107] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:04:06.282920  1571 raft_consensus.cc:2889] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:06.280158  1572 rpcz_store.cc:269] 0709 13:04:02.463424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.849396 (+385972us) service_pool.cc:224] Handling call
0709 13:04:06.261049 (+3411653us) spinlock_profiling.cc:243] Waited 2.43 s on lock 0x7b5400012554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:06.270988 (+  9939us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5594708224}
I20260709 13:04:06.284230  1720 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:04:06.285269  1569 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:06.286013  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2248) took 1050 ms. Trace:
I20260709 13:04:06.286311  1569 rpcz_store.cc:276] 0709 13:04:05.235784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.262309 (+1026525us) service_pool.cc:224] Handling call
0709 13:04:06.277119 (+ 14810us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.285753 (+  8634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.285962 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:06.274643  6401 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:06.242744  6161 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:06.291090  1687 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:06.287855  6401 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.293882  1572 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:06.287307  6395 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:06.294091  6401 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:06.257725  6271 consensus_queue.cc:1048] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:04:06.287925  6180 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:06.282100  6171 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.301694  1569 raft_consensus.cc:3055] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:06.305840  6518 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.059s
I20260709 13:04:06.308022  1572 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:04:06.293233  6508 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:06.324606  1569 raft_consensus.cc:740] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:06.295377  6287 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:04:06.332150  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 157633us
I20260709 13:04:06.335868  1569 consensus_queue.cc:260] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:06.325965  6286 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.329595  6439 consensus_queue.cc:1059] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:04:06.358045  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2250) took 4319 ms. Trace:
I20260709 13:04:06.316192  6171 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:06.356446  6497 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:06.357045  1569 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:06.357357  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 111515us
I20260709 13:04:06.358446  1571 rpcz_store.cc:276] 0709 13:04:02.038762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.549706 (+1510944us) service_pool.cc:224] Handling call
0709 13:04:06.278102 (+2728396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.294798 (+ 16696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.357775 (+ 62977us) spinlock_profiling.cc:243] Waited 2.72 s on lock 0x7b5400012550. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:06.357980 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6252267904}
I20260709 13:04:06.374573  1611 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:06.378885  1571 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:04:06.381481  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 87756.1us
I20260709 13:04:06.355132  6287 raft_consensus.cc:697] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:06.386598  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:06.390270  2348 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:06.392738  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 119381us
I20260709 13:04:06.373167  6439 consensus_queue.cc:1243] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:04:06.396340  1537 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:06.399101  1537 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:06.404142  6271 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:06.404330  1612 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:06.404644  6271 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.405041  6271 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:06.405839  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:06.409727  6508 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:06.410205  6508 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.410588  6508 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:06.413975  6361 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:06.414302  1685 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:06.414960  6361 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:06.416550  6396 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:06.315754  6421 consensus_queue.cc:237] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:06.363476  6286 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:06.389236  6181 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:04:06.429378  1538 leader_election.cc:336] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.977s (SENT)
I20260709 13:04:06.430138  1538 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:06.338327  6315 consensus_queue.cc:237] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:06.466127  1569 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:04:06.467924  1569 pending_rounds.cc:85] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 6
I20260709 13:04:06.468282  1569 pending_rounds.cc:107] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:04:06.468781  1569 raft_consensus.cc:2889] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:06.472527  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:06.401604  6439 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.421109  6508 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:06.396096  6180 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 13:04:06.486860  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 478) took 1039 ms. Trace:
I20260709 13:04:06.476035  6201 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:06.499060  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:06.492965  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 90984.6us
W20260709 13:04:06.493958  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:06.495072  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 80856.9us
I20260709 13:04:06.497102  1717 rpcz_store.cc:276] 0709 13:04:05.447703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.447929 (+   226us) service_pool.cc:224] Handling call
0709 13:04:06.486656 (+1038727us) spinlock_profiling.cc:243] Waited 63.6 ms on lock 0x7b5400053f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:06.486796 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":146204416}
I20260709 13:04:06.498449  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:06.501902  1685 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:06.466157  6271 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:06.481921  6181 raft_consensus.cc:697] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:06.504303  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:06.505069  1648 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:06.502784  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 138282us
W20260709 13:04:06.508054  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 102403us
I20260709 13:04:06.510128  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2249) took 1195 ms. Trace:
I20260709 13:04:06.510147  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:06.417546  6287 consensus_queue.cc:237] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:06.511062  1639 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:06.510754  1569 rpcz_store.cc:276] 0709 13:04:05.305423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.287018 (+981595us) service_pool.cc:224] Handling call
0709 13:04:06.301531 (+ 14513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.500893 (+199362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.501058 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:06.518271  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:06.522713  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:06.528100  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:06.543715  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.525038  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:06.544274  1719 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:04:06.546559  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:06.548395  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:06.549322  1721 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:04:06.560602  1570 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:04:06.509485  6395 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:06.712752  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.496615  6524 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:04:06.561856  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:06.715548  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 286232us
I20260709 13:04:06.722849  1648 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:06.496440  6439 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:06.543193  6161 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:06.745085  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 247350us
W20260709 13:04:06.546236  6453 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:04:06.743899  1612 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:06.746866  2314 leader_election.cc:336] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.595s (SENT)
I20260709 13:04:06.757421  1720 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
W20260709 13:04:06.759430  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.542344  6531 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:06.768932  1645 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:04:06.758494  2314 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:06.760910  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:06.776365  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:06.777870  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 230888us
W20260709 13:04:06.770887  6335 consensus_meta.cc:287] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.671s	user 0.007s	sys 0.005s
W20260709 13:04:06.778707  6349 consensus_meta.cc:287] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.586s	user 0.012s	sys 0.000s
I20260709 13:04:06.799853  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:06.801271  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 296225us
I20260709 13:04:06.802683  1643 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:06.804073  1537 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:06.798885  1644 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:06.812589  1645 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 12: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:06.781148  6494 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:06.777341  6533 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:06.772022  6453 consensus_queue.cc:1059] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
W20260709 13:04:06.812244  6325 consensus_meta.cc:287] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.701s	user 0.009s	sys 0.004s
W20260709 13:04:06.825289  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 78461.9us
I20260709 13:04:06.810714  6297 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.816291  6531 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:06.800237  6395 consensus_queue.cc:1048] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:04:06.816041  6180 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:06.837488  1538 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:06.802930  6349 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:06.835146  6531 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.834803  6286 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:06.858258  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.842773  6453 raft_consensus.cc:3055] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:06.863979  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:06.864668  1719 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:06.811846  6181 consensus_queue.cc:237] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:06.866878  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:06.867291  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 754) took 3608 ms. Trace:
I20260709 13:04:06.867801  1573 rpcz_store.cc:276] 0709 13:04:03.258586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.047574 (+1788988us) service_pool.cc:224] Handling call
0709 13:04:05.797385 (+749811us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.009162 (+211777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.867049 (+857887us) spinlock_profiling.cc:243] Waited 749 ms on lock 0x7b5400305f50. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:06.867227 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1723549568}
I20260709 13:04:06.872654  6349 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:06.874153  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 43833.4us
I20260709 13:04:06.875607  1538 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:06.832973  6297 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:06.842569  6180 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:06.879114  6508 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:06.879722  6180 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:06.879988  6508 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:06.869622  6286 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:04:06.882715  6286 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:06.884030  2347 raft_consensus.cc:2410] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:04:06.884473  6286 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:06.869699  6396 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:04:06.859871  6531 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:06.862991  6453 raft_consensus.cc:740] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:06.888440  6439 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:06.888793  6453 consensus_queue.cc:260] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:06.890126  6453 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:06.872805  6171 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:06.895982  1643 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:06.869866  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:06.838338  6533 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
W20260709 13:04:06.900537  2348 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.001s
I20260709 13:04:06.907312  2348 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
W20260709 13:04:06.908152  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.906723  1719 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:06.867586  6234 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:06.903785  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 394172us
I20260709 13:04:06.912716  6508 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.912917  6161 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.913332  6508 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:06.913385  6161 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:06.915638  6161 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:06.915715  6508 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:06.916224  6161 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.916707  6161 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:06.918862  6161 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:06.919569  6161 consensus_queue.cc:1048] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:06.911468  1685 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:06.925977  1639 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
W20260709 13:04:06.934142  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 197251us
I20260709 13:04:06.938623  1685 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:06.938822  6526 log.cc:927] Time spent T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.354s	user 0.001s	sys 0.001s
I20260709 13:04:06.966229  6171 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:06.966866  6344 consensus_meta.cc:287] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.807s	user 0.015s	sys 0.000s
I20260709 13:04:06.939005  6439 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.924396  6349 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:04:06.923486  6234 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:06.958819  6268 consensus_meta.cc:287] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.826s	user 0.013s	sys 0.000s
I20260709 13:04:06.980060  1612 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:04:06.947772  1538 connection.cc:256] RPC call timeout handler was delayed by -1.0484612050000237s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1647s for a response
I20260709 13:04:06.937656  6539 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:06.955348  6543 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:06.937845  6536 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:06.991231  6234 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:06.938495  6161 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
W20260709 13:04:06.994829  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 135706us
I20260709 13:04:06.995867  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:06.976486  6540 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:07.003301  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 71964us
I20260709 13:04:07.007220  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:07.008935  1537 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:07.013525  6546 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:07.014550  6546 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.015380  6546 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:07.018391  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:07.019469  1717 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:06.988740  6171 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:07.029358  2314 leader_election.cc:336] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.004s (SENT)
I20260709 13:04:07.031066  6171 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:06.999354  6508 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:07.040194  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.041316  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 758) took 2217 ms. Trace:
I20260709 13:04:07.041872  1565 rpcz_store.cc:276] 0709 13:04:04.824193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.133053 (+1308860us) service_pool.cc:224] Handling call
0709 13:04:06.202819 (+ 69766us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.292653 (+ 89834us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:06.292688 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:06.292736 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:06.292770 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.041034 (+748264us) spinlock_profiling.cc:243] Waited 85.7 ms on lock 0x7b54001b5b54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:07.041089 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:07.041249 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":197137792}
W20260709 13:04:07.045384  6491 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 1.744s	user 0.004s	sys 0.012s
I20260709 13:04:07.047994  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:07.053172  6540 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:04:07.057226  6401 consensus_meta.cc:287] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.721s	user 0.008s	sys 0.005s
I20260709 13:04:07.058140  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:07.057984  6401 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:06.984742  6439 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:07.061710  6439 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:06.986039  6268 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:07.062878  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:07.063794  1720 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:07.063946  1566 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:07.071038  6349 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:07.072057  1687 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:07.012841  6524 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:07.037799  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:07.082351  1717 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:07.049541  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:07.092794  1715 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:07.101579  1716 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 13:04:07.102821  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 142773us
I20260709 13:04:07.107805  1722 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:04:07.110644  6171 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:07.111677  1715 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:07.124601  6401 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:04:07.125595  6540 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:07.126319  6439 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:07.127027  6439 raft_consensus.cc:697] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:07.128283  6439 consensus_queue.cc:237] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:07.130059  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:07.125803  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 88206us
I20260709 13:04:07.138454  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:07.101715  1687 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:07.139124  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.139910  1687 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:07.143579  1687 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:07.074309  6310 raft_consensus.cc:3055] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:04:07.140789  6338 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.397s	user 0.011s	sys 0.006s
W20260709 13:04:07.168591  2314 leader_election.cc:336] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.208s (SENT)
I20260709 13:04:07.184875  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:07.187542  1715 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:07.192500  2314 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:07.190754  1685 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:07.192337  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 48877.7us
I20260709 13:04:07.083947  6508 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:07.157176  6310 raft_consensus.cc:740] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:04:07.199821  1538 leader_election.cc:336] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.547s (SENT)
I20260709 13:04:07.147171  6540 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:07.196627  1687 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:07.205011  1537 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:07.163216  6338 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.420s	user 0.011s	sys 0.006s
I20260709 13:04:07.146661  6401 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:07.076227  6224 consensus_queue.cc:260] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:07.206045  6344 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:07.194496  6349 raft_consensus.cc:697] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:07.227087  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:07.198110  6287 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:04:07.242182  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 73629.4us
I20260709 13:04:07.242450  6310 consensus_queue.cc:260] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:07.243662  5852 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:07.227994  6528 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.025s	sys 0.062s
W20260709 13:04:07.244623  6528 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.025s	sys 0.062s
I20260709 13:04:07.244140  5852 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.245150  5852 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:07.244104  6310 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:07.243685  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 84395us
I20260709 13:04:07.217204  6338 raft_consensus.cc:697] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:07.246901  1612 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:07.247462  6338 consensus_queue.cc:237] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:07.248185  6234 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:07.219261  6538 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:07.249603  1612 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:07.249958  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 59290.9us
I20260709 13:04:07.250912  1685 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:07.251402  1612 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:07.252548  6315 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:07.253391  6315 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:07.214361  6533 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:07.216919  6540 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:04:07.265036  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 65277.8us
I20260709 13:04:07.268805  1715 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:07.260985  6224 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:07.258474  6560 consensus_queue.cc:1175] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "130bc5ff39b847b1b3204626f284f838" responder_term: 12 status { last_received { term: 10 index: 10 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 11. Current term is 12. Ops: [11.10-11.10]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:04:07.289762  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.224133  6401 raft_consensus.cc:697] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:07.295374  6401 consensus_queue.cc:237] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:07.254562  6549 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.298070  6549 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:07.300462  6549 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:07.273105  1720 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:07.274818  6531 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:07.303364  6450 raft_consensus.cc:3055] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:07.291146  6540 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:07.303882  6450 raft_consensus.cc:740] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:07.304847  6450 consensus_queue.cc:260] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:07.305238  6531 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:07.276407  6349 consensus_queue.cc:237] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:07.305922  6450 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:07.278457  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:07.312337  1612 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:07.313839  6315 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:04:07.313980  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.313385  6234 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:07.316386  1612 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:07.317293  6234 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:07.318529  6527 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.479s	user 0.002s	sys 0.000s
I20260709 13:04:07.319933  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 479) took 1062 ms. Trace:
I20260709 13:04:07.321029  6508 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:07.254340  6287 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.321754  6287 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:07.320374  1714 rpcz_store.cc:276] 0709 13:04:06.257374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.257891 (+   517us) service_pool.cc:224] Handling call
0709 13:04:06.258230 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:06.258619 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:06.258638 (+    19us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:06.272428 (+ 13790us) log.cc:844] Serialized 26 byte log entry
0709 13:04:06.454148 (+181720us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:06.454198 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.454285 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:07.317870 (+863585us) spinlock_profiling.cc:243] Waited 9.49 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:07.319536 (+  1666us) raft_consensus.cc:1685] finished
0709 13:04:07.319581 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:07.319858 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6075,"raft.queue_time_us":951,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":11225,"spinlock_wait_cycles":21831936,"thread_start_us":152501,"threads_started":1,"wal-append.queue_time_us":497605}
I20260709 13:04:07.283005  6546 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:07.326679  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:07.330870  1715 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 11: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:04:07.329476  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 78663.3us
W20260709 13:04:07.337216  1538 leader_election.cc:336] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.726s (SENT)
W20260709 13:04:07.348223  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 143312us
I20260709 13:04:07.349012  1537 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:07.350031  6531 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:07.350838  6531 raft_consensus.cc:697] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:07.352159  6531 consensus_queue.cc:237] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:07.354044  1537 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:07.355268  6453 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:04:07.357668  1538 leader_election.cc:336] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.745s (SENT)
I20260709 13:04:07.358198  1538 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:07.360231  6508 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:07.356005  6453 raft_consensus.cc:697] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:07.336328  6396 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:07.334640  6494 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.375118  6494 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:07.382915  6494 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:07.384757  1612 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:07.341241  6344 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.032s
W20260709 13:04:07.400859  6517 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 1.360s	user 0.000s	sys 0.012s
I20260709 13:04:07.417495  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:07.418480  1569 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:07.418804  6540 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:07.421165  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:07.396379  6401 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:04:07.423947  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:07.424580  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:07.426249  1646 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:07.429105  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:07.432878  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2217) took 1005 ms. Trace:
I20260709 13:04:07.433092  1564 raft_consensus.cc:2410] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:04:07.421932  1573 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:07.433215  1646 rpcz_store.cc:276] 0709 13:04:06.429018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.415494 (+986476us) service_pool.cc:224] Handling call
0709 13:04:07.425883 (+ 10389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:07.432606 (+  6723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.432806 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:07.422195  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:07.397922  6537 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:07.446449  1565 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:07.446872  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2215) took 1046 ms. Trace:
I20260709 13:04:07.447901  1641 rpcz_store.cc:276] 0709 13:04:06.400752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.408497 (+1007745us) service_pool.cc:224] Handling call
0709 13:04:07.446790 (+ 38293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:07.451323  1569 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
W20260709 13:04:07.454849  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 70263us
I20260709 13:04:07.468725  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:07.469444  1569 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:04:07.451597  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:07.410485  6538 raft_consensus.cc:697] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:07.406316  6396 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:04:07.421609  6540 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:07.475032  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:07.426415  6546 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:07.476745  1569 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:07.477982  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:07.480482  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.481386  1612 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:07.431443  6497 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:04:07.430477  6533 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:04:07.485092  1567 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:04:07.486725  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:07.530856  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:07.542358  6565 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:04:07.542723  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:07.543746  6570 log.cc:927] Time spent T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:04:07.544353  6566 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.116s
I20260709 13:04:07.476224  6528 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.541291  6297 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:07.543351  6287 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:07.555739  1569 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:07.555851  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:07.560685  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:07.450850  6453 consensus_queue.cc:237] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:07.548058  6234 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:07.595069  6234 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:07.595500  6234 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:07.592722  6528 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:07.600452  6528 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:07.602607  2347 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55888 (request call id 580) took 4485 ms (client timeout 1628 ms). Trace:
W20260709 13:04:07.603066  2347 rpcz_store.cc:269] 0709 13:04:03.117274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.117858 (+   584us) service_pool.cc:224] Handling call
0709 13:04:07.602335 (+4484477us) spinlock_profiling.cc:243] Waited 3.71 s on lock 0x7b5400467654. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:07.602513 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8536821632}
I20260709 13:04:07.603407  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2252) took 2266 ms. Trace:
I20260709 13:04:07.604063  1572 rpcz_store.cc:276] 0709 13:04:05.336846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.297276 (+960430us) service_pool.cc:224] Handling call
0709 13:04:06.297614 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.325610 (+ 27996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.603054 (+1277444us) spinlock_profiling.cc:243] Waited 8.45 ms on lock 0x7b54001b6554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:07.603323 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19433856}
I20260709 13:04:07.576395  1571 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
W20260709 13:04:07.554936  6568 log.cc:927] Time spent T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:04:07.592284  6201 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:07.610249  1565 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:07.590480  6538 consensus_queue.cc:237] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:07.622373  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 150025us
I20260709 13:04:07.624059  1612 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:07.626842  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.628664  5852 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:07.625135  1612 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:07.633886  6537 raft_consensus.cc:2764] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:04:07.634778  1686 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:07.634990  6537 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:07.637193  6271 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:07.637342  6561 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:07.638213  6561 raft_consensus.cc:697] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:07.638960  6396 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.639513  6396 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:07.639318  6561 consensus_queue.cc:237] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:07.614207  1573 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:07.618130  6201 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:07.618530  6536 consensus_queue.cc:1059] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:07.652717  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:07.659267  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:07.667424  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2284) took 2903 ms. Trace:
I20260709 13:04:07.668112  1566 rpcz_store.cc:276] 0709 13:04:04.764445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.080444 (+1315999us) service_pool.cc:224] Handling call
0709 13:04:06.081029 (+   585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:07.064384 (+983355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.667003 (+602619us) spinlock_profiling.cc:243] Waited 983 ms on lock 0x7b5400406ad4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:07.667330 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2260043648}
I20260709 13:04:07.674170  6271 raft_consensus.cc:3055] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:07.674903  6271 raft_consensus.cc:740] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:07.674638  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:07.676925  6271 consensus_queue.cc:260] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:07.678560  6271 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:07.680568  2312 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:07.682147  6524 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.682628  6325 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:07.682806  6524 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:07.683579  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.653713  1573 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
W20260709 13:04:07.685845  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:07.689401  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.642623  6396 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:07.643441  6522 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.709908  1572 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:07.710443  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:07.711231  1568 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:04:07.696286  6560 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:04:07.734915  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 307453us
I20260709 13:04:07.735497  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:07.739753  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.743220  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:07.744783  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:07.745487  1641 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:07.746466  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:07.749045  2312 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:07.751992  1568 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:07.699290  6537 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:04:07.757819  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:07.763221  1572 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:04:07.767333  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:07.768190  1644 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:07.765988  6297 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:07.742374  6396 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:07.778164  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:07.779697  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:07.766292  6286 consensus_queue.cc:1059] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:04:07.785938  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:07.798275  1641 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:07.798560  1713 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 12: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:07.770416  6268 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:07.775888  6297 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.754225  6560 raft_consensus.cc:697] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:07.799882  2343 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:04:07.790942  5852 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:07.795060  1687 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:07.801156  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:07.721223  6539 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 13:04:07.815352  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 66410.4us
I20260709 13:04:07.816993  1643 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:07.896555  6287 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:07.879879  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:07.886725  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:07.897161  6543 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.742864  6338 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:04:07.876243  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:07.816749  6396 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:07.846315  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:07.880604  6538 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:04:07.799798  6297 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:07.858618  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:07.905001  6569 raft_consensus.cc:3055] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:07.903158  6421 raft_consensus.cc:3055] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:07.950093  6224 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.712294  6522 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:08.011436  1641 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
W20260709 13:04:07.955016  6587 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.084s	user 0.003s	sys 0.000s
I20260709 13:04:07.953413  6560 consensus_queue.cc:237] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:07.990074  6361 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.846429  1611 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:08.165829  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 319509us
I20260709 13:04:08.123584  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:08.169114  1564 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:07.857043  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 62125.3us
I20260709 13:04:08.012846  1644 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:04:08.013442  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:08.018174  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.025286  6201 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:08.033191  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:08.102279  6569 raft_consensus.cc:740] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:08.177917  1714 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:08.034029  6567 log.cc:927] Time spent T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.616s	user 0.002s	sys 0.003s
I20260709 13:04:07.964076  6234 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:08.035158  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:07.907395  6575 log.cc:927] Time spent T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:04:07.864606  6566 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:07.853433  6537 consensus_queue.cc:1059] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:04:08.119953  6508 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.868397  1717 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11. Ops: []
W20260709 13:04:08.164456  6551 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.910s	user 0.002s	sys 0.028s
W20260709 13:04:08.166541  6494 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.008s	sys 0.014s
I20260709 13:04:08.069610  1715 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:08.098685  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:08.102609  1570 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 13: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:04:08.104466  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.106777  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:08.108045  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:08.109268  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:08.110852  1567 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.010s	sys 0.009s
I20260709 13:04:08.038885  6271 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:08.161800  2344 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:08.120746  1713 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:08.100929  6538 consensus_queue.cc:1243] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer cf2f71592be8465fb18e36e15d0b2f89 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:04:08.170774  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:08.172150  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.213722  1716 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:08.172963  1566 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 13:04:08.168033  2314 leader_election.cc:336] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.143s (SENT)
W20260709 13:04:08.174495  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 230602us
I20260709 13:04:08.216101  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:08.184700  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:07.905848  6524 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:08.102047  6421 raft_consensus.cc:740] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:08.215672  2348 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:08.103309  6543 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:08.098112  6344 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 13:04:08.120288  6536 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:04:08.215041  1567 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.010s	sys 0.014s
I20260709 13:04:08.216524  1685 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:08.183697  1720 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:08.238329  1717 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:04:08.246837  1612 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:04:08.192672  6551 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.939s	user 0.002s	sys 0.029s
I20260709 13:04:08.183405  6287 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:04:08.257220  1538 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.597s (SENT)
I20260709 13:04:08.266260  1538 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:08.271685  6286 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:08.272228  6286 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:08.272578  6286 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:08.274092  6421 consensus_queue.cc:260] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:08.275511  6421 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:08.279110  6543 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:08.224418  6508 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:08.281872  6594 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:08.192487  6566 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:08.283558  6522 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:08.283803  6566 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:08.285399  6396 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:08.286003  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 118075us
W20260709 13:04:08.286631  2314 leader_election.cc:336] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.147s (SENT)
I20260709 13:04:08.287001  2314 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:08.288143  2314 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.161s (SENT)
I20260709 13:04:08.290037  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:08.289798  6536 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:08.290711  1641 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:04:08.292384  6536 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:08.292989  6569 consensus_queue.cc:260] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:08.294368  6569 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:08.297339  6201 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:08.298223  6344 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:08.301550  1570 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:04:08.283593  6508 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:08.302834  1570 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:04:08.305394  6597 log.cc:927] Time spent T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:04:08.185801  6584 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:08.262331  1564 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
W20260709 13:04:08.307140  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.315348  1713 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:08.318934  6234 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:08.319806  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 519885us
I20260709 13:04:08.320315  6539 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:08.323742  6297 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:08.215431  6494 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.009s	sys 0.015s
I20260709 13:04:08.328186  1648 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:04:08.328683  6361 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:08.374766  2348 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:08.377470  1720 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:08.259647  6537 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.387719  1715 raft_consensus.cc:2410] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:04:08.382442  2344 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:08.395992  1641 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:08.368487  1612 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:08.411911  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.265827  6287 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:08.119624  6224 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:08.344511  5852 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:08.329135  6584 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:08.241585  6271 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:08.438637  6224 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:08.429934  6234 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:04:08.442111  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 73752.8us
I20260709 13:04:08.474725  1612 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:08.456841  1643 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
W20260709 13:04:08.461689  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 204489us
I20260709 13:04:08.476115  1716 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:04:08.477463  1538 leader_election.cc:336] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.656s (SENT)
I20260709 13:04:08.478124  1538 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:08.434965  6287 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:08.521739  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:08.527494  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:08.528407  1537 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:08.469267  6584 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:08.528650  1720 raft_consensus.cc:2410] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 16 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 13:04:08.530238  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:08.531912  1717 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:08.538332  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:08.473587  6361 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:08.542526  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 316593us
I20260709 13:04:08.428670  6537 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:08.553730  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 76308.8us
I20260709 13:04:08.525141  6508 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.527945  5852 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:08.564864  1687 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:08.541288  6268 consensus_queue.cc:1059] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.047s
W20260709 13:04:08.642014  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.541611  6161 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:08.649978  6161 raft_consensus.cc:697] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:08.651134  6161 consensus_queue.cc:237] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:08.654621  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:08.655684  2348 raft_consensus.cc:2410] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:04:08.656539  6271 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:08.580449  6598 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.086s
W20260709 13:04:08.549000  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 332552us
I20260709 13:04:08.660053  6537 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:08.662444  6595 log.cc:927] Time spent T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:04:08.663046  6297 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:08.609442  1538 leader_election.cc:336] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.690s (SENT)
I20260709 13:04:08.668063  1685 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:08.581337  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:08.558226  6361 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:08.637848  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 76538.3us
W20260709 13:04:08.627233  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.671042  1568 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:04:08.671501  1687 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:08.699306  2312 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:08.698890  6539 raft_consensus.cc:697] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:08.700670  6297 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:08.701208  6297 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:08.701613  6297 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:08.701772  6539 consensus_queue.cc:237] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:08.704095  6421 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:08.704840  6421 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:08.677029  1538 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:08.732336  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 64398.4us
I20260709 13:04:08.665083  6569 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:08.735962  6569 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:08.744865  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 408542us
I20260709 13:04:08.667237  6201 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:04:08.740387  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 302666us
W20260709 13:04:08.751021  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.761879  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2297) took 1057 ms. Trace:
W20260709 13:04:08.770541  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.769773  1572 rpcz_store.cc:276] 0709 13:04:07.704213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.124146 (+419933us) service_pool.cc:224] Handling call
0709 13:04:08.188488 (+ 64342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:08.323047 (+134559us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:08.323081 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:08.323122 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:08.323156 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.761458 (+438302us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54001b3d54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:08.761541 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.761792 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":308991744}
I20260709 13:04:08.640686  6508 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:08.776134  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2227) took 1067 ms. Trace:
I20260709 13:04:08.776647  1646 rpcz_store.cc:276] 0709 13:04:07.708898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.709173 (+   275us) service_pool.cc:224] Handling call
0709 13:04:07.738989 (+ 29816us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:07.798844 (+ 59855us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:07.798950 (+   106us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:08.105145 (+306195us) log.cc:844] Serialized 52 byte log entry
0709 13:04:08.569028 (+463883us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:08.711965 (+142937us) log.cc:844] Serialized 14 byte log entry
0709 13:04:08.771674 (+ 59709us) log.cc:844] Serialized 14 byte log entry
0709 13:04:08.775756 (+  4082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.775851 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:08.775906 (+    55us) raft_consensus.cc:1685] finished
0709 13:04:08.775919 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.776071 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62734,"prepare.queue_time_us":141248,"raft.queue_time_us":14412,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":21568,"spinlock_wait_cycles":108652672,"thread_start_us":429416,"threads_started":2,"wal-append.queue_time_us":496009}
W20260709 13:04:08.742291  6592 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.217s	user 0.001s	sys 0.001s
I20260709 13:04:08.778128  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2234) took 1050 ms. Trace:
W20260709 13:04:08.783625  1714 consensus_meta.cc:287] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.605s	user 0.015s	sys 0.001s
I20260709 13:04:08.789886  1714 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
W20260709 13:04:08.800382  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.743726  6522 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:08.739588  6543 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:08.760296  6569 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:08.789438  1640 rpcz_store.cc:276] 0709 13:04:07.727940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.728107 (+   167us) service_pool.cc:224] Handling call
0709 13:04:07.750122 (+ 22015us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:07.762424 (+ 12302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:07.762472 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:07.845660 (+ 83188us) log.cc:844] Serialized 52 byte log entry
0709 13:04:08.120761 (+275101us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:08.503616 (+382855us) log.cc:844] Serialized 14 byte log entry
0709 13:04:08.522160 (+ 18544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.522236 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:08.777895 (+255659us) raft_consensus.cc:1685] finished
0709 13:04:08.777916 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.778064 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18856,"prepare.queue_time_us":68780,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":6045,"raft.queue_time_us":157022,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":25087,"thread_start_us":555099,"threads_started":2,"wal-append.queue_time_us":638236}
W20260709 13:04:08.835608  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 226241us
W20260709 13:04:08.836068  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.782234  6396 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:04:08.824340  6606 log.cc:927] Time spent T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
W20260709 13:04:08.845551  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 370599us
I20260709 13:04:08.852165  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:08.850976  1612 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:08.846393  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 309472us
I20260709 13:04:08.852916  1572 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:08.857874  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:08.857911  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:08.860653  6522 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:08.825728  6543 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:08.866762  2313 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:08.780006  6234 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:04:08.774802  6201 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:08.845070  6531 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.424s	user 0.031s	sys 0.041s
I20260709 13:04:08.844604  6494 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.878018  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:08.883276  2348 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 13: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:08.896034  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2640) took 1032 ms. Trace:
I20260709 13:04:08.896739  1713 rpcz_store.cc:276] 0709 13:04:07.863319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.864262 (+   943us) service_pool.cc:224] Handling call
0709 13:04:08.895763 (+1031501us) spinlock_profiling.cc:243] Waited 50.3 ms on lock 0x7b5400212854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:08.895949 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115696512}
W20260709 13:04:08.896745  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 74737.2us
I20260709 13:04:08.899539  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:08.900476  1567 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 13: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:08.905620  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:08.906996  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 56237.8us
I20260709 13:04:08.907878  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2281) took 1205 ms. Trace:
I20260709 13:04:08.908282  1573 rpcz_store.cc:276] 0709 13:04:07.702035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.724370 (+ 22335us) service_pool.cc:224] Handling call
0709 13:04:07.743741 (+ 19371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:07.744222 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:07.744262 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:07.744932 (+   670us) log.cc:844] Serialized 26 byte log entry
0709 13:04:08.183245 (+438313us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:08.183304 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.183389 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:08.907648 (+724259us) raft_consensus.cc:1685] finished
0709 13:04:08.907671 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.907826 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":151,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":5386,"thread_start_us":380733,"threads_started":1,"wal-append.queue_time_us":664333}
W20260709 13:04:08.881418  6531 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.461s	user 0.032s	sys 0.041s
W20260709 13:04:08.914142  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 242772us
I20260709 13:04:08.906524  1720 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:04:08.881044  6325 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:08.913985  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:08.875680  6543 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:08.919615  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:08.882594  6522 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:08.929770  1573 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:08.928067  1567 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:08.901597  6620 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:04:08.936444  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 202488us
W20260709 13:04:08.941138  2314 leader_election.cc:336] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.918s (SENT)
I20260709 13:04:08.941730  2314 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:08.911891  6524 raft_consensus.cc:3055] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:08.940466  1685 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:08.946677  1573 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:04:08.897567  6494 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:08.901260  6529 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
W20260709 13:04:08.946547  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 101181us
W20260709 13:04:08.964893  2314 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.943s (SENT)
I20260709 13:04:08.948074  6325 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:08.943825  6524 raft_consensus.cc:740] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:08.943533  6396 raft_consensus.cc:697] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:08.990433  6396 consensus_queue.cc:237] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:08.992830  6421 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:08.994385  6613 log.cc:927] Time spent T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:04:08.979285  1573 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:09.023458  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2295) took 1319 ms. Trace:
I20260709 13:04:09.012825  6201 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:09.013972  1538 leader_election.cc:336] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.647s (SENT)
W20260709 13:04:09.017599  6619 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:04:09.018774  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2284) took 1257 ms. Trace:
I20260709 13:04:09.031322  1564 rpcz_store.cc:276] 0709 13:04:07.761571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.334253 (+572682us) service_pool.cc:224] Handling call
0709 13:04:08.334569 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:08.399084 (+ 64515us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:08.399118 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:08.431232 (+ 32114us) log.cc:844] Serialized 26 byte log entry
0709 13:04:08.878896 (+447664us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:08.878979 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.879092 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:09.018497 (+139405us) raft_consensus.cc:1685] finished
0709 13:04:09.018524 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.018703 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63852,"raft.queue_time_us":64005,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":20894,"thread_start_us":410820,"threads_started":1,"wal-append.queue_time_us":504102}
W20260709 13:04:09.019685  6566 consensus_meta.cc:287] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.735s	user 0.014s	sys 0.000s
I20260709 13:04:09.034253  6566 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:09.037117  1719 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:09.027036  6633 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.038218  6633 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:09.041100  6633 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:09.023972  1565 rpcz_store.cc:276] 0709 13:04:07.703755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.031011 (+327256us) service_pool.cc:224] Handling call
0709 13:04:08.031455 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:08.036981 (+  5526us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:08.115378 (+ 78397us) log.cc:844] Serialized 14 byte log entry
0709 13:04:08.888928 (+773550us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:08.889506 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 13:04:08.890037 (+   531us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:08.890088 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.890184 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:09.023184 (+133000us) raft_consensus.cc:1685] finished
0709 13:04:09.023205 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.023385 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28359,"prepare.queue_time_us":82708,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":44,"raft.queue_time_us":138222,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":29370,"thread_start_us":673480,"threads_started":2,"wal-append.queue_time_us":471273}
I20260709 13:04:09.028683  1538 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:09.038218  6612 log.cc:927] Time spent T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:04:09.043095  6494 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:08.988814  6545 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.044090  6545 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:09.048043  6545 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:09.049074  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2296) took 1345 ms. Trace:
I20260709 13:04:09.049616  1571 rpcz_store.cc:276] 0709 13:04:07.703974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.120042 (+416068us) service_pool.cc:224] Handling call
0709 13:04:08.179158 (+ 59116us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:08.179377 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:08.179402 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:08.239499 (+ 60097us) log.cc:844] Serialized 26 byte log entry
0709 13:04:08.667661 (+428162us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:08.667710 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.667790 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:09.048775 (+380985us) raft_consensus.cc:1685] finished
0709 13:04:09.048816 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.048999 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":369551,"threads_started":1,"wal-append.queue_time_us":550332}
I20260709 13:04:09.032838  1612 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:09.028215  6629 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:09.052608  6629 raft_consensus.cc:697] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:09.052786  6633 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.053570  6633 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:09.070480  2314 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.743s (SENT)
I20260709 13:04:09.071472  2314 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:09.074072  6616 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.057983  6633 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:09.043509  6286 consensus_meta.cc:287] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.770s	user 0.010s	sys 0.005s
I20260709 13:04:09.056371  6508 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:09.058132  6617 raft_consensus.cc:2764] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:04:09.085448  1612 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:09.085785  1572 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 10.
I20260709 13:04:09.091080  1612 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:09.091876  6618 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:09.096626  6287 consensus_meta.cc:287] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.571s	user 0.013s	sys 0.000s
I20260709 13:04:09.097154  6287 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:09.079434  6616 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:09.082093  6524 consensus_queue.cc:260] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:09.094383  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2285) took 1300 ms. Trace:
I20260709 13:04:09.098425  1718 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:09.077802  6494 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:09.099493  6287 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:09.089208  6286 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:09.102648  6297 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:09.101504  6584 consensus_meta.cc:287] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.551s	user 0.006s	sys 0.003s
I20260709 13:04:09.103971  6569 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:09.106164  6618 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:09.107784  6234 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:09.054754  6629 consensus_queue.cc:237] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:09.124241  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.111564  1566 rpcz_store.cc:276] 0709 13:04:07.794306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.420160 (+625854us) service_pool.cc:224] Handling call
0709 13:04:08.458117 (+ 37957us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:08.524867 (+ 66750us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:08.565424 (+ 40557us) log.cc:844] Serialized 14 byte log entry
0709 13:04:08.865482 (+300058us) log.cc:844] Serialized 14 byte log entry
0709 13:04:08.881885 (+ 16403us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:08.903338 (+ 21453us) log.cc:844] Serialized 26 byte log entry
0709 13:04:08.903489 (+   151us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:08.903524 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.093331 (+189807us) spinlock_profiling.cc:243] Waited 1.14 ms on lock 0x7b18002dab40. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d662a6 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419
0709 13:04:09.093993 (+   662us) spinlock_profiling.cc:243] Waited 33 ms on lock 0x7b54004065d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:09.094045 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:09.094096 (+    51us) raft_consensus.cc:1685] finished
0709 13:04:09.094111 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.094303 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58203,"prepare.queue_time_us":54038,"prepare.run_cpu_time_us":123,"prepare.run_wall_time_us":45042,"raft.queue_time_us":55771,"raft.run_cpu_time_us":520,"raft.run_wall_time_us":35377,"spinlock_wait_cycles":94689536,"thread_start_us":163683,"threads_started":1,"wal-append.queue_time_us":376712}
I20260709 13:04:09.109382  6616 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:09.108891  1612 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:09.110898  6524 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:09.127403  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 187046us
I20260709 13:04:09.128773  1612 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:09.129709  1685 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:09.134618  6546 raft_consensus.cc:3055] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:09.135128  6546 raft_consensus.cc:740] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:09.136148  6546 consensus_queue.cc:260] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:09.122845  6584 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:09.139281  1648 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:09.140131  6286 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:09.127460  6234 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:09.142164  6622 log.cc:927] Time spent T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
W20260709 13:04:09.131844  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 210067us
I20260709 13:04:09.145505  1687 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:09.137418  6546 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:09.150955  1567 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:04:09.156427  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.157006  1647 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:09.159839  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.143496  6616 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:04:09.160634  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 241513us
I20260709 13:04:09.166585  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:09.166958  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:09.167470  6614 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.197s	user 0.003s	sys 0.000s
I20260709 13:04:09.167896  1642 raft_consensus.cc:2393] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:09.166957  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:09.162961  6543 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:09.171084  6566 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:09.171546  6543 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:09.165481  6584 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:09.165087  1687 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:09.174901  1685 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:09.181674  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:09.183437  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.184005  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.184217  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:09.184579  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.186056  1644 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:09.186538  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.186657  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.187948  1641 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 11: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:09.189029  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.186954  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:09.190376  1644 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:09.190883  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:09.192910  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:09.193161  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.202534  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:09.182483  2347 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:09.194016  1646 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 12.
I20260709 13:04:09.193967  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:09.206251  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.207479  1642 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 14: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:09.205873  1640 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 11: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:09.162662  6616 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:09.213582  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.214601  1640 raft_consensus.cc:2393] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:09.215539  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.216249  1640 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:09.217077  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.218179  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.219193  1644 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 12.
I20260709 13:04:09.174794  6287 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:09.220149  6287 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.220227  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.224460  6539 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:09.225109  6539 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:09.228808  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2250) took 1052 ms. Trace:
I20260709 13:04:09.209347  6201 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:09.230312  6201 raft_consensus.cc:697] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:04:09.231444  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 66445.3us
I20260709 13:04:09.231365  6201 consensus_queue.cc:237] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:09.221514  6616 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:09.234364  2343 pending_rounds.cc:85] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Aborting all ops after (but not including) 9
I20260709 13:04:09.234726  2343 pending_rounds.cc:107] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:04:09.235008  2343 raft_consensus.cc:2889] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:09.238615  6286 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:09.220752  6287 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:09.240273  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 488) took 1024 ms. Trace:
I20260709 13:04:09.244491  6396 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:09.245087  6396 raft_consensus.cc:697] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:09.245970  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.246944  1642 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:09.246042  6396 consensus_queue.cc:237] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:09.247902  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:09.243992  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 69174.8us
I20260709 13:04:09.249090  1642 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:09.250662  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.251271  1642 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:04:09.252177  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.252847  1642 raft_consensus.cc:2393] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:09.258616  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2644) took 1388 ms. Trace:
I20260709 13:04:09.227684  6539 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:09.259194  1719 rpcz_store.cc:276] 0709 13:04:07.870706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.871430 (+   724us) service_pool.cc:224] Handling call
0709 13:04:07.894755 (+ 23325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.037576 (+1142821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.258333 (+220757us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540029a9d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:09.258541 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2626811264}
I20260709 13:04:09.240978  1718 rpcz_store.cc:276] 0709 13:04:08.215676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.223816 (+  8140us) service_pool.cc:224] Handling call
0709 13:04:08.248797 (+ 24981us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.115155 (+866358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.240008 (+124853us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b5400213254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:09.240202 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1953516800}
W20260709 13:04:09.263089  6602 log.cc:927] Time spent T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.766s	user 0.002s	sys 0.000s
I20260709 13:04:09.264802  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2238) took 1511 ms. Trace:
W20260709 13:04:09.265491  6596 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.768s	user 0.000s	sys 0.004s
I20260709 13:04:09.265439  1645 rpcz_store.cc:276] 0709 13:04:07.753843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.770003 (+ 16160us) service_pool.cc:224] Handling call
0709 13:04:07.815218 (+ 45215us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:07.815647 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:07.815683 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:07.818771 (+  3088us) log.cc:844] Serialized 26 byte log entry
0709 13:04:07.908265 (+ 89494us) spinlock_profiling.cc:243] Waited 1.08 ms on lock 0x7b1800262060. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a250d1 00007f8248e40a0f
0709 13:04:08.249485 (+341220us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:08.249544 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.249686 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:09.262349 (+1012663us) spinlock_profiling.cc:243] Waited 7.73 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:09.264329 (+  1980us) raft_consensus.cc:1685] finished
0709 13:04:09.264362 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.264576 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":31088,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":5884,"spinlock_wait_cycles":20258944,"thread_start_us":340977,"threads_started":1,"wal-append.queue_time_us":643450}
I20260709 13:04:09.267258  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2229) took 1553 ms. Trace:
I20260709 13:04:09.267722  1639 rpcz_store.cc:276] 0709 13:04:07.713493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.713666 (+   173us) service_pool.cc:224] Handling call
0709 13:04:07.714207 (+   541us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:07.729196 (+ 14989us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:07.729229 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:07.742208 (+ 12979us) log.cc:844] Serialized 26 byte log entry
0709 13:04:08.240054 (+497846us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:08.240121 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.240803 (+   682us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:09.264819 (+1024016us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:09.266918 (+  2099us) raft_consensus.cc:1685] finished
0709 13:04:09.266941 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.267191 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3895,"raft.queue_time_us":8745,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":5211,"spinlock_wait_cycles":49274240,"thread_start_us":446803,"threads_started":1,"wal-append.queue_time_us":739227}
I20260709 13:04:09.269651  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.270855  1645 raft_consensus.cc:2410] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:04:09.229259  1648 rpcz_store.cc:276] 0709 13:04:08.176524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.366600 (+190076us) service_pool.cc:224] Handling call
0709 13:04:08.432122 (+ 65522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.139627 (+707505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.228467 (+ 88840us) spinlock_profiling.cc:243] Waited 707 ms on lock 0x7b54000a5fd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:09.228739 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1626009344}
I20260709 13:04:09.271718  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.271893  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:09.274081  2314 leader_election.cc:336] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.138s (SENT)
I20260709 13:04:09.284595  5852 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.285992  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2287) took 1425 ms. Trace:
I20260709 13:04:09.285187  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 770) took 1532 ms. Trace:
I20260709 13:04:09.286378  1569 rpcz_store.cc:276] 0709 13:04:07.861142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.455181 (+594039us) service_pool.cc:224] Handling call
0709 13:04:08.455570 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:08.455833 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:08.456221 (+   388us) log.cc:844] Serialized 14 byte log entry
0709 13:04:09.285571 (+829350us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:09.285629 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:09.285713 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.285797 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.285933 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":356726,"prepare.queue_time_us":653,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":62,"thread_start_us":472149,"threads_started":1,"wal-append.queue_time_us":563568}
I20260709 13:04:09.294697  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.295513  1644 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:09.286904  1570 rpcz_store.cc:276] 0709 13:04:07.752773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.304236 (+551463us) service_pool.cc:224] Handling call
0709 13:04:08.304507 (+   271us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:08.424609 (+120102us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:08.430064 (+  5455us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:08.471521 (+ 41457us) log.cc:844] Serialized 52 byte log entry
0709 13:04:08.873816 (+402295us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:08.905526 (+ 31710us) log.cc:844] Serialized 14 byte log entry
0709 13:04:09.284112 (+378586us) spinlock_profiling.cc:243] Waited 4.93 ms on lock 0x7b180039c940. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d12c0a 00007f824f452219 00007f8245d9aa40 00007f8245d954e4 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287 00007f8248ea0208
0709 13:04:09.284455 (+   343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.284585 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:09.284662 (+    77us) raft_consensus.cc:1685] finished
0709 13:04:09.284811 (+   149us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.285078 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82913,"prepare.queue_time_us":31464,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":4414,"raft.queue_time_us":473471,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":20867,"spinlock_wait_cycles":11349888,"thread_start_us":333423,"threads_started":1,"wal-append.queue_time_us":481042}
I20260709 13:04:09.297621  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:09.298308  1646 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:09.298368  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.285209  5852 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:09.372431  6297 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:09.389851  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 115825us
W20260709 13:04:09.391258  2314 leader_election.cc:336] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.158s (SENT)
I20260709 13:04:09.391692  2314 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:09.389370  6528 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:09.450250  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 436318us
I20260709 13:04:09.448485  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.452894  1648 raft_consensus.cc:2410] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 13:04:09.460394  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 359794us
W20260709 13:04:09.460636  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:09.461166  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.461261  1537 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:09.461865  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.462709  6508 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:04:09.464476  1538 leader_election.cc:336] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.765s (SENT)
W20260709 13:04:09.470360  1538 leader_election.cc:336] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.778s (SENT)
I20260709 13:04:09.470831  1538 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:09.472451  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.473855  6268 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:09.474902  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:09.475502  1639 raft_consensus.cc:2376] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11. Re-sending same reply.
I20260709 13:04:09.476480  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:09.482362  6637 log.cc:927] Time spent T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:04:09.451074  6286 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.487044  6286 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:09.489156  6286 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:09.475564  6268 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.490460  6531 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:09.490396  6268 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:09.490917  6531 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:09.492255  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:09.477560  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.493999  1639 raft_consensus.cc:2410] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:04:09.497584  1646 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:04:09.498955  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.499714  1646 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
W20260709 13:04:09.502348  1538 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.837s (SENT)
I20260709 13:04:09.503513  1640 raft_consensus.cc:2410] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:04:09.505884  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 114690us
I20260709 13:04:09.492480  1537 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:09.527189  2312 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:09.528499  1537 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:09.511272  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.531484  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.512861  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2243) took 1730 ms. Trace:
I20260709 13:04:09.509348  1686 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:09.532750  1643 rpcz_store.cc:276] 0709 13:04:07.782182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.040780 (+258598us) service_pool.cc:224] Handling call
0709 13:04:09.512557 (+1471777us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b54003566d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:09.512780 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":548930816}
W20260709 13:04:09.518546  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.524094  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:09.518281  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:09.535194  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.535557  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:09.536015  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.536811  1644 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 14: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:09.538244  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.538439  1570 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:04:09.539746  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:09.540355  1644 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:09.544094  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.547642  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:09.552589  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:09.525570  1538 leader_election.cc:336] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.248s (SENT)
I20260709 13:04:09.555610  1538 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:09.559782  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:09.560367  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 51140.3us
I20260709 13:04:09.568645  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.572793  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:09.574752  1648 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:09.582154  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.582921  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:09.501210  6531 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:09.587749  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:09.521908  6338 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:09.604732  1564 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:04:09.606245  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:09.606869  1564 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:09.561766  6161 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:09.610172  6161 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:09.610746  6161 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:09.612782  6268 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:09.615449  5852 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:09.615723  1641 raft_consensus.cc:1275] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:04:09.617292  1641 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:04:09.560961  6453 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:09.494912  6528 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:09.620906  6528 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:09.544273  6539 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.622011  6539 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:09.522344  6569 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:09.598605  1640 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:09.588351  6634 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:09.588083  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:09.581956  6636 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:09.600771  6287 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:09.520143  6349 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.603050  1571 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:04:09.645344  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:09.561261  6297 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:04:09.544548  6508 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:09.603452  6621 consensus_queue.cc:1059] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:04:09.589074  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.589509  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:09.583892  1566 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:04:09.652009  1642 raft_consensus.cc:2393] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:09.590873  1714 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:04:09.592602  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:09.654719  1570 raft_consensus.cc:2410] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:04:09.594445  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:09.623747  1647 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:04:09.657760  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:09.653023  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.660974  1641 raft_consensus.cc:2410] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:04:09.655948  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:09.597734  6560 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:04:09.632148  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 399393us
I20260709 13:04:09.600090  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.648732  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:09.648890  1644 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:09.667176  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:09.668887  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:09.669834  1566 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:09.523128  6551 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:09.663298  1640 raft_consensus.cc:2393] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:09.643464  6524 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:09.669946  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.650206  1572 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:04:09.671442  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:09.672621  1639 raft_consensus.cc:2393] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 12: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:09.664395  1648 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:09.679936  1567 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:09.680398  1643 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:09.559734  6616 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:09.676266  1644 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11. Ops: [7.5-10.7]
I20260709 13:04:09.647444  6636 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.665863  6453 raft_consensus.cc:697] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:09.711102  2314 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.513s (SENT) [suppressed 47 similar messages]
I20260709 13:04:09.711735  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:09.712646  1568 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:09.716944  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:09.526952  6271 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.657176  6234 consensus_queue.cc:1059] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:04:09.677125  6508 raft_consensus.cc:3055] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:09.710142  6636 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:09.755436  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:09.757315  6508 raft_consensus.cc:740] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:09.757398  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:09.758571  6508 consensus_queue.cc:260] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:09.759719  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:09.759966  6508 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:09.684973  6560 raft_consensus.cc:697] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:09.762897  6560 consensus_queue.cc:237] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:09.763298  6646 consensus_queue.cc:1059] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:09.710386  6566 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:04:09.780102  1564 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:09.781549  1645 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:09.656739  6349 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:09.682335  6634 raft_consensus.cc:697] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:09.683414  6620 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:09.805809  1572 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:04:09.808568  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.809394  1639 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:09.809461  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:09.810518  1573 raft_consensus.cc:1275] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:04:09.810968  1573 pending_rounds.cc:85] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 6
I20260709 13:04:09.811368  1573 pending_rounds.cc:107] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:04:09.811780  1573 raft_consensus.cc:2889] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:09.824687  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:09.825814  1572 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 12.
I20260709 13:04:09.838330  1567 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
W20260709 13:04:09.844000  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 44524.3us
I20260709 13:04:09.855966  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:09.838685  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 277768us
W20260709 13:04:09.869864  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.857069  1645 raft_consensus.cc:2376] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11. Re-sending same reply.
W20260709 13:04:09.841892  6608 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.042s	sys 0.108s
W20260709 13:04:09.875758  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 282656us
I20260709 13:04:09.768841  6224 consensus_queue.cc:1059] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 13:04:09.786696  6325 consensus_queue.cc:1059] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.080s
I20260709 13:04:09.889266  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.892549  1645 raft_consensus.cc:2393] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:04:09.892845  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.897678  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.899062  1645 raft_consensus.cc:2410] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 13:04:09.903875  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 344611us
I20260709 13:04:09.807583  6271 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:09.908329  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:09.918617  1648 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:09.912014  1641 raft_consensus.cc:1275] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:04:09.915833  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.882481  6629 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:09.891105  6608 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.043s	sys 0.109s
I20260709 13:04:09.882128  6569 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.924222  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.937551  1641 raft_consensus.cc:2410] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:04:09.935297  1644 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:04:09.942670  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.943539  1643 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:09.903187  6338 raft_consensus.cc:3055] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:09.886703  6524 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:04:09.968489  1648 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:09.974265  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:09.983980  1564 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:09.981982  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.920697  6624 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:09.989940  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.975908  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:09.995786  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.990759  1640 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:09.964720  6338 raft_consensus.cc:740] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:10.005991  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:10.010520  1717 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:10.014070  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.987084  6584 raft_consensus.cc:3055] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:09.987468  6268 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:09.978770  6287 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:10.017354  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:10.018293  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.965075  6569 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:10.024961  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:09.903493  6453 consensus_queue.cc:237] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:10.011261  6630 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.046919  6268 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:10.025267  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:10.015982  6201 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.022s	sys 0.039s
W20260709 13:04:10.054235  6201 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.023s	sys 0.040s
I20260709 13:04:10.001153  6531 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:10.016848  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:10.033840  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.038702  1571 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
W20260709 13:04:10.040252  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 152723us
I20260709 13:04:09.988063  6545 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:04:10.020149  1640 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:09.933504  6629 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:10.000097  6349 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:10.046000  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:10.046553  1641 raft_consensus.cc:2376] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11. Re-sending same reply.
I20260709 13:04:10.060593  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:10.085390  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:10.063376  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 50374.2us
I20260709 13:04:10.025980  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:10.080433  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:10.113196  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.024603  6584 raft_consensus.cc:740] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:10.024519  6539 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:10.138176  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:10.129889  1639 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:10.127703  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:10.129895  1647 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:10.130205  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:10.150990  6338 consensus_queue.cc:260] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:10.151870  6287 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:10.152338  6338 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:10.152719  1719 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
W20260709 13:04:10.153864  6641 log.cc:927] Time spent T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:04:10.058416  6616 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:10.147786  2312 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:10.163453  1564 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
W20260709 13:04:10.143409  6644 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.352s	user 0.002s	sys 0.000s
I20260709 13:04:10.042677  6657 consensus_queue.cc:1059] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.140s
I20260709 13:04:10.144796  6646 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:04:10.189836  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 130894us
W20260709 13:04:10.227397  1568 consensus_meta.cc:287] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.511s	user 0.012s	sys 0.001s
I20260709 13:04:10.228291  1568 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:10.231773  1685 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:10.206797  6643 log.cc:927] Time spent T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:04:10.238598  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:10.241837  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:10.234444  6657 consensus_queue.cc:1243] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer 066ee0109aba4465ba17fa65e14ef504 log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:10.234664  6325 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:10.261772  1538 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.000s	sys 0.004s
W20260709 13:04:10.262202  1538 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.000s	sys 0.004s
W20260709 13:04:10.260006  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 112337us
I20260709 13:04:10.264489  1685 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:10.266130  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:10.254412  6647 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.328s	user 0.000s	sys 0.001s
I20260709 13:04:10.278398  1643 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:04:10.253752  6524 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.249995  6271 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:10.285763  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 760251us
I20260709 13:04:10.287672  6674 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:10.298487  6674 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:10.299005  6674 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:10.300470  6545 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:10.300961  6508 consensus_meta.cc:287] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.541s	user 0.010s	sys 0.002s
I20260709 13:04:10.301056  6545 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.257792  6494 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
W20260709 13:04:10.271916  6161 consensus_meta.cc:287] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.661s	user 0.015s	sys 0.000s
I20260709 13:04:10.215134  6584 consensus_queue.cc:260] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:10.269397  6629 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:10.337860  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 71874.4us
W20260709 13:04:10.347911  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.367515  1685 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:10.338021  6545 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:10.374294  1685 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:10.267859  6325 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:10.348840  6650 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:10.362542  6584 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:10.394567  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 106218us
I20260709 13:04:10.305992  6524 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:10.386090  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.401201  2343 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 614) took 1159 ms. Trace:
W20260709 13:04:10.400601  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 136959us
W20260709 13:04:10.416343  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.418803  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:10.420007  1567 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:10.421504  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:10.423838  1567 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:10.408815  2343 rpcz_store.cc:276] 0709 13:04:09.233473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.233687 (+   214us) service_pool.cc:224] Handling call
0709 13:04:09.234127 (+   440us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:09.235901 (+  1774us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.235933 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:09.236513 (+   580us) log.cc:844] Serialized 52 byte log entry
0709 13:04:09.522631 (+286118us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:09.522702 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.522815 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.385199 (+862384us) raft_consensus.cc:1685] finished
0709 13:04:10.392040 (+  6841us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.392297 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4957,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"thread_start_us":285574,"threads_started":1,"wal-append.queue_time_us":529252}
I20260709 13:04:10.385367  6569 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:10.396167  6629 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:10.408226  6667 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:04:10.459576  6672 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:04:10.461201  1538 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.839s (SENT)
I20260709 13:04:10.461119  6271 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:10.445165  6642 log.cc:927] Time spent T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 13:04:10.467795  1538 leader_election.cc:336] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.969s (SENT)
W20260709 13:04:10.474407  1538 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.990s (SENT)
I20260709 13:04:10.474875  1538 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:10.476409  1538 leader_election.cc:336] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.056s (SENT)
I20260709 13:04:10.476984  1538 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:10.347020  6161 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:10.478509  1538 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.089s (SENT)
I20260709 13:04:10.479341  1538 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:10.478818  6667 raft_consensus.cc:697] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:10.475970  1567 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:10.486841  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:10.490376  1567 raft_consensus.cc:2393] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 15: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:10.480337  1647 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:10.491748  6686 consensus_queue.cc:1059] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:10.492733  6621 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:09.683434  6551 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:10.488745  5852 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:10.480055  6349 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:10.476647  6286 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.479415  6271 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:10.498086  6629 raft_consensus.cc:697] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:10.052891  6630 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:10.523605  6629 consensus_queue.cc:237] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:10.482656  6667 consensus_queue.cc:237] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:10.525044  6662 log.cc:927] Time spent T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.285s	user 0.003s	sys 0.000s
W20260709 13:04:10.527122  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.527779  6494 raft_consensus.cc:3055] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:10.528837  6494 raft_consensus.cc:740] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:10.522600  6286 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:10.529731  6494 consensus_queue.cc:260] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:10.535104  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 490) took 1349 ms. Trace:
I20260709 13:04:10.535830  1646 rpcz_store.cc:276] 0709 13:04:09.185578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.560819 (+375241us) service_pool.cc:224] Handling call
0709 13:04:09.561117 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:09.630200 (+ 69083us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.630253 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:09.645484 (+ 15231us) log.cc:844] Serialized 26 byte log entry
0709 13:04:09.921967 (+276483us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:09.922030 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.922123 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.526357 (+604234us) raft_consensus.cc:1685] finished
0709 13:04:10.526384 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.535027 (+  8643us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8280,"raft.queue_time_us":45046,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":13816,"thread_start_us":206441,"threads_started":1,"wal-append.queue_time_us":343676}
W20260709 13:04:10.513566  1538 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.156s (SENT)
I20260709 13:04:10.230607  6646 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:09.978099  6634 consensus_queue.cc:237] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:09.842010  6620 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:10.521837  1566 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:10.000874  6624 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:10.538923  1538 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:10.539587  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:10.539690  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:10.564018  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.564261  1647 raft_consensus.cc:1275] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:10.540045  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:10.566752  1646 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
W20260709 13:04:10.540439  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:10.541896  1714 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.028s	sys 0.092s
W20260709 13:04:10.568655  1714 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.028s	sys 0.092s
W20260709 13:04:10.570362  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.542846  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:10.543361  1564 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:04:10.537590  6664 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.362s	user 0.005s	sys 0.000s
I20260709 13:04:10.570444  6610 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:10.580806  6161 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:10.572223  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:10.584280  1567 raft_consensus.cc:1275] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:04:10.584704  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.573922  1648 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
W20260709 13:04:10.574703  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:10.586674  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:10.577256  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.579618  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:10.589903  1564 raft_consensus.cc:2410] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:04:10.591840  2346 raft_consensus.cc:2393] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 15: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:10.577754  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2295) took 1001 ms. Trace:
I20260709 13:04:10.594409  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:10.592921  1570 rpcz_store.cc:276] 0709 13:04:09.575938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.669862 (+ 93924us) service_pool.cc:224] Handling call
0709 13:04:09.670341 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:09.677097 (+  6756us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.677134 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:09.716154 (+ 39020us) log.cc:844] Serialized 26 byte log entry
0709 13:04:09.896638 (+180484us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:09.896690 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.896773 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.496779 (+600006us) spinlock_profiling.cc:243] Waited 4.42 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:10.577358 (+ 80579us) raft_consensus.cc:1685] finished
0709 13:04:10.577384 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.577667 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6171,"raft.queue_time_us":6297,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":28607,"spinlock_wait_cycles":10173696,"thread_start_us":88197,"threads_started":1,"wal-append.queue_time_us":281270}
I20260709 13:04:10.595052  1646 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:10.549075  6349 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:10.604560  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:10.610236  1567 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:04:10.614456  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:10.612525  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 99009us
W20260709 13:04:10.616884  1538 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.175s (SENT)
W20260709 13:04:10.619102  1538 leader_election.cc:336] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.207s (SENT)
I20260709 13:04:10.619778  1538 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:10.593745  6677 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.629824  1646 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
W20260709 13:04:10.639278  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.621562  6349 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:10.608198  6161 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:10.642843  1643 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:10.673275  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:10.678658  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:10.680577  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:10.625048  6638 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.010s	sys 0.005s
I20260709 13:04:10.686643  1640 raft_consensus.cc:1240] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:04:10.688349  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:10.689821  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:10.690781  1642 raft_consensus.cc:2393] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:10.642274  6508 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:10.692153  1641 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
W20260709 13:04:10.705003  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 295838us
I20260709 13:04:10.706147  2312 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:10.666178  6161 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.641556  6620 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:10.711309  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:10.711490  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:10.722857  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:10.700986  6638 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.010s	sys 0.006s
I20260709 13:04:10.724244  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:10.741566  1611 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:10.730238  1646 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:04:10.722690  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:10.779881  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:10.781261  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:10.782019  1645 raft_consensus.cc:2376] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11. Re-sending same reply.
I20260709 13:04:10.785199  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:10.800257  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:10.745296  6560 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:04:10.808955  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 189910us
I20260709 13:04:10.807796  1645 raft_consensus.cc:2393] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 12: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:04:10.818269  1538 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.341s (SENT)
I20260709 13:04:10.776497  6533 raft_consensus.cc:3055] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:10.822116  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:10.823623  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:10.788686  6659 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:04:10.830819  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.742360  6161 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:10.853046  1641 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:10.856734  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:10.674985  6677 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:10.837009  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:10.837778  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.871344  1642 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:04:10.778952  6234 consensus_queue.cc:1059] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:04:10.851588  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:10.797322  6687 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:10.863404  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 90708.1us
I20260709 13:04:10.823951  6560 raft_consensus.cc:697] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 15 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:10.874797  1645 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:04:10.835924  6224 raft_consensus.cc:3055] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:10.878468  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:10.874629  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 190049us
I20260709 13:04:10.834440  6349 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:10.881903  1538 leader_election.cc:336] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.887s (SENT)
I20260709 13:04:10.888945  1538 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:10.888708  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:10.840780  6533 raft_consensus.cc:740] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:10.885964  1639 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:04:10.788436  6361 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 13:04:10.853341  6659 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:10.936743  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.898123  6634 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:10.941780  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:10.942721  1569 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:10.942425  6533 consensus_queue.cc:260] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:10.943645  6533 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:10.944351  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:10.945230  1642 raft_consensus.cc:2410] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:04:10.943588  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:10.946096  1569 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:04:10.891037  6687 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:10.949055  6677 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:10.949115  6687 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:10.954927  6700 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
I20260709 13:04:10.955157  6268 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:10.956578  6688 log.cc:927] Time spent T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:04:10.960857  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 782) took 1020 ms. Trace:
W20260709 13:04:10.962252  6699 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:04:10.961450  1572 rpcz_store.cc:276] 0709 13:04:09.940540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.940834 (+   294us) service_pool.cc:224] Handling call
0709 13:04:09.941235 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.601937 (+660702us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:04:10.723935 (+121998us) log.cc:844] Serialized 14 byte log entry
0709 13:04:10.864411 (+140476us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:10.864490 (+    79us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:10.864553 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.959847 (+ 95294us) spinlock_profiling.cc:243] Waited 9.75 ms on lock 0x7b1800280f00. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d662a6 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419
0709 13:04:10.960503 (+   656us) spinlock_profiling.cc:243] Waited 660 ms on lock 0x7b54001b5654. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:10.960560 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.960765 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11559,"prepare.queue_time_us":76597,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":1101,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":10740,"spinlock_wait_cycles":1541477120,"thread_start_us":249076,"threads_started":2}
I20260709 13:04:10.964903  1571 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:04:10.965852  1571 pending_rounds.cc:85] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 5
I20260709 13:04:10.966180  1571 pending_rounds.cc:107] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:04:10.966485  1571 raft_consensus.cc:2889] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:10.970008  6161 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:10.970109  6551 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:10.970044  1572 raft_consensus.cc:1240] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: []
W20260709 13:04:10.977449  6636 consensus_meta.cc:287] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.182s	user 0.012s	sys 0.000s
I20260709 13:04:11.035861  6636 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:10.980656  6659 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:10.979856  6693 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:11.047109  6693 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:10.981900  1714 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 13:04:10.983889  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.053712  2314 leader_election.cc:336] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.129s (SENT)
W20260709 13:04:10.985805  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.058425  6636 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:10.987052  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:10.994048  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.062167  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 124004us
I20260709 13:04:11.065770  6551 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:11.068579  1685 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:11.073604  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:11.076709  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:11.077746  1718 raft_consensus.cc:2410] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:04:11.079406  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.081856  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:11.082765  1714 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:11.091143  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.095121  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:11.099156  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:11.101372  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:11.102326  1713 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:11.104192  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:11.105130  1713 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 19: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:04:11.107148  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.108902  6693 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:11.025843  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:11.111088  1564 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:04:11.112880  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:11.123621  1687 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:10.885238  6224 raft_consensus.cc:740] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:10.891166  6646 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:11.136745  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:10.895679  6325 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:11.147320  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 78949.2us
I20260709 13:04:11.148703  1685 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:11.139588  1538 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.545s (SENT)
I20260709 13:04:11.156164  1538 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:11.153205  1685 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:11.176435  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.145752  6674 consensus_meta.cc:287] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.846s	user 0.008s	sys 0.008s
I20260709 13:04:11.182883  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:11.183254  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:11.187868  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:11.189324  1564 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:04:11.190641  6674 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:11.192116  6697 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:11.193176  6697 raft_consensus.cc:697] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:11.144202  6633 raft_consensus.cc:2764] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:04:11.194355  6697 consensus_queue.cc:237] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:11.182583  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 56952.8us
I20260709 13:04:11.196717  1687 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:11.176138  6636 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:11.198364  6636 raft_consensus.cc:697] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:11.199560  6636 consensus_queue.cc:237] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:11.201133  6271 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:11.206774  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.204694  1718 raft_consensus.cc:2955] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Committing config change with OpId 10.9: config changed from index 6 to 9, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:04:11.179935  6338 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 1.027s	user 0.015s	sys 0.002s
I20260709 13:04:11.232524  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:11.208776  6646 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:11.371017  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.144930  6624 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:11.182531  6629 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:11.422574  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.373281  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 233749us
W20260709 13:04:11.399920  6709 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.429s	user 0.000s	sys 0.001s
I20260709 13:04:11.400730  1570 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:11.147912  6561 consensus_queue.cc:1048] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.212s
I20260709 13:04:11.425546  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2289) took 1851 ms. Trace:
I20260709 13:04:11.426057  1573 rpcz_store.cc:276] 0709 13:04:09.573745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.589722 (+ 15977us) service_pool.cc:224] Handling call
0709 13:04:09.602212 (+ 12490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.846831 (+244619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.425282 (+1578451us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b54002b8c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:11.425477 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":478290688}
W20260709 13:04:11.429592  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 219087us
W20260709 13:04:11.443683  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.368768  6551 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:04:11.489324  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.494170  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.516999  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.521857  1648 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.030s	sys 0.095s
W20260709 13:04:11.588510  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.564322  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.434984  6667 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.003s	sys 0.016s
W20260709 13:04:11.626957  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.649121  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 450625us
I20260709 13:04:11.750898  1687 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:11.358982  6633 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.241s
I20260709 13:04:11.753572  6633 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:11.754494  1687 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:11.418907  6629 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:11.729918  6533 consensus_meta.cc:287] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.774s	user 0.007s	sys 0.007s
I20260709 13:04:11.768759  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:11.559978  1571 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.006s	sys 0.008s
W20260709 13:04:11.589092  1648 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.031s	sys 0.096s
W20260709 13:04:11.776561  1571 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.007s	sys 0.009s
I20260709 13:04:11.777388  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2274) took 1176 ms. Trace:
I20260709 13:04:11.772372  6633 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:11.780510  1713 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:11.578099  1714 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:11.507797  6338 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:11.521559  6271 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:11.757613  6659 consensus_meta.cc:287] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.720s	user 0.011s	sys 0.002s
I20260709 13:04:11.202317  6560 consensus_queue.cc:237] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:11.628274  6667 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.005s	sys 0.016s
W20260709 13:04:11.615346  6234 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.002s	sys 0.013s
W20260709 13:04:11.710510  1685 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.034s	sys 0.057s
I20260709 13:04:11.720556  6706 consensus_queue.cc:1059] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:04:11.727205  6524 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:11.878690  6234 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.979s	user 0.002s	sys 0.014s
I20260709 13:04:11.671306  6674 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:11.734428  6618 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:11.681548  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 52476us
W20260709 13:04:11.751541  6648 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 1.886s	user 0.005s	sys 0.011s
W20260709 13:04:11.585500  1572 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.004s	sys 0.011s
I20260709 13:04:11.759624  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2321) took 1209 ms. Trace:
W20260709 13:04:11.907006  1572 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.004s	sys 0.012s
I20260709 13:04:11.770826  1722 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:11.907130  1568 rpcz_store.cc:276] 0709 13:04:10.550155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.624709 (+ 74554us) service_pool.cc:224] Handling call
0709 13:04:10.625091 (+   382us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:10.715054 (+ 89963us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:10.715099 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:10.715866 (+   767us) log.cc:844] Serialized 52 byte log entry
0709 13:04:11.146885 (+431019us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:11.147031 (+   146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.147267 (+   236us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:11.759235 (+611968us) raft_consensus.cc:1685] finished
0709 13:04:11.759271 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:11.759536 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16093,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"thread_start_us":506860,"threads_started":2,"wal-append.queue_time_us":876823}
W20260709 13:04:11.560631  1611 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.003s	sys 0.016s
I20260709 13:04:11.909852  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2322) took 1247 ms. Trace:
W20260709 13:04:11.909909  1611 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.018s	user 0.004s	sys 0.016s
I20260709 13:04:11.787091  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:11.680012  2314 leader_election.cc:336] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.513s (SENT)
I20260709 13:04:11.910291  1572 rpcz_store.cc:276] 0709 13:04:10.662581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.034390 (+371809us) service_pool.cc:224] Handling call
0709 13:04:11.034774 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:11.035026 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:11.064275 (+ 29249us) log.cc:844] Serialized 14 byte log entry
0709 13:04:11.909490 (+845215us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:11.909540 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:11.909586 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.909678 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:11.909793 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28546,"prepare.queue_time_us":28752,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"thread_start_us":844871,"threads_started":1}
I20260709 13:04:11.857295  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:11.778159  1648 rpcz_store.cc:276] 0709 13:04:10.601367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.603216 (+  1849us) service_pool.cc:224] Handling call
0709 13:04:10.605286 (+  2070us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.605553 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:10.626405 (+ 20852us) log.cc:844] Serialized 14 byte log entry
0709 13:04:10.899359 (+272954us) log.cc:844] Serialized 14 byte log entry
0709 13:04:11.776831 (+877472us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:11.776925 (+    94us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:11.777002 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.777097 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:11.777298 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13021,"prepare.queue_time_us":86568,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":103,"raft.queue_time_us":869601,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":247,"thread_start_us":1084830,"threads_started":2,"wal-append.queue_time_us":569499}
W20260709 13:04:11.911793  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 232012us
I20260709 13:04:11.860558  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:11.615875  6453 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.002s	sys 0.016s
I20260709 13:04:11.913236  1717 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
W20260709 13:04:11.875582  1647 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.194s	user 0.002s	sys 0.026s
W20260709 13:04:11.913966  1647 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.232s	user 0.002s	sys 0.026s
W20260709 13:04:11.910444  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 1.03213e+06us
I20260709 13:04:11.772493  6646 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:11.773622  6629 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:11.879021  1685 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.034s	sys 0.057s
I20260709 13:04:11.780541  6633 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:11.882874  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:11.791843  6584 consensus_meta.cc:287] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.362s	user 0.016s	sys 0.008s
W20260709 13:04:11.884819  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 130542us
I20260709 13:04:11.862480  6297 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:11.933598  1643 raft_consensus.cc:2410] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:04:11.871376  6566 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.914s	user 0.084s	sys 0.141s
I20260709 13:04:11.777866  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2320) took 1119 ms. Trace:
W20260709 13:04:11.869325  1716 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.004s	sys 0.014s
I20260709 13:04:11.914425  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:11.917620  6453 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.003s	sys 0.017s
I20260709 13:04:11.913264  1721 raft_consensus.cc:2393] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 12: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:11.946707  1717 raft_consensus.cc:2368] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11.
W20260709 13:04:11.881335  1715 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.048s	sys 0.107s
W20260709 13:04:11.947415  1715 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.049s	sys 0.107s
W20260709 13:04:11.918023  1614 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.004s	sys 0.021s
I20260709 13:04:11.870087  6271 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:11.862869  6338 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:11.872581  6659 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:11.937865  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:11.876433  1713 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
W20260709 13:04:11.940315  1716 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.004s	sys 0.014s
I20260709 13:04:11.915223  6234 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:11.919885  6718 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:11.920286  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 737065us
I20260709 13:04:11.932421  6616 raft_consensus.cc:3055] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:04:11.948544  1614 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.005s	sys 0.021s
I20260709 13:04:11.949311  1714 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 13: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:11.913347  6618 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:11.935734  6660 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.000s	sys 0.016s
W20260709 13:04:11.938939  6566 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.982s	user 0.085s	sys 0.141s
W20260709 13:04:11.943037  6545 consensus_meta.cc:287] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.556s	user 0.010s	sys 0.008s
I20260709 13:04:11.920190  6551 consensus_queue.cc:1059] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:04:11.948606  1687 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:11.966303  6533 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:11.880223  6522 raft_consensus.cc:2955] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Committing config change with OpId 10.9: config changed from index 6 to 9, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:04:11.942380  6297 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:11.956367  6616 raft_consensus.cc:740] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:11.976389  1646 raft_consensus.cc:2393] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:11.977100  1611 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:11.982249  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.985004  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:11.924494  6646 raft_consensus.cc:697] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:11.994324  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:12.012104  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.012760  1713 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:04:12.013463  1718 consensus_meta.cc:287] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.802s	user 0.008s	sys 0.004s
I20260709 13:04:11.965667  6584 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:11.976799  6660 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.001s	sys 0.016s
W20260709 13:04:12.024943  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.033929  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.035921  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 70675.5us
W20260709 13:04:12.038432  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 50316.3us
W20260709 13:04:12.042484  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 599707us
I20260709 13:04:12.050984  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:12.054251  1537 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:11.965119  6667 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 13:04:12.065256  1538 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:12.019629  6271 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:12.067332  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 777) took 2275 ms. Trace:
I20260709 13:04:11.984922  6528 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:04:12.072209  1566 rpcz_store.cc:276] 0709 13:04:09.792611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.803458 (+ 10847us) service_pool.cc:224] Handling call
0709 13:04:09.803945 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.570968 (+767023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.067034 (+1496066us) spinlock_profiling.cc:243] Waited 717 ms on lock 0x7b54001b6054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:12.067265 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1648698240}
I20260709 13:04:11.956459  6234 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:12.057077  6716 log.cc:927] Time spent T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.146s	user 0.004s	sys 0.001s
I20260709 13:04:11.965445  6718 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:12.059080  6621 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:11.987566  6545 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:12.110514  1719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 491) took 2103 ms (client timeout 1927 ms). Trace:
W20260709 13:04:12.120708  1719 rpcz_store.cc:269] 0709 13:04:09.989492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.989613 (+   121us) service_pool.cc:224] Handling call
0709 13:04:12.092266 (+2102653us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b54000230d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:12.092423 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":335181056}
W20260709 13:04:12.094455  6717 log.cc:927] Time spent T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 13:04:12.095100  6685 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:12.094228  6719 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.137176  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:12.094874  6722 consensus_queue.cc:1059] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.143s
I20260709 13:04:12.043248  6584 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:12.114746  6731 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:12.177109  1719 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 14. Ops: []
I20260709 13:04:12.140059  6633 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:12.147518  6531 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:12.070837  6533 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 13:04:12.131981  6727 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:12.145079  6287 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:12.160084  6621 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.207393  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:12.210425  1713 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:04:12.188808  1719 raft_consensus.cc:1240] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:12.189886  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:12.148377  6685 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:12.156427  6271 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:12.223361  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:12.206107  6727 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:12.225411  6297 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:12.225840  1721 raft_consensus.cc:1275] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:04:12.226469  1721 pending_rounds.cc:85] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 5
I20260709 13:04:12.226747  6727 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:12.226780  1721 pending_rounds.cc:107] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:04:12.184778  6731 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:12.228756  1721 raft_consensus.cc:2889] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:12.231746  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2273) took 1645 ms. Trace:
I20260709 13:04:12.232179  1647 rpcz_store.cc:276] 0709 13:04:10.586003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.586159 (+   156us) service_pool.cc:224] Handling call
0709 13:04:10.586532 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:10.599605 (+ 13073us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:10.599649 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:10.604060 (+  4411us) log.cc:844] Serialized 26 byte log entry
0709 13:04:11.914363 (+1310303us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:11.914405 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.914498 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:12.231531 (+317033us) raft_consensus.cc:1685] finished
0709 13:04:12.231550 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.231684 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12683,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":6688,"thread_start_us":1232334,"threads_started":1,"wal-append.queue_time_us":1486566}
W20260709 13:04:12.234354  6724 log.cc:927] Time spent T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:04:12.223991  6685 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:12.208678  6621 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:12.237193  6621 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:12.160333  6719 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:12.153395  6646 consensus_queue.cc:237] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:12.279712  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:12.223548  6725 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.250s	user 0.001s	sys 0.001s
W20260709 13:04:12.255416  6728 log.cc:927] Time spent T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.113s	user 0.001s	sys 0.001s
I20260709 13:04:12.095852  6616 consensus_queue.cc:260] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:12.273347  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 78087.8us
W20260709 13:04:12.294445  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.303884  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.271643  6618 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:12.303046  1686 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:12.355732  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 52777.5us
I20260709 13:04:12.363997  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2669) took 1291 ms. Trace:
I20260709 13:04:12.364624  1715 rpcz_store.cc:276] 0709 13:04:11.072288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.072609 (+   321us) service_pool.cc:224] Handling call
0709 13:04:11.073096 (+   487us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:11.149472 (+ 76376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:11.149506 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:11.177689 (+ 28183us) log.cc:844] Serialized 52 byte log entry
0709 13:04:11.947831 (+770142us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:11.947874 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.947939 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:12.363625 (+415686us) raft_consensus.cc:1685] finished
0709 13:04:12.363658 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.363914 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75813,"raft.queue_time_us":75957,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":26681,"thread_start_us":752627,"threads_started":1,"wal-append.queue_time_us":871912}
I20260709 13:04:12.345184  6531 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:04:12.358186  6671 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 2.015s	user 0.008s	sys 0.020s
I20260709 13:04:12.358933  6727 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:12.320905  6545 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:12.326079  6616 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:12.326897  6618 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:12.340097  6723 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.283s	user 0.000s	sys 0.002s
I20260709 13:04:12.384090  6618 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:12.385089  6618 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:12.357182  6551 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:12.386027  6551 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:12.380247  6271 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.358860  6297 consensus_queue.cc:1059] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:04:12.393085  6271 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:12.397976  6271 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:12.368043  6736 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:04:12.400701  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 452205us
I20260709 13:04:12.249878  6338 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:12.408386  1687 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:04:12.421550  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.434963  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.420452  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 499903us
I20260709 13:04:12.363112  6734 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:04:12.375993  6531 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:12.435850  6271 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:12.402271  6685 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:12.457952  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 346025us
W20260709 13:04:12.446048  6671 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:04:12.441912  6633 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:04:12.456921  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 391754us
I20260709 13:04:12.434394  6545 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:04:12.433777  6736 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:12.492082  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 147875us
I20260709 13:04:12.502187  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:12.506309  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:12.509279  1572 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:04:12.497673  6722 raft_consensus.cc:3055] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:12.516865  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:12.463753  6271 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:12.520078  6737 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:12.520519  6271 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:12.520799  6737 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:12.522330  6738 log.cc:927] Time spent T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:04:12.516889  6584 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:12.523761  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2679) took 1337 ms. Trace:
I20260709 13:04:12.524130  1716 rpcz_store.cc:276] 0709 13:04:11.186948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.187127 (+   179us) service_pool.cc:224] Handling call
0709 13:04:11.363443 (+176316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:11.949172 (+585729us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:11.949205 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:11.953487 (+  4282us) log.cc:844] Serialized 26 byte log entry
0709 13:04:12.420054 (+466567us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:12.420098 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.420188 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:12.523550 (+103362us) raft_consensus.cc:1685] finished
0709 13:04:12.523567 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.523701 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":62251,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":52194,"thread_start_us":987592,"threads_started":2,"wal-append.queue_time_us":453830}
I20260709 13:04:12.492760  6618 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:12.517534  6722 raft_consensus.cc:740] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:12.532486  6618 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:12.532915  6722 consensus_queue.cc:260] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:12.533773  1714 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:04:12.522338  1566 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:04:12.535192  6685 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:12.535306  6736 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:12.490648  6732 consensus_queue.cc:1059] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:04:12.545909  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:12.546635  6732 raft_consensus.cc:3055] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:12.547171  6732 raft_consensus.cc:740] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:12.506078  6551 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:12.535562  6531 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:12.562511  6737 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:12.544235  6629 consensus_meta.cc:287] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.611s	user 0.007s	sys 0.007s
I20260709 13:04:12.563597  6629 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:12.564911  1720 raft_consensus.cc:2368] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12.
I20260709 13:04:12.517812  1538 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:12.570664  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 53001.8us
I20260709 13:04:12.571799  1538 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:12.537242  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:12.538092  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.543896  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.576058  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.514566  6741 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:12.551424  6732 consensus_queue.cc:260] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:12.577907  6748 consensus_queue.cc:1059] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:12.578565  6732 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:12.579205  6748 raft_consensus.cc:3055] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:12.579579  6748 raft_consensus.cc:740] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:12.580416  6748 consensus_queue.cc:260] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:12.580749  1538 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:12.558495  1612 connection.cc:256] RPC call timeout handler was delayed by -4.40447892200001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17400000000000002s for a response
I20260709 13:04:12.559079  6752 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:12.557528  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 785) took 1832 ms. Trace:
I20260709 13:04:12.578151  6531 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:12.577180  6741 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:12.590107  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.610667  1612 connection.cc:256] RPC call timeout handler was delayed by -4.2820789219999824s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1876s for a response
W20260709 13:04:12.611332  1612 connection.cc:256] RPC call timeout handler was delayed by -4.1722789219999754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1998s for a response
W20260709 13:04:12.611778  1612 connection.cc:256] RPC call timeout handler was delayed by -4.1668789219999667s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20040000000000002s for a response
W20260709 13:04:12.612376  1612 connection.cc:256] RPC call timeout handler was delayed by -3.9544789219999643s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22400000000000003s for a response
W20260709 13:04:12.612859  1612 connection.cc:256] RPC call timeout handler was delayed by -3.9472789220000095s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22480000000000003s for a response
W20260709 13:04:12.613243  1612 connection.cc:256] RPC call timeout handler was delayed by -3.607078921999971s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2626s for a response
I20260709 13:04:12.614357  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:12.616314  1612 connection.cc:256] RPC call timeout handler was delayed by -3.387478921999957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28700000000000003s for a response
I20260709 13:04:12.616621  1566 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:12.618844  1538 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:12.621289  1538 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:12.622833  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:12.610780  1567 rpcz_store.cc:276] 0709 13:04:10.725408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.371725 (+646317us) service_pool.cc:224] Handling call
0709 13:04:11.505501 (+133776us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:12.050519 (+545018us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:12.050575 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:12.071652 (+ 21077us) log.cc:844] Serialized 26 byte log entry
0709 13:04:12.457609 (+385957us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:12.457684 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.457774 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:12.557254 (+ 99480us) raft_consensus.cc:1685] finished
0709 13:04:12.557279 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.557452 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77382,"raft.queue_time_us":123423,"raft.run_cpu_time_us":437,"raft.run_wall_time_us":17017,"thread_start_us":782847,"threads_started":2,"wal-append.queue_time_us":437754}
I20260709 13:04:12.624545  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:12.628244  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:12.628801  1612 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:04:12.587467  6715 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 1.212s	user 0.004s	sys 0.005s
I20260709 13:04:12.638319  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:12.582077  6748 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:12.574074  6742 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:12.640663  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:12.643806  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 133899us
I20260709 13:04:12.644261  1612 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:12.641945  1538 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:12.644930  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:12.646129  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:12.646814  1567 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:12.614368  6624 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:12.644196  1573 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:12.643119  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:12.627859  6453 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:12.604936  6630 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:12.654639  1570 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:12.631851  6494 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:12.660944  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:12.661623  6741 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:12.661815  6737 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:12.662405  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:12.663102  1567 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:12.663539  6620 raft_consensus.cc:3055] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:12.663955  6620 raft_consensus.cc:740] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:12.666548  1611 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:12.667143  6719 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:12.667985  6719 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:12.668403  6719 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:12.668808  6719 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:12.581848  6551 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:04:12.629048  6287 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:04:12.660799  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:12.630833  6633 raft_consensus.cc:697] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:12.689376  6633 consensus_queue.cc:237] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:12.645268  6361 raft_consensus.cc:2764] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:04:12.672449  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:12.693332  1570 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:12.651163  6522 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:12.671205  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 233573us
I20260709 13:04:12.696091  6629 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:12.701683  6727 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:12.623893  6752 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:12.702310  6161 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.707581  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:12.715525  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:12.716521  1639 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:12.716964  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:12.717765  1641 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:12.717904  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:12.718639  1639 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:09.990008   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:12.720556  1566 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:04:12.721993  1566 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:12.723094  1567 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:12.736995  1567 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:12.664933  6620 consensus_queue.cc:260] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:12.658705  6733 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:12.742933  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 97946.7us
W20260709 13:04:12.743924  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 138791us
I20260709 13:04:12.747202  6361 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:12.752161  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:12.679085  6624 raft_consensus.cc:697] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:12.703311  6287 consensus_queue.cc:1243] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:12.762555  1648 raft_consensus.cc:1275] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:04:12.640391  6722 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:12.774747  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 114060us
I20260709 13:04:12.768869  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:12.741321  6685 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:12.748736  6733 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:12.792944  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:12.798913  1612 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:12.797947  1648 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:12.772043  6620 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:12.813325  1538 leader_election.cc:336] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.558s (SENT)
I20260709 13:04:12.735828  6161 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:12.721897  6744 consensus_queue.cc:1048] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 13:04:12.814100  1648 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 13:04:12.826886  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2777) took 2173 ms (client timeout 1545 ms). Trace:
W20260709 13:04:12.827829  1643 rpcz_store.cc:269] 0709 13:04:10.660937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.661034 (+    97us) service_pool.cc:224] Handling call
0709 13:04:12.826592 (+2165558us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400182c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:12.826805 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2847060224}
I20260709 13:04:12.828318  1648 raft_consensus.cc:3055] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:12.828714  1648 raft_consensus.cc:740] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:12.815970  1639 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
W20260709 13:04:12.819000  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.818852  1538 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:12.822430  1687 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:12.775687  6646 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.834093  1716 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:12.797817  6733 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:12.837219  1648 consensus_queue.cc:260] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:12.844087  1538 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.892s (SENT)
I20260709 13:04:12.855021  1538 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:12.867158  1648 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:12.871651  1641 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
W20260709 13:04:12.874125  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 85451.6us
W20260709 13:04:12.876031  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.826813  6538 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.876708  1612 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:12.834857  6396 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:12.793910  6685 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:12.845199  6759 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:12.892966  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:12.894896  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 50883.3us
W20260709 13:04:12.897738  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 75404.6us
I20260709 13:04:12.898672  1687 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:12.875308  6551 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:12.902766  1538 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.998s (SENT)
I20260709 13:04:12.903982  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:12.906200  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.908700  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:12.909498  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:12.910313  1639 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:04:12.908834  1643 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:04:12.911469  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:12.912109  1639 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:04:11.939076  1571 rpcz_store.cc:276] 0709 13:04:10.658499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.970138 (+311639us) service_pool.cc:224] Handling call
0709 13:04:10.970631 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.970881 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:11.040166 (+ 69285us) log.cc:844] Serialized 14 byte log entry
0709 13:04:11.777303 (+737137us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:11.777391 (+    88us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:11.777476 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.777576 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:11.777780 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":68700,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":91998,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":267,"thread_start_us":736559,"threads_started":1,"wal-append.queue_time_us":898048}
I20260709 13:04:12.916227  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:12.916599  6758 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:12.894189  6396 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:12.917165  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:12.917255  6758 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:12.919086  6396 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:12.919494  6758 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:12.920264  1611 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:12.920740  6731 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:12.921253  1611 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:12.921561  6731 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:12.922156  6731 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:12.922684  6731 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:12.922992  1565 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2292) took 3348 ms (client timeout 2068 ms). Trace:
W20260709 13:04:12.923442  1565 rpcz_store.cc:269] 0709 13:04:09.574983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.589226 (+ 14243us) service_pool.cc:224] Handling call
0709 13:04:12.922786 (+3333560us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400406ad4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:12.922921 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2824036736}
I20260709 13:04:12.924692  1568 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:04:12.925889  6757 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:12.931838  1611 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:12.922394  6286 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:12.932996  1611 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:12.933068  6736 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:12.933743  6736 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:12.934109  6736 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:12.935457  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:12.936702  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:12.938879  6742 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:12.939177  1611 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:12.956327  6742 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:12.940487  1538 leader_election.cc:336] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.036s (SENT)
I20260709 13:04:12.957544  1538 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:12.958564  6732 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:12.939731  6618 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:12.939476  6757 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:12.961318  6757 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:12.961838  6757 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:12.962404  6742 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:12.911711  6741 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:12.966931  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:12.968163  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:12.968354  6706 raft_consensus.cc:3055] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:12.972626  6522 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:12.946749  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:12.975128  1571 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 12: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:04:12.976387  6701 log.cc:927] Time spent T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 2.147s	user 0.009s	sys 0.008s
I20260709 13:04:12.976637  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:12.978063  1565 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 18: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:12.978582  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:12.978916  6752 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:12.889971  6538 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:12.900121  6737 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:04:12.985052  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 86477.8us
I20260709 13:04:12.982263  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:12.983979  1643 raft_consensus.cc:2368] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 14. Current term is 15.
I20260709 13:04:12.986905  2348 raft_consensus.cc:2393] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 13: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:12.978937  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:12.984103  1639 raft_consensus.cc:2410] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:04:12.982312  1571 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:13.010511  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 50342.9us
I20260709 13:04:12.849589  6646 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:12.996150  1687 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:13.022243  1687 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:13.022909  6533 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:13.023310  1687 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:12.996603  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 57505.1us
I20260709 13:04:13.023885  6234 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:13.023358  6533 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:13.024703  6533 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:12.997296  6752 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:13.026670  6752 raft_consensus.cc:697] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:12.996292  6618 raft_consensus.cc:697] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:12.993781  6631 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:12.999539  6161 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:13.000139  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.002447  6747 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:12.969389  6706 raft_consensus.cc:740] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:12.985628  6721 log.cc:927] Time spent T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:04:13.035925  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:13.037899  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:13.038081  2348 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:04:13.037092  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:13.039877  1566 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:13.012205  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:13.013438  1538 leader_election.cc:336] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.134s (SENT)
I20260709 13:04:12.984768  6674 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:04:13.008765  6531 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:13.010110  6773 consensus_queue.cc:1048] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:04:13.016798  6768 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:12.902014  6624 consensus_queue.cc:237] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:12.992669  1648 raft_consensus.cc:1275] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:04:13.056385  1648 pending_rounds.cc:85] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 6
I20260709 13:04:13.057011  1648 pending_rounds.cc:107] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:04:13.057459  1648 raft_consensus.cc:2889] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:13.018715  6538 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:13.022502  6761 log.cc:927] Time spent T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 13:04:12.997174  6771 log.cc:927] Time spent T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:04:12.989768  6758 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:13.101518  6758 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.103744  6758 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:13.040760  1639 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:13.107909  6646 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:13.042814  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:13.108970  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.039165  6744 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:13.051630  1714 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:13.058604  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:13.116503  2349 raft_consensus.cc:2410] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:04:13.059980  1571 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:13.117245  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.118355  1648 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:04:13.119735  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.060053  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.061388  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.122033  1720 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:04:13.122599  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.067772  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:13.124063  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.124766  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.082111  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.082924  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.064265  6531 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.084501  6338 raft_consensus.cc:3055] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:04:13.087662  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:13.133680  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.088542  6533 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.092008  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.135001  1643 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:04:13.094069  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.135450  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.135979  1716 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:04:13.097681  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.032745  6286 consensus_queue.cc:1059] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:04:13.111515  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.139370  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.031730  6631 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:13.142990  1639 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:13.144351  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.145339  1639 raft_consensus.cc:2393] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:04:13.100626  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 116876us
W20260709 13:04:13.114176  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.038990  6746 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:13.124980  1720 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:13.030750  6618 consensus_queue.cc:237] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.129329  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.131189  1641 raft_consensus.cc:2410] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:04:13.131762  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.149565  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.150396  1643 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:13.150524  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.137203  1645 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:13.152084  1648 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:13.151564  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.143036  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:13.132805  6338 raft_consensus.cc:740] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:04:13.100592  2313 connection.cc:256] RPC call timeout handler was delayed by -1.7885298030000172s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.251s for a response
I20260709 13:04:13.172003  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.172955  1566 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:13.124003  6287 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.174346  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.174520  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.176160  1713 raft_consensus.cc:1275] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:13.177493  1714 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:13.177878  1643 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:13.180152  1713 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:13.181881  6560 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.182595  6560 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:13.184062  6287 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:13.136839  6744 raft_consensus.cc:697] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:13.185866  6686 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.186517  6686 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:13.187311  1564 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:13.186707  6744 consensus_queue.cc:237] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.189031  6686 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:13.189394  1570 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:04:13.152314  6631 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:13.136617  6706 consensus_queue.cc:260] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:13.195210  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 181506us
I20260709 13:04:13.202049  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.180266  1566 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 13:04:13.197897  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.199097  6224 consensus_queue.cc:260] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:13.194967  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.194555  6338 consensus_queue.cc:260] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:13.201890  1640 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:13.181018  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.208245  1643 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:13.206902  1645 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:13.206048  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:13.210453  1685 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:13.172273  6748 consensus_meta.cc:287] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.524s	user 0.005s	sys 0.007s
I20260709 13:04:13.221104  2314 leader_election.cc:400] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 14. Current term is 15.
I20260709 13:04:13.221747  2314 leader_election.cc:403] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:04:13.222813  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.211345  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.232156  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.232394  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.232630  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.233155  1716 raft_consensus.cc:2410] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:04:13.237692  1719 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 14: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:13.238372  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:13.207906  6719 consensus_meta.cc:287] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.539s	user 0.011s	sys 0.004s
I20260709 13:04:13.243117  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:13.244450  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.244897  6733 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:13.245726  6748 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.245534  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.246332  6748 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.247289  1719 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:13.247788  6733 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:13.248421  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.249981  1567 raft_consensus.cc:1240] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:13.250427  1719 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:13.249889  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:13.201187  6736 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.130200  6752 consensus_queue.cc:237] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.265177  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.257714  1685 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:13.184970  6560 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:13.212071  6706 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:13.265314  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.268450  1716 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:13.268751  1612 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:13.280144  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:13.276674  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:13.273461  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.223565  6551 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.231086  6758 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:13.254318  6538 raft_consensus.cc:3055] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:13.286770  1714 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:13.314646  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.315080  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.315800  1714 raft_consensus.cc:2393] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:13.284704  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.239497  6773 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:13.239286  6746 raft_consensus.cc:697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:13.252385  6646 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:13.273458  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.266340  6660 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:13.301916  6727 consensus_queue.cc:1059] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:13.303248  6396 consensus_queue.cc:1059] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:13.303937  1687 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:13.306080  6286 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:13.310036  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:13.318842  1643 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:04:13.298276  6768 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:13.240509  6494 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:04:13.317503  1639 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:13.312071  6538 raft_consensus.cc:740] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:13.278736  6234 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.342345  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.274335  6737 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:13.346844  6737 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.289130  6747 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:13.350687  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.292594  6560 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:04:13.287092  6741 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:13.352620  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:13.353998  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:13.353703  6768 raft_consensus.cc:697] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:13.354805  1713 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:13.334148  1573 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:13.293893  6685 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:13.334787  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.263688  6719 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.335282  6545 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:13.336519  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.320747  1612 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:13.366148  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 45565.5us
I20260709 13:04:13.338625  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.367045  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:13.368064  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:13.368952  6732 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:13.369755  6732 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.370491  6732 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:13.370853  6685 consensus_queue.cc:1048] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:13.374125  6742 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:13.358985  1643 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:13.376670  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.377573  1643 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:04:13.377864  1612 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:13.378732  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:13.378880  6620 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:13.379541  6361 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:13.379652  1612 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:13.379907  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:13.380213  1537 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:13.380800  6618 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:13.356146  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.296474  6551 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.380607  6361 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:13.381153  6685 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:13.381752  1639 raft_consensus.cc:2376] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13. Re-sending same reply.
I20260709 13:04:13.381968  6685 raft_consensus.cc:697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:13.382776  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.383116  6551 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:13.384061  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.384423  1713 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:13.383720  6685 consensus_queue.cc:237] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.385289  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.386682  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:13.389197  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.345270  6234 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.391901  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.360126  1716 raft_consensus.cc:2376] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12. Re-sending same reply.
I20260709 13:04:13.312635  6758 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:13.364579  1714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:13.400566  1714 raft_consensus.cc:606] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
W20260709 13:04:13.330826  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 127954us
I20260709 13:04:13.402700  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:13.403622  2313 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:13.398464  1537 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:13.395526  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:13.381165  1643 raft_consensus.cc:2410] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:04:13.325773  6759 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
W20260709 13:04:13.395263  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 174424us
I20260709 13:04:13.405881  2314 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:13.406807  1612 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:13.410084  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:13.411003  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:13.412091  1643 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:13.413023  1612 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:13.333376  6660 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.364851  6545 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.415896  1537 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:13.416019  2314 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:13.329566  6773 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:13.432304  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.433022  6752 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:13.417479  6361 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:13.444644  6758 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:13.433688  6773 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:13.446092  6782 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:13.360319  6768 consensus_queue.cc:237] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.447376  5852 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:13.448103  5852 raft_consensus.cc:697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:13.417858  6494 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:13.418526  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:13.418393  6737 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:13.422034  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:13.407779  6522 consensus_queue.cc:1059] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:04:13.402268  6746 consensus_queue.cc:237] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:13.424618  6719 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:13.393778  6757 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.425806  6781 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:13.426390  6736 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:13.426570  2313 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:13.413798  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.433943  1571 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:04:13.470754  1571 pending_rounds.cc:85] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 4
I20260709 13:04:13.471377  1571 pending_rounds.cc:107] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:04:13.471833  1571 raft_consensus.cc:2889] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:13.415050  6551 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:13.436854  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:13.428705  6731 consensus_meta.cc:287] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.505s	user 0.015s	sys 0.000s
I20260709 13:04:13.478751  6731 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.401916  6533 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:13.439486  6730 raft_consensus.cc:3055] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:13.441717  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2322) took 2874 ms. Trace:
I20260709 13:04:13.442638  6234 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:13.443445  6759 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:13.482157  6730 raft_consensus.cc:740] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:13.434185  6545 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:13.430238  6538 consensus_queue.cc:260] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.423434  6732 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:13.451846  1716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:13.489118  1644 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 13: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:13.489342  1716 raft_consensus.cc:606] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:13.452370  6494 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.450893  1645 raft_consensus.cc:2376] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11. Re-sending same reply.
I20260709 13:04:13.449041  5852 consensus_queue.cc:237] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.497924  6494 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:13.462939  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:13.461707  1572 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:13.468183  1643 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
W20260709 13:04:13.474159  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.482524  1648 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 15: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:13.502017  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.502920  1643 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:13.504041  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.505050  1643 raft_consensus.cc:2410] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:04:13.482218  1569 rpcz_store.cc:276] 0709 13:04:10.566956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.950405 (+383449us) service_pool.cc:224] Handling call
0709 13:04:10.950709 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.997894 (+2047185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:12.997927 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:12.997977 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:12.998039 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.441411 (+443372us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b54001270d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:13.441464 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:13.441607 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4707968256}
I20260709 13:04:13.485307  1573 raft_consensus.cc:1275] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
W20260709 13:04:13.509291  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:13.493883  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 81025.5us
I20260709 13:04:13.490979  6781 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:13.512355  6560 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 14. Current term is 15.
I20260709 13:04:13.514461  1573 raft_consensus.cc:1275] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:04:13.499858  6631 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.517882  1569 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:13.519033  6631 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:13.477443  6551 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:13.520601  6551 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:13.522217  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.522940  1645 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:13.523172  6732 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:13.489787  6731 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:13.525336  1645 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:04:13.534292  1645 pending_rounds.cc:85] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 5
I20260709 13:04:13.534574  1645 pending_rounds.cc:107] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:04:13.534816  1645 raft_consensus.cc:2889] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:13.536512  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:13.537216  1645 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:13.533571  1612 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:13.528060  6782 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:13.537422  1716 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:13.541304  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:13.545599  1612 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:13.549538  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 123099us
W20260709 13:04:13.550209  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 85829.8us
I20260709 13:04:13.550944  2313 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:13.555109  2313 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:13.554643  6736 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:13.568300  6782 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:13.568691  6747 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:13.571062  2349 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:13.572113  1612 leader_election.cc:400] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12.
W20260709 13:04:13.572829  1646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2780) took 2849 ms (client timeout 1892 ms). Trace:
I20260709 13:04:13.572968  1642 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:13.573750  1612 leader_election.cc:403] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 13:04:13.573844  1646 rpcz_store.cc:269] 0709 13:04:10.741521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.779476 (+ 37955us) service_pool.cc:224] Handling call
0709 13:04:13.572630 (+2793154us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400184554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:13.572767 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2742910336}
W20260709 13:04:13.563480  6792 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.080s	user 0.000s	sys 0.002s
I20260709 13:04:13.575309  1646 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:04:13.575766  1612 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:13.578290  1646 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:04:13.578665  1722 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:13.559417  6731 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:13.581573  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:13.581722  1646 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:04:13.583387  1646 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:04:13.559666  6798 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:13.568825  6234 consensus_queue.cc:1048] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:04:13.589489  2348 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:04:13.538666  6730 consensus_queue.cc:260] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.592031  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:13.574537  6660 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:13.592154  1612 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:13.588562  6736 raft_consensus.cc:697] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:13.691694  2348 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:13.692306  1713 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:04:13.693403  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:13.695860  2342 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:04:13.707213  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 115152us
I20260709 13:04:13.586184  6798 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.627220  6737 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.579396  6719 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12.
W20260709 13:04:13.687144  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.699867  1716 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:13.576746  6560 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:04:13.592368  6731 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:13.721662  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:13.722936  5852 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:13.725350  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2335) took 1095 ms. Trace:
I20260709 13:04:13.725931  6551 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.725857  1570 rpcz_store.cc:276] 0709 13:04:12.629421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.694461 (+ 65040us) service_pool.cc:224] Handling call
0709 13:04:13.189103 (+494642us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:13.189960 (+   857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.725139 (+535179us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b5400012050. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:13.725283 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1136582016}
I20260709 13:04:13.729050  6551 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:13.730628  6736 consensus_queue.cc:237] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.591266  6730 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:13.687978  1646 pending_rounds.cc:85] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 5
I20260709 13:04:13.736867  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:13.738232  1646 pending_rounds.cc:107] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:04:13.738580  1646 raft_consensus.cc:2889] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:13.705526  2343 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:13.709560  1612 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:13.740574  6718 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:13.741299  6718 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:13.706630  6746 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:13.717985  6737 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.712455  6798 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:13.702576  6790 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.139s	user 0.000s	sys 0.005s
I20260709 13:04:13.737851  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:13.706810  6660 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:13.744050  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.747207  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:13.749500  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:13.738559  1716 raft_consensus.cc:2368] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 15. Current term is 16.
I20260709 13:04:13.762871  2348 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 11: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:04:13.752537  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 65269.1us
I20260709 13:04:13.758272  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.771720  1713 raft_consensus.cc:2368] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 15. Current term is 16.
I20260709 13:04:13.763484  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:13.772768  1716 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:13.767323  1612 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:13.770447  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:13.734854  6757 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:13.778574  1612 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:13.769248  5852 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:13.769953  6551 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.764117  6631 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:04:13.803608  6624 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:13.781715  6722 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:13.804360  6737 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:13.744957  6667 consensus_queue.cc:1059] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:04:13.800745  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.806152  1716 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:13.778925  6731 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:13.818971  6667 raft_consensus.cc:3055] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:04:13.803763  6794 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:04:13.819697  6667 raft_consensus.cc:740] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:13.786360  6660 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:13.821806  1719 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:04:13.757277  6746 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.818993  1537 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:13.823815  6732 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:13.818607  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.824378  6732 raft_consensus.cc:697] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:13.819249  6731 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:13.825055  6731 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:13.771225  6286 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:04:13.824091  1719 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:13.825398  6732 consensus_queue.cc:237] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:13.835237  6770 log.cc:927] Time spent T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.919s	user 0.004s	sys 0.001s
I20260709 13:04:13.812206  6551 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:13.777571  6697 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:04:13.838754  6697 consensus_queue.cc:1243] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer cf2f71592be8465fb18e36e15d0b2f89 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:04:13.838860  6551 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:13.839251  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.840241  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2286) took 1126 ms. Trace:
I20260709 13:04:13.840636  1642 rpcz_store.cc:276] 0709 13:04:12.714065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.714216 (+   151us) service_pool.cc:224] Handling call
0709 13:04:13.572781 (+858565us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:13.579324 (+  6543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.840028 (+260704us) spinlock_profiling.cc:243] Waited 825 ms on lock 0x7b5400184550. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:13.840183 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1897177344}
I20260709 13:04:13.840018  1645 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16.
I20260709 13:04:13.843973  1720 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: []
W20260709 13:04:13.847452  1717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2297) took 1933 ms (client timeout 1797 ms). Trace:
I20260709 13:04:13.848047  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:13.811657  5852 raft_consensus.cc:697] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:13.850096  2313 leader_election.cc:400] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 15. Current term is 16.
I20260709 13:04:13.851135  2313 leader_election.cc:400] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 15. Current term is 16.
I20260709 13:04:13.850728  5852 consensus_queue.cc:237] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:13.854058  6560 consensus_queue.cc:1059] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:04:13.855683  6271 raft_consensus.cc:3055] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:13.856276  6271 raft_consensus.cc:740] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:13.856748  6560 raft_consensus.cc:3055] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:13.857208  6560 raft_consensus.cc:740] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:13.857987  6524 raft_consensus.cc:3055] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:13.858444  6524 raft_consensus.cc:740] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:13.859302  6524 consensus_queue.cc:260] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:13.859970  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2678) took 2694 ms. Trace:
I20260709 13:04:13.860304  6524 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:13.860914  1719 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:13.861578  1719 raft_consensus.cc:606] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:13.862890  6287 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:13.848125  1717 rpcz_store.cc:269] 0709 13:04:11.913676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.914154 (+   478us) service_pool.cc:224] Handling call
0709 13:04:13.843626 (+1929472us) spinlock_profiling.cc:243] Waited 5.97 ms on lock 0x7b54004079d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:13.847307 (+  3681us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13723008}
I20260709 13:04:13.868005  1642 raft_consensus.cc:1275] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:13.871345  6748 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:13.824175  6746 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:13.820937  6667 consensus_queue.cc:260] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:13.873772  6667 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:13.877768  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.878789  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:13.878839  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2283) took 1173 ms. Trace:
I20260709 13:04:13.880610  1640 rpcz_store.cc:276] 0709 13:04:12.705551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.709200 (+  3649us) service_pool.cc:224] Handling call
0709 13:04:13.878603 (+1169403us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b54002aadd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:13.878766 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1026858240}
I20260709 13:04:13.882017  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:13.849365  1644 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 13:04:13.883879  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:13.885062  1644 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: []
W20260709 13:04:13.905730  1538 leader_election.cc:336] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.078s (SENT)
I20260709 13:04:13.860445  1718 rpcz_store.cc:276] 0709 13:04:11.165882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.166045 (+   163us) service_pool.cc:224] Handling call
0709 13:04:11.166489 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:11.166808 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:12.028803 (+861995us) log.cc:844] Serialized 14 byte log entry
0709 13:04:12.602672 (+573869us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b18000ad860. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f824c87d3c7 00007f8248e8f9a7 00007f8248e9f0bf 00007f8248e9f038
0709 13:04:12.779281 (+176609us) log.cc:844] Serialized 14 byte log entry
0709 13:04:13.859411 (+1080130us) spinlock_profiling.cc:243] Waited 7.79 ms on lock 0x7b18000ad860. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287
0709 13:04:13.859537 (+   126us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:13.859634 (+    97us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:13.859695 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.859762 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:13.859905 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":248582,"prepare.queue_time_us":424496,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":67705,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":17643,"spinlock_wait_cycles":24983296,"thread_start_us":621388,"threads_started":3,"wal-append.queue_time_us":440891}
I20260709 13:04:13.919883  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:13.922662  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.816299  2343 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:13.929287  1538 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:13.898255  1687 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:13.942778  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 44670.6us
I20260709 13:04:13.943892  1687 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:13.945768  1687 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:13.946058  6746 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:13.946974  6746 raft_consensus.cc:697] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:13.947625  6529 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:13.948060  6746 consensus_queue.cc:237] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.948799  6697 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:13.949396  6697 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:13.949522  6529 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:13.949801  6744 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:13.949968  6697 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:13.956902  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.959084  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:13.960953  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 45113.6us
I20260709 13:04:13.964119  2340 pending_rounds.cc:85] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Aborting all ops after (but not including) 9
I20260709 13:04:13.964583  2340 pending_rounds.cc:107] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:04:13.964943  1644 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:13.782313  6494 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:13.879781  6731 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:13.969978  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:13.977819  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:13.964896  2340 raft_consensus.cc:2889] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:13.972432  1642 raft_consensus.cc:1240] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: [13.6-13.6]
I20260709 13:04:13.973134  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:13.976233  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.943792  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:13.983314  2344 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 12: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:13.984459  1718 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:04:13.892906  6730 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.005508  1717 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:04:13.857414  6271 consensus_queue.cc:260] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:13.979808  6733 consensus_queue.cc:1059] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
W20260709 13:04:14.031036  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 83322.3us
I20260709 13:04:14.041383  1713 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 13:04:14.050901  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 84320.4us
I20260709 13:04:14.036051  6730 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:14.054265  6545 consensus_meta.cc:287] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.565s	user 0.014s	sys 0.000s
I20260709 13:04:14.054941  6545 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:14.040163  1687 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:14.056406  6730 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:14.057189  6545 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:14.036660  6659 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:14.061391  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2300) took 1005 ms. Trace:
I20260709 13:04:14.061973  1645 rpcz_store.cc:276] 0709 13:04:13.055863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.845891 (+790028us) service_pool.cc:224] Handling call
0709 13:04:13.846974 (+  1083us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:14.059046 (+212072us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:14.059081 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:14.059138 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:14.059187 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.061053 (+  1866us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b5400097254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:14.061110 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.061303 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":487121152}
I20260709 13:04:14.021207  6758 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:14.022110  6781 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:04:14.082867  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 170071us
I20260709 13:04:14.084193  2314 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:14.065579  6731 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:14.082115  6729 raft_consensus.cc:2764] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 13:04:14.142673  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.095134  6271 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:14.082618  6494 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:14.016947  6560 consensus_queue.cc:260] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:14.120308  6674 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:14.156710  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2340) took 1200 ms. Trace:
I20260709 13:04:14.158350  1564 rpcz_store.cc:276] 0709 13:04:12.955341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.188229 (+232888us) service_pool.cc:224] Handling call
0709 13:04:13.188539 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:13.188777 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:13.225922 (+ 37145us) log.cc:844] Serialized 14 byte log entry
0709 13:04:13.838074 (+612152us) spinlock_profiling.cc:243] Waited 8.51 ms on lock 0x7b1800374560. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a2523a 00007f8248e8fa2a
0709 13:04:14.022242 (+184168us) log.cc:844] Serialized 14 byte log entry
0709 13:04:14.022629 (+   387us) log.cc:844] Serialized 14 byte log entry
0709 13:04:14.144519 (+121890us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:14.144577 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:14.144641 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.144734 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.155570 (+ 10836us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":815,"prepare.queue_time_us":111419,"prepare.run_cpu_time_us":266,"prepare.run_wall_time_us":16728,"raft.queue_time_us":104570,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":36501,"spinlock_wait_cycles":99665024,"thread_start_us":304242,"threads_started":2,"wal-append.queue_time_us":828072}
I20260709 13:04:14.139856  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:14.093340  6545 consensus_queue.cc:1059] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:04:14.172247  6815 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.095767  6781 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:14.173602  6815 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:14.176141  6781 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:14.184214  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2298) took 1145 ms. Trace:
I20260709 13:04:14.184789  1646 rpcz_store.cc:276] 0709 13:04:13.038288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.588431 (+550143us) service_pool.cc:224] Handling call
0709 13:04:13.594989 (+  6558us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:13.742977 (+147988us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:13.743011 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:13.743606 (+   595us) log.cc:844] Serialized 52 byte log entry
0709 13:04:13.887398 (+143792us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:13.887442 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.887524 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:14.183907 (+296383us) raft_consensus.cc:1685] finished
0709 13:04:14.183933 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.184135 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3176,"raft.queue_time_us":4700,"raft.run_cpu_time_us":601,"raft.run_wall_time_us":9242,"thread_start_us":107545,"threads_started":1,"wal-append.queue_time_us":372896}
I20260709 13:04:14.187178  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:14.194056  6287 raft_consensus.cc:3055] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:04:14.195636  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.095645  6744 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:04:14.153062  6674 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:14.140094  6748 consensus_queue.cc:1059] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.028s
W20260709 13:04:14.219929  6810 log.cc:927] Time spent T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:04:14.221823  6338 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:14.194604  6287 raft_consensus.cc:740] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:14.226776  6287 consensus_queue.cc:260] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:14.228091  6287 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:14.204710  6560 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:14.154353  6812 consensus_queue.cc:1059] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:04:14.229875  6689 raft_consensus.cc:3055] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:14.230291  6689 raft_consensus.cc:740] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:14.231114  6689 consensus_queue.cc:260] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:14.233851  6798 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:14.214968  6674 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:14.235122  2342 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 14: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:14.236660  2349 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:04:14.239598  6624 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:14.239498  6748 consensus_queue.cc:1243] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:04:14.234807  1568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2331) took 1607 ms (client timeout 1505 ms). Trace:
W20260709 13:04:14.242899  1568 rpcz_store.cc:269] 0709 13:04:12.627819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.627981 (+   162us) service_pool.cc:224] Handling call
0709 13:04:14.234530 (+1606549us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b54002ba554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:14.234683 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":679631104}
I20260709 13:04:14.261320  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:14.273030  1642 raft_consensus.cc:2393] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:14.259155  6689 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:14.261148  6697 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:14.297160  6815 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:14.312538  6798 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:14.353505  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:14.353062  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:14.359512  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 430290us
I20260709 13:04:14.360198  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:14.362444  1538 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:14.363193  1642 raft_consensus.cc:2410] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:04:14.364943  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:14.367604  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:14.371845  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 277828us
I20260709 13:04:14.376142  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:14.377511  1645 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 7.
W20260709 13:04:14.378648  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 148048us
I20260709 13:04:14.379896  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:14.379916  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.385842  1719 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:14.379453  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:14.388558  1645 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:14.388849  1720 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:14.369411  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:14.398507  6531 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:14.405010  6531 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.406574  6531 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:14.412135  6531 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:14.432659  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.433528  6697 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:14.446862  6531 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:14.461417  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2330) took 1529 ms. Trace:
I20260709 13:04:14.464617  1565 rpcz_store.cc:276] 0709 13:04:12.928614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.020003 (+ 91389us) service_pool.cc:224] Handling call
0709 13:04:14.235753 (+1215750us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:14.236055 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:14.236094 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:14.236158 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:14.236218 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.236302 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.457719 (+221417us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54002ba550. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:14.457958 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2794926464}
I20260709 13:04:14.474982  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:14.479986  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2347) took 1270 ms. Trace:
I20260709 13:04:14.480669  1572 rpcz_store.cc:276] 0709 13:04:13.209128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.353709 (+144581us) service_pool.cc:224] Handling call
0709 13:04:14.479355 (+1125646us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b5400014d54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:14.479614 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245474048}
W20260709 13:04:14.483242  1641 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 503) took 1574 ms (client timeout 1513 ms). Trace:
I20260709 13:04:14.484226  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:14.486084  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:14.483923  1641 rpcz_store.cc:269] 0709 13:04:12.908285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.908429 (+   144us) service_pool.cc:224] Handling call
0709 13:04:14.482608 (+1574179us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b54002db754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:14.482952 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509476096}
I20260709 13:04:14.487327  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2795) took 1500 ms. Trace:
I20260709 13:04:14.488307  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:14.488929  5852 consensus_queue.cc:1059] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:04:14.490350  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.490793  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:14.491164  5852 raft_consensus.cc:3055] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:14.491999  5852 raft_consensus.cc:740] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:14.492641  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:14.490914  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 501) took 1436 ms. Trace:
W20260709 13:04:14.494150  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 46346.7us
I20260709 13:04:14.487396  1642 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:14.493491  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2308) took 2300 ms. Trace:
I20260709 13:04:14.495836  1538 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:14.494943  1721 rpcz_store.cc:276] 0709 13:04:13.054179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.054437 (+   258us) service_pool.cc:224] Handling call
0709 13:04:14.490390 (+1435953us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b5400247554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:14.490715 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1061738752}
I20260709 13:04:14.488137  1648 rpcz_store.cc:276] 0709 13:04:12.986782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.171705 (+184923us) service_pool.cc:224] Handling call
0709 13:04:14.486764 (+1315059us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b5400097c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:14.487166 (+   402us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":711749888}
I20260709 13:04:14.499658  1722 rpcz_store.cc:276] 0709 13:04:12.192912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.193043 (+   131us) service_pool.cc:224] Handling call
0709 13:04:12.193369 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:13.579065 (+1385696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.492804 (+913739us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400213754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:14.493199 (+   395us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3185681792}
I20260709 13:04:14.499727  5852 consensus_queue.cc:260] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:14.512971  5852 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:14.518988  2343 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:14.520340  1572 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:14.525506  1640 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:14.528213  1641 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:14.532006  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:14.529474  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:14.536048  1572 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:04:14.538563  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:14.538771  1648 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:14.541893  6731 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.542974  6731 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:14.537212  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:14.551326  1565 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:04:14.541921  1571 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:04:14.558601  1648 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:55704
I20260709 13:04:14.572916  1648 raft_consensus.cc:606] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:14.573822  1648 raft_consensus.cc:614] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:14.577417  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:14.584993  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 62622.1us
I20260709 13:04:14.611392  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:14.611392  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:14.614149  2343 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:04:14.615453  6733 consensus_queue.cc:1059] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:14.616300  6828 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:14.618266  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.621562  1641 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:14.622201  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:14.623301  1720 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
W20260709 13:04:14.623910  6772 log.cc:927] Time spent T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.600s	user 0.003s	sys 0.001s
I20260709 13:04:14.621583  1639 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:14.611543  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:14.629606  6271 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:14.595098  6531 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:14.631485  6271 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:14.636231  6809 log.cc:927] Time spent T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.587s	user 0.001s	sys 0.000s
I20260709 13:04:14.636896  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:14.637995  6531 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:14.653223  1645 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
W20260709 13:04:14.653386  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 53055us
I20260709 13:04:14.652819  1642 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:14.635774  6271 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:14.644389  6807 log.cc:927] Time spent T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.579s	user 0.000s	sys 0.004s
I20260709 13:04:14.667775  1643 raft_consensus.cc:1240] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:04:14.627816  6828 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:14.675385  1565 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:14.676285  6828 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:14.679805  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:14.693606  1715 raft_consensus.cc:2410] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 14 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 13:04:14.687251  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:04:14.687315  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 321827us
I20260709 13:04:14.692003  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2285) took 1978 ms. Trace:
I20260709 13:04:14.694681  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:14.695202  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:14.695252  6828 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:04:14.689901  6813 log.cc:927] Time spent T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.630s	user 0.002s	sys 0.002s
I20260709 13:04:14.699335  1572 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:04:14.701730  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2316) took 1469 ms. Trace:
I20260709 13:04:14.703835  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:14.703012  1715 rpcz_store.cc:276] 0709 13:04:13.232578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.233735 (+  1157us) service_pool.cc:224] Handling call
0709 13:04:14.679316 (+1445581us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b580009db98. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60e9c 00007f8245d60a5b 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:04:14.701660 (+ 22344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3167488}
I20260709 13:04:14.711539  6224 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:14.713819  6828 consensus_queue.cc:1243] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer cf2f71592be8465fb18e36e15d0b2f89 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:14.713955  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:14.715816  6833 consensus_queue.cc:1059] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:04:14.717334  6833 raft_consensus.cc:3055] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:14.717782  6833 raft_consensus.cc:740] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:14.719478  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2704) took 1132 ms. Trace:
I20260709 13:04:14.719594  6833 consensus_queue.cc:260] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:14.720997  6833 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:14.723640  1687 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:14.724406  6396 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:14.724726  1715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:14.692613  6674 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:14.725521  1715 raft_consensus.cc:606] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:14.727406  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:14.728663  1566 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:14.725625  6667 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:04:14.729527  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.730285  6667 raft_consensus.cc:697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:14.734845  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:14.735766  2343 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:04:14.736542  6741 raft_consensus.cc:3055] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:14.736730  2344 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:14.737139  6741 raft_consensus.cc:740] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:14.731947  6667 consensus_queue.cc:237] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:14.739389  6747 raft_consensus.cc:3055] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:14.739995  6747 raft_consensus.cc:740] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:14.740494  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:14.741550  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:14.742066  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:14.742760  1722 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:04:14.744385  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.745054  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2312) took 1548 ms. Trace:
I20260709 13:04:14.744432  2343 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:14.747495  6806 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.690s	user 0.001s	sys 0.000s
I20260709 13:04:14.726629  6731 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:14.750787  6820 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:14.752035  6820 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:14.756029  6820 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:14.758121  6674 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:14.759135  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2309) took 1561 ms. Trace:
I20260709 13:04:14.759646  1644 rpcz_store.cc:276] 0709 13:04:13.172581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.973213 (+800632us) service_pool.cc:224] Handling call
0709 13:04:13.973687 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:14.554109 (+580422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:14.555353 (+  1244us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:14.555423 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:14.555476 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.733322 (+177846us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b54001c0554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:14.733373 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.733613 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1312818048}
I20260709 13:04:14.761062  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2706) took 1034 ms. Trace:
I20260709 13:04:14.762032  1644 raft_consensus.cc:3055] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:14.762504  1644 raft_consensus.cc:740] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:14.720042  1714 rpcz_store.cc:276] 0709 13:04:13.586993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.587139 (+   146us) service_pool.cc:224] Handling call
0709 13:04:13.587425 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:13.594081 (+  6656us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:13.594139 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:13.594764 (+   625us) log.cc:844] Serialized 26 byte log entry
0709 13:04:13.881719 (+286955us) spinlock_profiling.cc:243] Waited 8.53 ms on lock 0x7b180018dba0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a250d1 00007f8248e40a0f
0709 13:04:14.013426 (+131707us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:14.013478 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.013567 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:14.668337 (+654770us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:14.718781 (+ 50444us) raft_consensus.cc:1685] finished
0709 13:04:14.718806 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.719088 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1708,"raft.queue_time_us":1874,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":95449,"spinlock_wait_cycles":73923712,"thread_start_us":131470,"threads_started":1,"wal-append.queue_time_us":366240}
I20260709 13:04:14.771906  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:14.761502  1716 rpcz_store.cc:276] 0709 13:04:13.726146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.840514 (+114368us) service_pool.cc:224] Handling call
0709 13:04:13.841022 (+   508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:13.841480 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:13.971308 (+129828us) log.cc:844] Serialized 14 byte log entry
0709 13:04:14.226699 (+255391us) spinlock_profiling.cc:243] Waited 3.22 ms on lock 0x7b18004525c0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a2523a 00007f8248e8fa2a
0709 13:04:14.542827 (+316128us) log.cc:844] Serialized 14 byte log entry
0709 13:04:14.587451 (+ 44624us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:14.708735 (+121284us) log.cc:844] Serialized 26 byte log entry
0709 13:04:14.708911 (+   176us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:14.708957 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.760513 (+ 51556us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:14.760646 (+   133us) raft_consensus.cc:1685] finished
0709 13:04:14.760662 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.760873 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58289,"prepare.queue_time_us":128288,"prepare.run_cpu_time_us":205,"prepare.run_wall_time_us":14646,"raft.queue_time_us":123020,"raft.run_cpu_time_us":647,"raft.run_wall_time_us":27173,"spinlock_wait_cycles":7402368,"thread_start_us":271114,"threads_started":2,"wal-append.queue_time_us":322584}
I20260709 13:04:14.777330  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:14.778465  1572 raft_consensus.cc:2410] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:04:14.780452  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:14.782316  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:14.784063  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:14.745501  1639 rpcz_store.cc:276] 0709 13:04:13.196744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.413471 (+216727us) service_pool.cc:224] Handling call
0709 13:04:14.744822 (+1331351us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54000cf3d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:14.744974 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2646330624}
I20260709 13:04:14.795224  2346 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:04:14.741472  6747 consensus_queue.cc:260] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:14.797178  6741 consensus_queue.cc:260] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:14.798768  6741 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:14.799469  6753 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.799492  1644 consensus_queue.cc:260] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:14.800364  6753 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:14.801071  1644 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:14.766520  6271 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:14.801769  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2305) took 1670 ms. Trace:
I20260709 13:04:14.697490  1647 rpcz_store.cc:276] 0709 13:04:12.713731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.738019 (+ 24288us) service_pool.cc:224] Handling call
0709 13:04:12.738477 (+   458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:12.763126 (+ 24649us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:12.763199 (+    73us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:12.777343 (+ 14144us) log.cc:844] Serialized 26 byte log entry
0709 13:04:12.906758 (+129415us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:12.906797 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.906867 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:14.622733 (+1715866us) spinlock_profiling.cc:243] Waited 8.71 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:14.646866 (+ 24133us) raft_consensus.cc:1685] finished
0709 13:04:14.646899 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.691552 (+ 44653us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24040,"raft.queue_time_us":24302,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":19240,"spinlock_wait_cycles":20025216,"thread_start_us":78839,"threads_started":1,"wal-append.queue_time_us":210625}
I20260709 13:04:14.806617  1640 rpcz_store.cc:276] 0709 13:04:13.131146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.896046 (+764900us) service_pool.cc:224] Handling call
0709 13:04:14.507309 (+611263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:14.526319 (+ 19010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.801378 (+275059us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b5400097c50. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:14.801682 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1180858240}
I20260709 13:04:14.829849  1538 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:14.814394  6271 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.784767  6731 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.803187  6820 consensus_queue.cc:1059] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:04:14.844110  6271 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:14.847911  6736 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.007s	sys 0.016s
I20260709 13:04:14.833689  6828 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:04:14.874106  1647 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:04:14.876720  6287 consensus_meta.cc:287] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.648s	user 0.023s	sys 0.010s
W20260709 13:04:14.879542  1611 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.797s (SENT) [suppressed 33 similar messages]
I20260709 13:04:14.882043  1714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
W20260709 13:04:14.883831  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:14.884654  6560 consensus_meta.cc:287] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.656s	user 0.014s	sys 0.014s
I20260709 13:04:14.885519  1564 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:14.885921  6823 consensus_queue.cc:1059] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:14.887145  6566 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:14.887918  6566 consensus_queue.cc:1243] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:04:14.888852  6812 raft_consensus.cc:3055] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:14.889390  6812 raft_consensus.cc:740] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:04:14.890733  6840 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:04:14.890995  6812 consensus_queue.cc:260] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:14.892897  6768 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.238s	user 0.009s	sys 0.011s
W20260709 13:04:14.893335  6768 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.426s	user 0.009s	sys 0.011s
W20260709 13:04:14.882339  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 80153us
I20260709 13:04:14.896780  2343 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:14.876794  1570 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:14.921820  1644 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:14.893417  1686 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:04:14.897898  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.896183  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:14.919625  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 52217us
W20260709 13:04:14.926855  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.933727  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:14.935897  1572 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:04:14.937117  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:14.937741  6731 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:14.938333  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:14.939776  1564 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:04:14.940115  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 797) took 2239 ms. Trace:
I20260709 13:04:14.940475  1567 rpcz_store.cc:276] 0709 13:04:12.700583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.738373 (+ 37790us) service_pool.cc:224] Handling call
0709 13:04:12.771178 (+ 32805us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:13.250807 (+479629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.939606 (+1688799us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b5400014354. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:14.939920 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1100176256}
I20260709 13:04:14.940940  6731 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:14.946538  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:14.951012  1722 raft_consensus.cc:2368] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 12.
I20260709 13:04:14.901732  1714 raft_consensus.cc:606] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
W20260709 13:04:14.897300  6836 log.cc:927] Time spent T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:04:14.958446  1566 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
W20260709 13:04:14.900590  6736 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.883s	user 0.008s	sys 0.017s
I20260709 13:04:14.937780  1573 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:14.894061  6624 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:14.897374  6820 raft_consensus.cc:3055] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:14.894199  6812 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:14.971612  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.968194  6624 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.988485  6624 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:14.989831  2312 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:14.991866  2312 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:14.993993  6686 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:14.995889  6646 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:14.996595  6646 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:14.997089  6646 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:15.004745  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 111514us
W20260709 13:04:15.006559  6732 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.893s	user 0.037s	sys 0.017s
W20260709 13:04:15.007062  6732 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.943s	user 0.037s	sys 0.018s
I20260709 13:04:15.006884  6636 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:15.010087  6286 consensus_queue.cc:1059] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:15.010915  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:15.011945  1568 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 14: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:15.013926  6531 raft_consensus.cc:3055] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:15.014336  6531 raft_consensus.cc:740] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:14.993402  2312 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:15.018994  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 516) took 1205 ms. Trace:
I20260709 13:04:15.019480  1643 rpcz_store.cc:276] 0709 13:04:13.813386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.502813 (+689427us) service_pool.cc:224] Handling call
0709 13:04:14.657192 (+154379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:14.670306 (+ 13114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:15.018755 (+348449us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54002dbc50. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:15.018923 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":262149376}
I20260709 13:04:14.967056  6820 raft_consensus.cc:740] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:15.029150  1538 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:15.031332  1571 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:15.033529  6271 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:15.037832  6736 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:15.038663  6736 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:15.040926  6736 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:15.044327  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:15.046511  6271 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:15.047474  2346 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55890 (request call id 479) took 3865 ms (client timeout 1802 ms). Trace:
W20260709 13:04:15.049285  2346 rpcz_store.cc:269] 0709 13:04:11.182425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.182602 (+   177us) service_pool.cc:224] Handling call
0709 13:04:15.047164 (+3864562us) spinlock_profiling.cc:243] Waited 3.6 s on lock 0x7b5400600254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:15.047351 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8281447296}
W20260709 13:04:15.050933  6809 log.cc:927] Time spent T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:04:15.054446  6748 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.055258  6748 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:15.050639  2314 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:15.058800  6759 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.059526  6759 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:15.061905  6689 raft_consensus.cc:3055] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:15.062388  6689 raft_consensus.cc:740] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:15.062530  1612 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:15.063576  6689 consensus_queue.cc:260] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:15.065017  6689 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:15.015458  6531 consensus_queue.cc:260] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:15.067013  6531 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:15.068722   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:04:15.070147  6759 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:15.079859  6538 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:15.080911  6748 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:15.083494  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.083632  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:15.090334  6646 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:15.071746  6820 consensus_queue.cc:260] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:15.094336  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.094743  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 215395us
W20260709 13:04:15.102248  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.107918  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.108423  6759 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:15.105966  6224 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:15.101964  6286 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.110132  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2326) took 1031 ms. Trace:
I20260709 13:04:15.111927  6781 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:15.113046  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 62550us
I20260709 13:04:15.107625  6560 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:15.110142  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:04:15.109352  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 134842us
I20260709 13:04:15.139307  2346 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:15.116039  6624 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:15.107546  6815 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:15.117122  6753 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:15.102942  6618 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:04:15.145162  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:15.135612  1648 rpcz_store.cc:276] 0709 13:04:14.078595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.577210 (+498615us) service_pool.cc:224] Handling call
0709 13:04:14.593516 (+ 16306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:14.708459 (+114943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:14.708522 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:14.763087 (+ 54565us) log.cc:844] Serialized 26 byte log entry
0709 13:04:14.960378 (+197291us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:14.960434 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.960531 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:15.109742 (+149211us) raft_consensus.cc:1685] finished
0709 13:04:15.109764 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:15.110043 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8025,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":27315,"thread_start_us":189581,"threads_started":1,"wal-append.queue_time_us":344467}
I20260709 13:04:15.157663  1714 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:15.159384  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 166067us
I20260709 13:04:15.161455   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:15.178624  2312 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:15.142529  6560 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:15.180373  1566 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:15.185151  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:15.186090  1565 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:15.195777  6161 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:15.201195  6682 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:15.201757  1538 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:15.202204  6682 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:15.203240  6538 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.203644  1538 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:15.203809  6538 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:15.205950  6533 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.206317  1716 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:04:15.206915  1716 pending_rounds.cc:85] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 5
I20260709 13:04:15.207347  1716 pending_rounds.cc:107] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:04:15.207697  1716 raft_consensus.cc:2889] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:15.208442  6271 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:15.209035  6271 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:15.209431  6271 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:15.206739  6533 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:15.205236  1538 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:15.229715  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.232285  6682 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:15.233256  6747 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:15.237694  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.238785  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 58340.4us
I20260709 13:04:15.183192  6560 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:15.240631  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:15.243742  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 72236.5us
I20260709 13:04:15.208657  6646 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:15.201326  1685 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:15.177327  6736 consensus_queue.cc:1059] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:04:15.180709  6753 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:04:15.176918  6286 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:15.265909  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.183604  6781 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:15.272051  6833 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:15.193228  6815 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:15.265554  6224 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:15.272864  6753 consensus_queue.cc:1243] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:15.307685  6781 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:15.311496  6815 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:15.297703  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:15.299415  5852 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:15.303597  6533 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:15.276077  6453 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:04:15.322225  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 44490.7us
I20260709 13:04:15.321931  1721 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:04:15.323367  2346 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:04:15.332288  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:15.338724  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 137542us
I20260709 13:04:15.340651  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:15.341509  2346 raft_consensus.cc:2368] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 18. Current term is 19.
I20260709 13:04:15.330122  6815 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:15.352459  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 147338us
I20260709 13:04:15.357929  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:15.370265  1686 leader_election.cc:400] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 18. Current term is 19.
I20260709 13:04:15.370687  1686 leader_election.cc:403] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:04:15.350687  6753 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.375501  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:15.377774  1538 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:15.363802  6781 consensus_queue.cc:1059] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:04:15.378482  6798 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:15.378867  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:15.364979  6732 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:15.385462  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 112622us
W20260709 13:04:15.388615  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.395242  6815 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:15.402776  6697 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:15.383790  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:15.380630  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:15.371531  6624 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:15.341223  5852 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:15.411296  6753 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:15.414275  6286 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:15.416123  5852 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:15.417021  1567 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:15.417732  1569 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:15.418169  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:15.420434  1566 raft_consensus.cc:1275] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:15.421478  6697 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:15.423810  6697 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:15.426685  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 49103.4us
I20260709 13:04:15.427966  6828 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:15.438041  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.438688  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.439471  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:15.451429  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:15.452564  1714 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:04:15.453596  1722 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:04:15.460119  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 50261.5us
I20260709 13:04:15.464013  2340 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:04:15.456424  1568 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:04:15.470476  1572 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:15.465011  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:15.483248  1565 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:15.464592  2340 pending_rounds.cc:85] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Aborting all ops after (but not including) 10
I20260709 13:04:15.485582  2340 pending_rounds.cc:107] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:04:15.486912  2340 raft_consensus.cc:2889] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:15.490267  1612 leader_election.cc:400] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 12.
I20260709 13:04:15.491319  1612 leader_election.cc:403] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:04:15.500590  1538 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:15.467427  6624 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:15.509785  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:15.510969  6732 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:15.511090  6629 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 18. Current term is 19.
I20260709 13:04:15.507541  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2327) took 1428 ms. Trace:
I20260709 13:04:15.517678  1647 pending_rounds.cc:85] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 9
I20260709 13:04:15.518118  1647 pending_rounds.cc:107] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:04:15.518373  1647 raft_consensus.cc:2889] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:15.518592  6820 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:15.513831  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.523473  1645 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:15.523635  6863 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:15.524197  6863 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:15.524600  6863 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:15.524663  6798 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:15.525218  6798 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:15.525861  6748 raft_consensus.cc:3055] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:15.526557  6748 raft_consensus.cc:740] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:15.527583  6748 consensus_queue.cc:260] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:15.529117  6748 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:15.534879  1643 raft_consensus.cc:1275] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:15.535907  6697 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:15.536957  1470 catalog_manager.cc:5697] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 6 to 8, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 8 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:15.538335  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 82357.1us
I20260709 13:04:15.539285  6741 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:15.539790  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:15.541795  6731 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:15.542505  6731 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:15.525338  6736 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.544808  6736 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:15.545794  6224 raft_consensus.cc:3055] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:15.546303  6224 raft_consensus.cc:740] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:15.521709  6753 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:15.542183  1470 catalog_manager.cc:5697] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 3 to 4, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 4 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:15.551744  5852 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:15.557842  6618 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:15.558362  6636 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.558531  6618 raft_consensus.cc:697] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:15.558660  6759 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:15.559098  6636 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:15.559650  6618 consensus_queue.cc:237] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:15.520067  6624 raft_consensus.cc:697] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:15.562083  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:15.562021  6624 consensus_queue.cc:237] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:15.562937  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:15.563527  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:15.563627  2340 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:15.562378  6815 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:15.567592  1686 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:15.521850  6732 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:15.517339  1641 rpcz_store.cc:276] 0709 13:04:14.078912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.622970 (+544058us) service_pool.cc:224] Handling call
0709 13:04:14.623859 (+   889us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:04:14.657054 (+ 33195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:14.657098 (+    44us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:04:14.680273 (+ 23175us) log.cc:844] Serialized 104 byte log entry
0709 13:04:14.827936 (+147663us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:15.152522 (+324586us) log.cc:844] Serialized 14 byte log entry
0709 13:04:15.153190 (+   668us) log.cc:844] Serialized 14 byte log entry
0709 13:04:15.154676 (+  1486us) log.cc:844] Serialized 14 byte log entry
0709 13:04:15.167169 (+ 12493us) log.cc:844] Serialized 14 byte log entry
0709 13:04:15.396720 (+229551us) spinlock_profiling.cc:243] Waited 4.23 ms on lock 0x7b1800364720. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:04:15.506746 (+110026us) spinlock_profiling.cc:243] Waited 4.46 ms on lock 0x7b1800364720. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d12c0a 00007f824f452219 00007f8245d9aa40 00007f8245d954e4 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287 00007f8248ea0208
0709 13:04:15.507036 (+   290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:15.507151 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:15.507219 (+    68us) raft_consensus.cc:1685] finished
0709 13:04:15.507239 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:15.507460 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":679347,"prepare.queue_time_us":355510,"prepare.run_cpu_time_us":154,"prepare.run_wall_time_us":157,"raft.queue_time_us":392487,"raft.run_cpu_time_us":1043,"raft.run_wall_time_us":57872,"spinlock_wait_cycles":19980288,"thread_start_us":421515,"threads_started":2,"wal-append.queue_time_us":205063}
I20260709 13:04:15.587342  6638 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:15.592670  6638 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.604118  6638 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:15.598487  6659 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:15.606958  6659 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:15.610282  6859 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.611148  6859 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:15.618635  6860 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:04:15.619374  6659 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:15.622112  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.569115  6636 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:15.628239  6862 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
W20260709 13:04:15.630976  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 91338.2us
W20260709 13:04:15.634115  6531 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.567s	user 0.011s	sys 0.004s
I20260709 13:04:15.635430  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:15.636835  6538 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:15.638556  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:15.589047  6667 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:04:15.649147  1641 raft_consensus.cc:1275] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:15.602270  6616 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.600787  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:15.650982  6616 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:15.652007  1570 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
W20260709 13:04:15.649039  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 161740us
I20260709 13:04:15.661691  6731 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:15.662725  6753 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:15.662966  6736 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:15.663569  6753 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:15.666301  1470 catalog_manager.cc:5697] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 7 to 9, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 9 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:15.670632  1470 catalog_manager.cc:5697] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:15.673012  1644 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
W20260709 13:04:15.667770  1471 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38666 (request call id 215) took 31284 ms (client timeout 15000 ms). Trace:
I20260709 13:04:15.653235  6616 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:15.678992  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:15.673535  1644 pending_rounds.cc:85] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 6
I20260709 13:04:15.682248  1644 pending_rounds.cc:107] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:04:15.682592  1644 raft_consensus.cc:2889] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:15.687445  1646 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:04:15.684461  2312 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:15.698207  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:15.699429  1721 raft_consensus.cc:2393] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 10: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:04:15.712127  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 56577.6us
W20260709 13:04:15.714485  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.714936  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.716516  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:15.717918  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:15.718863  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:15.709587  1639 pending_rounds.cc:85] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 7
I20260709 13:04:15.720271  1639 pending_rounds.cc:107] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:04:15.721225  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:15.722518  6861 log.cc:927] Time spent T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 13:04:15.722690  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:15.723413  2347 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:04:15.723599  6224 consensus_queue.cc:260] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:15.724956  6224 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:15.724514  1470 catalog_manager.cc:5697] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:15.726388  1470 catalog_manager.cc:5697] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 6 to 9, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 9 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:15.581976  6560 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:15.730023  6697 raft_consensus.cc:993] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: : Instructing follower 066ee0109aba4465ba17fa65e14ef504 to start an election
I20260709 13:04:15.730659  6667 raft_consensus.cc:1081] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Signalling peer 066ee0109aba4465ba17fa65e14ef504 to start an election
I20260709 13:04:15.730948  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:15.731164  6560 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:15.674947  1471 rpcz_store.cc:269] 0709 13:03:44.400867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.401185 (+   318us) service_pool.cc:224] Handling call
0709 13:03:44.645746 (+244561us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b54002b0054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248def090 00007f8248e8df3a 00007f824ec441cc 00007f824ed4237e 00007f824d2032a4 00007f824d203232 00007f824d203161 00007f824d2030dc 00007f824d202392 00007f82471c048c 00007f82471bf8c6
0709 13:04:15.667384 (+31021638us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:56.930539 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:57.205299 (+274760us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:57.245820 (+ 40521us) write_op.cc:435] Acquired schema lock
0709 13:03:57.249991 (+  4171us) tablet.cc:662] Decoding operations
0709 13:03:57.499546 (+249555us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:57.499661 (+   115us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:57.499690 (+    29us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:03:57.514005 (+ 14315us) tablet.cc:701] Row locks acquired
0709 13:03:57.524121 (+ 10116us) write_op.cc:260] PREPARE: finished
0709 13:03:57.546453 (+ 22332us) write_op.cc:270] Start()
0709 13:03:57.559300 (+ 12847us) write_op.cc:276] Timestamp: P: 1783602237535915 usec, L: 0
0709 13:03:57.576100 (+ 16800us) op_driver.cc:348] REPLICATION: starting
0709 13:03:57.614177 (+ 38077us) log.cc:844] Serialized 7944 byte log entry
0709 13:03:58.801799 (+1187622us) spinlock_profiling.cc:243] Waited 7.26 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8245d7b0ac 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b
0709 13:03:59.320512 (+518713us) op_driver.cc:464] REPLICATION: finished
0709 13:03:59.893255 (+572743us) spinlock_profiling.cc:243] Waited 5.56 ms on lock 0x7b54002b0054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e8232a 00007f8248e07f14 00007f8248e07ec3 00007f8248e07e38 00007f8248e07df3 00007f8248e070dc 00007f8248e30e16 00007f8248e18cbb 00007f8248e0339d 00007f8248e06d75
0709 13:04:00.105142 (+211887us) write_op.cc:301] APPLY: starting
0709 13:04:00.105258 (+   116us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:01.394596 (+1289338us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:01.394649 (+    53us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:02.907450 (+1512801us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:02.907975 (+   525us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=40,mrs_lookups=0
0709 13:04:02.908056 (+    81us) write_op.cc:312] APPLY: finished
0709 13:04:02.972825 (+ 64769us) log.cc:844] Serialized 180 byte log entry
0709 13:04:03.193988 (+221163us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:03.198464 (+  4476us) write_op.cc:454] Released schema lock
0709 13:04:03.198609 (+   145us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:03.200548 (+  1939us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:03.200612 (+    64us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":21788,"reported_tablets":53,"spinlock_wait_cycles":3054336,"child_traces":[["op",{"apply.queue_time_us":784271,"apply.run_cpu_time_us":84949,"apply.run_wall_time_us":3094951,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87767,"num_ops":20,"prepare.queue_time_us":338809,"prepare.run_cpu_time_us":20534,"prepare.run_wall_time_us":1906235,"raft.queue_time_us":244025,"raft.run_cpu_time_us":35476,"raft.run_wall_time_us":588067,"replication_time_us":1743047,"spinlock_wait_cycles":29475968,"thread_start_us":1919298,"threads_started":5,"wal-append.queue_time_us":1641621,"wal-append.run_cpu_time_us":24629,"wal-append.run_wall_time_us":2640545}]]}
I20260709 13:04:15.721249  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:15.742457  1715 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:15.717011  1640 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:04:15.744390  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:15.751474  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.745297  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:15.720458  1639 raft_consensus.cc:2889] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:15.749789  1721 raft_consensus.cc:1217] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Deduplicated request from leader. Original: 9.6->[11.7-13.8]   Dedup: 11.7->[13.8-13.8]
I20260709 13:04:15.754791  1721 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 13:04:15.761435  1573 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:15.728065  1470 catalog_manager.cc:5697] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 6 to 7, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 7 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:15.750255  1716 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19.
W20260709 13:04:15.772822  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 57065.5us
I20260709 13:04:15.774264  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:15.781098  1538 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:15.709863  6674 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:15.749359  6271 consensus_meta.cc:287] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.539s	user 0.013s	sys 0.001s
I20260709 13:04:15.788362  1565 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:15.790066  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:15.794694  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.795920  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:15.798636  1641 raft_consensus.cc:2410] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:04:15.798004  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:15.798134  6747 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.801616  6747 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:15.803733  6747 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:15.805787  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:15.806594  1714 raft_consensus.cc:2393] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 13: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:15.797981  1611 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:15.807920  2314 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:15.813877  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:15.787362  6741 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 12.
I20260709 13:04:15.747347  6667 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:04:15.791507  6674 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.817360  1644 raft_consensus.cc:2393] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 7: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:15.819887  1565 raft_consensus.cc:1240] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:15.823333  1645 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:04:15.813133  1470 catalog_manager.cc:5697] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 7 to 9, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 9 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:04:15.797013  6560 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.824674  1470 catalog_manager.cc:5697] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 6 to 10, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 10 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:15.833473  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.834229  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:15.826428  1612 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:15.834786  6781 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:15.792531  6616 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:04:15.821146  6736 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:15.798458  6271 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:15.806782  6659 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.826805  6674 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:15.858527  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:15.863991  1721 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:04:15.869711  1713 raft_consensus.cc:2410] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:04:15.844411  6618 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:15.875654  1470 catalog_manager.cc:5697] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:15.859928  6736 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:15.897116  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 116132us
I20260709 13:04:15.891927  1685 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:15.848104  6747 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:15.915290  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 89503.8us
W20260709 13:04:15.919140  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.878641  6833 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.582s	user 0.007s	sys 0.006s
I20260709 13:04:15.878715  6618 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.914395  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:15.921558  1611 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:15.894881  1470 catalog_manager.cc:5697] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:04:15.920935  1564 consensus_meta.cc:287] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.502s	user 0.014s	sys 0.000s
I20260709 13:04:15.923454  1564 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:15.859745  6560 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:15.929980  1537 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:15.899736  6736 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:15.933561  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:15.936986  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:15.936121  1538 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:15.959872  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.925845  6833 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:15.925298  6618 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:15.928619  6624 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.936726  6741 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:15.887122  6659 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:15.976042  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:15.951361  6646 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:16.001317  6624 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:15.993499  6828 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:16.003539  6828 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:16.003741  6659 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:16.004009  6828 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:16.004593  6624 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:16.004714  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 128791us
W20260709 13:04:16.002140  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 66114.8us
I20260709 13:04:16.005944  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:16.006857  1685 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:16.008622  6538 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:16.009171  6624 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:16.009326  6685 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:16.009845  6624 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.010226  6859 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:16.010478  6624 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:16.013852  6646 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:16.014436  6646 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:16.014256  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:16.014667  1685 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:15.931520  6732 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:16.018181  6732 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:16.018229  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:16.019078  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 99632.1us
I20260709 13:04:16.019946  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:16.020033  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:16.021584  6859 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:16.023067  6868 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.079s	user 0.002s	sys 0.000s
I20260709 13:04:15.935184  6753 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:16.025839  6732 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.027498  6732 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:16.030444  1538 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:16.031070  6560 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:15.943688  6531 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.094s
I20260709 13:04:16.031996  1470 catalog_manager.cc:5697] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 5 to 10, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 10 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:16.033844  6531 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:16.035984  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:16.035616  1470 catalog_manager.cc:5697] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 6 to 9, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 9 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:16.036749  1564 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
W20260709 13:04:16.036473  6865 log.cc:927] Time spent T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:04:16.038036  1565 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
I20260709 13:04:16.038318  2312 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:16.039652  6753 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:16.040019  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:16.040490  6753 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.040992  6753 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:16.041560  2312 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:16.042621  6873 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:16.043126  6873 raft_consensus.cc:697] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:16.044188  6873 consensus_queue.cc:237] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:16.009208  6864 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:16.046298  6864 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.046722  6864 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:16.049767  6667 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:04:16.050426  6866 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:04:16.050525  6667 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:16.052711  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 178001us
W20260709 13:04:16.054859  6871 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:04:16.032819  1538 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:16.034957  6732 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:16.070328  1611 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:16.072017  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.977342  6748 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:16.038460  6867 log.cc:927] Time spent T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:04:16.098078  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:16.095975  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:16.101182  2342 raft_consensus.cc:1813] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Handling vote request from an unknown peer f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:16.102013  2342 raft_consensus.cc:2410] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 12 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:04:15.988029  6638 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:16.103173  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 70529.8us
I20260709 13:04:16.103871  6667 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:16.105018  6638 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:16.094760  6722 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:16.106006  6815 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:16.106619  6815 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:16.110121  1538 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:16.075384  1470 catalog_manager.cc:5697] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 5 to 8, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 8 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:16.105012  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 51423.3us
I20260709 13:04:16.075634  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:16.103404  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:16.116521  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 101932us
I20260709 13:04:16.118170  1722 raft_consensus.cc:2393] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 14: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:16.068091  6685 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:16.074699  6820 consensus_meta.cc:287] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.527s	user 0.006s	sys 0.007s
I20260709 13:04:16.072537  6759 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.139165  6747 consensus_queue.cc:1048] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:16.120221  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:16.139357  6759 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:16.141716  1685 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:16.143193  1685 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:16.095150  6659 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:16.116164  6863 consensus_meta.cc:287] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.591s	user 0.010s	sys 0.002s
I20260709 13:04:16.144447  6863 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:16.103585  6859 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:16.145951  1611 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:16.146880  6863 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:16.146549  6533 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:16.147673  1611 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:16.141726  6743 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:04:16.148649  6722 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:16.125800  6161 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:16.136770  1648 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:04:16.108877  6815 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:16.131989  6538 consensus_queue.cc:1048] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:04:16.115056  6748 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:16.138355  5852 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:16.149443  1611 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:16.139386  1470 catalog_manager.cc:5697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:15.998715  6798 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:16.149905  6722 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:16.208590  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:16.204075  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:16.210254  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 60958.1us
I20260709 13:04:16.205528  1570 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:16.149556  6812 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:16.150183  6773 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:16.209249  6748 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:16.211628  1611 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:16.212980  6748 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:16.214507  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:16.215607  2344 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:16.217204  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:16.218626  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:16.219383  1569 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:04:16.145871  6685 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:16.222018  6616 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:16.222422  2342 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:16.211658  6812 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.223539  6812 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:16.170318  6659 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:16.223245  6736 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:16.224710  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:16.149849  6286 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:16.225471  1573 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 9.
I20260709 13:04:16.227494  1538 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:16.229677  5852 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:16.231949  1571 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:16.235266  6616 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:16.203536  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:16.237430  1568 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:16.246719  2347 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
W20260709 13:04:16.251847  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.153605  6161 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:16.066643  6636 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.270421  1612 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:16.122201  6732 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:16.271730  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:16.285750  1715 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:16.289991  2313 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:16.277783  6722 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.285717   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:16.242512  6271 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:16.292497  2313 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:16.292729  1611 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:16.296062  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 68929.2us
I20260709 13:04:16.301940  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:16.302758  1538 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:16.252252  6286 raft_consensus.cc:697] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:16.278975  6161 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:16.298327  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:16.251988  6773 raft_consensus.cc:697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:16.297796  1720 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:04:16.224326  6659 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:16.304280  2313 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:16.133481  6638 consensus_queue.cc:1059] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.164s
I20260709 13:04:16.304843  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:16.252080  6759 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:16.325448  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:16.335059  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.304270  6859 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:16.340279  6638 raft_consensus.cc:3055] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:16.340776  6638 raft_consensus.cc:740] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:16.341770  6638 consensus_queue.cc:260] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:16.343209  6638 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:16.327097  6873 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:16.346596  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:16.347155  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:16.352636  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 50005.4us
I20260709 13:04:16.353685  1538 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:16.355172  1538 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:16.326508  6748 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:16.358157  6624 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:16.368851  6646 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:16.322355  6629 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:16.388976  6747 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:16.389456  6747 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.363130  1568 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:16.390319  6747 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:16.361310  6659 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:16.359163  6828 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:16.365290  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:16.365760  6859 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.392755  1569 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:16.367880  6798 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:16.365512  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:16.371165  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:16.297693  6722 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:16.373852  6685 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:16.374919  6286 consensus_queue.cc:237] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:16.336220  6732 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:16.377240  6736 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:16.403177  1722 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:16.377827  6781 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:16.395880  6859 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:16.409124  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 54107.9us
I20260709 13:04:16.411190  1538 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:16.378511  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:16.358355  6773 consensus_queue.cc:237] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:16.335901  1470 catalog_manager.cc:5697] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 7 to 11, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 11 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:16.385190  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:16.385785  6271 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:16.385797  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:16.386111  6224 consensus_meta.cc:287] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.661s	user 0.007s	sys 0.007s
I20260709 13:04:16.387020  6533 consensus_queue.cc:1059] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:16.388391  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:16.283810  6636 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:16.391243  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:16.363256  1572 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:40670
I20260709 13:04:16.392933  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:16.384181  1685 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:16.392099  5852 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:16.427518  1572 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:16.396399  1570 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:16.399087  1565 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:04:16.380968  6759 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:16.403005  1564 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 14. Current term is 15. Ops: []
W20260709 13:04:16.429203  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45103.5us
I20260709 13:04:16.428349  1572 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:16.345345  6812 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:16.431694  1685 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:16.438776  1685 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:16.402642  6659 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:16.440841  5852 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:16.440109  1470 catalog_manager.cc:5697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 6 to 10, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 10 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:16.448807  1570 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:16.437565  6743 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:04:16.452529  6624 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:16.437703  6689 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:04:16.443497  6815 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:16.456396  6636 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:16.462423  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:16.506673  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 95703.8us
I20260709 13:04:16.437350  6538 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:16.505889  6636 consensus_queue.cc:1059] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:04:16.511610  6646 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:16.512766  6722 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:16.453233  6287 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:16.513865  6287 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:16.514295  6287 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:16.511638  1470 catalog_manager.cc:5697] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:04:16.538041  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 215942us
I20260709 13:04:16.515813  6538 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:16.542498  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:16.485441  6659 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:16.535053  6674 consensus_meta.cc:287] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.643s	user 0.012s	sys 0.002s
I20260709 13:04:16.565583  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:16.532472  6815 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.538187  6833 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:16.573491  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:16.574278  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 290734us
I20260709 13:04:16.545514  6538 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:16.579245  1611 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:16.545595  6722 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:16.535909  6743 consensus_queue.cc:1059] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:04:16.584216  1538 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:16.588680  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:16.583288  6616 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:16.536942  6753 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:16.568580  6883 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:04:16.638737  1611 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:16.640436  2347 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 12: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:16.572685  6674 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:16.631778  6618 consensus_meta.cc:287] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.557s	user 0.012s	sys 0.000s
I20260709 13:04:16.674273  1639 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:04:16.675305  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:16.577133  6815 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:16.684703  1565 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:16.592320  6885 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.095s
I20260709 13:04:16.691215  6674 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:16.692296  6878 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:04:16.692657  6859 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:16.716679  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 143742us
I20260709 13:04:16.718613  6859 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:16.719547  6864 consensus_meta.cc:287] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.672s	user 0.015s	sys 0.000s
I20260709 13:04:16.673940  6883 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:16.720032  6864 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:16.696828  6689 raft_consensus.cc:3055] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:16.721843  6689 raft_consensus.cc:740] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:16.722921  6863 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:16.697748  2346 raft_consensus.cc:2393] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:16.654369  6828 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:16.698856  6833 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:16.597892  6722 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:16.726711  6833 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:16.728356  6636 raft_consensus.cc:3055] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:16.728852  6636 raft_consensus.cc:740] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:16.730226  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:16.730032  6636 consensus_queue.cc:260] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:16.732120  6636 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:16.732651  1537 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:16.734107  5852 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:16.734234  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:16.734524  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:16.734984  1643 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:04:16.736183  2313 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:16.737118  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.737661  6859 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:16.738059  1641 raft_consensus.cc:1275] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:16.737203  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:16.736068  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:16.739220  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:16.739806  1646 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:16.722862  6689 consensus_queue.cc:260] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:16.734525  5852 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.744470  5852 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:16.745558  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:16.752848  1564 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
W20260709 13:04:16.747041  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:16.764678  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.665027  6732 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:16.790644  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:16.791785  1567 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 20: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:16.793006  6812 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:16.799283  6753 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:16.803732  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:16.804580  1568 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:16.811569  1567 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:04:16.812392  6879 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 13:04:16.814229  6667 raft_consensus.cc:3055] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:16.814656  6667 raft_consensus.cc:740] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:16.815771  6667 consensus_queue.cc:260] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:16.819896  6667 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:16.703814  6618 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:16.823345  1644 raft_consensus.cc:1240] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:16.832989  6722 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:16.837914  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 199331us
W20260709 13:04:16.856482  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:16.872973  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 137049us
I20260709 13:04:16.873494  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:16.877178  1686 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:16.875123  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.875972  1647 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:16.882817  6689 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:16.884939  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:16.894079  1567 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 20: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:16.900111  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:16.914938  1713 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 20: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:16.895810  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:16.917619  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:16.917955  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:16.918391  1643 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:16.918608  2347 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 20: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:16.892681  1572 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:16.921941  1611 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:16.926278  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:16.927170  1645 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:16.926502  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:16.930177  1565 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:16.942492  1716 raft_consensus.cc:1275] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:16.895336  6732 raft_consensus.cc:697] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:16.852468  6859 consensus_queue.cc:1059] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.137s
I20260709 13:04:16.956904  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:16.963724  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 48595.8us
I20260709 13:04:16.965648  1640 raft_consensus.cc:2410] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:04:16.919555  6646 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.967062  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2350) took 1076 ms. Trace:
I20260709 13:04:16.980325  1646 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:16.946457  6743 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.922782  6659 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:16.993242  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:16.994124  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:17.010253  2347 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 20: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:16.998875  1639 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:04:17.012586  1569 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:16.950389  6812 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:17.015480  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:16.961922  6722 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:17.023913  1611 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:17.024197  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:16.976953  6864 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:17.007457  6618 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:16.977533  1718 rpcz_store.cc:276] 0709 13:04:15.778968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.805849 (+ 26881us) service_pool.cc:224] Handling call
0709 13:04:15.827969 (+ 22120us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:15.854431 (+ 26462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:15.854477 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:16.002414 (+147937us) spinlock_profiling.cc:243] Waited 3.53 ms on lock 0x7b2c00013bb8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824777ea11 00007f824778e372 00007f8248ec0a01 00007f8248e8543a 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2
0709 13:04:16.002966 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 13:04:16.004335 (+  1369us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b180046c180. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d98522 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:04:16.398037 (+393702us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:16.398083 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.400311 (+  2228us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:16.811791 (+411480us) spinlock_profiling.cc:243] Waited 2.93 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:16.855361 (+ 43570us) raft_consensus.cc:1685] finished
0709 13:04:16.855396 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:16.855581 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7329,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":17425,"spinlock_wait_cycles":21939200,"thread_start_us":394587,"threads_started":1,"wal-append.queue_time_us":531802}
I20260709 13:04:17.006247  6646 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:17.023295  6638 consensus_meta.cc:287] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.677s	user 0.009s	sys 0.009s
I20260709 13:04:16.955823  6629 consensus_queue.cc:1059] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 13:04:17.042984  1572 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:17.035706  1611 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:17.034780  6161 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.696s	user 0.015s	sys 0.004s
I20260709 13:04:17.054838  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:17.055771  1646 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:17.032725  6828 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:17.015955  6743 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:17.034014  6722 raft_consensus.cc:697] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:17.041679  6883 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:17.043089  6893 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:17.085549  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 181192us
W20260709 13:04:17.093441  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 216379us
I20260709 13:04:17.097019  1645 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
W20260709 13:04:17.084102  6748 consensus_meta.cc:287] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.699s	user 0.016s	sys 0.003s
I20260709 13:04:17.098276  1565 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 12.
I20260709 13:04:17.088891  1685 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:17.051328  5852 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:17.061260  6618 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:17.113083  5852 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:17.081830  6883 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.119903  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:17.108695  6287 consensus_meta.cc:287] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.594s	user 0.011s	sys 0.006s
I20260709 13:04:17.140573  6287 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:17.143752  6287 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:17.144659  6287 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:17.145207  6287 raft_consensus.cc:697] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:17.117264  6748 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:17.122532  1645 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:17.068816  6828 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:17.126019  6743 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:17.127012  6646 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:17.130072  6722 consensus_queue.cc:237] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:17.138320  6895 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
W20260709 13:04:17.139475  6898 log.cc:927] Time spent T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:04:17.078523  6732 consensus_queue.cc:237] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:17.153287  1568 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:04:17.060417  6161 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:17.093786  6893 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:17.122275  6815 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:17.121766  6883 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:17.176901  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:17.180387  2312 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:17.196588  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:17.199265  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:17.199810  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:17.200611  1565 raft_consensus.cc:2393] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 13: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:17.198467  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:17.162124  6224 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.207043  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:17.209152  1567 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 12: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:17.209321  1564 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:17.217598  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:17.219646  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:17.222057  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:17.222889  1572 raft_consensus.cc:2393] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 13: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:17.225574  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:17.236594  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:17.237003  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:17.237391  1713 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:17.238642  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:17.239554  5852 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:17.240656  1718 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 20: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:17.248109  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:17.248898  1716 raft_consensus.cc:3055] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:17.249361  1716 raft_consensus.cc:740] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:17.249378  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:17.261150  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:17.265347  1564 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:17.273237  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:17.286639  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:17.287786  1647 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:17.212095  6618 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:17.250859  6747 consensus_meta.cc:287] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.860s	user 0.016s	sys 0.000s
I20260709 13:04:17.306854  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:17.307070  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:17.313752  1641 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:17.309010  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:17.314510  1645 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:17.289951  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:17.317328  1572 raft_consensus.cc:2393] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 12: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:17.326639  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:17.327695  1647 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:17.276538  6659 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.331678  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:17.283310  6674 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:04:17.336859  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 248097us
I20260709 13:04:17.320261  1716 consensus_queue.cc:260] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:17.315618  6731 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:17.339251  1716 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:17.339489  6731 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:17.346704  1718 raft_consensus.cc:1275] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:17.338003  1685 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:17.348279  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 167965us
I20260709 13:04:17.349551  2312 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:17.350099  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:17.352980  1647 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:17.353155  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 243861us
I20260709 13:04:17.225245  6224 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:17.359012  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:17.361600  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:17.362715  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:17.363515  1716 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:17.269241  6815 consensus_queue.cc:1048] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 13:04:17.319516  6743 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:04:17.375531  1611 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:17.363543  1642 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:17.335044  6748 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:17.392843  6748 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:17.267599  6287 consensus_queue.cc:237] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:17.394761  2347 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:17.352443  6885 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:17.401024  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:17.401532  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:17.361094  6781 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:17.322811  6747 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:17.405555  2346 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
W20260709 13:04:17.409958  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 72086.1us
I20260709 13:04:17.335750  6646 consensus_queue.cc:1048] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:04:17.411492  1538 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:17.333153  6618 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:17.419477  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:17.416998  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 86478.5us
I20260709 13:04:17.437911  1648 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:17.440200  1538 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:17.414392  6616 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:17.443621  6863 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:17.446636  6616 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.447623  6616 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:17.449121  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:17.481300  1717 raft_consensus.cc:2393] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 14: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:17.358340  6659 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:17.436985  6689 consensus_meta.cc:287] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.554s	user 0.014s	sys 0.000s
I20260709 13:04:17.484944  1569 raft_consensus.cc:1240] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 7. Current term is 8. Ops: []
W20260709 13:04:17.487010  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 46941.8us
I20260709 13:04:17.434790  6646 consensus_queue.cc:1243] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:04:17.503026  1538 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:17.522028  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 84535.7us
I20260709 13:04:17.525302  1538 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:17.527859  1538 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:17.538714  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 138508us
I20260709 13:04:17.498579  6286 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:17.483390  6908 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:17.420046  6885 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:17.545469  1538 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:17.550596  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 175239us
W20260709 13:04:17.551265  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 197088us
W20260709 13:04:17.529451  6900 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.364s	user 0.000s	sys 0.002s
I20260709 13:04:17.529143  6863 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:17.559276  5852 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:17.564513  5852 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:17.565181  5852 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:17.561008  1538 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:17.568385  6913 log.cc:927] Time spent T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:04:17.567929  6932 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:17.569972  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2403) took 1108 ms. Trace:
I20260709 13:04:17.570465  1566 rpcz_store.cc:276] 0709 13:04:16.461412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.825156 (+363744us) service_pool.cc:224] Handling call
0709 13:04:16.825556 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:16.829432 (+  3876us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:16.829519 (+    87us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:16.872705 (+ 43186us) log.cc:844] Serialized 26 byte log entry
0709 13:04:17.297824 (+425119us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:17.297874 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:17.297954 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:17.569682 (+271728us) raft_consensus.cc:1685] finished
0709 13:04:17.569708 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:17.569894 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":993,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":270,"thread_start_us":411420,"threads_started":1,"wal-append.queue_time_us":499556}
I20260709 13:04:17.575471  6885 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:17.540297  6828 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:17.577627  2340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55890 (request call id 506) took 1046 ms. Trace:
I20260709 13:04:17.551679  6271 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:17.549430  6798 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.605029  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:17.549512  6893 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:17.550621  6286 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:17.587349  6903 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 13:04:17.588105  6161 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:17.588797  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:17.588960  6919 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.562315  6815 consensus_queue.cc:1059] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:04:17.597934  6773 raft_consensus.cc:993] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: : Instructing follower 066ee0109aba4465ba17fa65e14ef504 to start an election
I20260709 13:04:17.578429  2340 rpcz_store.cc:276] 0709 13:04:16.531366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.531565 (+   199us) service_pool.cc:224] Handling call
0709 13:04:16.546456 (+ 14891us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:16.558961 (+ 12505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:16.558994 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:16.573797 (+ 14803us) log.cc:844] Serialized 26 byte log entry
0709 13:04:16.907149 (+333352us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:16.907190 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.907438 (+   248us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:17.577246 (+669808us) raft_consensus.cc:1685] finished
0709 13:04:17.577285 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:17.577531 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11741,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":10424,"thread_start_us":286514,"threads_started":1,"wal-append.queue_time_us":523617}
I20260709 13:04:17.530966  6910 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:17.605103  1648 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:17.599908  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2387) took 1238 ms. Trace:
I20260709 13:04:17.516918  6909 consensus_queue.cc:1048] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
W20260709 13:04:17.625248  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:17.756378  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.576828  6885 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:17.646409  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 95674.6us
W20260709 13:04:17.737476  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 179415us
I20260709 13:04:17.740715  5852 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:17.670881  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:17.737453  2342 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
W20260709 13:04:17.624115  6833 consensus_meta.cc:287] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.897s	user 0.022s	sys 0.011s
I20260709 13:04:17.581007  6863 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:17.789402  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 230443us
W20260709 13:04:17.893059  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 332117us
I20260709 13:04:17.893921  1611 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:17.737593  6616 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:17.786520  6271 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.752563  6732 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:17.894549  5852 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:17.650820  6798 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:17.895762  6538 consensus_meta.cc:287] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.302s	user 0.016s	sys 0.005s
I20260709 13:04:17.896152  6616 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:17.896265  6538 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:17.652371  6286 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:17.897156  6616 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:17.897882  1721 raft_consensus.cc:1240] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:17.606352  6917 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.136s
I20260709 13:04:17.756799  6919 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:17.877475  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:17.877998  1716 raft_consensus.cc:2410] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:04:17.879634  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:17.653851  6748 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:17.624435  6674 raft_consensus.cc:697] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:17.886049  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:17.886394  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:17.745687  6781 raft_consensus.cc:697] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:17.752851  1571 rpcz_store.cc:276] 0709 13:04:16.361831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.362246 (+   415us) service_pool.cc:224] Handling call
0709 13:04:16.362641 (+   395us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:16.389959 (+ 27318us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:16.568109 (+178150us) log.cc:844] Serialized 14 byte log entry
0709 13:04:16.703016 (+134907us) spinlock_profiling.cc:243] Waited 7.42 ms on lock 0x7b180017dac0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a2523a 00007f8248e8fa2a
0709 13:04:16.961251 (+258235us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:17.026633 (+ 65382us) log.cc:844] Serialized 52 byte log entry
0709 13:04:17.026821 (+   188us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:17.027091 (+   270us) log.cc:844] Serialized 14 byte log entry
0709 13:04:17.586263 (+559172us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b180017dac0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d12c0a 00007f824f452219 00007f8245d9aa40 00007f8245d954e4 00007f8245d95455 00007f824c87d3c7 00007f8248e8f9a7 00007f8248e9f0bf 00007f8248e9f038
0709 13:04:17.586548 (+   285us) log.cc:844] Serialized 14 byte log entry
0709 13:04:17.599530 (+ 12982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:17.599630 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:17.599676 (+    46us) raft_consensus.cc:1685] finished
0709 13:04:17.599688 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:17.599844 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":648832,"prepare.queue_time_us":795436,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":130,"raft.queue_time_us":76839,"raft.run_cpu_time_us":663,"raft.run_wall_time_us":72344,"spinlock_wait_cycles":23767296,"thread_start_us":435236,"threads_started":2,"wal-append.queue_time_us":493458}
I20260709 13:04:17.894691  1538 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:17.915409  1718 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
W20260709 13:04:17.899785  6920 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.133s	user 0.001s	sys 0.001s
I20260709 13:04:17.895633  6732 raft_consensus.cc:697] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:17.919409  1538 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:17.931107  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 323140us
I20260709 13:04:17.883877  6927 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:17.934659  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 47951.8us
W20260709 13:04:17.919175  6911 log.cc:927] Time spent T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
I20260709 13:04:17.939435  1687 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:17.899323  6863 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:17.939880  1537 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:17.935853  1538 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:17.941888  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 64524.8us
W20260709 13:04:17.942981  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49163.2us
I20260709 13:04:17.943708  1611 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:17.914907  6747 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:17.930958  6743 raft_consensus.cc:1081] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Signalling peer 066ee0109aba4465ba17fa65e14ef504 to start an election
I20260709 13:04:17.929409  6864 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:17.918737  6618 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:17.931867  6932 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.905337  6885 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:17.963883  1687 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:17.901495  6833 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:17.934913  6928 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
W20260709 13:04:18.014619  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 64589.8us
I20260709 13:04:17.948854  6908 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:17.938539  6689 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:17.936298  6815 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:18.008530   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:04:18.010546  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 47583.9us
I20260709 13:04:17.905069  6271 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:17.946782  6659 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:18.014853  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 79171.2us
I20260709 13:04:18.027146  1538 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:18.015405  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:18.028795  1568 raft_consensus.cc:2410] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:04:18.015836  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:18.030128  1573 raft_consensus.cc:2410] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:04:18.042265  6618 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:18.042567  6748 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:18.060292  6939 log.cc:927] Time spent T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:04:18.060144  6748 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.061779  6926 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:18.015384  6940 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:18.062839  6864 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:18.062651  6940 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.048254  1537 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:17.964924  6916 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:18.046325  1647 consensus_meta.cc:287] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.668s	user 0.010s	sys 0.004s
I20260709 13:04:18.077171  1647 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:18.092674  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:18.008417  6929 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:18.044853  6747 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:17.944885  6910 consensus_queue.cc:1048] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.131s
I20260709 13:04:18.045114  6815 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:18.122877  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:18.124095  1647 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:18.067250  6908 raft_consensus.cc:697] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:18.069739  6931 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:18.133121  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 189490us
I20260709 13:04:17.945421  6927 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.170702  1611 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:18.181661  6938 log.cc:927] Time spent T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:04:18.183260  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:18.011631  6932 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:18.051062  6689 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.119378  6833 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:18.223026  6946 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:04:18.232214  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 49155.8us
W20260709 13:04:18.239861  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 276064us
I20260709 13:04:18.131634  6815 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:18.240573  1538 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:18.241184  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 70625.1us
I20260709 13:04:18.242051  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:18.243498  1538 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:18.066439  6674 consensus_queue.cc:237] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:18.184391  6667 consensus_meta.cc:287] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.364s	user 0.016s	sys 0.002s
I20260709 13:04:18.129365  6732 consensus_queue.cc:237] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.242090  1611 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:18.239288  1647 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 12.
I20260709 13:04:18.049256  6781 consensus_queue.cc:237] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:18.259479  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:18.192520  6538 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:18.238643  6942 raft_consensus.cc:2764] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 13:04:18.252151  6937 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 13:04:18.233158  6820 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.324811  5852 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:18.327070  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:18.245008  6946 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:18.333873  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 159079us
W20260709 13:04:18.334506  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.329463  6940 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:18.260309  6864 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:18.262950  6616 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:18.335412  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 94894.9us
I20260709 13:04:18.334880  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:18.242897  6812 consensus_queue.cc:1059] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.095s
I20260709 13:04:18.346529  6748 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:18.337684  6946 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:18.394934  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 49593.3us
I20260709 13:04:18.370301  6864 raft_consensus.cc:697] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:18.409001  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 166998us
I20260709 13:04:18.410363  1611 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:18.344352  6820 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:18.400159  6636 consensus_meta.cc:287] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.667s	user 0.016s	sys 0.004s
I20260709 13:04:18.417120  1611 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:18.395610  6859 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:18.422580  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2383) took 1054 ms. Trace:
I20260709 13:04:18.424527  1572 rpcz_store.cc:276] 0709 13:04:17.368660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.377066 (+  8406us) service_pool.cc:224] Handling call
0709 13:04:17.396160 (+ 19094us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:17.396443 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:17.396473 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:17.412063 (+ 15590us) log.cc:844] Serialized 26 byte log entry
0709 13:04:17.638191 (+226128us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:17.638238 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:17.638331 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:18.422363 (+784032us) raft_consensus.cc:1685] finished
0709 13:04:18.422386 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:18.422513 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":156801,"threads_started":1,"wal-append.queue_time_us":497821}
I20260709 13:04:18.411175  6919 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:18.422803  1611 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:18.449929  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 115117us
I20260709 13:04:18.448675  1611 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:18.458283  1611 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:18.420704  6748 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:18.415885  6942 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:18.415948  6674 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:18.419039  6624 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:18.429544  6929 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.454778  6753 raft_consensus.cc:3055] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:18.460104  6959 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:18.456060  6271 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:18.468014  6944 log.cc:927] Time spent T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.066s	user 0.000s	sys 0.003s
I20260709 13:04:18.380254  6908 consensus_queue.cc:237] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.457458  6638 raft_consensus.cc:3055] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:04:18.539162  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 81011.3us
I20260709 13:04:18.587389  1611 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:18.535569  6933 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.544401  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2363) took 2003 ms. Trace:
I20260709 13:04:18.595970  1611 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:18.467641  6748 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:18.558419  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 102920us
I20260709 13:04:18.471607  6917 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:18.479379  6812 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:18.450362  6859 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:18.474746  1687 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:18.538265  6753 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:18.536020  6952 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:04:18.583209  6883 consensus_meta.cc:287] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.373s	user 0.013s	sys 0.004s
I20260709 13:04:18.596735  1721 rpcz_store.cc:276] 0709 13:04:16.541035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.541563 (+   528us) service_pool.cc:224] Handling call
0709 13:04:16.567295 (+ 25732us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:17.898387 (+1331092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.543978 (+645591us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400248954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:18.544301 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3059821952}
I20260709 13:04:18.579449  6271 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:18.563002  6907 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.526296  6624 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.580044  6659 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:18.579214  6927 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:18.601617  6748 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:18.623710  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:18.518929  6674 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:18.617733  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 143355us
I20260709 13:04:18.586048  6638 raft_consensus.cc:740] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:18.562359  6951 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:18.584815  6906 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.603806  6948 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:18.623939  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:18.624902  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:18.555444  6953 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.200s
I20260709 13:04:18.639104  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:18.639395  1718 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:18.518554  6942 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:18.526175  6929 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.640137  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:18.610124  6538 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.618531  6932 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:18.643364  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:18.621359  6945 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:18.584906  6864 consensus_queue.cc:237] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.667343  6945 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:18.656889  6947 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.327s	user 0.000s	sys 0.002s
I20260709 13:04:18.609323  6917 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:18.634729  6952 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.642813  6958 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:18.617879  6812 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:18.688311  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 92481.2us
I20260709 13:04:18.660776  6722 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:18.647051  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:18.682060  6964 log.cc:927] Time spent T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 13:04:18.635210  6885 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:18.626157  6955 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:18.602329  6933 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:18.682621  6974 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.621438  6883 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.682262  6966 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:18.642994  6271 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.716599  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:18.716985  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:18.685461  6538 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.731765  1538 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:18.727142  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:18.631398  6907 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:18.711165  6820 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:18.734745  6971 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:18.744812  6820 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:18.745357  6820 raft_consensus.cc:697] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:18.746017  6812 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:18.746629  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:18.746308  6820 consensus_queue.cc:237] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.747931  6883 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:18.748351  1715 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
W20260709 13:04:18.750280  6950 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 13:04:18.750775  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:18.750959  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:18.728039  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:18.752329  1648 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:18.708302  6936 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:18.647302  6906 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.644800  6948 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:18.714376  6927 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:04:18.734720  6956 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.167s	user 0.000s	sys 0.002s
W20260709 13:04:18.734892  6954 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.283s	user 0.000s	sys 0.002s
W20260709 13:04:18.736238  6960 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.278s	user 0.000s	sys 0.002s
I20260709 13:04:18.737780  6978 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.738826  6731 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:18.772174  6947 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:04:18.670346  6753 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:18.743023  6967 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:04:18.782768  6953 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:18.722090  6962 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:18.724428  6982 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:18.801911  6965 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:18.786892  6936 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.817546  6965 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:18.802837  6978 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.820096  6936 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:18.820578  6965 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:18.820916  6968 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:18.821225  6978 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:18.821557  6968 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.823676  6968 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:18.729933  6955 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:18.824285  6907 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:18.824544  6286 consensus_meta.cc:287] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.926s	user 0.012s	sys 0.002s
I20260709 13:04:18.825009  6286 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.741820  6973 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.731217  6685 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:18.703697  6638 consensus_queue.cc:260] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.797441  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:18.779497  6931 consensus_meta.cc:287] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.542s	user 0.006s	sys 0.006s
I20260709 13:04:18.746675  6812 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:18.706081  6952 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.799072  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:18.802245  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:18.870146  1572 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
W20260709 13:04:18.735065  6949 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.287s	user 0.001s	sys 0.001s
I20260709 13:04:18.704882  6958 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.800508  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:18.804545  1648 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:18.883211  1568 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:04:18.741923  6975 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:18.895829  6975 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:18.817099  6982 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.905329  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:18.828703  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 106450us
W20260709 13:04:18.799223  6863 consensus_meta.cc:287] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.845s	user 0.003s	sys 0.010s
I20260709 13:04:18.838599  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:18.849865  1714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:18.800761  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:18.913389  1564 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:18.865468  6722 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:18.866786  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 135115us
I20260709 13:04:18.920491  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:18.867560  1573 raft_consensus.cc:2410] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:04:18.866506  2313 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:18.725728  6966 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:18.873519  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:18.924919  1571 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:18.814107  6953 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.723331  6974 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.884248  1645 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: [9.6-12.8]
I20260709 13:04:18.853120  1687 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:18.904927  6960 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:04:18.921437  1564 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:18.935809  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:18.936587  1572 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:04:18.880422  6931 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:18.928231  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 62577us
I20260709 13:04:18.914234  1714 raft_consensus.cc:606] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:18.876483  6955 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:18.902859  6624 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:18.932912  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:18.872215  6685 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.932751  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2380) took 1638 ms. Trace:
W20260709 13:04:18.955658  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 102702us
I20260709 13:04:18.913671  6674 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:18.884094  6812 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.940297  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:18.924237  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:18.854985  6962 consensus_queue.cc:1059] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:04:18.976549  1648 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:18.976824  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:18.977595  2347 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:18.958810  1564 raft_consensus.cc:2410] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:04:18.977964  1538 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:18.953961  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:18.925185  6959 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:18.944438  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:18.985054  5852 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.931975  6968 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:18.963495  6996 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:18.992125  1639 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:04:18.963761  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:18.964404  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:18.968544  6971 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:18.970850  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:18.964404  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:18.926220  6885 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:18.956386  1567 rpcz_store.cc:276] 0709 13:04:17.294618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.294776 (+   158us) service_pool.cc:224] Handling call
0709 13:04:17.311679 (+ 16903us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:17.347021 (+ 35342us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:17.598074 (+251053us) log.cc:844] Serialized 14 byte log entry
0709 13:04:18.121977 (+523903us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:18.187743 (+ 65766us) log.cc:844] Serialized 26 byte log entry
0709 13:04:18.206663 (+ 18920us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:18.206722 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.206814 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:18.932424 (+725610us) raft_consensus.cc:1685] finished
0709 13:04:18.932471 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:18.932691 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":324895,"prepare.queue_time_us":242696,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":8770,"raft.queue_time_us":139704,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":42792,"thread_start_us":455975,"threads_started":2,"wal-append.queue_time_us":537896}
I20260709 13:04:18.954380  1570 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:04:18.990558  1687 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:18.934665  6936 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.934684  6953 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:18.985808  5852 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.999104  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2760) took 1399 ms. Trace:
I20260709 13:04:18.985320  1571 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:18.937507  6859 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:18.970511  6271 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:18.917109  6863 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:18.979969  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:19.019429  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:19.021239  2347 raft_consensus.cc:2410] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:04:19.021772  2342 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
W20260709 13:04:19.023568  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.013156  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:19.014403  1644 raft_consensus.cc:1217] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Deduplicated request from leader. Original: 9.6->[11.7-14.9]   Dedup: 11.7->[12.8-14.9]
I20260709 13:04:18.956089  6624 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:19.026072  6624 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.029095  6624 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.018031  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.018468  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.031370  1644 pending_rounds.cc:85] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 7
I20260709 13:04:19.031814  1644 pending_rounds.cc:107] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:04:19.032173  1644 raft_consensus.cc:2889] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:18.951995  6955 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.033581  7002 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:19.034320  7002 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.034926  6958 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:18.999825  6927 consensus_queue.cc:1059] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:04:19.035993  6917 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:18.940989  6722 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.036576  7002 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:18.988623  6629 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:18.869424  6973 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:19.036458  6948 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:19.003705  1717 rpcz_store.cc:276] 0709 13:04:17.599933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.600452 (+   519us) service_pool.cc:224] Handling call
0709 13:04:17.650450 (+ 49998us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:17.877689 (+227239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:17.877738 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:17.889720 (+ 11982us) log.cc:844] Serialized 26 byte log entry
0709 13:04:18.018402 (+128682us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:18.018596 (+   194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.019003 (+   407us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:18.998862 (+979859us) raft_consensus.cc:1685] finished
0709 13:04:18.998885 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:18.999032 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74499,"raft.queue_time_us":165846,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":12983,"thread_start_us":70509,"threads_started":1,"wal-append.queue_time_us":327293}
W20260709 13:04:19.043903  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 66053.5us
I20260709 13:04:18.956343  6618 consensus_queue.cc:1059] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:04:19.019387  6885 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.054052  1538 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:19.058748  6994 log.cc:927] Time spent T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:04:19.042060  6999 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:19.059448  6812 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:19.059743  6933 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.061101  1645 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:19.061404  6974 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.062465  6962 raft_consensus.cc:3055] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:19.090451  6962 raft_consensus.cc:740] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:19.043728  6271 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:19.066423  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 826) took 2184 ms. Trace:
I20260709 13:04:19.003072  6982 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:19.043442  6967 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:04:18.999217  6968 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.076912  1538 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:19.019942  6919 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:19.082284  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:19.082427  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:19.045809  6978 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:19.130321  2347 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:19.086370  6945 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.088852  6995 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:19.089780  6989 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.092110  6962 consensus_queue.cc:260] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.052273  6629 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.095337  1569 rpcz_store.cc:276] 0709 13:04:16.882282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.032320 (+150038us) service_pool.cc:224] Handling call
0709 13:04:17.041678 (+  9358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:17.499829 (+458151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.050536 (+1550707us) spinlock_profiling.cc:243] Waited 442 ms on lock 0x7b5400307854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:19.066346 (+ 15810us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1016691968}
I20260709 13:04:19.142374  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2854) took 2063 ms. Trace:
I20260709 13:04:19.097754  6864 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:19.002132  6996 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:19.110623  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.176645  1645 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:19.177920  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.178689  1645 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:19.179834  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.180532  1645 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:19.112129  1538 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:19.182317  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 70286.8us
I20260709 13:04:19.182504  6910 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:19.183086  6910 raft_consensus.cc:697] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:19.184057  6910 consensus_queue.cc:237] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.015443  6936 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.196209  6929 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:19.144553  6966 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:19.143019  1646 rpcz_store.cc:276] 0709 13:04:17.079395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.079676 (+   281us) service_pool.cc:224] Handling call
0709 13:04:17.080268 (+   592us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:17.104834 (+ 24566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:17.104884 (+    50us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:04:18.126194 (+1021310us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b2c000120e8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824777ea11 00007f824778e372 00007f8248ec0a01 00007f8248e8543a 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2
0709 13:04:18.138498 (+ 12304us) log.cc:844] Serialized 78 byte log entry
0709 13:04:18.431884 (+293386us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:18.526676 (+ 94792us) log.cc:844] Serialized 14 byte log entry
0709 13:04:18.714912 (+188236us) log.cc:844] Serialized 14 byte log entry
0709 13:04:18.877322 (+162410us) log.cc:844] Serialized 14 byte log entry
0709 13:04:18.925725 (+ 48403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.935086 (+  9361us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:19.142101 (+207015us) raft_consensus.cc:1685] finished
0709 13:04:19.142126 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:19.142311 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":121839,"prepare.queue_time_us":153396,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":2756,"raft.queue_time_us":359588,"raft.run_cpu_time_us":818,"raft.run_wall_time_us":43422,"spinlock_wait_cycles":13045376,"thread_start_us":443566,"threads_started":3,"wal-append.queue_time_us":362007}
I20260709 13:04:19.119374  6990 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.130023  6978 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.153753  6286 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:19.154397  6685 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.155514  6689 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:19.156538  6863 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.108109  6906 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:19.160579  6815 consensus_meta.cc:287] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.837s	user 0.012s	sys 0.001s
I20260709 13:04:19.140192  6995 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:19.142707  6629 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:19.165321  6161 raft_consensus.cc:3055] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:19.105276  6942 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:19.083317  6999 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.161863  6982 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:19.131122  6271 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:19.127123  6952 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.198539  2340 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:19.129510  6973 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:19.209793  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.140576  6989 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.237069  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.211371  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2852) took 2171 ms. Trace:
I20260709 13:04:19.239468  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.240290  6936 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:19.227348  6748 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.587s	user 0.013s	sys 0.000s
I20260709 13:04:19.222347  6996 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:19.140035  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 57751us
I20260709 13:04:19.264146  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:19.238643  6978 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:19.256101  1645 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:19.253332  1643 rpcz_store.cc:276] 0709 13:04:17.034188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.034386 (+   198us) service_pool.cc:224] Handling call
0709 13:04:17.043945 (+  9559us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:17.080236 (+ 36291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:17.080291 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:17.103523 (+ 23232us) log.cc:844] Serialized 26 byte log entry
0709 13:04:17.316566 (+213043us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:17.382658 (+ 66092us) log.cc:844] Serialized 14 byte log entry
0709 13:04:17.382996 (+   338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:17.383081 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:19.166688 (+1783607us) spinlock_profiling.cc:243] Waited 7.67 ms on lock 0x7b1800243640. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:04:19.172201 (+  5513us) raft_consensus.cc:1685] finished
0709 13:04:19.205240 (+ 33039us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:19.205492 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7067,"prepare.queue_time_us":470316,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":310,"raft.run_cpu_time_us":379,"raft.run_wall_time_us":24076,"spinlock_wait_cycles":17648000,"thread_start_us":213913,"threads_started":2,"wal-append.queue_time_us":281700}
W20260709 13:04:19.262238  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 240365us
I20260709 13:04:19.247174  6161 raft_consensus.cc:740] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:19.272197  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:19.273999  1538 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:19.237862  6990 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:19.280803  6990 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:19.282681  1646 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:19.286499  1538 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:19.241415  6815 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.287755  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.295137  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.298619  1716 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 12.
I20260709 13:04:19.291240  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.288702  1719 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:19.302626  1719 raft_consensus.cc:606] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:19.303076  1719 raft_consensus.cc:614] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:19.291522  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.300107  2313 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:19.296844  1538 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:19.309940  6224 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.270638  6982 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:19.249759  6953 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:19.311180  6863 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:19.301530  1645 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:19.321115  6161 consensus_queue.cc:260] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.304234  1717 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16.
I20260709 13:04:19.317303  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.337913  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:19.338761  2313 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:19.338837  1715 raft_consensus.cc:2410] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:04:19.318636  6995 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.324859  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:19.340458  1643 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:19.289664  6271 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:19.267498  6748 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:19.346597  2313 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:19.319088  6999 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.350240  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.351325  1644 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
W20260709 13:04:19.327700  6946 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.924s	user 0.011s	sys 0.005s
I20260709 13:04:19.353012  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.312156  6966 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:19.356644  6966 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:19.341302  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.361326  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.361514  1719 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19.
I20260709 13:04:19.330946  6996 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:19.362022  1645 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:19.356298  1538 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:19.345770  6995 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:19.283525  6936 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:19.363528  6161 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:19.316525  6958 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:19.364285  6958 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.364720  6958 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:19.360483  6966 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:19.284829  6990 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:19.363746  6936 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:19.337069  6863 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:19.367586  6942 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:19.357079  6224 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:19.378412  6990 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.363044  6996 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:19.383141  6996 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:19.387522  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.388351  5852 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.389003  6989 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.389439  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.355090  6271 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.390226  1644 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:19.392179  6271 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.393182  6863 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:19.372888  2313 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:19.372592  1641 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:04:19.373281  1715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:19.380671  6942 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.395884  1715 raft_consensus.cc:606] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:19.396627  1715 raft_consensus.cc:614] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:19.362146  6815 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.405509  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:19.406492  1645 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:19.411537  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.353076  6946 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:19.378064  6936 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:19.365741  6995 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.359799  6629 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.414424  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.396176  6942 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:19.434127  7013 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:04:19.436364  6629 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:19.411710  6224 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:19.441604  1643 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:19.463939  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.430894  6995 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:19.467327  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:19.468412  2347 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:04:19.466522  1643 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:19.468426  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:19.446201  6224 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:19.449059  6815 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:19.476155  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.470908  1686 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:19.477833  1643 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:19.451894  6885 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.478816  6815 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.476720  6224 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:19.482942  6990 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.486477  6748 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:19.492775  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.481365  6286 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:19.495184  6286 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.495541  6286 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:19.503253  6968 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.504750  6722 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.481031  6996 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:19.476338  6970 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.007s	sys 0.009s
W20260709 13:04:19.512367  6970 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.007s	sys 0.009s
I20260709 13:04:19.514930  6638 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:19.517009  6987 raft_consensus.cc:3055] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:19.523821  2340 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:19.525452  2347 raft_consensus.cc:1240] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:19.532924  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.533843  1640 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:19.533318  1645 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:19.535907  6863 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.451017  6946 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:19.513718  6996 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:19.537423  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 58868.1us
I20260709 13:04:19.474658  1644 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:19.538363  1648 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:19.549296  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.540953  1647 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 13: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:19.544786  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:19.546188  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.555529  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:19.546974  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.469990  6629 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.558394  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:19.517520  6987 raft_consensus.cc:740] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:19.559813  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:19.571688  2340 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:19.559788  1643 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:04:19.572999  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.562249  1647 raft_consensus.cc:1275] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:19.481156  6815 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.577165  1643 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:19.578068  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.578236  1686 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:19.545361  6659 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:19.588907  6659 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:19.592005  6659 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:19.594012  6929 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:19.594551  6929 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:19.595216  6929 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:19.597596  1647 raft_consensus.cc:1240] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:04:19.584295  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.585615  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:19.600421  2344 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:19.605799  2343 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:04:19.603797  1538 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:19.584877  1686 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:19.607074  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.608422  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:19.609874  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.521620  6942 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.610663  1713 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:19.613917  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.613957  6928 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:19.617362  6943 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:19.617568  6928 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:19.581264  6946 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:19.619540  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:19.559046  6286 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:19.593503  6636 raft_consensus.cc:3055] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:19.626434  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.628209  2343 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:19.633769  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.634330  1644 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:04:19.635442  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.571614  6916 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:19.636588  6916 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.638537  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.638801  1568 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:19.639190  1717 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:19.639581  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.640625  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:19.641224  1568 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:19.641450  1571 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:19.646073  6618 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:19.646595  1538 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:19.646906  6618 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:19.649569  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:19.631963  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.650406  1568 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:19.650774  1713 raft_consensus.cc:2410] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 15 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 13:04:19.651772  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.653589  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:19.654054  1564 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:40670
I20260709 13:04:19.654286  1567 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:19.655901  1572 raft_consensus.cc:1240] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:19.658293  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:19.639084  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:19.658979  1568 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:19.653048  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.576568  6629 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:19.673306  5852 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:19.636884  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.673909  1538 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:19.662742  1715 raft_consensus.cc:2368] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14.
I20260709 13:04:19.662626  6828 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:19.678061  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:19.621922  6928 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:19.667588  1713 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:19.663051  6636 raft_consensus.cc:740] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:19.683132  5852 raft_consensus.cc:697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:19.619052  6936 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:19.695888  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.682912  1716 raft_consensus.cc:2368] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17.
I20260709 13:04:19.705876  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.710835  1644 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:19.685755  1714 raft_consensus.cc:2368] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 17. Current term is 18.
I20260709 13:04:19.681663  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:19.711308  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:19.712252  1647 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:04:19.698820  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.698194  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.717449  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.718853  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.719553  1643 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:04:19.720525  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.710181  5852 consensus_queue.cc:237] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.723743  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.629042  6946 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.725978  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.725989  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.729405  1643 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:19.678851  6828 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:19.615388  6987 consensus_queue.cc:260] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.679726  6961 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:04:19.735391  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.754065  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.732553  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:19.752127  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 44252us
I20260709 13:04:19.753993  1645 raft_consensus.cc:1240] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:19.795588  1648 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:04:19.766129  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:19.618260  6943 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.770345  6958 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.775031  6286 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:19.722361  6798 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.678562  6916 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:19.779702  1567 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:04:19.783088  6618 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:19.693877  6999 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:19.791801  1720 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:19.792912  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.757956  1714 raft_consensus.cc:1275] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 6. (term mismatch)
I20260709 13:04:19.819067  1714 pending_rounds.cc:85] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 5
I20260709 13:04:19.796123  1538 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:19.800076  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:19.803459  1713 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 16: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:19.810026  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:19.820109  1565 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:04:19.801628  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2390) took 1009 ms. Trace:
I20260709 13:04:19.813310  2344 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:19.819511  1714 pending_rounds.cc:107] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 13.6
I20260709 13:04:19.819182  1567 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:19.821486  1714 raft_consensus.cc:2889] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:19.792140  6828 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:19.823895  1565 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:19.727088  6978 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.829576  1566 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:19.826208  1644 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:19.732368  6224 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:19.819348  1647 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:19.828616  1612 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:19.838963  1612 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:19.843055  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:19.800359  6961 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:19.853940  6951 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:19.855207  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:19.856491  6659 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:19.856931  6659 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:19.857307  6659 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:19.820142  6966 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.821972  6636 consensus_queue.cc:260] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.861428  6636 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:19.862612  6966 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.863848  1570 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:19.864329  6958 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.818034  6999 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:19.865100  6999 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:19.869345  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.870384  6942 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.873164  1564 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:19.874012  1564 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:19.851970  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:19.827515  6987 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:19.880800  1612 leader_election.cc:400] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 17. Current term is 18.
I20260709 13:04:19.891597  1612 leader_election.cc:403] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:04:19.889183  1572 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:19.880653  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:19.892125  2346 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:19.899811  1643 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 13: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:19.855559  6961 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:19.914145  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:19.916142  6629 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:19.889899  6951 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:19.917598  6936 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:19.920212  6943 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.826025  6618 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 13:04:19.926934  1564 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:19.942588  6946 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.944929  6224 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:19.902748  6989 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:19.948932  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 50667.7us
I20260709 13:04:19.954409  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:19.956229  2314 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:19.959014  2314 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:19.962533  2314 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:19.963881  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:19.964870  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:19.927241  6951 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:19.967079  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:19.967871  1567 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:19.964665  6798 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:20.004520  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:20.007714  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:19.967808  6996 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:19.967311  6978 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:19.970321  6674 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:19.972659  6951 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:19.961263  6224 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:19.999730  6798 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.999150  6955 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:20.030468  6828 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:20.031927  1565 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:20.038632  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.043650  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.046481  1568 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:20.026921  6798 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:20.042598  6955 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.059026  6955 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:20.059933  6828 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:20.061291  6955 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:20.062008  6928 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:20.064442  6928 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:20.066368  1611 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:20.068075  6955 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:20.068526  6955 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:20.068814  1611 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:20.069523  6955 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:20.075115  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.075994  1572 raft_consensus.cc:2393] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:20.076929  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:20.078869  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.079751  1570 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 12: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:20.079491  6955 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:20.081532  6955 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:20.082795  6953 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:20.087208  7023 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:04:20.089802  6863 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:20.098565  1611 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:20.099630  6883 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:20.100095  6883 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:20.100478  6883 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:20.102141  7025 log.cc:927] Time spent T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:04:20.101068  6977 consensus_queue.cc:1059] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:04:20.067248  7022 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:04:20.104593  1611 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:20.119792  1611 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:20.122915  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.124655  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:20.077556  1572 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:04:19.967777  6929 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:20.130209  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.121479  1611 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:20.147495  1611 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:20.148864  1611 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:20.152506  1611 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:20.133891  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 411935us
I20260709 13:04:20.155459  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:20.159325  1570 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:04:20.157845  1567 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:20.108536  6985 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:04:20.147612  6995 consensus_meta.cc:287] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.672s	user 0.014s	sys 0.001s
I20260709 13:04:20.146145  6955 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:20.148036  6963 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:20.146229  6224 raft_consensus.cc:3055] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:20.166354  1611 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:20.150736  6987 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:20.158874  6936 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:20.174803  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.156260  6748 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:20.185354  6538 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:20.186698  6659 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:20.188969  6659 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:20.177934  6224 raft_consensus.cc:740] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:20.190567  6224 consensus_queue.cc:260] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:20.191757  6224 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:20.181151  1565 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:20.194494  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:20.195062  1565 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 17
W20260709 13:04:20.209823  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 43559us
I20260709 13:04:20.163591  6985 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.219476  1565 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:04:20.173844  6968 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:20.174607  6955 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:20.224632  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:20.220492  1572 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:20.224066  1685 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:20.228176  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.182569  6748 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:20.228929  1572 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:20.216986  6985 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:20.211262  7010 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:20.172688  6995 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:20.217877  6731 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:04:20.233832  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 267466us
W20260709 13:04:20.302488  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:20.297425  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.304665  2314 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:20.227831  6629 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.306602  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:20.311364  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:20.315379  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:20.334257  1648 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:20.226661  6955 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:20.322274  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:20.305881  7010 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:20.326470  7030 log.cc:927] Time spent T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 13:04:20.327519  7032 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:04:20.221375  6968 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:20.299932  6987 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:20.332814  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.363897  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:20.232420  1685 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:20.336242  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.232537  6953 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:20.235193  6748 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:20.340001  2342 raft_consensus.cc:2368] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17.
I20260709 13:04:20.339169  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:20.346964  1572 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: []
W20260709 13:04:20.269537  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.366034  1567 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:20.227960  6636 consensus_queue.cc:1059] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.135s
I20260709 13:04:20.414825  2344 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 13: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:20.222922  6538 consensus_queue.cc:1059] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.115s
I20260709 13:04:20.367985  6978 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.370622  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:20.338583  6977 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:04:20.416260  2346 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:20.230573  6646 consensus_queue.cc:1059] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.139s
I20260709 13:04:20.320873  6731 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:20.334973  6224 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:20.369292  6968 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:20.412236  6928 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:20.418434  6999 consensus_meta.cc:287] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.553s	user 0.013s	sys 0.000s
I20260709 13:04:20.446417  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:20.451397  7010 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:20.452093  6929 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:20.420027  6978 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:20.377924  6638 consensus_queue.cc:1059] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:04:20.338151  6629 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:20.524909  6629 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:04:20.455307  6995 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:20.434070  6953 raft_consensus.cc:3055] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:20.459447  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:20.459609  6975 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:20.380465  6987 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:20.528622  6953 raft_consensus.cc:740] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:20.517010  6999 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:20.463443  6883 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:20.529665  6953 consensus_queue.cc:260] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:20.530970  6953 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:20.464702  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:20.531728  2344 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:20.532585  6999 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:20.463650  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:20.493886  6731 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:20.467597  6962 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:20.468467  6923 consensus_queue.cc:1059] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:20.469224  6636 consensus_queue.cc:1059] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:04:20.471375  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 675397us
I20260709 13:04:20.472340  6952 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:20.477041  6271 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.477315  6908 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.479557  1537 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:20.583740  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 104266us
I20260709 13:04:20.480219  6943 consensus_queue.cc:1059] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:20.490401  1648 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:20.492594  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.496456  1611 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:20.512686  2346 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:04:20.429045  6989 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 17. Current term is 18.
I20260709 13:04:20.596781  6989 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:20.597800  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 101514us
I20260709 13:04:20.515790  1570 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:20.517078  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.518034  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.520748  1572 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:04:20.600252  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 166330us
I20260709 13:04:20.521054  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.522959  6638 raft_consensus.cc:3055] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:20.534392  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:20.535281  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 2867) took 1456 ms. Trace:
I20260709 13:04:20.442344  6781 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:20.551043  2349 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 20: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:20.512796  6618 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.513903  6815 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:20.620056  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.567265  2344 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:20.480170  6863 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:04:20.526095  7010 raft_consensus.cc:3055] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:20.581873  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.572024  1538 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:20.601559  1567 raft_consensus.cc:2393] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:04:20.628679  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 56756.2us
I20260709 13:04:20.628480  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.616496  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.614993  1687 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:20.617453  1646 rpcz_store.cc:276] 0709 13:04:19.079295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.236715 (+157420us) service_pool.cc:224] Handling call
0709 13:04:20.535102 (+1298387us) spinlock_profiling.cc:243] Waited 29.4 ms on lock 0x7b54001f2a54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:20.535229 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67590400}
I20260709 13:04:20.566323  1647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:55704
I20260709 13:04:20.632519  1647 raft_consensus.cc:606] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:20.632952  1566 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 16: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:20.632987  1647 raft_consensus.cc:614] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:20.598646  6820 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:20.654546  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:20.654913  1538 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:20.655283  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:20.636159  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:20.627276  7010 raft_consensus.cc:740] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:20.603494  6952 raft_consensus.cc:697] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:20.638652  6974 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:20.599365  6908 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:20.643735  6987 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:20.666909  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.593842  6271 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:19.820951  1640 rpcz_store.cc:276] 0709 13:04:18.791918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.535865 (+743947us) service_pool.cc:224] Handling call
0709 13:04:19.536208 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:19.536408 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:19.536763 (+   355us) log.cc:844] Serialized 14 byte log entry
0709 13:04:19.801262 (+264499us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:19.801358 (+    96us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:19.801410 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.801481 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:19.801578 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":190,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":106,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":7000,"thread_start_us":264212,"threads_started":1}
I20260709 13:04:20.617100  6638 raft_consensus.cc:740] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:20.648819  6929 raft_consensus.cc:3055] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:20.624226  6618 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:20.627866  6781 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:20.662065  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.663815  1687 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:20.661751  6928 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:20.679281  1565 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
W20260709 13:04:20.680245  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 447905us
I20260709 13:04:20.681340  1570 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:04:20.678061  1687 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:20.682504  1685 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:20.682569  1687 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:20.685194  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 222668us
I20260709 13:04:20.652417  6828 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:20.602797  6946 consensus_queue.cc:1059] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 13:04:20.660548  6820 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.690646  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:20.693190  1718 raft_consensus.cc:2393] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:20.702744  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.704078  1567 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 13: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:20.673885  6863 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:20.707758  1642 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:20.674219  6929 raft_consensus.cc:740] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:20.627671  6815 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:20.687076  6921 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:20.680915  6928 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.675879  6978 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:20.691363  6828 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.713008  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.724056  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:20.715936  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:20.705760  6820 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:20.711310  6999 raft_consensus.cc:3055] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:20.735793  1565 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:20.732386  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:20.736425  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.717590  6975 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:04:20.825451  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 105621us
I20260709 13:04:20.714344  6961 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:04:20.732522  7031 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.218s	user 0.000s	sys 0.002s
I20260709 13:04:20.832046  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:20.860469  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.861121  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.719600  6921 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:20.725266  6629 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:20.718621  6863 raft_consensus.cc:697] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:20.863018  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2779) took 1230 ms. Trace:
W20260709 13:04:20.863862  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.726115  6928 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:20.727432  6828 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:20.825518  6951 consensus_meta.cc:287] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.798s	user 0.004s	sys 0.009s
I20260709 13:04:20.735370  6946 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.716822  7038 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:20.878512  1716 rpcz_store.cc:276] 0709 13:04:19.632500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.639368 (+  6868us) service_pool.cc:224] Handling call
0709 13:04:20.857776 (+1218408us) spinlock_profiling.cc:243] Waited 42.4 ms on lock 0x7b5400247f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:20.862910 (+  5134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97619840}
I20260709 13:04:20.871445  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:20.896018  1718 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:20.868441  6966 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.880213  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.823603  6958 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:20.735091  6999 raft_consensus.cc:740] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:20.892865  1642 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:20.737799  6987 raft_consensus.cc:3055] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:04:20.877357  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 194977us
I20260709 13:04:20.913757  1685 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:20.914466  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:20.899255  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:20.871012  6921 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:20.903497  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:20.719970  7010 consensus_queue.cc:260] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:20.891069  6629 raft_consensus.cc:697] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:04:20.948009  6798 consensus_meta.cc:287] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.841s	user 0.012s	sys 0.002s
I20260709 13:04:20.948398  6629 consensus_queue.cc:237] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:20.950687  6863 consensus_queue.cc:237] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:20.957795  7043 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:04:20.913116  6958 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:20.969595  6883 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:20.912314  6987 raft_consensus.cc:740] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:20.974915  6999 consensus_queue.cc:260] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:20.975195  6987 consensus_queue.cc:260] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:20.924007  2342 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55890 (request call id 519) took 1369 ms (client timeout 1819 ms). Trace:
I20260709 13:04:20.933139  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2391) took 2060 ms. Trace:
I20260709 13:04:20.872634  6961 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:20.895403  7038 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:20.976811  2342 rpcz_store.cc:269] 0709 13:04:19.555061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.555306 (+   245us) service_pool.cc:224] Handling call
0709 13:04:20.923731 (+1368425us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b540035f2d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:20.923899 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1801941760}
I20260709 13:04:20.935657  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:21.003190  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2395) took 1365 ms. Trace:
I20260709 13:04:21.003573  1717 rpcz_store.cc:276] 0709 13:04:19.637298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.935382 (+1298084us) service_pool.cc:224] Handling call
0709 13:04:21.003122 (+ 67740us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:20.871390  7037 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:20.940048  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:20.976253  6999 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:20.921902  6753 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:21.015211  1570 raft_consensus.cc:2393] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:20.934132  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:21.016238  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:20.979714  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2425) took 1319 ms. Trace:
I20260709 13:04:21.017010  1565 raft_consensus.cc:1240] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:20.986635  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:20.996461  6781 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:21.000631  6923 raft_consensus.cc:3055] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:20.977257  1645 rpcz_store.cc:276] 0709 13:04:18.873136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.536790 (+663654us) service_pool.cc:224] Handling call
0709 13:04:19.537278 (+   488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:19.795795 (+258517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.932832 (+1137037us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54000a6ed4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:20.933063 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":497939712}
I20260709 13:04:20.948848  6798 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:21.022907  6923 raft_consensus.cc:740] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:21.025022  6798 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:21.025313  2346 raft_consensus.cc:1240] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:04:20.900893  6946 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:21.031569  6946 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:21.032148  6946 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.032601  6946 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:20.977027  6987 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:20.862207  6638 consensus_queue.cc:260] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.017160  1569 rpcz_store.cc:276] 0709 13:04:19.659937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.668892 (+  8955us) service_pool.cc:224] Handling call
0709 13:04:19.680254 (+ 11362us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:19.684878 (+  4624us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:19.684966 (+    88us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:19.750561 (+ 65595us) log.cc:844] Serialized 52 byte log entry
0709 13:04:20.117088 (+366527us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:20.117156 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.117271 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:20.979337 (+862066us) raft_consensus.cc:1685] finished
0709 13:04:20.979372 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.979632 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42368,"raft.queue_time_us":9811,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":11552,"spinlock_wait_cycles":774400,"thread_start_us":288306,"threads_started":1,"wal-append.queue_time_us":630577}
I20260709 13:04:21.059762  6618 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:20.928997  6731 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:21.148968  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 274182us
W20260709 13:04:21.149544  7042 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
W20260709 13:04:21.038300  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 124697us
I20260709 13:04:20.887490  6975 raft_consensus.cc:697] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:21.039978  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:21.152117  1714 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:21.041888  1718 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:21.042253  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2387) took 1433 ms. Trace:
I20260709 13:04:21.153852  1722 rpcz_store.cc:276] 0709 13:04:19.608625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.609552 (+   927us) service_pool.cc:224] Handling call
0709 13:04:20.933933 (+1324381us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b580009db98. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:04:21.042180 (+108247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3704704}
I20260709 13:04:20.899649  6955 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.009692  7038 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:20.911299  6966 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:20.889973  6929 consensus_queue.cc:260] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.020897  6943 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.064081  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:21.064836  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:20.883538  6952 consensus_queue.cc:237] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.068070  6798 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.023943  6923 consensus_queue.cc:260] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.179224  6798 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:21.180225  6923 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:21.069048  1564 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:21.182166  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:40670 (request call id 2422) took 1528 ms. Trace:
I20260709 13:04:21.182560  1564 rpcz_store.cc:276] 0709 13:04:19.653570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.653927 (+   357us) service_pool.cc:224] Handling call
0709 13:04:20.918797 (+1264870us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b5400012554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e7ca3d 00007f824de4ccd0 00007f82473d1254 00007f82473d11e2 00007f82473d1111 00007f82473d108c 00007f82473d0342 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:21.068892 (+150095us) spinlock_profiling.cc:243] Waited 92.5 ms on lock 0x7b4c0006f200. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8247151a20 00007f8247162a29 00007f824716313f 00007f82471272ab 00007f8247154f3c 00007f82471553fe 00007f82473e288b 00007f8248de1495 00007f8248e3230f 00007f8248e7ca55 00007f824de4ccd0
0709 13:04:21.182041 (+113149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":714684928}
I20260709 13:04:21.183075  6946 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:21.162271  6928 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.185720  6798 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:21.186821  6928 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:21.048256  7041 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:21.189947  7041 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:21.190440  7041 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:21.192687  6978 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:21.194901  7046 consensus_queue.cc:1059] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:21.082265  6958 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:21.196939  7044 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.197110  6965 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:21.047395  7037 raft_consensus.cc:697] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:04:21.103396  7021 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 1.011s	user 0.009s	sys 0.002s
I20260709 13:04:21.107918  2342 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:21.108317  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:21.108976  1537 leader_election.cc:400] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17.
I20260709 13:04:21.208173  1537 leader_election.cc:403] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:04:21.208732  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 99936.6us
I20260709 13:04:21.109100  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:21.110219  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:21.110991  6908 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:21.124401  2346 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:04:21.126196  6722 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.126252  6883 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.127267  6936 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:21.127920  6953 consensus_meta.cc:287] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.596s	user 0.010s	sys 0.002s
I20260709 13:04:21.127923  6659 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:21.054086  6616 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:21.129328  1645 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:21.129448  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:21.130185  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:21.130353  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:21.225672  1646 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:21.226284  1648 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 15: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:04:21.133028  6989 consensus_meta.cc:287] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.536s	user 0.003s	sys 0.010s
I20260709 13:04:21.138716  5852 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:21.140935  1639 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.542s	user 0.003s	sys 0.008s
I20260709 13:04:21.033926  6828 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.160087  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:21.212167  6958 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17.
I20260709 13:04:21.154764  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:21.261103  1565 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:21.164752  1685 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:21.165376  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:21.262615  1642 raft_consensus.cc:2368] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16.
I20260709 13:04:21.262998  1646 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:21.178133  6638 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:21.266691  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:21.197741  7044 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:21.198204  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:21.198513  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:21.270895  1722 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 14: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:21.199731  1713 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:21.200363  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:21.202931  6943 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:21.203174  2312 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.163s (SENT) [suppressed 1 similar messages]
I20260709 13:04:21.189250  1687 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:21.206566  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:21.203433  6985 consensus_meta.cc:287] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.881s	user 0.014s	sys 0.000s
I20260709 13:04:21.173223  6731 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:21.228250  1720 raft_consensus.cc:2410] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:04:21.241276  1640 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 14: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:21.245383  1639 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:04:21.245848  5852 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.247486  1570 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 14: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:20.897382  6951 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.250279  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:21.257776  6989 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:04:21.223055  6968 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.712s	user 0.011s	sys 0.003s
I20260709 13:04:21.223701  6946 consensus_queue.cc:1059] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:04:21.224349  6936 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.259296  6815 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:21.331090  6271 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:21.332860  1647 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:21.259409  6828 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:21.258553  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:21.175511  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:21.217190  6908 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:21.264258  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 99575us
I20260709 13:04:21.348366  1685 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:21.264513  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:21.084859  6833 raft_consensus.cc:3055] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:21.267485  1642 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:21.240221  6616 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:21.202160  1686 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:21.272689  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:21.272442  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:21.388679  1685 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:21.389693  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 187600us
I20260709 13:04:21.391587  6646 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:21.392153  6646 raft_consensus.cc:697] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:21.392587  6732 raft_consensus.cc:3055] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:21.392967  6732 raft_consensus.cc:740] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:21.393908  6732 consensus_queue.cc:260] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:21.242244  6975 consensus_queue.cc:237] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.395289  6732 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:21.395771  7010 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:21.398713  6966 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:21.399724  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:21.393064  6646 consensus_queue.cc:237] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:21.400964  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:21.403421  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:21.410573  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:21.415223  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2410) took 1110 ms. Trace:
I20260709 13:04:21.415875  1639 rpcz_store.cc:276] 0709 13:04:20.304401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.306357 (+  1956us) service_pool.cc:224] Handling call
0709 13:04:21.414969 (+1108612us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54001f3954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:21.415135 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":610928384}
I20260709 13:04:21.404198  2346 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:04:21.423075  1639 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:21.425350  1639 raft_consensus.cc:1275] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:21.322427  7047 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:21.433092  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:21.434043  1713 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:21.437081  1685 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:21.277597  1612 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:21.224825  6961 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:21.283799  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.443192  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:21.296190  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 107013us
I20260709 13:04:21.443977  1685 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:21.299466  1648 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 8.
I20260709 13:04:21.445957  1648 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:21.445923  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:21.222553  6929 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:21.306486  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:21.464457  1685 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:21.460275  1713 raft_consensus.cc:2393] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 14: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:21.427958  6743 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:21.471462  7045 consensus_queue.cc:1059] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:04:21.459640  6636 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:21.472671  6943 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:21.476666  6961 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.477581  6961 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.478266  6957 raft_consensus.cc:3055] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:21.478854  6957 raft_consensus.cc:740] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:21.479794  6957 consensus_queue.cc:260] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.480504  6961 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:21.481102  6957 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:21.482586  1686 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:21.485687  1716 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:21.487990  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:21.488864  1716 raft_consensus.cc:2410] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:04:21.462204  6533 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:21.489871  6917 consensus_queue.cc:1059] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:21.490402  6533 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.489854  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:21.220732  6883 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.491173  6533 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:21.469182  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:21.493681  6883 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:21.220940  6722 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.493955  6731 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:21.459209  6976 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:21.494793  6731 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:21.442405  6989 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:21.496009  6722 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:21.280097  7037 consensus_queue.cc:237] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:21.497228  6731 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:21.494706  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:21.498658  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:21.286010  6618 consensus_queue.cc:1059] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:04:21.323990  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:21.324255  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:21.325004  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:21.325238  6968 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:21.326268  6951 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:21.326316  6946 raft_consensus.cc:3055] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:21.327052  5852 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:21.328482  6936 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:21.270581  6905 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:21.334969  6271 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:21.537139  6271 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.210567  6965 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.227442  6659 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:21.344141  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.351030  1569 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 12: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:21.271764  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:21.540774  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.541326  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:21.542764  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:21.356367  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:21.357060  6908 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:21.351679  6833 raft_consensus.cc:740] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:21.358866  7040 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:21.359577  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:21.554373  1647 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:21.361434  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:21.499195  6636 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:21.362751  7044 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:21.561318  1720 raft_consensus.cc:2393] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 13: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:21.364982  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:21.298803  6985 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.366806  6955 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:21.375794  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:21.565627  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:21.440883  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 163360us
I20260709 13:04:21.286836  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:21.378947  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2393) took 2504 ms. Trace:
I20260709 13:04:21.566100  1646 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:21.566303  1571 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:21.566458  1612 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:21.566454  1641 rpcz_store.cc:276] 0709 13:04:18.874207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.539516 (+665309us) service_pool.cc:224] Handling call
0709 13:04:20.535234 (+995718us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:20.535642 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:20.535670 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:20.552288 (+ 16618us) log.cc:844] Serialized 52 byte log entry
0709 13:04:20.872497 (+320209us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:20.872540 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.872612 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:21.151165 (+278553us) raft_consensus.cc:1685] finished
0709 13:04:21.151185 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.378379 (+227194us) spinlock_profiling.cc:243] Waited 979 ms on lock 0x7b54001f2a50. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:21.378737 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4717,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":14524,"spinlock_wait_cycles":2251428224,"thread_start_us":244714,"threads_started":1,"wal-append.queue_time_us":337017}
I20260709 13:04:21.489930  6616 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:21.578583  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:21.508083  1642 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:21.580240  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:21.319226  1565 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:04:21.581005  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:21.513088  1611 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:21.538472  1566 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:21.583652  1612 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:21.584095  1720 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:21.505049  6722 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.362254  7046 consensus_queue.cc:1059] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:21.587138  6271 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:21.557121  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 279808us
I20260709 13:04:21.455750  6893 consensus_queue.cc:1059] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.080s
W20260709 13:04:21.208328  6978 consensus_queue.cc:1175] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "066ee0109aba4465ba17fa65e14ef504" responder_term: 13 status { last_received { term: 13 index: 9 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:04:21.502877  6989 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:21.461876  6538 consensus_queue.cc:1059] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.090s
W20260709 13:04:21.547309  6748 consensus_meta.cc:287] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.125s	user 0.015s	sys 0.000s
I20260709 13:04:21.499748  1573 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 14: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:04:21.504354  1538 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.043s	sys 0.094s
W20260709 13:04:21.615162  1538 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.043s	sys 0.095s
I20260709 13:04:21.528836  7010 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:21.628799  1645 raft_consensus.cc:2393] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 13: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:04:21.574092  6753 consensus_meta.cc:287] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.554s	user 0.006s	sys 0.006s
I20260709 13:04:21.574530  1646 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
W20260709 13:04:21.575610  6999 consensus_meta.cc:287] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.556s	user 0.009s	sys 0.002s
I20260709 13:04:21.576771  6965 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:21.577119  6951 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:21.578171  6629 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.581545  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:21.645032  1568 raft_consensus.cc:2368] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16.
I20260709 13:04:21.601085  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:21.536369  6946 raft_consensus.cc:740] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:21.606127  1720 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:21.607722  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:21.646220  1565 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 20.
I20260709 13:04:21.608052  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:21.548233  6976 raft_consensus.cc:697] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:04:21.609329  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 406197us
I20260709 13:04:21.623963  1538 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:21.560025  6636 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:21.635681  2349 raft_consensus.cc:1240] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:04:21.664106  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:21.665253  1720 raft_consensus.cc:2410] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:04:21.667405  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 43574us
I20260709 13:04:21.613819  6798 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.669363  6636 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:21.671281  1715 raft_consensus.cc:3055] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:21.678712  1715 raft_consensus.cc:740] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:21.634131  7010 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.642408  6995 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.634744  6883 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:21.645747  6961 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:21.655107  6618 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:21.681267  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:21.607191  6616 consensus_queue.cc:1048] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:04:21.670317  6999 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:21.688277  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:21.692610  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 228276us
W20260709 13:04:21.693048  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 180085us
I20260709 13:04:21.613447  6722 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.634459  6989 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.633803  6748 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.705470  1611 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:21.692514  1537 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:21.695019  1715 consensus_queue.cc:260] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.706586  1715 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:04:21.683308  6995 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:21.708587  2343 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:21.686455  7010 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:21.708244  1647 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:04:21.700062  1687 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:21.682304  7047 raft_consensus.cc:697] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:21.714776  1611 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:21.712256  1685 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:21.719185  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 135599us
I20260709 13:04:21.727584  6271 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:21.736855  6271 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.732182  1611 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:21.663177  6629 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.743310  1644 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:21.738386  1720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:21.745630  1720 raft_consensus.cc:606] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:21.697821  6999 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:21.714059  6820 consensus_meta.cc:287] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.888s	user 0.013s	sys 0.008s
I20260709 13:04:21.739848  1611 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:21.746317  1720 raft_consensus.cc:614] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:21.713035  6961 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:21.680437  6798 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:21.741496  1537 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:21.717566  7038 consensus_meta.cc:287] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.542s	user 0.006s	sys 0.006s
I20260709 13:04:21.713353  6773 raft_consensus.cc:3055] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:21.755394  1538 leader_election.cc:400] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16.
I20260709 13:04:21.677650  6833 consensus_queue.cc:260] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.762672  7047 consensus_queue.cc:237] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.763872  6833 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:21.766278  7055 log.cc:927] Time spent T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:04:21.765843  7038 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.752624  6999 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:21.767795  7060 log.cc:927] Time spent T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:21.768324  7038 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:21.768975  6974 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:21.769204  6271 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:21.769603  6773 raft_consensus.cc:740] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:21.770557  6773 consensus_queue.cc:260] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.771770  6773 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:21.762018  1537 leader_election.cc:400] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14.
I20260709 13:04:21.775590  1537 leader_election.cc:403] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:21.767613  1718 raft_consensus.cc:2368] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13.
I20260709 13:04:21.711726  6946 consensus_queue.cc:260] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:21.776547  6946 raft_consensus.cc:3060] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:21.738662  6743 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:21.782099  6743 raft_consensus.cc:697] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:04:21.782989  5852 consensus_queue.cc:1175] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f5f4b2707a9647e5bb99e065068dea2b" responder_term: 14 status { last_received { term: 11 index: 9 } last_committed_idx: 9 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:04:21.783121  6743 consensus_queue.cc:237] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.785319  6629 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:21.742724  6883 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:21.788573  1537 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:21.761895  1538 leader_election.cc:403] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:21.782161  6271 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14.
I20260709 13:04:21.743827  6618 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:21.789878  6943 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:21.790202  6753 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:04:21.817310  6989 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:21.793967  1717 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:21.795313  1538 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:21.800546  6995 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:21.803954  6953 raft_consensus.cc:3055] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:04:21.806977  2343 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 472) took 1843 ms (client timeout 1558 ms). Trace:
I20260709 13:04:21.789598  6985 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:21.821065  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:21.823446  1537 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:21.798448  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2415) took 1492 ms. Trace:
I20260709 13:04:21.727319  7048 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:21.835150  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:21.852914  5852 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:04:21.852821  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 57650.7us
W20260709 13:04:21.855937  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 116194us
I20260709 13:04:21.740164  6965 consensus_queue.cc:1059] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.103s
W20260709 13:04:21.820156  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 107992us
I20260709 13:04:21.836867  7061 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:21.811206  7063 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.061s	user 0.003s	sys 0.000s
I20260709 13:04:21.857553  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:21.878676  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:21.879801  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:21.810793  6659 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:21.904865  1644 rpcz_store.cc:276] 0709 13:04:20.305921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.330904 (+ 24983us) service_pool.cc:224] Handling call
0709 13:04:20.364971 (+ 34067us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.743780 (+1378809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.798172 (+ 54392us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54001fead4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:21.798377 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3169579392}
I20260709 13:04:21.795025  6952 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16.
I20260709 13:04:21.906293  1720 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 14: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:21.871084  1571 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:21.860944  2344 raft_consensus.cc:2410] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:04:21.815770  6975 raft_consensus.cc:3055] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:04:21.857975  2343 rpcz_store.cc:269] 0709 13:04:19.963290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.963463 (+   173us) service_pool.cc:224] Handling call
0709 13:04:21.806748 (+1843285us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b54006d0454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:21.806888 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4010853760}
I20260709 13:04:21.810565  6968 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:21.800704  6987 consensus_meta.cc:287] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.655s	user 0.010s	sys 0.000s
W20260709 13:04:21.980283  6929 consensus_meta.cc:287] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.519s	user 0.004s	sys 0.004s
I20260709 13:04:21.916741  6271 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:21.852804  6638 consensus_meta.cc:287] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.569s	user 0.008s	sys 0.005s
W20260709 13:04:21.921636  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 98327.5us
I20260709 13:04:21.982373  1721 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:21.982560  1537 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:21.857512  7052 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:04:21.923566  1611 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:21.923650  6974 consensus_queue.cc:1059] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:04:21.929064  6962 consensus_meta.cc:287] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.370s	user 0.012s	sys 0.003s
I20260709 13:04:21.931442  6965 raft_consensus.cc:3055] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:21.994010  6965 raft_consensus.cc:740] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:21.996140  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:21.996807  1646 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:21.996994  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:21.997464  1642 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:04:21.997797  1639 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:21.998301  1641 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:21.998610  6731 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:21.869786  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 153465us
I20260709 13:04:21.998905  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:21.914496  6966 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.999801  1640 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:21.857726  6953 raft_consensus.cc:740] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:21.745391  6976 consensus_queue.cc:237] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:22.000198  6966 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.000962  6953 consensus_queue.cc:260] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.002012  6953 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:22.002749  7051 log.cc:927] Time spent T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.276s	user 0.000s	sys 0.002s
I20260709 13:04:21.915518  6936 raft_consensus.cc:3055] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:21.959230  6946 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:04:21.921250  7067 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:21.964035  6636 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:21.965276  7041 consensus_meta.cc:287] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.774s	user 0.012s	sys 0.001s
I20260709 13:04:21.854123  6533 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:22.015290  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:21.929484  6999 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.952716  6908 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:21.970242  6748 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:21.971828  6732 consensus_meta.cc:287] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.576s	user 0.013s	sys 0.000s
W20260709 13:04:21.974356  6921 consensus_meta.cc:287] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.964s	user 0.011s	sys 0.003s
I20260709 13:04:21.759196  6820 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:21.981422  1714 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:04:21.992126  7057 log.cc:927] Time spent T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:04:21.995481  6965 consensus_queue.cc:260] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:21.999014  6883 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:21.999285  6731 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:22.003639  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 86090.9us
I20260709 13:04:21.997805  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:21.958386  7065 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:22.005721  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 82227.8us
I20260709 13:04:22.005955  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:22.006398  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.963851  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:21.993219  7068 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:22.005625  2312 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:22.011272  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:22.075848  1643 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.013871  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:22.014647  1642 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:04:22.077335  1639 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:22.016237  1641 raft_consensus.cc:2410] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:04:21.937426  1717 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:22.019942  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:21.967128  1715 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:22.022043  6974 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:22.009191  6995 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:04:22.022567  6952 raft_consensus.cc:3055] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:22.124852  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.126312  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.126873  1714 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:22.128094  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:21.950242  6975 raft_consensus.cc:740] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:04:22.129536  7058 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:04:22.129886  6975 consensus_queue.cc:260] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:22.131508  6975 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:22.131665  6995 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:22.114989  6974 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:22.132927  6974 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:22.023975  6271 consensus_queue.cc:1059] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:21.967260  6798 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:22.027714  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:22.135200  1644 raft_consensus.cc:2410] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
W20260709 13:04:22.137852  6917 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 16 is higher.
I20260709 13:04:22.138244  6271 raft_consensus.cc:3055] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:22.138782  6271 raft_consensus.cc:740] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:22.139740  6271 consensus_queue.cc:260] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:22.025204  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 174274us
I20260709 13:04:22.140962  6271 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:22.007046  6987 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:22.027069  1537 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:22.160933  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 133932us
I20260709 13:04:22.161948  1537 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:22.162973  1714 raft_consensus.cc:2368] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16.
I20260709 13:04:22.017231  6905 raft_consensus.cc:3055] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:22.034029  1538 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:22.176296  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 142362us
I20260709 13:04:22.036078  1611 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:22.018900  7041 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.040607  1720 raft_consensus.cc:2410] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:04:22.079591  1640 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:22.030264  6921 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.125088  6952 raft_consensus.cc:740] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:22.027515  5852 raft_consensus.cc:697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:22.200615  5852 consensus_queue.cc:237] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.141278  2314 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:22.142218  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:22.147487  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.006855  6936 raft_consensus.cc:740] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:04:22.165723  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.169943  6636 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:22.171711  6820 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:22.074862  6966 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:22.173404  6533 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:22.174224  6722 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:22.036804  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:22.220827  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 215265us
I20260709 13:04:22.181452  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:22.204696  7072 log.cc:927] Time spent T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:04:22.186367  6943 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:22.178470  1538 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:22.073833  6965 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:22.195957  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 159972us
I20260709 13:04:22.229179  1611 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:22.209293  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:22.074540  7065 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:22.212986  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 180271us
W20260709 13:04:22.215190  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 73989.6us
I20260709 13:04:22.216917  1716 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:22.178953  1643 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:22.222132  1647 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:22.174882  6946 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:22.222820  2312 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:22.175206  6905 raft_consensus.cc:740] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:22.227403  1719 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:22.231572  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 53194.7us
I20260709 13:04:22.224011  6908 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:22.231645  1611 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:22.199997  6987 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:22.211829  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.243875  1538 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:22.247815  1720 raft_consensus.cc:2410] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:04:22.236035  1612 leader_election.cc:400] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13.
I20260709 13:04:22.241127  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:22.241534  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:22.248097  1538 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:22.237715  6962 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:22.233193  6820 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:22.246502  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:22.248632  1564 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:04:22.231070  6943 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:22.248670  1568 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 20.
I20260709 13:04:22.241887  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:22.206765  6863 consensus_queue.cc:1048] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:04:22.224294  7075 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:22.249279  1612 leader_election.cc:403] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:04:22.119365  7068 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.252357  6271 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:22.248984  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:22.260897  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.275302  1611 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:22.255908  1538 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:22.265722  1572 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:04:22.281993  1720 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:22.268989  1567 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:04:22.256366  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:22.283241  1565 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 8.
I20260709 13:04:22.284651  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:22.285221  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 548) took 1074 ms. Trace:
I20260709 13:04:22.234879  6722 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:22.287339  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 51456.3us
I20260709 13:04:22.324653  6271 raft_consensus.cc:697] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:22.330642  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:22.331722  6968 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:22.332235  1612 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:22.333163  6968 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:22.333698  1612 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:22.333863  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.334722  6968 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:22.334832  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:22.336962  6968 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:22.337457  6968 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:22.338385  1612 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:22.340312  1612 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:22.337469  6616 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:22.341804  6955 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:22.342211  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:22.342532  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:22.342860  1647 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:04:22.343104  1643 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 7.
W20260709 13:04:22.327299  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 127520us
I20260709 13:04:22.327838  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:22.345315  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.345965  1720 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:22.346873  1720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:22.341245  6659 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:22.238204  6946 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.347461  6659 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:22.347672  6946 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.347958  6659 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:22.348122  2314 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:22.349799  2314 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:22.352931  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.232180  6966 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:04:22.285730  1648 rpcz_store.cc:276] 0709 13:04:21.234637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.468275 (+233638us) service_pool.cc:224] Handling call
0709 13:04:21.520178 (+ 51903us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:21.740601 (+220423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.740680 (+    79us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:21.754014 (+ 13334us) log.cc:844] Serialized 52 byte log entry
0709 13:04:21.943436 (+189422us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:22.035550 (+ 92114us) log.cc:844] Serialized 14 byte log entry
0709 13:04:22.037486 (+  1936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.037586 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:22.284990 (+247404us) raft_consensus.cc:1685] finished
0709 13:04:22.285011 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:22.285162 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25972,"prepare.queue_time_us":264906,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":63,"raft.queue_time_us":78170,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":60208,"thread_start_us":399079,"threads_started":3,"wal-append.queue_time_us":250758}
I20260709 13:04:22.347481  1720 raft_consensus.cc:606] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:22.356837  6271 consensus_queue.cc:237] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:22.324782  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49611.4us
I20260709 13:04:22.243391  6798 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:22.334895  1713 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:04:22.251020  6936 consensus_queue.cc:260] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:22.284291  6987 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:22.382447  6936 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:22.267374  6962 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:22.385294  6962 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:22.265326  6820 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:22.267957  6921 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:22.387777  6820 consensus_queue.cc:1059] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:22.388643  6820 consensus_queue.cc:1243] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:22.353946  6883 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:04:22.389137  7010 consensus_meta.cc:287] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.650s	user 0.002s	sys 0.009s
I20260709 13:04:22.382339  6987 raft_consensus.cc:697] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:22.389822  7010 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:22.268124  6943 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:22.391664  1722 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 21: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:22.391959  6946 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.342206  6955 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:22.394026  6955 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.271057  6731 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:22.378409  6995 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:22.264747  6952 consensus_queue.cc:260] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.386744  6999 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:22.395015  1720 raft_consensus.cc:614] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:04:22.413370  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.395852  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.358964  1612 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:22.414391  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 55523us
I20260709 13:04:22.414889  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.258873  7065 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:22.416501  1612 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:22.389400  6883 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:22.328583  7075 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.390542  6987 consensus_queue.cc:237] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.412675  6995 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.263304  6905 consensus_queue.cc:260] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.420182  6952 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.421473  1720 raft_consensus.cc:1275] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
W20260709 13:04:22.431803  7040 consensus_meta.cc:287] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.868s	user 0.005s	sys 0.008s
I20260709 13:04:22.394505  6863 raft_consensus.cc:3055] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:22.353690  7067 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:22.395640  1611 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:22.430001  7070 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:04:22.428624  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.436607  6883 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:22.439142  2313 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:22.442250  1721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2786) took 1711 ms (client timeout 1811 ms). Trace:
W20260709 13:04:22.469280  1721 rpcz_store.cc:269] 0709 13:04:20.731120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.732011 (+   891us) service_pool.cc:224] Handling call
0709 13:04:22.442071 (+1710060us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400213754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:22.442188 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2864192384}
I20260709 13:04:22.470187  2313 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:22.470688  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.397311  7048 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:04:22.443766  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.444856   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:04:22.406178  6798 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.408892  6748 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.449023  1720 pending_rounds.cc:85] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 6
I20260709 13:04:22.477180  1720 pending_rounds.cc:107] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:04:22.421919  1713 raft_consensus.cc:2393] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 13: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:04:22.456121  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 60576us
I20260709 13:04:22.284238  6908 raft_consensus.cc:697] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:22.438058  6731 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:22.454917  6905 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.432623  1571 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:04:22.455137  6863 raft_consensus.cc:740] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:22.448032  7065 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:22.464598  7010 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:22.465679  6616 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:22.465886  7041 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:22.430079  6962 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:22.467253  7068 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:22.468014  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.444480  7074 raft_consensus.cc:2804] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:22.450430  6995 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:22.476522  1721 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:04:22.508525  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2807) took 1160 ms. Trace:
I20260709 13:04:22.508868  1721 rpcz_store.cc:276] 0709 13:04:21.349184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.471721 (+1122537us) service_pool.cc:224] Handling call
0709 13:04:22.474437 (+  2716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.508243 (+ 33806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.508468 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:22.510222  1721 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:22.510761  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 534) took 1042 ms. Trace:
I20260709 13:04:22.511049  1721 rpcz_store.cc:276] 0709 13:04:21.468083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.509781 (+1041698us) service_pool.cc:224] Handling call
0709 13:04:22.510065 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.510548 (+   483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.510704 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:22.477459  1720 raft_consensus.cc:2889] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:22.460140  7040 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:22.468995  7070 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.512995  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2803) took 1267 ms. Trace:
I20260709 13:04:22.513052  7070 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:22.514441  6908 consensus_queue.cc:237] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.444977  6974 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.519214  6974 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:22.481341  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.520737  1713 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:22.521497   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:22.484082  1611 leader_election.cc:400] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16.
I20260709 13:04:22.522378  1611 leader_election.cc:403] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 13:04:22.522874  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 44724.1us
I20260709 13:04:22.523032  6722 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16.
I20260709 13:04:22.523989  6943 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:22.525233  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 536) took 1003 ms. Trace:
I20260709 13:04:22.441365  6659 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.471823  6946 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:22.477864  6798 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:22.485389  6748 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.482115  6731 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.485721  7048 raft_consensus.cc:697] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:22.528048  6953 consensus_meta.cc:287] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.003s
I20260709 13:04:22.529458  7075 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:22.529832  6962 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.521953  1713 raft_consensus.cc:1240] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:22.532665  1722 rpcz_store.cc:276] 0709 13:04:21.521457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.542513 (+ 21056us) service_pool.cc:224] Handling call
0709 13:04:22.524963 (+982450us) spinlock_profiling.cc:243] Waited 848 ms on lock 0x7b5400213254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:22.525154 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1950366976}
I20260709 13:04:22.513399  1720 rpcz_store.cc:276] 0709 13:04:21.245794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.420849 (+1175055us) service_pool.cc:224] Handling call
0709 13:04:22.421149 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.512783 (+ 91634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.512946 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:22.485271  7065 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.540705  1718 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 20.
I20260709 13:04:22.544267  1715 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
W20260709 13:04:22.487309  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 143429us
I20260709 13:04:22.531265  6753 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:22.550504  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 294709us
W20260709 13:04:22.551920  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 81845.6us
I20260709 13:04:22.557847  1717 raft_consensus.cc:2410] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 13:04:22.545657  7066 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.011s	sys 0.019s
I20260709 13:04:22.556464  7065 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:22.559228  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:22.560741  1538 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:22.550552  6731 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:22.561399  7066 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.012s	sys 0.019s
I20260709 13:04:22.583144  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2784) took 1876 ms. Trace:
I20260709 13:04:22.570932  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:22.574057  1537 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:22.591439  1612 leader_election.cc:400] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16.
I20260709 13:04:22.577836  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:22.583748  1719 rpcz_store.cc:276] 0709 13:04:20.707005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.707223 (+   218us) service_pool.cc:224] Handling call
0709 13:04:20.707604 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.243558 (+1535954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.579443 (+335885us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b5400246b54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:22.583071 (+  3628us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3489459456}
I20260709 13:04:22.601679  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2409) took 1400 ms. Trace:
I20260709 13:04:22.602268  1716 rpcz_store.cc:276] 0709 13:04:21.200854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.554894 (+354040us) service_pool.cc:224] Handling call
0709 13:04:21.555163 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.235445 (+680282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.601296 (+365851us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b54002c60d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:22.601551 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1521444736}
I20260709 13:04:22.568547  6753 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:04:22.604089  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 156982us
I20260709 13:04:22.604926  2346 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:04:22.598390  6946 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:04:22.599188  6820 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:22.598459  6974 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:04:22.596449  7045 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.593341  6953 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.606282  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:22.622068  2312 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:22.623268  2340 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:04:22.622069  1612 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:22.612624  7048 consensus_queue.cc:237] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.608824  6820 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:22.633955  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.604988  6968 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.634193  6820 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.636385  6968 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:22.570955  6943 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:22.614574  7067 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13.
I20260709 13:04:22.614267  7045 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.641400  6995 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:22.642165  7045 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:22.615031  6953 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.643934  6995 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:22.645164  6953 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:22.645656  6748 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:22.645848  6798 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:22.637144  6968 consensus_queue.cc:1059] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:22.619081  6962 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:22.620710  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:04:22.647991  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 45109.7us
I20260709 13:04:22.648617  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.654488  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:22.657685  1571 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:22.660241  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:22.667393  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.667186  2340 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:22.669309  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.671412  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:22.672497  1648 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:04:22.553213  6863 consensus_queue.cc:260] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.683439  2348 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 21: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:22.675391  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.656430  6953 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.695374  6917 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:04:22.695587  1719 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 21: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:22.695528  6953 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:22.683555  7067 consensus_queue.cc:1059] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:04:22.655047  7089 consensus_queue.cc:1048] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:04:22.684810  6271 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:22.699852  6271 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.700444  6271 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:22.702139  6863 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.687501  6955 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.688944  6798 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:22.691731  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:22.703541  1567 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:22.706241  7067 raft_consensus.cc:3055] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:22.706728  7067 raft_consensus.cc:740] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:22.707569  7067 consensus_queue.cc:260] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.708014  6955 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:22.694540  6659 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:22.694753  1648 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:04:22.712584  1568 raft_consensus.cc:1240] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: []
W20260709 13:04:22.684772  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 124157us
W20260709 13:04:22.712657  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 47463.3us
I20260709 13:04:22.714707  2314 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:22.713250  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.661744  6753 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:22.716877  1538 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:22.717440  1722 raft_consensus.cc:2393] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:22.718991  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.721416  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:22.722236  1722 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:22.725463  1571 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:04:22.734184  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:22.719767  1720 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:04:22.738880  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:22.738813  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:22.740833  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:22.742479  1567 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 21.
I20260709 13:04:22.741536  2346 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:22.741240  1612 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:22.672487  6748 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:22.697325  7065 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.676044  6962 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:22.793263  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:22.770731  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:22.794459  1720 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.750891  6271 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.736541  6798 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:22.796726  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:22.752491  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:22.770449  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:22.799904  1716 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:22.765049  6974 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:22.798185  1570 raft_consensus.cc:2410] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:04:22.667637  7070 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:22.794235  1565 raft_consensus.cc:2376] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11. Re-sending same reply.
I20260709 13:04:22.808993  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:22.810518  1565 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:22.748620  6943 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:22.815729  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:22.752974  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:22.772121  6753 raft_consensus.cc:697] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:22.749869  1538 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:22.801306  1566 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:22.751986  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:22.806311  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:22.806823  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.818480  1572 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 21: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:22.765210  6946 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:22.750475  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:22.780686  6955 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:22.824357  1564 raft_consensus.cc:2368] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14.
I20260709 13:04:22.817899  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:22.748196  7089 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:04:22.825770  1570 raft_consensus.cc:1240] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:22.796056  6748 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.829023  1564 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:22.829737  1566 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:22.831735  1566 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
W20260709 13:04:22.821533  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 71798.1us
I20260709 13:04:22.833355  1566 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:22.834829  1538 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:22.836601  2343 raft_consensus.cc:2368] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16.
I20260709 13:04:22.827283  6798 raft_consensus.cc:697] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:04:22.840903  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 316815us
I20260709 13:04:22.841187  6271 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:22.820109  6943 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.841703  1611 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:22.846994  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.848205  2346 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:22.822609  7082 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:22.853458  1565 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:22.852686  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.860102  1714 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:22.861479  6271 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:22.854301  1716 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:22.854007  1612 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:22.861727  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.861830  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:22.863287  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:22.861704  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.864866  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:22.865427  1717 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:22.865749  6271 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:04:22.866204  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.749351  6917 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.107s
I20260709 13:04:22.838060  6955 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.869283  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:22.869959  1572 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:22.843143  7074 raft_consensus.cc:697] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:22.867775  1714 raft_consensus.cc:2410] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:04:22.868626  1538 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:22.875550  1538 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:22.870636  1612 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:22.844436  6943 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:22.877760  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.893298  1718 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:22.890859  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:22.894120  1714 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 13: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:22.900696  1716 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.900776  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.838434  6946 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.877120  6659 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:22.837037  6753 consensus_queue.cc:237] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:22.863332  6962 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:22.859380  7082 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:22.896692  6271 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:22.918423  6271 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:22.918823  6271 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:22.839351  6974 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:04:22.879575  6908 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:22.840646  6748 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.876546  6955 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:22.906888  1643 raft_consensus.cc:1275] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:04:22.897248  6936 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:04:22.894886  7085 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.280s	user 0.005s	sys 0.000s
I20260709 13:04:22.942147  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:22.942783  1644 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.943717  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:22.944916  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:22.945547  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.909332  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:22.946105  1639 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.910552  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:22.913455  1572 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:22.914438  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:22.915666  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:22.917667  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:22.948256  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:22.921125  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:22.948302  1641 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:22.950170  1643 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:22.879812  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:22.951280  1715 raft_consensus.cc:2376] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14. Re-sending same reply.
I20260709 13:04:22.922443  6953 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:22.924057  7065 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:22.926069  6908 raft_consensus.cc:697] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:22.928458  2313 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:22.932067  1719 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:22.933457  7087 consensus_queue.cc:1059] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:22.935883  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:22.977049  1648 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:22.936611  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:22.940677  1717 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:22.948982  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:22.978513  1647 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:04:22.978961  1718 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:22.920881  7082 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:22.934846  6955 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:22.963061  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.894613  6820 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:22.955797  1716 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
W20260709 13:04:22.984266  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 55888.2us
I20260709 13:04:22.989838  1566 raft_consensus.cc:1275] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:04:23.002656  1612 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:23.003459  1718 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:23.004904  1714 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:23.006143  1643 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:23.007270  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:23.008064  1643 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:23.008850  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:23.009382  1643 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:23.011044  1722 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:22.968520  6936 raft_consensus.cc:697] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 13 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:23.005290  1612 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:23.014067  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:23.015429  6271 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:23.012540  1639 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:23.018913  1565 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:23.027678  2346 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:04:23.027808  1566 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:23.022958  1612 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:23.031316  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:23.034353  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:23.035373  2344 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:23.036227  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:22.957434  6974 consensus_queue.cc:1048] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
W20260709 13:04:23.021548  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.021695  1715 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:04:23.019315  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:22.906541  6798 consensus_queue.cc:237] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:23.041676  1569 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:23.046897  1722 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 21: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:22.916054  7074 consensus_queue.cc:237] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:23.051118  6271 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:23.051718  1566 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:23.058401  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:23.020967  7090 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:04:23.060542  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.021103  6995 raft_consensus.cc:3055] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:23.049628  1644 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:23.067152  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:23.135010  1645 raft_consensus.cc:2393] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 15: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:23.056334  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:23.069193  1647 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:23.074065  1643 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:23.137216  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:23.137470  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:23.139215  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:23.078081  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.083546  6616 raft_consensus.cc:3055] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:23.050246  7070 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:23.110795  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:23.061017  6968 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:23.142537  2312 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:23.148769  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:23.166535  1573 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:04:23.148963  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:23.149581  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:23.101212  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:23.141662  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:23.150559  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:23.151477  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:23.153852  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:23.159614  1646 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:04:23.115978  7067 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:23.168203  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:23.133556  7094 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:23.188395  1571 raft_consensus.cc:2368] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9.
I20260709 13:04:23.207017  1566 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 14: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:23.197386  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:23.189318  1569 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:23.191394  1648 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:23.133563  6995 raft_consensus.cc:740] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:23.196724  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:23.212618  1570 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:04:23.170012  1720 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:23.171744  1643 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:23.214882  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:23.215746  1643 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:23.216706  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:23.215987  1646 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16.
I20260709 13:04:23.197508  1573 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:23.219218  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:23.164708  7065 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:23.146154  6616 raft_consensus.cc:740] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:23.066028  6731 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:23.225242  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.225914  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:23.255931  1572 raft_consensus.cc:2393] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 13: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:23.231096  1611 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:23.220481  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:23.239796  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:23.245458  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:23.257766  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:23.252313  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:23.257694  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:23.260403  1566 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:04:23.270720  1570 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:23.196967  6659 raft_consensus.cc:697] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:04:23.276153  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45199.1us
W20260709 13:04:23.242411  6883 consensus_meta.cc:287] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.766s	user 0.005s	sys 0.008s
I20260709 13:04:23.290163  7108 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:23.290520  7101 log.cc:927] Time spent T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:04:23.290918  7108 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:23.246858  6953 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 13:04:23.292805  6722 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:23.294745  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.240260  6968 raft_consensus.cc:697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:23.309278  2314 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:23.311092  1645 raft_consensus.cc:2368] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14.
I20260709 13:04:23.296381  1611 leader_election.cc:400] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9.
I20260709 13:04:23.312695  1611 leader_election.cc:403] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:23.315408  1714 raft_consensus.cc:1240] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:23.285264  6962 raft_consensus.cc:697] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:23.206502  6936 consensus_queue.cc:237] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:23.302945  1641 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:23.338460  1641 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:04:23.307281  1538 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:23.318796  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:23.320919  1612 leader_election.cc:400] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16.
I20260709 13:04:23.340204  1612 leader_election.cc:403] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:23.331946  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:23.341022  1566 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:23.335348  1648 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:23.147457  6908 consensus_queue.cc:237] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:23.349217  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:23.352823  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:23.354070  1573 raft_consensus.cc:2410] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:04:23.349807  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:23.351717  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 44531.9us
I20260709 13:04:23.273590  7094 raft_consensus.cc:697] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:23.355865  1538 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:23.363658  1538 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:23.280156  7065 raft_consensus.cc:697] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 15 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:04:23.303990  7099 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
W20260709 13:04:23.359083  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 63118.1us
I20260709 13:04:23.364769  5852 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:23.378576  5852 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:23.379166  5852 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:23.379801  1611 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:23.340406  6952 consensus_meta.cc:287] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.884s	user 0.012s	sys 0.000s
I20260709 13:04:23.397092  1611 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:23.400328  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:23.403820  1572 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 21: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:23.369074  7075 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:23.294222  6883 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:23.374336  1641 raft_consensus.cc:1240] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:23.357700  7066 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.375020  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:23.364779  1538 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:23.377836  7097 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
I20260709 13:04:23.386942  7104 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 8. Current term is 9.
I20260709 13:04:23.304870  6946 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:23.387962  7092 log.cc:927] Time spent T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
W20260709 13:04:23.389120  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 68419us
I20260709 13:04:23.390024  7107 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16.
I20260709 13:04:23.400931  7103 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:23.400300  6917 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.464291  6917 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:23.468405  6917 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:23.368341  1648 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 12. Ops: []
I20260709 13:04:23.471951  1648 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:23.403096  5852 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:23.478463  5852 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:23.415448  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:23.479537  1717 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:23.479394  6968 consensus_queue.cc:237] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:23.459796  7066 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:23.483774  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2422) took 1469 ms. Trace:
I20260709 13:04:23.418341  1714 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:04:23.421805  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:23.484259  1721 rpcz_store.cc:276] 0709 13:04:22.014718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.511697 (+496979us) service_pool.cc:224] Handling call
0709 13:04:22.530981 (+ 19284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:22.532076 (+  1095us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:22.532105 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:22.532577 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 13:04:22.857624 (+325047us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:22.857707 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.860341 (+  2634us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:23.483538 (+623197us) raft_consensus.cc:1685] finished
0709 13:04:23.483564 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:23.483708 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2980,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":4969,"thread_start_us":290215,"threads_started":1,"wal-append.queue_time_us":454453}
I20260709 13:04:23.485827  1720 raft_consensus.cc:2393] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 13: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:23.423548  1715 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:23.486076  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:23.433050  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 68358.8us
I20260709 13:04:23.457417  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:23.435664  1685 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:23.488654  1571 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:23.488219  1538 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:23.388002  6820 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:23.346222  6748 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:23.437340  1566 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:23.425875  7075 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:23.486932  1722 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:23.391741  7084 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:23.453811  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:23.493151  1538 leader_election.cc:400] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14.
I20260709 13:04:23.519932  1538 leader_election.cc:403] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
W20260709 13:04:23.494483  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.520819  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:23.495078  1721 raft_consensus.cc:2393] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:23.501569  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:23.521998  1715 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:23.502067  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:23.504393  1573 raft_consensus.cc:1240] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:23.505599  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:23.506551  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:23.527964  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:23.528749  1720 raft_consensus.cc:1813] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Handling vote request from an unknown peer f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:23.529348  1720 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 13: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:23.494927  5852 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:04:23.498791  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 63231.4us
I20260709 13:04:23.530900  1685 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:23.508299  6659 consensus_queue.cc:237] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:23.507125  1648 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:23.516783  7094 consensus_queue.cc:237] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:23.428922  7089 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.520169  1719 raft_consensus.cc:1275] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:04:23.534245  5852 raft_consensus.cc:697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:23.391695  7110 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:23.346462  6995 consensus_queue.cc:260] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:23.451902  1641 raft_consensus.cc:1275] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:04:23.486189  6952 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:23.542416  6820 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:23.543330  1571 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:23.678180  1641 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:04:23.543591  1721 raft_consensus.cc:1275] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:23.462878  7108 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:23.520272  7099 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:04:23.555650  1648 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:23.557049  1569 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:23.563230  1568 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:23.534849  1685 leader_election.cc:400] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14.
I20260709 13:04:23.509904  6917 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:23.688910  1685 leader_election.cc:403] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:04:23.689884  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 156412us
I20260709 13:04:23.523489  7075 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:23.519575  7070 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:23.590854  1720 raft_consensus.cc:3055] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:23.721261  1720 raft_consensus.cc:740] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:23.722705  1720 consensus_queue.cc:260] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:23.537104  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 394654us
I20260709 13:04:23.441413  6955 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:23.640869  5852 consensus_queue.cc:237] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:23.544380  6863 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.544109  6905 raft_consensus.cc:2764] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:04:23.637542  7084 raft_consensus.cc:697] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:23.552769  7048 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.687408  7103 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:04:23.657809  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.485255  6616 consensus_queue.cc:260] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:23.833017  6616 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:23.643748  6976 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.838320  6976 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:23.666261  7111 log.cc:927] Time spent T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:04:23.843597  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2832) took 1094 ms. Trace:
I20260709 13:04:23.601019  7090 raft_consensus.cc:697] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:23.847299  7090 consensus_queue.cc:237] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:23.679615  1641 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:04:23.680240  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 384375us
W20260709 13:04:23.850160  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.640318  7110 raft_consensus.cc:697] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:23.851979  7110 consensus_queue.cc:237] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:23.654024  7112 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.136s	user 0.001s	sys 0.001s
I20260709 13:04:23.571851  7089 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:23.859416  7089 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:23.748178  6271 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:23.861271  6271 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:23.864156  6271 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:23.793911  1648 raft_consensus.cc:1275] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:23.543897  6962 consensus_queue.cc:237] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:23.794648  1717 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:23.860266  7089 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.844173  1716 rpcz_store.cc:276] 0709 13:04:22.749315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.014330 (+265015us) service_pool.cc:224] Handling call
0709 13:04:23.046529 (+ 32199us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:23.055702 (+  9173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:23.055745 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:23.064340 (+  8595us) log.cc:844] Serialized 26 byte log entry
0709 13:04:23.210291 (+145951us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:23.210351 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.210452 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:23.843238 (+632786us) raft_consensus.cc:1685] finished
0709 13:04:23.843275 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:23.843498 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13909,"raft.queue_time_us":8796,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":11738,"thread_start_us":145559,"threads_started":1,"wal-append.queue_time_us":348795}
I20260709 13:04:23.794947  1720 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:23.808678  1715 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:23.642899  6946 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.879066  6946 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:23.881796  6946 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:23.817294  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:23.826314  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 135440us
I20260709 13:04:23.883104  1721 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:23.883368  6976 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:23.884244  6976 consensus_queue.cc:1059] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:23.893781  1721 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:23.906111  6976 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:23.873586  7089 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:23.907825  6976 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:23.723842  6743 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14.
I20260709 13:04:23.715803  6952 raft_consensus.cc:697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:23.637715  7065 consensus_queue.cc:237] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:23.936059  6743 consensus_queue.cc:1059] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:04:23.940886  1611 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.026s	sys 0.046s
W20260709 13:04:23.941294  1611 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.026s	sys 0.046s
W20260709 13:04:23.941838  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 544862us
I20260709 13:04:23.942634  1611 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:23.944795  1611 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:23.800015  7070 raft_consensus.cc:697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:23.952562  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:23.950965  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:23.953258  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:23.953243  1571 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:23.953799  1569 raft_consensus.cc:1813] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Handling vote request from an unknown peer f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:23.954857  1569 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 13: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
W20260709 13:04:23.954967  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.955808  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:23.810175  6987 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:04:23.973963  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 248304us
I20260709 13:04:23.829488  6731 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:23.950659  6798 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.045s	sys 0.041s
I20260709 13:04:23.931517  7108 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.943997  6955 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:23.963464  7114 log.cc:927] Time spent T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:04:23.917491  7089 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:23.951256  6722 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:23.946993  7106 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.997025  1720 raft_consensus.cc:1275] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:04:24.003465  1645 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:24.007305  1687 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:23.951632  6748 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:23.814000  6863 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:23.978385  6905 raft_consensus.cc:3055] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:04:23.985349  7099 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:04:23.923333  7048 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:23.995203  6798 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.045s	sys 0.042s
I20260709 13:04:24.042033  1687 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:24.064347  6976 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:24.064934  7118 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:04:24.064831  1687 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:24.066747  1687 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:24.067584  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2933) took 1148 ms. Trace:
I20260709 13:04:23.992885  7104 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:24.048450  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:23.997946  7131 raft_consensus.cc:3055] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:04:24.063349  7105 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.425s	user 0.000s	sys 0.001s
I20260709 13:04:24.065029  6976 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.063835  6962 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.124078  1572 raft_consensus.cc:1275] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:04:24.041551  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 156959us
I20260709 13:04:24.067960  1646 rpcz_store.cc:276] 0709 13:04:22.919636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.247161 (+327525us) service_pool.cc:224] Handling call
0709 13:04:23.247475 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:23.247827 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:23.247858 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:23.248302 (+   444us) log.cc:844] Serialized 26 byte log entry
0709 13:04:23.783434 (+535132us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:23.783497 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.783594 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:24.067350 (+283756us) raft_consensus.cc:1685] finished
0709 13:04:24.067371 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.067526 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39326,"raft.queue_time_us":25405,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":14176,"thread_start_us":264001,"threads_started":1,"wal-append.queue_time_us":783987}
I20260709 13:04:24.041414  6883 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:24.066998  7117 log.cc:927] Time spent T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:04:23.983970  6987 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:24.119477  1565 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:04:24.056901  7106 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:24.121765  7104 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:24.068271  6731 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:24.143086  7123 log.cc:927] Time spent T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:04:24.068750  1687 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:24.143435  6731 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:24.059854  6905 raft_consensus.cc:740] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:24.145391  6905 consensus_queue.cc:260] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:24.147083  6905 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:24.147773  1569 raft_consensus.cc:3055] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:24.148254  1569 raft_consensus.cc:740] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:24.135272  1646 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:24.150852  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2446) took 1110 ms. Trace:
I20260709 13:04:24.121083  7070 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:24.151252  1646 rpcz_store.cc:276] 0709 13:04:23.040648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.134422 (+1093774us) service_pool.cc:224] Handling call
0709 13:04:24.134814 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:24.150402 (+ 15588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.150696 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:24.115803  7131 raft_consensus.cc:740] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:24.021472  6955 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:24.037762  7116 consensus_queue.cc:1059] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.114s
I20260709 13:04:24.068019  7066 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:24.133844  6962 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:24.152997  1646 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:24.012928  7108 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:24.067133  6616 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:04:24.155253  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.160979  1571 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:24.073035  6952 consensus_queue.cc:237] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:24.220294  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.149103  1569 consensus_queue.cc:260] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:24.164194  7082 consensus_meta.cc:287] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.150s	user 0.016s	sys 0.000s
I20260709 13:04:24.058424  7084 consensus_queue.cc:237] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:24.223918  1569 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:24.140798  7126 log.cc:927] Time spent T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:04:24.210779  7138 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:24.229041  7138 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:24.212956  7114 log.cc:927] Time spent T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:04:24.231510  7082 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:24.206974  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2448) took 1073 ms. Trace:
I20260709 13:04:24.231812  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:24.232831  1564 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:24.233731  1571 raft_consensus.cc:2410] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:04:24.234858  7082 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:24.234946  1646 rpcz_store.cc:276] 0709 13:04:23.133833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.152346 (+1018513us) service_pool.cc:224] Handling call
0709 13:04:24.152763 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:24.198394 (+ 45631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.206858 (+  8464us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:24.162016  7096 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:24.238715  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 113678us
I20260709 13:04:24.209929  6917 raft_consensus.cc:697] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:24.131356  6976 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:24.246415  2340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 697) took 1239 ms. Trace:
W20260709 13:04:24.254324  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 185647us
I20260709 13:04:24.253243  2340 rpcz_store.cc:276] 0709 13:04:22.999269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.999583 (+   314us) service_pool.cc:224] Handling call
0709 13:04:23.000074 (+   491us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:23.014706 (+ 14632us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:23.014737 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:23.027792 (+ 13055us) log.cc:844] Serialized 52 byte log entry
0709 13:04:23.391562 (+363770us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:23.461746 (+ 70184us) log.cc:844] Serialized 14 byte log entry
0709 13:04:23.676416 (+214670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.676523 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:24.234670 (+558147us) raft_consensus.cc:1685] finished
0709 13:04:24.235037 (+   367us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.238113 (+  3076us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14025,"prepare.queue_time_us":54000,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":1830,"raft.queue_time_us":364743,"raft.run_cpu_time_us":479,"raft.run_wall_time_us":155309,"thread_start_us":537206,"threads_started":2,"wal-append.queue_time_us":599054}
I20260709 13:04:24.256695  1687 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:24.243312  7066 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:24.269867  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:24.237613  7139 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:24.277941  1569 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:04:24.279838  1571 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:24.269145  6731 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:24.293887  6917 consensus_queue.cc:237] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:24.252782  7096 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:24.294548  7124 log.cc:927] Time spent T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:04:24.241379  6616 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:24.343879  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 79566us
I20260709 13:04:24.344576  2349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 700) took 1328 ms. Trace:
I20260709 13:04:24.321269  7145 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:24.357275  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:24.357151  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:24.345005  2349 rpcz_store.cc:276] 0709 13:04:23.015734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.015911 (+   177us) service_pool.cc:224] Handling call
0709 13:04:23.016321 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:23.286954 (+270633us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:23.378363 (+ 91409us) log.cc:844] Serialized 14 byte log entry
0709 13:04:23.661420 (+283057us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:23.803505 (+142085us) log.cc:844] Serialized 26 byte log entry
0709 13:04:23.827185 (+ 23680us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:23.827343 (+   158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.827509 (+   166us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:24.344325 (+516816us) raft_consensus.cc:1685] finished
0709 13:04:24.344350 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.344508 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14686,"prepare.queue_time_us":49091,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":278924,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":14909,"spinlock_wait_cycles":13113088,"thread_start_us":537421,"threads_started":2,"wal-append.queue_time_us":741532}
I20260709 13:04:24.347093  7082 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:24.386147  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2923) took 1482 ms. Trace:
I20260709 13:04:24.346354  7137 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:24.352391  6863 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:24.268479  7131 consensus_queue.cc:260] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:24.342449  6952 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:24.398599  6917 raft_consensus.cc:2764] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:04:24.400471  7106 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:24.402379  7143 log.cc:927] Time spent T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 13:04:24.403787  7129 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:04:24.404732  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:24.415251  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 158635us
I20260709 13:04:24.441988  1687 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:24.358554  7145 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.407176  1640 rpcz_store.cc:276] 0709 13:04:22.903239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.903544 (+   305us) service_pool.cc:224] Handling call
0709 13:04:22.935814 (+ 32270us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:04:23.059404 (+123590us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:23.059436 (+    32us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:04:23.068603 (+  9167us) log.cc:844] Serialized 104 byte log entry
0709 13:04:23.523268 (+454665us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:23.571009 (+ 47741us) log.cc:844] Serialized 14 byte log entry
0709 13:04:23.783884 (+212875us) log.cc:844] Serialized 14 byte log entry
0709 13:04:23.786247 (+  2363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.786397 (+   150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:24.363347 (+576950us) raft_consensus.cc:1685] finished
0709 13:04:24.363381 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.385947 (+ 22566us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":47497,"raft.queue_time_us":66495,"raft.run_cpu_time_us":497,"raft.run_wall_time_us":92487,"spinlock_wait_cycles":8420736,"thread_start_us":444444,"threads_started":2,"wal-append.queue_time_us":510260}
I20260709 13:04:24.382999  7048 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:24.405416  7119 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.282s	user 0.002s	sys 0.000s
I20260709 13:04:24.390070  7096 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:24.435976  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 166832us
I20260709 13:04:24.452214  1685 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:24.438153  7067 consensus_meta.cc:287] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.240s	user 0.011s	sys 0.008s
I20260709 13:04:24.415493  7082 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:24.451061  6955 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:24.437491  7131 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:24.530040  7082 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:24.531473  6943 consensus_meta.cc:287] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.632s	user 0.008s	sys 0.007s
I20260709 13:04:24.532054  6943 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:24.533516  1567 raft_consensus.cc:2393] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 16: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:24.494244  1640 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:24.448119  6987 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:24.542307  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2837) took 1122 ms. Trace:
I20260709 13:04:24.451138  7145 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:24.436048  6952 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:24.505898  7141 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:04:24.508612  7134 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:24.510615  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2833) took 1760 ms. Trace:
I20260709 13:04:24.574885  1718 rpcz_store.cc:276] 0709 13:04:22.750032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.019900 (+269868us) service_pool.cc:224] Handling call
0709 13:04:23.020311 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:23.135831 (+115520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:23.135866 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:23.140773 (+  4907us) log.cc:844] Serialized 26 byte log entry
0709 13:04:23.573700 (+432927us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:23.573770 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.573877 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:24.510328 (+936451us) raft_consensus.cc:1685] finished
0709 13:04:24.510355 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.510540 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":115073,"raft.queue_time_us":117247,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":11981,"thread_start_us":325406,"threads_started":1,"wal-append.queue_time_us":800453}
I20260709 13:04:24.512771  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2449) took 1291 ms. Trace:
I20260709 13:04:24.577800  1572 rpcz_store.cc:276] 0709 13:04:23.221143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.524314 (+303171us) service_pool.cc:224] Handling call
0709 13:04:23.795409 (+271095us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:24.124714 (+329305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.512506 (+387792us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b54002238d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:24.512701 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":754827520}
I20260709 13:04:24.513767  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2484) took 1273 ms. Trace:
I20260709 13:04:24.579146  1565 rpcz_store.cc:276] 0709 13:04:23.240072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.525189 (+285117us) service_pool.cc:224] Handling call
0709 13:04:23.793830 (+268641us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:24.137534 (+343704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.513513 (+375979us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54002251d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:24.513704 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":747994752}
W20260709 13:04:24.516858  7113 log.cc:927] Time spent T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.854s	user 0.000s	sys 0.001s
I20260709 13:04:24.521618  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2430) took 1572 ms. Trace:
I20260709 13:04:24.580547  1645 rpcz_store.cc:276] 0709 13:04:22.949325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.339879 (+390554us) service_pool.cc:224] Handling call
0709 13:04:23.369583 (+ 29704us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:24.038168 (+668585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.521337 (+483169us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b54000a69d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:24.521544 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1457405824}
I20260709 13:04:24.523332  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2839) took 1101 ms. Trace:
I20260709 13:04:24.582558  1722 rpcz_store.cc:276] 0709 13:04:23.421815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.421965 (+   150us) service_pool.cc:224] Handling call
0709 13:04:23.462224 (+ 40259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:23.519035 (+ 56811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.523058 (+1004023us) spinlock_profiling.cc:243] Waited 24.2 ms on lock 0x7b540038fed4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:24.523266 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55660800}
I20260709 13:04:24.527020  1571 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:04:24.527853  6616 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:24.420936  7137 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:24.472816  7108 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:24.477941  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 94239.2us
I20260709 13:04:24.542932  1719 rpcz_store.cc:276] 0709 13:04:23.419599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.419764 (+   165us) service_pool.cc:224] Handling call
0709 13:04:23.420047 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:23.554570 (+134523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.542012 (+987442us) spinlock_profiling.cc:243] Waited 99.7 ms on lock 0x7b54000541d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:24.542229 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":229339520}
I20260709 13:04:24.544936  1537 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:24.600302  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 55642.1us
I20260709 13:04:24.601951  1537 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:24.605453  1643 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 15 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:24.607434  1646 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 13 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:24.441037  7066 raft_consensus.cc:697] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:24.570813  6976 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:24.572849  1566 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:04:24.590473  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:24.591147  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 139065us
I20260709 13:04:24.613991  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:24.599404  7075 consensus_meta.cc:287] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.804s	user 0.012s	sys 0.003s
I20260709 13:04:24.589980  7152 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:24.611805  1639 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:24.558462  6917 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:24.609485  6820 consensus_meta.cc:287] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.914s	user 0.014s	sys 0.000s
I20260709 13:04:24.605443  7082 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:24.613503  1537 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:24.607858  7139 raft_consensus.cc:697] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:24.609299  6863 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:24.638180  7075 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:24.641475  6943 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:24.646044  7084 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:24.663609  6743 consensus_queue.cc:1059] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:04:24.666200  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 52326.5us
I20260709 13:04:24.667454  7163 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:24.667276  6952 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:24.668434  7152 raft_consensus.cc:697] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:24.594973  7134 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:24.669467  7137 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:24.640076  6820 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14.
I20260709 13:04:24.693882  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:24.694936  2340 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:24.695861  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:24.699528  7134 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:24.699846  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:24.706833  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.701277  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2926) took 1793 ms. Trace:
I20260709 13:04:24.701728  6636 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:04:24.707834  1642 rpcz_store.cc:276] 0709 13:04:22.908330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.908449 (+   119us) service_pool.cc:224] Handling call
0709 13:04:22.908770 (+   321us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:23.071608 (+162838us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:23.071655 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:23.072397 (+   742us) log.cc:844] Serialized 52 byte log entry
0709 13:04:23.297920 (+225523us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:23.354352 (+ 56432us) log.cc:844] Serialized 14 byte log entry
0709 13:04:23.398441 (+ 44089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.697296 (+1298855us) spinlock_profiling.cc:243] Waited 64.2 ms on lock 0x7b5400099554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:24.697394 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:24.697473 (+    79us) raft_consensus.cc:1685] finished
0709 13:04:24.697665 (+   192us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.701186 (+  3521us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44528,"prepare.queue_time_us":59839,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":214170,"raft.run_cpu_time_us":379,"raft.run_wall_time_us":29871,"spinlock_wait_cycles":147762432,"thread_start_us":322944,"threads_started":2,"wal-append.queue_time_us":444677}
I20260709 13:04:24.689244  6953 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:04:24.712139  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:24.714102  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:24.715371  2340 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
W20260709 13:04:24.720129  1567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 859) took 1572 ms (client timeout 2029 ms). Trace:
W20260709 13:04:24.728557  1567 rpcz_store.cc:269] 0709 13:04:23.147295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.151183 (+  3888us) service_pool.cc:224] Handling call
0709 13:04:24.719831 (+1568648us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54001b5b54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:24.720043 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3094466816}
I20260709 13:04:24.730471  7139 consensus_queue.cc:237] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:24.706259  7154 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.069s	user 0.005s	sys 0.000s
I20260709 13:04:24.747958  7067 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:24.748572  1639 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2447) took 1701 ms (client timeout 1731 ms). Trace:
W20260709 13:04:24.749084  1639 rpcz_store.cc:269] 0709 13:04:23.047526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.055865 (+  8339us) service_pool.cc:224] Handling call
0709 13:04:24.748214 (+1692349us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54000a5fd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:24.748502 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3285928832}
I20260709 13:04:24.749809  6616 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.750572  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:24.750366  6616 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:24.751549  1639 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:24.751833  6955 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.752975  6616 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:24.732015  6962 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:24.724951  6798 raft_consensus.cc:3055] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:24.755290  6798 raft_consensus.cc:740] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:24.756533  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2936) took 1315 ms. Trace:
I20260709 13:04:24.737932  7158 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:24.717304  7137 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
W20260709 13:04:24.740156  7150 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.330s	user 0.001s	sys 0.000s
I20260709 13:04:24.740804  6917 consensus_queue.cc:1059] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:24.774811  6905 raft_consensus.cc:3055] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:24.775336  6905 raft_consensus.cc:740] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:04:24.776628  7093 log.cc:927] Time spent T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.628s	user 0.011s	sys 0.004s
I20260709 13:04:24.776363  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:24.776329  6905 consensus_queue.cc:260] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:24.777438  2344 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:24.777827  6905 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:24.778406  1647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2421) took 1831 ms (client timeout 2155 ms). Trace:
I20260709 13:04:24.778573  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:24.778887  1647 rpcz_store.cc:269] 0709 13:04:22.946789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.136928 (+190139us) service_pool.cc:224] Handling call
0709 13:04:24.778169 (+1641241us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54002db254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:24.778337 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2543379712}
I20260709 13:04:24.715811  7163 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:04:24.741710  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:24.742048  1685 connection.cc:256] RPC call timeout handler was delayed by -1.0303273630000263s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1593s for a response
W20260709 13:04:24.795312  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.742394  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:24.796043  1645 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 10.
I20260709 13:04:24.742503  7131 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:24.743042  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:24.798903  1648 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:24.743722  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:24.799767  1612 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:24.800171  1642 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:24.744098  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:24.801761  1640 raft_consensus.cc:2368] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14.
I20260709 13:04:24.745990  7108 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.733573  7161 raft_consensus.cc:3055] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:24.746487  6987 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:24.757400  1646 rpcz_store.cc:276] 0709 13:04:23.441433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.247988 (+806555us) service_pool.cc:224] Handling call
0709 13:04:24.248577 (+   589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:24.607916 (+359339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.756117 (+148201us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b54000a64d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:24.756439 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":824807680}
I20260709 13:04:24.763530  1719 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:24.763379  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:24.813303  1722 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:24.766180  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:24.766578  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:24.768801  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:24.719959  7104 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:24.717164  6636 consensus_queue.cc:1048] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:04:24.756325  6798 consensus_queue.cc:260] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:24.794677  6962 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:24.811219  7109 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:24.782733  1687 connection.cc:256] RPC call timeout handler was delayed by -1.2748071749999781s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2155s for a response
I20260709 13:04:24.769263  1686 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:24.687980  7160 consensus_queue.cc:1048] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:04:24.752667  6955 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:24.814560  1718 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:24.814761  1720 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:24.814829  1721 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:04:24.780112  6743 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.829842  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:24.794601  6936 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.780681  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:24.808938  6968 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:24.798292  6917 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.748749  7067 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:24.841089  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:24.802551  7137 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:24.837374  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 68199.1us
I20260709 13:04:24.701803  7066 consensus_queue.cc:237] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:24.841413  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:24.842712  1687 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:24.809168  7161 raft_consensus.cc:740] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:24.812013  7158 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:24.778712  7145 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:24.847508  1713 raft_consensus.cc:2410] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:04:24.812071  7108 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:24.815909  6987 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:24.827980  6976 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:04:24.832777  7075 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:24.833596  7152 consensus_queue.cc:237] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:24.872498  7153 consensus_queue.cc:1048] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:04:24.836555  7131 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.878541  1716 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:04:24.868446  6743 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:24.883635  1714 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2446) took 1438 ms (client timeout 1549 ms). Trace:
W20260709 13:04:24.928761  1714 rpcz_store.cc:269] 0709 13:04:23.445281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.485192 (+ 39911us) service_pool.cc:224] Handling call
0709 13:04:24.883367 (+1398175us) spinlock_profiling.cc:243] Waited 885 ms on lock 0x7b5400249354. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:24.883551 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2035589888}
W20260709 13:04:24.929919  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.882521  6995 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:24.917876  7114 log.cc:927] Time spent T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
W20260709 13:04:24.886706  7140 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.672s	user 0.002s	sys 0.000s
I20260709 13:04:24.846280  6953 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:24.846524  7144 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.889196  6636 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:04:24.879712  7165 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:04:24.953908  6955 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:24.905252  7067 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:24.965966  6953 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.644690  1537 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:24.725680  1568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2478) took 1778 ms (client timeout 1593 ms). Trace:
I20260709 13:04:25.115471  6953 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:25.019622  7108 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:25.085353  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:25.086859  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:25.088786  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:25.154886  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:25.155442  1571 raft_consensus.cc:1240] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:25.156098  1567 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:04:24.807559  1612 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:25.085059  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:25.103801  1572 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:04:25.113588  6636 raft_consensus.cc:697] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:24.845034  7109 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:25.217042  7109 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:24.922024  7137 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.225759  7176 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:25.227212  7176 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:24.971302  7144 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.023411  7145 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:25.084945  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:24.878289  7158 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.258993  7158 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:24.893332  7161 consensus_queue.cc:260] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:25.127460  6743 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:25.261183  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.261579  7103 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.262255  7103 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:25.262315  7161 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:25.271471  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 463909us
W20260709 13:04:25.272687  7146 log.cc:927] Time spent T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.863s	user 0.006s	sys 0.000s
I20260709 13:04:25.274246  7094 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:25.275238  7094 raft_consensus.cc:697] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:04:25.116212  1568 rpcz_store.cc:269] 0709 13:04:22.947756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.947970 (+   214us) service_pool.cc:224] Handling call
0709 13:04:24.725403 (+1777433us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b5400013454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:24.725571 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1039937280}
I20260709 13:04:25.278272  7104 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:25.279574  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2473) took 2454 ms. Trace:
I20260709 13:04:25.280196  1564 rpcz_store.cc:276] 0709 13:04:22.825013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.833226 (+  8213us) service_pool.cc:224] Handling call
0709 13:04:22.833550 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:24.233242 (+1399692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.278836 (+1045594us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54002247d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:25.279293 (+   457us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3217917824}
I20260709 13:04:24.837500  6962 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.169394  1565 raft_consensus.cc:2393] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:24.868759  6936 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:25.282552  7065 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:25.284299  6962 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:25.175048  6636 consensus_queue.cc:237] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:24.868882  6917 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.164710  6908 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:25.276882  7094 consensus_queue.cc:237] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:25.179183  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.182754  1566 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:25.288199  6820 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.191272  6955 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.289063  6820 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.289299  6955 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.290134  7159 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.290956  7159 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.291848  6955 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:25.190699  6987 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:24.922051  7131 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:25.179677  6616 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:25.187080  7068 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.206549  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2934) took 2270 ms. Trace:
I20260709 13:04:25.196374  6953 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:25.305261  1644 rpcz_store.cc:276] 0709 13:04:22.936034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.320804 (+384770us) service_pool.cc:224] Handling call
0709 13:04:23.321225 (+   421us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:23.386623 (+ 65398us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:23.386981 (+   358us) log.cc:844] Serialized 14 byte log entry
0709 13:04:23.637966 (+250985us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:04:23.653758 (+ 15792us) log.cc:844] Serialized 78 byte log entry
0709 13:04:23.653997 (+   239us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:24.736818 (+1082821us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b18001a70a0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f824c87d3c7 00007f8248e8f9a7 00007f8248e9f0bf 00007f8248e9f038
0709 13:04:24.834842 (+ 98024us) log.cc:844] Serialized 14 byte log entry
0709 13:04:25.205299 (+370457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.205493 (+   194us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:25.205743 (+   250us) raft_consensus.cc:1685] finished
0709 13:04:25.205757 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:25.206117 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":300593,"prepare.queue_time_us":1087010,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":95,"raft.queue_time_us":418418,"raft.run_cpu_time_us":753,"raft.run_wall_time_us":10270,"spinlock_wait_cycles":9950080,"thread_start_us":416254,"threads_started":3,"wal-append.queue_time_us":575412}
I20260709 13:04:25.308140  7184 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:25.199105  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 354675us
I20260709 13:04:25.308372  6616 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:25.172614  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49648.4us
W20260709 13:04:25.167790  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 520547us
I20260709 13:04:25.311411  6616 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:25.311159  6798 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:25.312265  7176 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:25.312887  1537 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:25.186426  7067 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:25.314186  7067 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:25.315419  7068 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.316757  7067 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:25.317607  1687 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:25.284934  6936 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:25.289572  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.208269  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2931) took 2296 ms. Trace:
I20260709 13:04:25.322275  1643 rpcz_store.cc:276] 0709 13:04:22.911537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.244384 (+332847us) service_pool.cc:224] Handling call
0709 13:04:23.244729 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:24.606010 (+1361281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.207522 (+601512us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54001f4854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:25.208027 (+   505us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3129108864}
I20260709 13:04:25.282351  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:25.325057  7131 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:25.324846  1687 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:25.325717  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 76920us
I20260709 13:04:25.326637  7145 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:25.326778  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 63116.3us
I20260709 13:04:25.327155  7145 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.327970  7145 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:25.321034  7163 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:25.309289  7184 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.329054  7163 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:25.329556  7163 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:25.329958  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 47941us
I20260709 13:04:25.330354  7144 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:25.296906  7137 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:25.330892  1612 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:25.331660  7184 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:25.332155  1612 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:25.327092  1722 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:25.333943  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:25.291905  6820 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:25.293857  6908 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.357257  1611 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:25.377326  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 43901.5us
I20260709 13:04:25.387844  1687 leader_election.cc:400] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14.
I20260709 13:04:25.380194  6936 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:25.375098  7068 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:25.403398  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.408960  7185 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:25.259317   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:25.395411  6953 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:25.412397  6908 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:25.418365  7159 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:25.418910  6917 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:25.411000  7184 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:25.408337  7133 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:25.409976  1639 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.015s	sys 0.018s
W20260709 13:04:25.409691  1573 consensus_meta.cc:287] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.590s	user 0.012s	sys 0.008s
W20260709 13:04:25.423799  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.400734  6936 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:25.397291  7067 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:25.423549  7068 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.438100  6820 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.421734  6616 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:25.439987  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 111704us
I20260709 13:04:25.483793  7133 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:25.482059  6820 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.436976  7103 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.487178  6820 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:25.487380  7103 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:25.441466  1566 raft_consensus.cc:3055] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:25.488246  1566 raft_consensus.cc:740] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:25.488662  1573 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
W20260709 13:04:25.442301  6968 consensus_meta.cc:287] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.576s	user 0.006s	sys 0.020s
I20260709 13:04:25.444172  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:25.446071  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 84125us
I20260709 13:04:25.446384  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:25.448923  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:25.429183  7048 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:04:25.475950  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.422657  7185 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.470644  7184 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.482057  7068 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:25.499872  7184 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:25.495297  6995 consensus_meta.cc:287] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.525s	user 0.010s	sys 0.009s
I20260709 13:04:25.490477  1565 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:04:25.476894  6936 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:25.476497  7067 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.489604  1566 consensus_queue.cc:260] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:25.507555  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.498819  7185 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:25.513124  1611 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:25.520727  7068 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:25.521626  6820 consensus_queue.cc:1059] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:25.522547  6943 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.523530  6820 raft_consensus.cc:3055] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:25.523334  6943 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.509977  7067 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:25.509346  7048 raft_consensus.cc:697] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:25.531221  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:25.524060  6820 raft_consensus.cc:740] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:25.509567  6968 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:25.528260  1687 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:25.538659  7152 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:25.539542  7152 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.541970  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:25.543695  6636 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:25.544229  6636 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.544611  6636 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:25.533532  7133 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:25.545854  6976 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.546687  6976 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.559486  1568 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:25.563586  7133 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:25.564553  7159 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.554548  7068 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:25.565527  7070 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:25.565385  7159 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.565174  7133 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:25.568343  7133 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:25.565181  7115 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:04:25.590770  6820 consensus_queue.cc:260] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:25.585340  7187 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.059s	user 0.002s	sys 0.000s
I20260709 13:04:25.582407  6976 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:25.595644  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 99679.2us
I20260709 13:04:25.599517  6908 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:25.584335  7133 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 13:04:25.602095  6974 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:04:25.546689  7048 consensus_queue.cc:237] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:25.609371  1611 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:25.610627  6968 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:25.612092  6968 consensus_queue.cc:1059] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:25.613586  6968 raft_consensus.cc:3055] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:25.614138  6968 raft_consensus.cc:740] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:25.615931  6968 consensus_queue.cc:260] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:25.620167  1573 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2453) took 1667 ms (client timeout 2197 ms). Trace:
W20260709 13:04:25.620841  1573 rpcz_store.cc:269] 0709 13:04:23.952735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.952947 (+   212us) service_pool.cc:224] Handling call
0709 13:04:25.619859 (+1666912us) spinlock_profiling.cc:243] Waited 756 ms on lock 0x7b54004074d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:25.620058 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1739939584}
W20260709 13:04:25.622838  6905 consensus_meta.cc:287] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.844s	user 0.017s	sys 0.010s
W20260709 13:04:25.623466  7191 log.cc:927] Time spent T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.141s	user 0.000s	sys 0.007s
I20260709 13:04:25.623943  7152 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:25.624828  6968 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:25.613904  1611 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:25.634305  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 53064.3us
I20260709 13:04:25.634918  7068 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:25.636494  1611 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:25.640147  7070 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:04:25.658813  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:25.660321  1722 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:25.661321  7197 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:25.604144  6976 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.602574  7075 consensus_queue.cc:1059] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.133s
I20260709 13:04:25.673571  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:25.640476  6820 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:25.674388  1722 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:25.633236  7074 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.631752  6995 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 13:04:25.677976  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:25.678537  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:25.679442  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:25.667059  6943 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:25.684036  7159 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:25.684114  2346 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:04:25.681178  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:25.695309  1537 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:25.697096  7189 log.cc:927] Time spent T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.160s	user 0.003s	sys 0.000s
I20260709 13:04:25.697553  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:25.702099  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:25.702564  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:25.706378  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:25.710419  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:25.711639  1719 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16.
I20260709 13:04:25.724255  7194 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.137s
I20260709 13:04:25.727027  1716 raft_consensus.cc:2410] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 16 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:04:25.727445  6974 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:25.728005  1713 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:25.728232  6974 raft_consensus.cc:697] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:25.729557  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:25.730595  2348 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:04:25.736847  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:25.733069  6798 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:04:25.743593  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:25.744563  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:25.746652  1611 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:25.750447  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 59030.1us
I20260709 13:04:25.756306  2344 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:04:25.674073  6976 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.760850  2313 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:25.688416  7132 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 13:04:25.774752  2313 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:25.689841  7163 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.719394  7074 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:25.805521  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 72609.9us
I20260709 13:04:25.770778  6798 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:25.809340  7094 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:25.812038  1713 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:25.811733  7094 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:25.732420  1466 catalog_manager.cc:5697] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 11 to 12, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 12 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:25.815550  7182 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:25.819886  6936 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.822919  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:25.823693  1714 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:04:25.824841  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:25.826439  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:25.826892  6636 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.831136  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:25.832073  7163 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:25.826589  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:25.800647  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 73850.9us
I20260709 13:04:25.796125  7115 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:25.796300  6955 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:25.845788  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:25.847893  7108 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.746486  6974 consensus_queue.cc:237] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.848769  7108 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:25.852830  7103 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:25.809803  6798 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:25.865540  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:25.875826  7198 consensus_queue.cc:1059] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:04:25.876590  7103 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.877411  7152 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.879410  7103 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:25.878865  7152 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.877945  7163 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.901985  7163 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.881546  6636 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:25.883476  7184 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.908222  7184 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:25.867043  1721 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:25.844058  6616 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.888576  7068 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.885834  6962 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.901098  7203 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.909370  6616 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.909471  7068 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.821476  7185 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:25.910606  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 483) took 1150 ms. Trace:
I20260709 13:04:25.911072  2346 rpcz_store.cc:276] 0709 13:04:24.760564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.760731 (+   167us) service_pool.cc:224] Handling call
0709 13:04:25.910342 (+1149611us) spinlock_profiling.cc:243] Waited 904 ms on lock 0x7b54005424d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:25.910490 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2079992576}
I20260709 13:04:25.912326  7068 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:25.841117  7182 raft_consensus.cc:697] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:25.916122  7075 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:25.916632  7152 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:25.909667  6962 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.918782  6976 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:25.919565  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49061us
I20260709 13:04:25.920382  6936 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:25.922093  7074 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:25.922806  7198 consensus_queue.cc:1243] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:04:25.902380  1722 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:04:25.871582  6798 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:25.910154  7203 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.927973  7163 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:04:25.932281  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:25.955195  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.935847  2348 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:04:25.937785  7204 consensus_queue.cc:1048] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:25.913740  7182 consensus_queue.cc:237] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:25.941426  6820 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:25.978252  6820 consensus_queue.cc:1243] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer cf2f71592be8465fb18e36e15d0b2f89 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:25.941695  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:25.924345  6616 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:25.972651  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.945158  7185 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:25.994199  1611 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:25.996738  1721 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:04:25.984973  7198 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.001019  1720 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:04:26.000173  1718 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:04:26.008991  2313 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:26.009826  6636 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.995729  7202 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:25.988426  7094 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:25.985018  6936 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:04:25.996059  7161 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.009060  7067 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:26.036486  1611 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:26.031046  7145 consensus_meta.cc:287] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.702s	user 0.010s	sys 0.006s
I20260709 13:04:26.038936  7161 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:26.040992  7067 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:26.018855  7198 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:26.042510  1643 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:04:26.059830  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 118309us
I20260709 13:04:26.046037  1646 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:26.046809  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.049301  1640 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:04:26.048910  6798 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:26.052423  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.053948  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.054651  6636 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:26.073827  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:26.072217  1642 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 12: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:26.075281  6636 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:26.076435  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.076835  1647 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:04:26.080237  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 47183.3us
W20260709 13:04:26.056100  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.082263  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.083093  1642 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:26.084362  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.085448  7202 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:26.085870  7094 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:26.086393  7202 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:26.087554  7094 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:26.090224  7202 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:26.085124  1642 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 12: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:26.091147  7161 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:26.092903  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.094748  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:26.098088  1642 raft_consensus.cc:2393] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:26.100304  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.101107  1569 raft_consensus.cc:1275] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:04:26.101265  1537 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:26.106211  7203 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:26.056986  7075 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:26.063031  7145 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:26.106972  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:26.107262  7075 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:26.108438  1570 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:04:26.111408  7198 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:26.111824  7115 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:26.113225  7108 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:26.113502  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.114516  2314 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:26.114949  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:26.115091  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:26.115782  1642 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:26.117102  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:26.117835  7109 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:26.117362  1566 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:26.115630  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.117808  1642 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:26.075326  1646 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 13: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:26.113281  1612 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:26.101545  1569 pending_rounds.cc:85] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 10
I20260709 13:04:26.018096  7132 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:26.042534  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.013768  7185 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:26.070262  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.138901  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:26.139722  1612 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:26.071352  1645 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:04:26.071538  6962 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:26.072106  6798 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:26.066350  1648 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:04:26.029989  7074 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 13:04:26.136662  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:26.146366  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:26.151098  2348 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:26.134936  1644 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:04:26.129249  1569 pending_rounds.cc:107] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:04:26.070410  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.153949  1569 raft_consensus.cc:2889] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:26.137837  1641 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:26.144587  1612 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:26.145718  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.146122  7210 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:26.131245  6995 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:26.148602  1648 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:26.148747  7075 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:26.147047  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:26.155988  1645 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:26.154151  1643 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:26.140404  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:26.137998  7132 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.135188  6820 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:26.150876  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.184063  1640 raft_consensus.cc:2393] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 16: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:26.186436  1646 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:26.189110  1641 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:26.172435  6974 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.159657  7145 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:26.169241  7210 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:26.200022  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 55595us
I20260709 13:04:26.201746  1644 raft_consensus.cc:1275] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:26.201704  1612 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:26.203742  2346 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:26.185894  7132 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:26.207528  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:26.208258  1715 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:26.187152  6987 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:26.213598  6987 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.220319  1642 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
W20260709 13:04:26.221179  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 224215us
I20260709 13:04:26.214126  6987 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:26.213785  6798 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:04:26.228173  1642 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:04:26.230640  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.233305  1612 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:26.201309  7210 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:26.240648  1647 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:26.214702  6962 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:26.243484  1715 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17.
I20260709 13:04:26.246101  1643 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:04:26.255023  1642 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:26.254587  1686 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:04:26.255940  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 61428.3us
I20260709 13:04:26.256929  1642 raft_consensus.cc:1275] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:26.257869  7161 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:26.258553  7161 raft_consensus.cc:697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:26.259586  7161 consensus_queue.cc:237] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:26.206154  7185 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:04:26.281843  7108 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:26.282198  7213 log.cc:927] Time spent T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:04:26.282743  7108 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:26.288697  7108 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:26.232689  6974 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:26.292349  1470 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38662 (request call id 210) took 39720 ms (client timeout 15000 ms). Trace:
W20260709 13:04:26.296515  7208 log.cc:927] Time spent T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.254s	user 0.002s	sys 0.000s
I20260709 13:04:26.297993  1466 catalog_manager.cc:5697] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 9 to 11, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 11 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:26.301359  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:26.300454  1466 catalog_manager.cc:5697] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 9 to 14, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 14 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:26.303184  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:26.299793  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:26.243613  7067 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:26.252667  6962 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:26.319154  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 76278.7us
I20260709 13:04:26.320187  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:26.321558  1612 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:26.326303  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:26.334653  1647 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:26.341888  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:26.343811  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:26.292533  6974 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:26.322273  6962 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:26.316162  7067 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.326566  7185 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:26.352849  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:26.358238  1466 catalog_manager.cc:5697] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 5 to 11, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 11 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:04:26.341820  7108 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:26.368167  1713 raft_consensus.cc:2393] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 15: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:26.365221  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:26.366868  7212 log.cc:927] Time spent T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.119s	user 0.004s	sys 0.001s
I20260709 13:04:26.393841  1566 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:26.394801  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.396433  1647 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:26.351082  6908 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:26.402904  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 59236.1us
I20260709 13:04:26.404461  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:26.409083  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.410180  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:26.411442  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.411756  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:26.412568  6908 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:26.413372  1466 catalog_manager.cc:5697] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 7 to 10, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 10 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:26.415472  1466 catalog_manager.cc:5697] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 7 to 9, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:26.420306  7161 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.429286  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:26.442395  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:26.448165  2313 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:26.448916  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.364646  7067 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:26.452812  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:26.421078  7161 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:26.455207  1647 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:26.463184  1612 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:26.434952  7185 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:26.468384  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:26.435527  7182 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:26.469228  1718 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:26.450584  7108 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:26.464083  7094 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:26.485847  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.487260  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:26.468493  7109 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:26.493969  1645 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:26.468187  7133 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:26.482244  6616 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:26.417433  1466 catalog_manager.cc:5697] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 7 to 10, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 10 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:26.482944  7108 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:26.518851  1722 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:26.491904  6908 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:26.523926  1538 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:26.542222  1640 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:26.541473  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 93463.3us
I20260709 13:04:26.546828  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.555523  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:26.556656  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:26.557724  1722 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:26.558635  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:26.560554  1718 raft_consensus.cc:1275] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:26.560705  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:26.519047  6616 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:26.561940  6616 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:26.562433  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:26.306221  1470 rpcz_store.cc:269] 0709 13:03:46.571592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.580355 (+  8763us) service_pool.cc:224] Handling call
0709 13:04:26.291873 (+39711518us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:17.447810 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:17.448589 (+   779us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:17.448671 (+    82us) write_op.cc:435] Acquired schema lock
0709 13:04:17.448695 (+    24us) tablet.cc:662] Decoding operations
0709 13:04:17.551641 (+102946us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:17.569850 (+ 18209us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:17.569893 (+    43us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:04:17.571854 (+  1961us) tablet.cc:701] Row locks acquired
0709 13:04:17.571918 (+    64us) write_op.cc:260] PREPARE: finished
0709 13:04:17.572161 (+   243us) write_op.cc:270] Start()
0709 13:04:17.572363 (+   202us) write_op.cc:276] Timestamp: P: 1783602257572067 usec, L: 0
0709 13:04:17.572504 (+   141us) op_driver.cc:348] REPLICATION: starting
0709 13:04:17.573615 (+  1111us) log.cc:844] Serialized 7118 byte log entry
0709 13:04:18.626073 (+1052458us) op_driver.cc:464] REPLICATION: finished
0709 13:04:19.393994 (+767921us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8245d7b0ac 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f824c8bbef0
0709 13:04:19.725791 (+331797us) write_op.cc:301] APPLY: starting
0709 13:04:19.775847 (+ 50056us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:20.371813 (+595966us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:20.371892 (+    79us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:21.818827 (+1446935us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:21.852580 (+ 33753us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 13:04:21.902999 (+ 50419us) write_op.cc:312] APPLY: finished
0709 13:04:22.140320 (+237321us) log.cc:844] Serialized 164 byte log entry
0709 13:04:22.285858 (+145538us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:22.397714 (+111856us) write_op.cc:454] Released schema lock
0709 13:04:22.431414 (+ 33700us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:22.461430 (+ 30016us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:22.461473 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":286422,"reported_tablets":54,"spinlock_wait_cycles":48384,"child_traces":[["op",{"apply.queue_time_us":1041305,"apply.run_cpu_time_us":77816,"apply.run_wall_time_us":2763040,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79403,"num_ops":18,"prepare.queue_time_us":305657,"prepare.run_cpu_time_us":192253,"prepare.run_wall_time_us":456934,"raft.queue_time_us":239934,"raft.run_cpu_time_us":18909,"raft.run_wall_time_us":907311,"replication_time_us":1035231,"spinlock_wait_cycles":5714688,"thread_start_us":1375586,"threads_started":5,"wal-append.queue_time_us":709819,"wal-append.run_cpu_time_us":26360,"wal-append.run_wall_time_us":2585543}]]}
I20260709 13:04:26.510290  6974 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:26.569437  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 45613.7us
I20260709 13:04:26.525099  7108 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:26.545262  7133 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:26.591378  7133 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:26.580111  6908 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:04:26.633654  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 50991.4us
I20260709 13:04:26.632730  1647 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:26.638899  1640 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:26.656952  1538 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:26.657445  1640 raft_consensus.cc:1240] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 13:04:26.638557  7221 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.140s	user 0.002s	sys 0.000s
I20260709 13:04:26.673009  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.679222  1645 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:26.634536  6908 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.688956  2314 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:26.692427  7109 raft_consensus.cc:697] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 17 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:26.701058  1721 raft_consensus.cc:1275] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:04:26.701380  7109 consensus_queue.cc:237] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:26.702451  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:26.703153  1721 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:26.706085  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:26.707698  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:26.708359  1714 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:26.709225  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:26.710047  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.715560  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.717401  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:26.717439  1719 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:26.612924  6974 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:26.716312  1645 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:26.717263  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:26.732197  7075 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:04:26.732542  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 77847.1us
I20260709 13:04:26.734440  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:26.736186  7218 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.237s	user 0.000s	sys 0.003s
I20260709 13:04:26.736083  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.736965  1642 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:26.738080  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.739382  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.739976  7161 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:26.740177  1642 raft_consensus.cc:2410] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:04:26.746235  1466 catalog_manager.cc:5697] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 6 to 11, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 11 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:26.749677  7115 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:26.750362  7115 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:26.750810  7115 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:26.759111  7090 consensus_queue.cc:1048] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:04:26.764089  7182 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:26.710172  6908 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:26.711201  7184 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:04:26.774621  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.774971  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:26.747882  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:26.778208  1646 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.004s
I20260709 13:04:26.778729  1646 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:26.783460  2314 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:26.769371  6987 consensus_meta.cc:287] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.532s	user 0.011s	sys 0.000s
I20260709 13:04:26.795070  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.787459  1645 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:26.802721  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.805325  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:26.811152  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:26.820840  1720 raft_consensus.cc:2393] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 11: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:26.811895  1640 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:04:26.833164  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.832742  1538 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:26.834067  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:26.837567  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:26.839510  1717 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:26.840350  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:26.838415  1714 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:26.845628  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:26.831717  6995 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.655s	user 0.012s	sys 0.003s
I20260709 13:04:26.852864  1720 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 13: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:26.857426  1640 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:04:26.841394  1685 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.610s (SENT) [suppressed 19 similar messages]
W20260709 13:04:26.863253  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:26.864190  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.807812  6987 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:26.874728  2344 pending_rounds.cc:85] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Aborting all ops after (but not including) 7
I20260709 13:04:26.875228  2344 pending_rounds.cc:107] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:04:26.875614  2344 raft_consensus.cc:2889] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:26.878038  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:26.879132  1720 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:26.881422  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:26.832242  7203 consensus_queue.cc:1059] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:04:26.881151  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:26.892624  1640 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:26.857771  6995 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:26.894802  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:26.896309  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:26.896662  1720 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:26.899991  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:26.862212  7075 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:26.909518  1640 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 13: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:26.909660  1648 raft_consensus.cc:1240] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:26.875365  7152 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.912925  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:26.874405  7094 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:26.928695  1640 raft_consensus.cc:2393] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 11: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:26.852233  6962 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:26.936167  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:26.937134  1569 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:26.890257  7235 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:26.957152  7205 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.755s	user 0.005s	sys 0.000s
I20260709 13:04:26.958806  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.895921  7145 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 13:04:26.946079  7203 raft_consensus.cc:3055] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:26.943575  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:26.967551  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:26.910292  7075 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.777652  1466 catalog_manager.cc:5697] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 4 to 9, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:27.008628  1721 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
W20260709 13:04:27.024508  1641 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.006s	sys 0.003s
I20260709 13:04:27.033456  1639 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:04:26.961114  7217 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.684s	user 0.004s	sys 0.000s
I20260709 13:04:26.939258  7094 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:26.978144  7132 consensus_meta.cc:287] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.753s	user 0.007s	sys 0.007s
W20260709 13:04:26.967096  7227 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:04:27.040302  1641 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.006s	sys 0.003s
I20260709 13:04:27.045104  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:27.047719  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:27.046360  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:26.988566  7235 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:27.051947  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:26.973337  7203 raft_consensus.cc:740] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:27.032832  6636 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.055596  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:27.023468  7075 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:27.059425  1640 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
W20260709 13:04:27.060792  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 277438us
I20260709 13:04:27.075031  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:27.080359  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 264749us
I20260709 13:04:26.976518  7152 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:27.108342  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.116827  1640 raft_consensus.cc:1275] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:27.116302  1538 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:27.063231  7235 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:27.086869  7145 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.116827  1641 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:27.053562  7132 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:27.140533  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:27.137364  6974 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:27.172338  2346 raft_consensus.cc:2368] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16.
I20260709 13:04:27.176008  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:27.183178  1640 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:27.177124  1569 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:27.076956  6636 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:27.199951  1641 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:27.221697  6636 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:27.226963  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2465) took 1011 ms. Trace:
W20260709 13:04:27.227681  6616 consensus_meta.cc:287] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.665s	user 0.012s	sys 0.002s
I20260709 13:04:27.228534  6616 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:27.229810  7245 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.195451  6974 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.233594  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:27.178819  1466 catalog_manager.cc:5697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:27.235039  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.235692  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.236644  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.261580  1566 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:27.236786  6908 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:27.238756  1570 raft_consensus.cc:1240] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:04:27.240917  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.231974  7152 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:27.227766  1643 rpcz_store.cc:276] 0709 13:04:26.215407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.586785 (+371378us) service_pool.cc:224] Handling call
0709 13:04:26.630186 (+ 43401us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:26.630725 (+   539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.630755 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:04:26.637869 (+  7114us) log.cc:844] Serialized 78 byte log entry
0709 13:04:27.050487 (+412618us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:27.050541 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.050637 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:27.226621 (+175984us) raft_consensus.cc:1685] finished
0709 13:04:27.226645 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.226889 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":887,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":7817,"thread_start_us":322224,"threads_started":1,"wal-append.queue_time_us":483257}
I20260709 13:04:27.259912  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:27.274653  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:27.187774  7145 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:27.273898  1571 raft_consensus.cc:2376] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14. Re-sending same reply.
I20260709 13:04:27.244800  6987 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:27.259083  7067 consensus_meta.cc:287] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.792s	user 0.014s	sys 0.002s
W20260709 13:04:27.266793  7108 consensus_meta.cc:287] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.635s	user 0.010s	sys 0.004s
I20260709 13:04:27.289243  1718 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:27.304841  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 188639us
I20260709 13:04:27.306856  1573 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:27.259704  6962 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:27.308399  1685 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.593s (SENT)
I20260709 13:04:27.324023  1685 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:27.308002  1717 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:04:27.303932  7244 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.173s	user 0.000s	sys 0.002s
I20260709 13:04:27.304196  7245 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:27.319542  7233 log.cc:927] Time spent T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.471s	user 0.006s	sys 0.002s
I20260709 13:04:27.259529  7133 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:27.210884  7203 consensus_queue.cc:260] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:27.267108  6974 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:27.377461  7132 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:27.361987  7133 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:27.390025  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2952) took 1333 ms. Trace:
I20260709 13:04:27.361238  1538 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:27.392256  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:27.394258  7241 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
W20260709 13:04:27.395121  7247 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:04:27.360780  7161 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:27.349498  6616 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:27.307885  7067 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:27.363595  1564 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 16: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:27.320940  7108 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:27.408071  1640 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:27.403249  1648 rpcz_store.cc:276] 0709 13:04:26.056859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.188088 (+131229us) service_pool.cc:224] Handling call
0709 13:04:26.200840 (+ 12752us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.927817 (+726977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.363727 (+435910us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b5400183154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:27.389936 (+ 26209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1612133248}
I20260709 13:04:27.418286  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.477370  1718 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:27.421777  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:27.465845  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 107774us
I20260709 13:04:27.486794  1571 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:04:27.407524  7244 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
W20260709 13:04:27.490136  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 183149us
I20260709 13:04:27.494982  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:27.418716  7115 consensus_meta.cc:287] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.667s	user 0.018s	sys 0.000s
I20260709 13:04:27.503715  7268 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:27.503484  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.504153  7203 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:27.504261  1641 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:27.504308  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:27.504632  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:27.506074  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:27.509125  1643 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
W20260709 13:04:27.505980  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 82429.9us
I20260709 13:04:27.419670  7259 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.509501  1572 raft_consensus.cc:2376] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11. Re-sending same reply.
I20260709 13:04:27.509155  1639 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:27.495812  2340 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:27.523430  2343 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:27.523869  6636 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:27.494482  6962 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:04:27.524466  6636 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:27.523154  7259 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:27.526247  6908 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:27.529987  1717 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:04:27.484149  6936 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:27.525487  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.533509  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:27.542739  1648 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:27.543982  1647 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:27.537581  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.541180  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.546463  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.548038  1646 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 15: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:27.548089  1571 raft_consensus.cc:2393] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 13: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:27.550096  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:27.547747  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 62445.5us
I20260709 13:04:27.558001  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.550957  1568 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:27.403363  7235 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:27.559687  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.516582  7268 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:27.611769  1612 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.900s	user 0.010s	sys 0.005s
W20260709 13:04:27.612442  1612 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.010s	sys 0.005s
W20260709 13:04:27.612820  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 901314us
I20260709 13:04:27.614722  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:27.615948  1568 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:27.617590  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55890 (request call id 535) took 1316 ms. Trace:
I20260709 13:04:27.618166  2346 rpcz_store.cc:276] 0709 13:04:26.300801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.301021 (+   220us) service_pool.cc:224] Handling call
0709 13:04:27.610204 (+1309183us) spinlock_profiling.cc:243] Waited 869 ms on lock 0x7b54005b1154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:27.617431 (+  7227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1998959360}
I20260709 13:04:27.619764  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:27.622834  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 79112.2us
I20260709 13:04:27.626828  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.628736  1612 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:27.628911  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.626631  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.633877  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:27.634904  1569 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 11: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:27.519954  7115 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:27.649791  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.653590  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:27.659178  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 85882.9us
I20260709 13:04:27.575496  7268 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:27.660459  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 882) took 1406 ms. Trace:
I20260709 13:04:27.663435  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:27.557595  6908 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.093s
I20260709 13:04:27.670881  1566 raft_consensus.cc:1275] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:04:27.668471  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 135495us
I20260709 13:04:27.570531  6936 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:27.679505  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:27.686843  1573 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:27.665900  1567 rpcz_store.cc:276] 0709 13:04:26.242547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.242734 (+   187us) service_pool.cc:224] Handling call
0709 13:04:26.254643 (+ 11909us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.325418 (+ 70775us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:26.445872 (+120454us) log.cc:844] Serialized 14 byte log entry
0709 13:04:26.670032 (+224160us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:26.670699 (+   667us) log.cc:844] Serialized 26 byte log entry
0709 13:04:26.671139 (+   440us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:26.671188 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.671291 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:27.318884 (+647593us) spinlock_profiling.cc:243] Waited 3.46 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:27.619148 (+300264us) raft_consensus.cc:1685] finished
0709 13:04:27.649108 (+ 29960us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.649304 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42051,"prepare.queue_time_us":249744,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":5316,"raft.queue_time_us":84754,"raft.run_cpu_time_us":376,"raft.run_wall_time_us":33300,"spinlock_wait_cycles":7957888,"thread_start_us":266713,"threads_started":2,"wal-append.queue_time_us":376429}
I20260709 13:04:27.680450  1612 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:27.693179  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.695989  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:27.692369  1685 leader_election.cc:336] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.731s (SENT)
I20260709 13:04:27.695006  1571 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:27.776481  1569 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:04:27.778888  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.787971  1565 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:04:27.790918  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 98479.6us
I20260709 13:04:27.691373  7259 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:27.830796  1685 leader_election.cc:336] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.928s (SENT)
I20260709 13:04:27.831269  1685 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:27.837585  1572 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:27.558389  1466 catalog_manager.cc:5697] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:04:27.855836  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.842818  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.859561  2348 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:27.886761  1567 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:27.862504  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.890782  1565 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:27.867888  7245 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:27.839824  7259 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:27.896754  7259 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:27.899865  7259 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:27.870960  6987 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:27.900949  6987 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:27.903849  6987 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:27.833352  7145 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:27.877493  6616 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:27.905284  6616 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:27.905889  6616 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:27.878952  7115 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:27.781956  6974 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:27.888864  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.893241  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:27.922014  1565 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:27.923290  7075 consensus_meta.cc:287] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.826s	user 0.015s	sys 0.000s
I20260709 13:04:27.982400  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:27.984386  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.989687  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.990985  1569 raft_consensus.cc:2410] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:27.991962  1567 raft_consensus.cc:2410] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:04:27.998965  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:27.980909  7108 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:28.007685  7263 log.cc:927] Time spent T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.250s	user 0.000s	sys 0.002s
I20260709 13:04:27.988065  7067 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:28.008109  7236 log.cc:927] Time spent T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 1.018s	user 0.004s	sys 0.008s
I20260709 13:04:27.988965  7133 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:27.988878  7252 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:04:28.008097  7248 log.cc:927] Time spent T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.463s	user 0.002s	sys 0.000s
I20260709 13:04:28.026104  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:28.018829  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:28.023952  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 363913us
W20260709 13:04:28.020098  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 189324us
W20260709 13:04:28.054231  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.024721  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 552) took 1635 ms. Trace:
I20260709 13:04:28.055732  1715 rpcz_store.cc:276] 0709 13:04:26.387688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.406083 (+ 18395us) service_pool.cc:224] Handling call
0709 13:04:26.443907 (+ 37824us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.462857 (+ 18950us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:04:26.836153 (+373296us) log.cc:844] Serialized 14 byte log entry
0709 13:04:27.336485 (+500332us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.759519 (+423034us) spinlock_profiling.cc:243] Waited 3.49 ms on lock 0x7b2c00013bb8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824777ea11 00007f824778e372 00007f8248ec0a01 00007f8248e8543a 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2
0709 13:04:27.837758 (+ 78239us) log.cc:844] Serialized 26 byte log entry
0709 13:04:27.837956 (+   198us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:27.838001 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.838084 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.003121 (+165037us) raft_consensus.cc:1685] finished
0709 13:04:28.016190 (+ 13069us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.016362 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14888,"prepare.queue_time_us":518916,"prepare.run_cpu_time_us":177,"prepare.run_wall_time_us":4009,"raft.queue_time_us":94074,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":6849,"spinlock_wait_cycles":8022912,"thread_start_us":800886,"threads_started":3,"wal-append.queue_time_us":422856}
I20260709 13:04:28.059156  7275 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.059942  7275 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:28.001118  7145 consensus_queue.cc:1059] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:04:28.050814  1569 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:27.991514  7075 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:28.011157  7283 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:04:28.067904  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 389347us
I20260709 13:04:28.008278  7281 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:28.072299  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:28.047961  7267 log.cc:927] Time spent T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.413s	user 0.002s	sys 0.000s
I20260709 13:04:28.008677  7246 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:28.034031  7235 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:28.008025  7277 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.393s
I20260709 13:04:28.087785  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:28.094699  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:28.090979  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 569) took 1409 ms. Trace:
I20260709 13:04:28.093552  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:28.007917  7242 consensus_queue.cc:1048] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.411s
I20260709 13:04:28.076215  1538 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:28.008267  7278 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.211s
I20260709 13:04:28.212010  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:28.220152  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:28.220363  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:28.221115  1717 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 15: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:28.223125  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:28.224674  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:28.132071  7269 log.cc:927] Time spent T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.329s	user 0.000s	sys 0.000s
W20260709 13:04:28.131470  7276 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
W20260709 13:04:28.117545  7271 log.cc:927] Time spent T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.342s	user 0.001s	sys 0.001s
I20260709 13:04:28.136358  6936 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:28.133482  2346 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 17 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:28.133188  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2479) took 1448 ms. Trace:
I20260709 13:04:28.127815  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2862) took 1251 ms. Trace:
I20260709 13:04:28.207831  1642 rpcz_store.cc:276] 0709 13:04:26.682029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.742006 (+ 59977us) service_pool.cc:224] Handling call
0709 13:04:26.742364 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.114997 (+372633us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.115046 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.115668 (+   622us) log.cc:844] Serialized 26 byte log entry
0709 13:04:27.403307 (+287639us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:27.403412 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.403561 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.090683 (+687122us) raft_consensus.cc:1685] finished
0709 13:04:28.090709 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.090898 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":120061,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":18712,"thread_start_us":577393,"threads_started":2,"wal-append.queue_time_us":408607}
W20260709 13:04:28.227203  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.229954  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:28.100955  1568 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
W20260709 13:04:28.152612  7210 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.905s	user 0.011s	sys 0.003s
I20260709 13:04:28.145421  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:28.156694  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:28.231709  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 144046us
I20260709 13:04:28.163228  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:28.194154  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 57964.1us
I20260709 13:04:28.168895  1572 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:28.179087  1565 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:28.211923  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:28.213459  7152 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.226104  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:28.245568  1569 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:28.245488  7152 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:28.246613  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:28.138211  1721 raft_consensus.cc:1275] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:28.247210  1569 raft_consensus.cc:2376] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14. Re-sending same reply.
I20260709 13:04:28.248891  6936 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:28.249296  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:28.228063  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2865) took 1314 ms. Trace:
I20260709 13:04:28.249889  1722 rpcz_store.cc:276] 0709 13:04:26.914137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.096282 (+182145us) service_pool.cc:224] Handling call
0709 13:04:27.127143 (+ 30861us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.346770 (+219627us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.346808 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.358056 (+ 11248us) log.cc:844] Serialized 26 byte log entry
0709 13:04:27.505746 (+147690us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:27.505786 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.507758 (+  1972us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.227859 (+720101us) raft_consensus.cc:1685] finished
0709 13:04:28.227882 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.228004 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9419,"raft.queue_time_us":86685,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"thread_start_us":345374,"threads_started":2,"wal-append.queue_time_us":310092}
I20260709 13:04:28.250993  1716 raft_consensus.cc:2393] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 14: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:28.250563  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:28.252023  1715 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:04:28.130630  7265 log.cc:927] Time spent T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:04:28.256197  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:28.233084  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:28.256925  1717 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
W20260709 13:04:28.237519  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 166915us
I20260709 13:04:28.238701  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:28.129760  7262 consensus_queue.cc:1059] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.553s
I20260709 13:04:28.239373  7278 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:04:28.240053  7094 consensus_meta.cc:287] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.183s	user 0.012s	sys 0.002s
I20260709 13:04:28.227407  1713 rpcz_store.cc:276] 0709 13:04:26.876358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.892670 (+ 16312us) service_pool.cc:224] Handling call
0709 13:04:26.926321 (+ 33651us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.954582 (+ 28261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.954639 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:26.955614 (+   975us) log.cc:844] Serialized 26 byte log entry
0709 13:04:27.244315 (+288701us) spinlock_profiling.cc:243] Waited 38.7 ms on lock 0x7b1800368920. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a250d1 00007f8248e40a0f
0709 13:04:27.560872 (+316557us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:27.560934 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.561031 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.127418 (+566387us) raft_consensus.cc:1685] finished
0709 13:04:28.127442 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.127734 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12902,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":147,"spinlock_wait_cycles":88989056,"thread_start_us":316337,"threads_started":1,"wal-append.queue_time_us":1078842}
I20260709 13:04:28.227274  1645 rpcz_store.cc:276] 0709 13:04:26.684640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.801629 (+116989us) service_pool.cc:224] Handling call
0709 13:04:26.807665 (+  6036us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.808021 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.808049 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:26.839003 (+ 30954us) log.cc:844] Serialized 26 byte log entry
0709 13:04:27.162976 (+323973us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:27.163042 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.163147 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.132892 (+969745us) raft_consensus.cc:1685] finished
0709 13:04:28.132922 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.133118 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":88460,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":39565,"thread_start_us":278962,"threads_started":1,"wal-append.queue_time_us":480198}
I20260709 13:04:28.241851  6974 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:28.242880  7210 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:28.244130  7286 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:28.248425  7152 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:28.249430  6936 raft_consensus.cc:697] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:04:28.122205  7254 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.273s	user 0.000s	sys 0.000s
I20260709 13:04:28.097023  7145 consensus_queue.cc:1243] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:28.241377  1537 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:28.253750  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 555556us
I20260709 13:04:28.299120  7287 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:28.250707  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:28.088292  7108 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:28.259771  1568 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:04:28.259902  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:28.310559  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:28.311357  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:28.088119  7283 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:28.204689  7246 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:28.262799  1567 raft_consensus.cc:2393] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 9: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:28.077253  7067 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.186s
I20260709 13:04:28.267277  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2870) took 1292 ms. Trace:
W20260709 13:04:28.266040  1612 connection.cc:256] RPC call timeout handler was delayed by -1.54291897600001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:04:28.295468  7286 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:28.403882  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 97639.4us
I20260709 13:04:28.261350  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:28.404825   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:28.311268  1570 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:28.271667  7242 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.264050  7245 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:04:28.314489  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55890 (request call id 541) took 1457 ms. Trace:
I20260709 13:04:28.322511  1569 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:28.294063  6987 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:28.267680  6908 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:04:28.333540  1567 raft_consensus.cc:1275] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:28.291206  7094 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:28.403851  1568 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16.
I20260709 13:04:28.303524  7275 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:28.295966  7259 consensus_queue.cc:1048] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:04:28.306491  7287 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:28.333814  7108 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:28.411331  1565 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:04:28.410470  7293 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:28.491216  1564 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2503) took 1256 ms (client timeout 1507 ms). Trace:
I20260709 13:04:28.271325  7278 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:28.545522  1564 rpcz_store.cc:269] 0709 13:04:27.234646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.234772 (+   126us) service_pool.cc:224] Handling call
0709 13:04:28.489628 (+1254856us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400190fd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:28.491078 (+  1450us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233615104}
I20260709 13:04:28.493292  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2487) took 1789 ms. Trace:
I20260709 13:04:28.495657  1569 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:28.313969  7262 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.175s
I20260709 13:04:28.142252  1466 catalog_manager.cc:5697] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:04:28.549877  2312 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:28.331269  7246 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:28.488102  2344 rpcz_store.cc:276] 0709 13:04:26.854547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.854681 (+   134us) service_pool.cc:224] Handling call
0709 13:04:26.874463 (+ 19782us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:26.912595 (+ 38132us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.912631 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:26.951011 (+ 38380us) log.cc:844] Serialized 52 byte log entry
0709 13:04:27.240055 (+289044us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b180007bac0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a250d1 00007f8248e40a0f
0709 13:04:27.386040 (+145985us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:27.386099 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.386199 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.307758 (+921559us) raft_consensus.cc:1685] finished
0709 13:04:28.307775 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.311466 (+  3691us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15650,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":6834,"spinlock_wait_cycles":47235968,"thread_start_us":123190,"threads_started":1,"wal-append.queue_time_us":727782}
I20260709 13:04:28.538888  1571 raft_consensus.cc:2410] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 13:04:28.499400  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 161124us
I20260709 13:04:28.546756  1719 rpcz_store.cc:276] 0709 13:04:26.704008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.845138 (+141130us) service_pool.cc:224] Handling call
0709 13:04:26.861884 (+ 16746us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.886532 (+ 24648us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.886574 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:26.900533 (+ 13959us) log.cc:844] Serialized 26 byte log entry
0709 13:04:27.271506 (+370973us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:27.271572 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.271677 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.492873 (+1221196us) raft_consensus.cc:1685] finished
0709 13:04:28.492895 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.493220 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":734,"raft.queue_time_us":114,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":13816,"thread_start_us":283998,"threads_started":1,"wal-append.queue_time_us":744035}
W20260709 13:04:28.411839  6616 consensus_meta.cc:287] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.505s	user 0.014s	sys 0.000s
I20260709 13:04:28.337314  1714 rpcz_store.cc:276] 0709 13:04:26.975301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.390192 (+414891us) service_pool.cc:224] Handling call
0709 13:04:27.390598 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.391563 (+   965us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:27.391598 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.392714 (+  1116us) log.cc:844] Serialized 26 byte log entry
0709 13:04:27.511354 (+118640us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:27.511416 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.511511 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.266902 (+755391us) raft_consensus.cc:1685] finished
0709 13:04:28.266929 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.267095 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2423,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"thread_start_us":110001,"threads_started":1,"wal-append.queue_time_us":265236}
W20260709 13:04:28.554215  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 149393us
I20260709 13:04:28.623142  1568 raft_consensus.cc:3055] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:04:28.630033  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 388752us
I20260709 13:04:28.536051  6987 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:28.551910  7268 consensus_meta.cc:287] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.864s	user 0.016s	sys 0.000s
I20260709 13:04:28.499100  7277 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.586030  7235 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:28.492239  7280 consensus_queue.cc:1059] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.142s
I20260709 13:04:28.585932  6974 raft_consensus.cc:697] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:28.648752  1568 raft_consensus.cc:740] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:28.534343  6936 consensus_queue.cc:237] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:28.662124  1687 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:28.665711  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:28.650738  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 336401us
I20260709 13:04:28.585415  7108 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:28.661303  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:28.554378  7293 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:28.652187  7075 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:28.673215  1714 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 21.
I20260709 13:04:28.685819  7258 consensus_queue.cc:1059] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
W20260709 13:04:28.687678  7291 log.cc:927] Time spent T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:04:28.695412  7252 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:28.696174  7252 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:28.699044  6987 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:28.700500  7094 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:28.701498  7290 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:28.701709  1466 catalog_manager.cc:5697] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 9 to 11, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 11 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:28.703145  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:28.704478  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:28.705127  1720 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:28.705068  1568 consensus_queue.cc:260] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:28.706208  7288 log.cc:927] Time spent T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.392s	user 0.000s	sys 0.001s
I20260709 13:04:28.699048  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:28.707026  2348 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55890 (request call id 539) took 2149 ms (client timeout 1921 ms). Trace:
W20260709 13:04:28.707430  2348 rpcz_store.cc:269] 0709 13:04:26.557991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.558172 (+   181us) service_pool.cc:224] Handling call
0709 13:04:28.706770 (+2148598us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400800054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:28.706893 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2989942656}
I20260709 13:04:28.707824  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:28.706267  1568 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:28.711328  1537 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:28.703684  1466 catalog_manager.cc:5697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:28.713368  1537 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:28.493602  7242 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:28.720981  7296 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:28.721262  7287 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:28.665575  7245 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:04:28.707770  7283 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:28.722628  7283 consensus_queue.cc:1059] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:28.723634  7283 consensus_queue.cc:1243] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer 066ee0109aba4465ba17fa65e14ef504 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:04:28.653800  6616 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:28.724296  7108 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:28.724977  6974 consensus_queue.cc:237] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:28.727347  7293 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:28.728987  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:28.729779  1714 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:28.729959  7245 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:28.730913  7280 raft_consensus.cc:3055] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:28.731282  7280 raft_consensus.cc:740] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:28.731362  1714 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
W20260709 13:04:28.681854  7203 consensus_meta.cc:287] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.162s	user 0.010s	sys 0.005s
I20260709 13:04:28.678136  7145 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:28.675724  7235 raft_consensus.cc:697] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:28.657953  7268 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:28.788842  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 55815.8us
W20260709 13:04:28.772748  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 88029.6us
I20260709 13:04:28.717746  7094 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:28.774724  1573 raft_consensus.cc:2410] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:04:28.780309  1720 raft_consensus.cc:2410] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:04:28.722514  1466 catalog_manager.cc:5697] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 12 to 14, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 14 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:28.819211  1568 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:28.730187  6636 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:28.818476  1538 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:28.732513  7294 consensus_queue.cc:1059] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:04:28.663718  7277 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:28.831552  1537 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:28.774534  7145 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.732316  7280 consensus_queue.cc:260] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:28.846971  1719 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:28.773984  7261 consensus_queue.cc:1059] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.106s
I20260709 13:04:28.839504  7145 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:28.820122  7203 consensus_queue.cc:1048] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:04:28.828891  7238 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
W20260709 13:04:28.866869  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 162818us
I20260709 13:04:28.822273  7094 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:28.875047  7286 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:28.873698  1685 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:28.895076  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 63576.9us
I20260709 13:04:28.896193  1685 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:28.889734  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.890812  7108 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.862351  7294 consensus_queue.cc:1243] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:04:28.845054  7275 consensus_queue.cc:1059] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:04:28.892719  7075 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:28.902638  1538 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:28.907282  1685 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:28.867622  7210 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:28.903752  1537 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:28.874163  7301 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:28.915328  1685 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:28.914497  6636 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:28.919617  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:28.920348  7268 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:28.913120  7299 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:28.876205  7235 consensus_queue.cc:237] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:28.921408  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:28.931656  1642 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:28.912405  7309 consensus_queue.cc:1059] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:04:28.922161  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:28.938778  1641 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:28.938675  1537 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:28.922897  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:28.923781  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:28.923908  7094 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:28.911387  7258 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:28.920740  1645 raft_consensus.cc:1275] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:28.924721  7245 raft_consensus.cc:2764] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:04:28.925618  7296 consensus_queue.cc:1059] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:28.926172  7293 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:28.936264  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 252343us
I20260709 13:04:28.939908  1640 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:28.946058  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:28.943058  7258 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:28.947885  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:28.949898  6974 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:28.965971  1643 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:28.965521  7258 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:28.961843  1645 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:04:28.944643  1537 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:28.953300  7308 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:28.953686  1466 catalog_manager.cc:5697] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:28.980382  1641 raft_consensus.cc:2410] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:28.928140  7278 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:28.956817  7274 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.090s	user 0.003s	sys 0.007s
I20260709 13:04:28.945806  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:28.957684  7252 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:28.927423  7301 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.999312  7301 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:28.999881  7280 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:29.000311  1643 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
W20260709 13:04:28.960482  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.002357  7261 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:29.003644  1538 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:28.986991  7290 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:29.007308  7290 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:28.976917  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:28.955273  6616 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:28.933255  7075 raft_consensus.cc:697] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:28.915141  7108 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:28.996438  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:28.975777  7293 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.016810  1648 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
I20260709 13:04:28.998725  7252 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:29.009727  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 65125.9us
I20260709 13:04:29.016053  1538 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:29.007756  7290 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:29.005554  7270 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.909s	user 0.000s	sys 0.002s
I20260709 13:04:29.001386  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:28.959439  7268 consensus_queue.cc:1059] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:04:28.990693  7238 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.032986  1538 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:29.015316  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:29.017596  7094 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:04:29.026966  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:29.028882  7242 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:29.030581  6636 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:29.047595  1648 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:29.047853  6636 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:29.048911  1714 raft_consensus.cc:3055] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:29.049328  1714 raft_consensus.cc:740] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:29.008371  6974 raft_consensus.cc:3055] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:29.031325  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2491) took 1054 ms. Trace:
I20260709 13:04:29.018656  7293 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:29.041205  1640 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:29.043396  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:29.016934  7296 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:29.023999  7299 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:29.063700  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:29.017894  7286 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:28.953344  7246 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:29.063071  1570 rpcz_store.cc:276] 0709 13:04:27.976868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.405719 (+428851us) service_pool.cc:224] Handling call
0709 13:04:28.406145 (+   426us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:28.632650 (+226505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:28.632704 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:28.661571 (+ 28867us) log.cc:844] Serialized 52 byte log entry
0709 13:04:28.942438 (+280867us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:28.942497 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.942592 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:29.031124 (+ 88532us) raft_consensus.cc:1685] finished
0709 13:04:29.031139 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.031269 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27415,"raft.queue_time_us":168723,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":4518,"spinlock_wait_cycles":132480,"thread_start_us":319915,"threads_started":2,"wal-append.queue_time_us":334752}
I20260709 13:04:29.059818  6616 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.038355  7309 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:29.072186  1538 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:29.020071  7278 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:04:29.019523  7258 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:29.062891  6974 raft_consensus.cc:740] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:29.083134  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 119821us
I20260709 13:04:29.085989  6908 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:04:29.082340  7295 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:04:29.092643  1466 catalog_manager.cc:5697] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 10 to 13, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 13 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:29.085373  1537 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:29.085318  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:29.068406  1714 consensus_queue.cc:260] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:29.077713  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:29.116005  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 570) took 2413 ms. Trace:
I20260709 13:04:29.222234  1643 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:29.116811  1687 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:04:29.223526  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 106837us
I20260709 13:04:29.224939  1687 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:29.082199  7262 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:29.134532  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:29.084060  7286 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:29.059437  7238 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:29.141664  7309 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:29.073952  7299 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.117818  6616 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:29.152917  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 80834.6us
W20260709 13:04:29.177814  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 54530.1us
I20260709 13:04:29.203148  7161 raft_consensus.cc:3055] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:29.179000  7251 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.119118  7242 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:29.136667  7305 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:29.181958  7296 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:29.219530  7268 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:29.185688  7258 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:04:29.073199  7235 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:29.190084  7108 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:29.128763  7243 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:29.111325  7075 consensus_queue.cc:237] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:29.251461  7161 raft_consensus.cc:740] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:29.197464  1641 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:04:29.200354  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2472) took 2815 ms. Trace:
I20260709 13:04:29.201437  7246 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:29.201762  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.202386  7152 raft_consensus.cc:3055] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:29.294018  7152 raft_consensus.cc:740] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:29.295586  7152 consensus_queue.cc:260] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:29.297096  7152 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:29.297623  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2497) took 1041 ms. Trace:
I20260709 13:04:29.298309  1719 rpcz_store.cc:276] 0709 13:04:28.255877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.657935 (+402058us) service_pool.cc:224] Handling call
0709 13:04:29.297340 (+639405us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b54003a39d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:29.297506 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":414177152}
I20260709 13:04:29.298924  6616 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:29.299428  2342 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:29.300220  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:29.202562  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:04:29.303082  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:29.304181  1685 leader_election.cc:336] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.507s (SENT)
W20260709 13:04:29.304891  1685 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.570s (SENT)
I20260709 13:04:29.203369  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:29.306434  1612 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:29.204017  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:29.204293  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:29.309379  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:29.204174  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:29.205958  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:29.311264  1566 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:04:29.312410  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:29.209139  1640 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 12.
I20260709 13:04:29.313899  6616 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:29.314735  6616 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.315110  6616 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:29.315294  1640 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:04:29.209277  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:29.318570  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:29.319591  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:29.181456  6636 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:29.211946  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:29.214035  1645 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:29.207976  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:29.214725  1642 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:04:29.220906  1611 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:29.322932  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 102135us
I20260709 13:04:29.323304  2346 raft_consensus.cc:2393] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 12: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:29.325081  1611 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:29.222477  1644 rpcz_store.cc:276] 0709 13:04:26.682376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.754373 (+ 71997us) service_pool.cc:224] Handling call
0709 13:04:26.754748 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.081223 (+326475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.081274 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.394695 (+313421us) spinlock_profiling.cc:243] Waited 6.18 ms on lock 0x7b2c000120e8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824777ea11 00007f824778e372 00007f8248ec0a01 00007f8248e8543a 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2
0709 13:04:27.395190 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 13:04:27.553211 (+158021us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:27.553575 (+   364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.553689 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.955707 (+1402018us) spinlock_profiling.cc:243] Waited 5.74 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:29.095370 (+139663us) raft_consensus.cc:1685] finished
0709 13:04:29.095406 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.095619 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115268,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":11130,"spinlock_wait_cycles":27415808,"thread_start_us":483446,"threads_started":2,"wal-append.queue_time_us":405885}
I20260709 13:04:29.326654  1644 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:04:29.327515  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2517) took 1682 ms. Trace:
I20260709 13:04:29.327829  1644 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:29.328022  1572 rpcz_store.cc:276] 0709 13:04:27.644798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.859309 (+214511us) service_pool.cc:224] Handling call
0709 13:04:27.979977 (+120668us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:28.239136 (+259159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.327323 (+1088187us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54002b8754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:29.327465 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":433139456}
W20260709 13:04:29.228384  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 143130us
I20260709 13:04:29.229614  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:29.330302  2343 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:29.330325  1537 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:29.230185  1646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2482) took 2401 ms (client timeout 1922 ms). Trace:
I20260709 13:04:29.232188  1648 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:29.235328  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:29.235351  1687 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:29.333403  1564 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:04:29.238812  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:29.334468  1716 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:29.250622  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 273859us
I20260709 13:04:29.255216  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:29.335554  7287 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:29.336114  2314 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:29.215516  7278 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.183902  6974 consensus_queue.cc:260] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:29.337800  7325 log.cc:927] Time spent T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:04:29.337553  7278 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:29.338095  6974 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:29.338698  7323 consensus_queue.cc:1048] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 13:04:29.339959  7278 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:29.233920  7262 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.341063  7321 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:29.229475  6908 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:04:29.245707  7316 log.cc:927] Time spent T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:04:29.243848  7286 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:29.270121  1720 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:29.341840  7185 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:29.341601  1611 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:29.346269  1537 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:29.315338  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:29.313089  1566 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
W20260709 13:04:29.331589  1646 rpcz_store.cc:269] 0709 13:04:26.811471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.814847 (+  3376us) service_pool.cc:224] Handling call
0709 13:04:29.212426 (+2397579us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b54001f4354. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:29.212608 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1636025728}
W20260709 13:04:29.362609  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 127389us
I20260709 13:04:29.363526  2314 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:29.366344  7323 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:29.365700  1687 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:29.369861  7323 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:29.370939  7323 raft_consensus.cc:697] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:29.372301  7323 consensus_queue.cc:237] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:29.195359  1466 catalog_manager.cc:5697] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 8 to 10, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 10 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:29.246415  7299 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:29.266049  7304 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:29.290251  1647 rpcz_store.cc:276] 0709 13:04:26.385257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.654885 (+269628us) service_pool.cc:224] Handling call
0709 13:04:26.655219 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.544508 (+889289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.200094 (+1655586us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b54002dc154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:29.200291 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2043582848}
I20260709 13:04:29.340890  7300 raft_consensus.cc:3055] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:29.274181  7296 raft_consensus.cc:697] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:29.240847  7290 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:29.272186  7235 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:29.341445  7262 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:29.381106  1611 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:29.268874  1643 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:29.341670  7321 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.369469  1687 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:29.394616  1570 raft_consensus.cc:2393] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:29.399152  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:29.442217  1648 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:29.413858  7300 raft_consensus.cc:740] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:29.404273  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:29.386610  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:29.410991  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:29.272234  7242 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:29.407359  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:29.452919  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 83562.4us
I20260709 13:04:29.258831  7251 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:29.284960  7268 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:29.424810  1713 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:29.427860  1647 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:29.426272  7235 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:29.392746  7278 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:04:29.392554  6936 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:04:29.434190  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 53212.5us
I20260709 13:04:29.388511  7203 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:29.391300  7115 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:29.414589  7304 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.443681  1648 raft_consensus.cc:1275] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:29.400599  2346 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:29.447386  1720 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:29.381968  7252 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.045s
W20260709 13:04:29.451341  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 64965.3us
I20260709 13:04:29.493165  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:29.450395  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 101702us
W20260709 13:04:29.495481  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 189981us
W20260709 13:04:29.496171  1685 leader_election.cc:336] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.610s (SENT)
I20260709 13:04:29.376024  7259 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:29.453886  1687 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:29.454478  1715 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:29.423096  7232 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:29.283182  7305 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:29.464695  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 86705.8us
I20260709 13:04:29.287406  7243 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:29.406991  6908 consensus_queue.cc:1059] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:04:29.430296  1646 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:29.465224  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2532) took 1427 ms. Trace:
I20260709 13:04:29.515100  1567 rpcz_store.cc:276] 0709 13:04:28.040136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.535922 (+495786us) service_pool.cc:224] Handling call
0709 13:04:28.536437 (+   515us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:28.771456 (+235019us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:28.771503 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:28.800534 (+ 29031us) log.cc:844] Serialized 26 byte log entry
0709 13:04:28.995502 (+194968us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:29.062524 (+ 67022us) log.cc:844] Serialized 14 byte log entry
0709 13:04:29.117850 (+ 55326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.117996 (+   146us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:29.464974 (+346978us) raft_consensus.cc:1685] finished
0709 13:04:29.465014 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.465164 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53275,"prepare.queue_time_us":49763,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":176426,"raft.run_cpu_time_us":381,"raft.run_wall_time_us":37082,"spinlock_wait_cycles":851840,"thread_start_us":323974,"threads_started":2,"wal-append.queue_time_us":240979}
I20260709 13:04:29.289877  1641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:55704
I20260709 13:04:29.434968  7108 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:29.443328  7185 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:29.478000  1611 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:29.518460  2343 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:29.519850  1611 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:29.437898  7321 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:29.471276  7339 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:29.491938  7304 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:29.515010  7301 consensus_meta.cc:287] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.515s	user 0.012s	sys 0.000s
I20260709 13:04:29.486928  7115 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:29.523530  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2498) took 1258 ms. Trace:
I20260709 13:04:29.510428  7287 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:04:29.497864  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 44074.5us
I20260709 13:04:29.527391  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:29.464172  7094 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:29.504825  1685 leader_election.cc:336] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.590s (SENT)
W20260709 13:04:29.568022  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.591368  2344 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:29.591544  1687 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:04:29.591408  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 86604.6us
I20260709 13:04:29.587384  1718 rpcz_store.cc:276] 0709 13:04:28.265044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.954501 (+689457us) service_pool.cc:224] Handling call
0709 13:04:28.954887 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:29.295357 (+340470us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b1800465100. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f8248e7d060 00007f8248e7dbfb 00007f8248e8864b
0709 13:04:29.295588 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.295628 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:29.296278 (+   650us) log.cc:844] Serialized 26 byte log entry
0709 13:04:29.499640 (+203362us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:29.499679 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.499748 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:29.513732 (+ 13984us) raft_consensus.cc:1685] finished
0709 13:04:29.513767 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.523451 (+  9684us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6653,"raft.queue_time_us":29140,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":20652,"spinlock_wait_cycles":6660992,"thread_start_us":232303,"threads_started":2,"wal-append.queue_time_us":215212}
I20260709 13:04:29.539366  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:29.515055  7258 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:29.524408  7232 raft_consensus.cc:697] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:29.503106  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:29.613740  2348 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 9.
I20260709 13:04:29.534811  7339 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.527305  7210 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:29.415231  7261 consensus_queue.cc:1059] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
W20260709 13:04:29.630666  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 126287us
I20260709 13:04:29.528494  6636 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:29.637194  1641 raft_consensus.cc:606] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:29.593683  7115 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:29.619609  7259 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:29.618886  7258 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:29.661693  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 141928us
I20260709 13:04:29.633276  7287 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:29.669795  1685 leader_election.cc:336] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.623s (SENT)
W20260709 13:04:29.659663  7327 log.cc:927] Time spent T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.142s	user 0.002s	sys 0.000s
W20260709 13:04:29.670848  1685 leader_election.cc:336] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.791s (SENT)
I20260709 13:04:29.639379  7339 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:29.673516  1611 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:29.648157  7309 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:29.679862  1685 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:29.678241  6636 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:29.592666  7301 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:29.631793  7281 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:04:29.663519  7259 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:29.666918  7258 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:29.672574  7145 consensus_meta.cc:287] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.803s	user 0.012s	sys 0.002s
I20260709 13:04:29.688309  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 889) took 1453 ms. Trace:
I20260709 13:04:29.686575  1538 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:29.644723  7278 consensus_queue.cc:1059] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:04:29.684507  7094 consensus_queue.cc:1048] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
W20260709 13:04:29.730633  7336 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:04:29.728451  1569 rpcz_store.cc:276] 0709 13:04:28.234624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.583928 (+349304us) service_pool.cc:224] Handling call
0709 13:04:28.584254 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:28.634889 (+ 50635us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:28.634923 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:28.634963 (+    40us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:28.635094 (+   131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.687936 (+1052842us) spinlock_profiling.cc:243] Waited 50.4 ms on lock 0x7b54001275d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:29.688017 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.688225 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116014848}
W20260709 13:04:29.760519  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 89705.2us
W20260709 13:04:29.798852  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:29.848980  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 163765us
I20260709 13:04:29.686137  6936 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:29.605623  7296 consensus_queue.cc:237] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:29.919665  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 246278us
I20260709 13:04:29.686770  6636 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:29.921546  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2980) took 1001 ms. Trace:
I20260709 13:04:30.019784  1642 rpcz_store.cc:276] 0709 13:04:28.919818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.199515 (+279697us) service_pool.cc:224] Handling call
0709 13:04:29.214443 (+ 14928us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:29.332974 (+118531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.921241 (+588267us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b54001f4350. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:29.921455 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33460096}
I20260709 13:04:29.718191  7287 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:29.936146  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 302048us
I20260709 13:04:29.727202  7259 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:29.934479  1538 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:30.015767  1611 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:30.031379  1687 connection.cc:256] RPC call timeout handler was delayed by -2.6051601030000029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1922s for a response
W20260709 13:04:30.039871  1716 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.692s	user 0.003s	sys 0.005s
I20260709 13:04:30.040293  1716 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:29.939680  7242 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:29.920269  7203 consensus_queue.cc:1059] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.112s
W20260709 13:04:30.026429  7280 consensus_meta.cc:287] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.026s	user 0.010s	sys 0.006s
I20260709 13:04:30.047659  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:30.051883  2344 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:30.021833  6636 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:30.057564  1611 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:29.917807  7145 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:30.017866  7251 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:30.060825  1718 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:30.073716  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2985) took 1144 ms. Trace:
I20260709 13:04:30.081378  1645 rpcz_store.cc:276] 0709 13:04:28.934168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.324578 (+390410us) service_pool.cc:224] Handling call
0709 13:04:29.324883 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:29.325326 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:29.325359 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:29.325906 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 13:04:29.484175 (+158269us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:29.484236 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.484344 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:30.073013 (+588669us) raft_consensus.cc:1685] finished
0709 13:04:30.073079 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.073539 (+   460us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11748,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":82,"thread_start_us":157788,"threads_started":1,"wal-append.queue_time_us":259531}
I20260709 13:04:30.063722  7249 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:04:30.066219  7305 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:30.067801  7350 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:04:30.069228  1717 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:30.070804  7301 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:30.070892  7243 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:30.022912  7268 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:30.322809  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:30.073542  1611 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:30.053825  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 119422us
I20260709 13:04:29.652369  1466 catalog_manager.cc:5697] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 6 to 9, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:30.288756  1722 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:30.330497  1538 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:30.331907  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 258457us
I20260709 13:04:30.337548  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:30.338213  1611 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:30.339033  1564 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.936s	user 0.009s	sys 0.004s
I20260709 13:04:30.057724  6936 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:30.348933  1564 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:30.351049  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2501) took 1143 ms. Trace:
I20260709 13:04:30.037246  7278 consensus_queue.cc:1059] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.282s
I20260709 13:04:30.351991  1564 rpcz_store.cc:276] 0709 13:04:29.207486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.207752 (+   266us) service_pool.cc:224] Handling call
0709 13:04:30.350941 (+1143189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:29.915637  7232 consensus_queue.cc:237] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:30.074534  7252 raft_consensus.cc:3055] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:30.318496  7203 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:30.289527  7067 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:30.349761  1611 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:30.388195  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:30.397615  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 48032.7us
I20260709 13:04:30.399714  1611 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:30.428455  2314 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.051s	user 0.040s	sys 0.064s
W20260709 13:04:30.429098  2314 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.052s	user 0.040s	sys 0.064s
W20260709 13:04:30.429507  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 1.05213e+06us
I20260709 13:04:30.353593  6936 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.043381  7287 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:30.447853  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:30.361702  6962 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:30.362178  7305 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:30.381490  7252 raft_consensus.cc:740] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:30.370411  7356 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:30.413488  7067 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.414075  7242 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:30.456297  6936 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:30.464870  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 98511us
I20260709 13:04:30.491250  1718 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16.
W20260709 13:04:30.506876  7335 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.820s	user 0.002s	sys 0.000s
W20260709 13:04:30.508572  7333 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.554s	user 0.000s	sys 0.003s
W20260709 13:04:30.513960  7330 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.674s	user 0.000s	sys 0.001s
I20260709 13:04:30.514827  7358 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:30.518555  7343 log.cc:927] Time spent T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.487s	user 0.000s	sys 0.000s
I20260709 13:04:30.460358  7278 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:30.520345  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 45626.7us
I20260709 13:04:30.527066  7356 raft_consensus.cc:697] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:30.547212  7356 consensus_queue.cc:237] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:30.471865  7067 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:30.554250  2315 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.043s	sys 0.049s
W20260709 13:04:30.561455  2315 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.182s	user 0.043s	sys 0.049s
I20260709 13:04:30.503149  7285 raft_consensus.cc:3055] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:30.521706  7350 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:30.533017  7278 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.574309  7145 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:30.570385  1466 catalog_manager.cc:5697] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 8 to 9, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:30.550161  7352 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.577947  7352 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:30.580907  7352 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:30.454310  7349 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.841s
I20260709 13:04:30.583088  7349 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:30.584812  7349 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:04:30.590807  7293 consensus_meta.cc:287] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.508s	user 0.012s	sys 0.000s
I20260709 13:04:30.591472  7293 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:30.598001  7161 consensus_queue.cc:260] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:30.599459  7161 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:30.474674  7301 consensus_queue.cc:1059] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
W20260709 13:04:30.610939  1570 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2503) took 1401 ms (client timeout 1686 ms). Trace:
W20260709 13:04:30.615602  1570 rpcz_store.cc:269] 0709 13:04:29.209040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.308854 (+ 99814us) service_pool.cc:224] Handling call
0709 13:04:30.610713 (+1301859us) spinlock_profiling.cc:243] Waited 73.8 ms on lock 0x7b540041c8d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:30.610876 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169710848}
W20260709 13:04:30.610985  2348 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 495) took 1441 ms (client timeout 1913 ms). Trace:
W20260709 13:04:30.616827  2348 rpcz_store.cc:269] 0709 13:04:29.169499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.202144 (+ 32645us) service_pool.cc:224] Handling call
0709 13:04:30.610850 (+1408706us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b54005424d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:30.610938 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":429014912}
W20260709 13:04:30.569777  7295 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.156s	user 0.003s	sys 0.017s
W20260709 13:04:30.610913  1686 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.222s	user 0.014s	sys 0.025s
W20260709 13:04:30.619556  1686 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.231s	user 0.014s	sys 0.025s
W20260709 13:04:30.597754  7338 log.cc:927] Time spent T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.733s	user 0.001s	sys 0.000s
W20260709 13:04:30.613420  1536 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.054s	sys 0.081s
I20260709 13:04:30.569059  7285 raft_consensus.cc:740] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:30.618638  7295 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.205s	user 0.003s	sys 0.018s
W20260709 13:04:30.626452  1536 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.055s	sys 0.081s
I20260709 13:04:30.603410  7278 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:30.630398  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 1.25156e+06us
I20260709 13:04:30.546298  6936 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:30.641510  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:30.633939  6908 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.007s	sys 0.029s
W20260709 13:04:30.633773  6987 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.216s	user 0.008s	sys 0.028s
W20260709 13:04:30.638024  7262 consensus_meta.cc:287] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.189s	user 0.008s	sys 0.005s
I20260709 13:04:30.613113  7349 consensus_queue.cc:1059] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:04:30.577070  1466 catalog_manager.cc:5697] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:04:30.645363  6616 consensus_meta.cc:287] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.327s	user 0.013s	sys 0.003s
W20260709 13:04:30.655547  6908 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.007s	sys 0.029s
I20260709 13:04:30.665416  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:30.669883  1716 raft_consensus.cc:2410] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:04:30.671756  1713 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:30.657482  6987 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.255s	user 0.009s	sys 0.029s
I20260709 13:04:30.645052  7293 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:30.602357  7252 consensus_queue.cc:260] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:30.679522  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:30.680231  1722 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:30.642582  7287 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:30.676960  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:30.683856  1716 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:30.644767  7145 consensus_queue.cc:1059] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
W20260709 13:04:30.685235  1573 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2519) took 2991 ms (client timeout 1778 ms). Trace:
W20260709 13:04:30.685693  1573 rpcz_store.cc:269] 0709 13:04:27.679793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.695726 (+ 15933us) service_pool.cc:224] Handling call
0709 13:04:30.670793 (+2975067us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54001b4254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:30.671154 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2426922880}
W20260709 13:04:30.665115  6974 consensus_meta.cc:287] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.326s	user 0.009s	sys 0.007s
I20260709 13:04:30.657918  7365 raft_consensus.cc:3055] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:30.689177  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2988) took 1747 ms. Trace:
I20260709 13:04:30.691381  1644 rpcz_store.cc:276] 0709 13:04:28.941944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.328972 (+387028us) service_pool.cc:224] Handling call
0709 13:04:29.329337 (+   365us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:29.342661 (+ 13324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.342696 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:29.343274 (+   578us) log.cc:844] Serialized 52 byte log entry
0709 13:04:29.429777 (+ 86503us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:29.429841 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.429928 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:30.688757 (+1258829us) raft_consensus.cc:1685] finished
0709 13:04:30.688909 (+   152us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.689114 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7504,"raft.run_cpu_time_us":47,"raft.run_wall_time_us":47,"thread_start_us":86057,"threads_started":1,"wal-append.queue_time_us":176964}
I20260709 13:04:30.696285  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:30.685002  7115 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 1.023s	user 0.010s	sys 0.003s
I20260709 13:04:30.699745  1713 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:30.664779  6616 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:30.710810  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 182101us
I20260709 13:04:30.662199  7262 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:30.715659  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 316038us
W20260709 13:04:30.700845  1537 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.239s	user 0.015s	sys 0.026s
W20260709 13:04:30.698722  7286 consensus_meta.cc:287] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.317s	user 0.015s	sys 0.000s
I20260709 13:04:30.691244  7365 raft_consensus.cc:740] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:30.720171  1537 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.258s	user 0.015s	sys 0.026s
I20260709 13:04:30.664351  7285 consensus_queue.cc:260] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:30.745924  7299 consensus_meta.cc:287] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.319s	user 0.008s	sys 0.006s
I20260709 13:04:30.746358  7285 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:30.748854  7258 consensus_meta.cc:287] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.828s	user 0.013s	sys 0.000s
I20260709 13:04:30.702256  6908 raft_consensus.cc:3055] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:04:30.717976  7348 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.878s	user 0.002s	sys 0.000s
I20260709 13:04:30.752465  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2984) took 1823 ms. Trace:
I20260709 13:04:30.702099  1466 catalog_manager.cc:5697] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 9 to 10, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 10 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:04:30.749470  7258 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:30.756410  7238 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:30.755779  1466 catalog_manager.cc:5697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 11 to 12, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 12 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
W20260709 13:04:30.760310  7259 consensus_meta.cc:287] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.717s	user 0.010s	sys 0.000s
I20260709 13:04:30.716590  7252 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:30.727312  1611 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:30.766747  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 44067.4us
I20260709 13:04:30.767409  7185 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:30.768126  1611 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:30.721971  6987 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:04:30.769325  7185 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:30.769846  7185 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:30.770313  7185 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:30.772012  1611 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:30.773506  6987 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:30.775611  1611 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:30.775414  7259 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:30.718418  7363 raft_consensus.cc:3055] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:30.778107  7363 raft_consensus.cc:740] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:30.779330  7363 consensus_queue.cc:260] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:30.780866  7363 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:30.783807  6616 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:30.785084  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:30.727613  7235 consensus_meta.cc:287] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 1.238s	user 0.009s	sys 0.007s
I20260709 13:04:30.786993  7235 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:30.731966  7365 consensus_queue.cc:260] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:30.728999  7323 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.006s	sys 0.020s
I20260709 13:04:30.742720  7290 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:30.703926  7115 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:30.743044  7152 consensus_meta.cc:287] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.445s	user 0.016s	sys 0.001s
I20260709 13:04:30.750259  6908 raft_consensus.cc:740] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:30.725533  7261 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.054s	user 0.000s	sys 0.015s
I20260709 13:04:30.754215  1640 rpcz_store.cc:276] 0709 13:04:28.929072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.316144 (+387072us) service_pool.cc:224] Handling call
0709 13:04:29.316497 (+   353us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:29.316978 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.317006 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:29.357920 (+ 40914us) log.cc:844] Serialized 52 byte log entry
0709 13:04:29.533094 (+175174us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:29.533215 (+   121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.533297 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:30.752185 (+1218888us) raft_consensus.cc:1685] finished
0709 13:04:30.752210 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.752393 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9626,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":104,"thread_start_us":105926,"threads_started":1,"wal-append.queue_time_us":238145}
W20260709 13:04:30.763113  1717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2504) took 1702 ms (client timeout 1686 ms). Trace:
W20260709 13:04:30.796797  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:30.813529  7261 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.144s	user 0.000s	sys 0.016s
I20260709 13:04:30.764518  7258 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:30.815248  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:30.815810  7115 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:30.816169  1569 raft_consensus.cc:2410] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
W20260709 13:04:30.816550  7323 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.009s	sys 0.025s
I20260709 13:04:30.818208  6908 consensus_queue.cc:260] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:30.819448  6908 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:30.819262  1717 rpcz_store.cc:269] 0709 13:04:29.061074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.077202 (+ 16128us) service_pool.cc:224] Handling call
0709 13:04:30.762855 (+1685653us) spinlock_profiling.cc:243] Waited 847 ms on lock 0x7b540029a9d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:30.762968 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1947399040}
W20260709 13:04:30.820843  7353 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.787s	user 0.001s	sys 0.000s
I20260709 13:04:30.822469  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 2995) took 1571 ms. Trace:
I20260709 13:04:30.822917  1643 rpcz_store.cc:276] 0709 13:04:29.250747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.427898 (+177151us) service_pool.cc:224] Handling call
0709 13:04:29.428285 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:29.449334 (+ 21049us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.449368 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:29.475716 (+ 26348us) log.cc:844] Serialized 26 byte log entry
0709 13:04:29.648766 (+173050us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:29.649179 (+   413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.649501 (+   322us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:30.820191 (+1170690us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:30.822129 (+  1938us) raft_consensus.cc:1685] finished
0709 13:04:30.822158 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.822401 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11408,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":9051,"spinlock_wait_cycles":3312000,"thread_start_us":88794,"threads_started":1,"wal-append.queue_time_us":171141}
I20260709 13:04:30.826195  7280 raft_consensus.cc:3055] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:30.826726  7280 raft_consensus.cc:740] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:30.826869  7370 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.827551  7370 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:30.828899  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:30.720118  7360 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:30.828756  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 572) took 1957 ms. Trace:
I20260709 13:04:30.830001  7360 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:30.830585  1639 rpcz_store.cc:276] 0709 13:04:28.870950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.871160 (+   210us) service_pool.cc:224] Handling call
0709 13:04:28.897467 (+ 26307us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:29.042866 (+145399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.042917 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:29.068453 (+ 25536us) log.cc:844] Serialized 52 byte log entry
0709 13:04:29.247561 (+179108us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:29.504722 (+257161us) log.cc:844] Serialized 14 byte log entry
0709 13:04:29.646011 (+141289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.828296 (+1182285us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54002dbc54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:30.828353 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:30.828405 (+    52us) raft_consensus.cc:1685] finished
0709 13:04:30.828515 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.828692 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19226,"prepare.queue_time_us":31142,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":23890,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":61500,"spinlock_wait_cycles":333173632,"thread_start_us":506740,"threads_started":3,"wal-append.queue_time_us":421992}
I20260709 13:04:30.832441  7360 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:30.776824  6987 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:30.829303  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:30.833014  6987 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.833427  6987 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:30.833433  1644 raft_consensus.cc:2376] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15. Re-sending same reply.
I20260709 13:04:30.834002  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:30.834771  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:30.835582  1643 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:30.836396  7368 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.836889  7368 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:30.839659  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:30.840389  1644 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 13: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:30.841194  6936 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:30.841548  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:30.839018  7368 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:30.844455  7365 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:30.847378  1646 raft_consensus.cc:1240] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:04:30.721947  7286 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:30.849784  7286 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:30.797222  7268 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:30.790102  2347 raft_consensus.cc:2393] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 12: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:30.850924  7268 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:30.850869  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:30.797056  7294 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.854360  7294 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:30.855403  7268 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:30.856654  7294 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:30.843485  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:30.857697  1646 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:04:30.857746  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:30.858628  1716 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 16: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:30.789580  7235 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:30.746742  7299 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:30.861347  7299 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:30.862399  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2501) took 1824 ms. Trace:
I20260709 13:04:30.830042  7280 consensus_queue.cc:260] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:30.862835  1721 rpcz_store.cc:276] 0709 13:04:29.037654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.323407 (+285753us) service_pool.cc:224] Handling call
0709 13:04:29.323707 (+   300us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:29.324096 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.324122 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:29.324720 (+   598us) log.cc:844] Serialized 52 byte log entry
0709 13:04:30.861824 (+1537104us) spinlock_profiling.cc:243] Waited 4.32 ms on lock 0x7b18003f37c0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a250d1
0709 13:04:30.861987 (+   163us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:30.862022 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.862110 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:30.862149 (+    39us) raft_consensus.cc:1685] finished
0709 13:04:30.862159 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.862334 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7359,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":30682,"spinlock_wait_cycles":9936640,"thread_start_us":62490,"threads_started":1,"wal-append.queue_time_us":152262}
I20260709 13:04:30.864704  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:30.813458  7277 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:30.791499  7300 consensus_queue.cc:260] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:30.867173  7300 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:30.732363  7262 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:30.863755  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:30.862547  1639 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:04:30.868533  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:30.870682  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:30.864854  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:30.874653  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:30.738271  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 1.33098e+06us
I20260709 13:04:30.806581  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2508) took 1722 ms. Trace:
I20260709 13:04:30.876495  1715 rpcz_store.cc:276] 0709 13:04:29.083823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.344612 (+260789us) service_pool.cc:224] Handling call
0709 13:04:29.344990 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.497857 (+152867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.806175 (+1308318us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b5400212354. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:30.806472 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":251244416}
I20260709 13:04:30.876569  1537 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:30.842259  6987 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:30.857216  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:30.880801  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:30.889160  1647 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:30.889606  1570 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:04:30.890647  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:30.892709  1645 raft_consensus.cc:2393] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 12: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:30.898047  1713 raft_consensus.cc:2410] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 16 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 13:04:30.765353  7321 consensus_meta.cc:287] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.205s	user 0.004s	sys 0.008s
I20260709 13:04:30.905419  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:30.905812  7262 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:30.906100  2341 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:30.906239  7262 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:30.906689  7262 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:30.795868  7341 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 1.159s	user 0.022s	sys 0.027s
I20260709 13:04:30.900331  1569 raft_consensus.cc:1275] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:30.902607  1716 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:04:30.898492  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:30.907424  1687 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:30.899698  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:30.727581  1685 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.791s	user 0.012s	sys 0.007s
W20260709 13:04:30.915016  1685 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.978s	user 0.012s	sys 0.008s
I20260709 13:04:30.888926  7275 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:30.917167  7283 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:30.918288  1687 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:30.919199  7152 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:30.919652  1687 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:30.920907  1687 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:30.921171  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:30.912825  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:30.913291  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:30.922410  1644 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 15: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:30.908231  1537 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:30.914852  1640 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:30.909329  1573 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 14: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:30.914824  1645 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:30.924479  2344 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:30.910353  1611 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:30.915812  1685 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.201s (SENT)
I20260709 13:04:30.925429  1643 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:04:30.924388  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 501) took 1693 ms. Trace:
I20260709 13:04:30.912715  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:30.900759  7299 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.920544  7152 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:30.935442  1685 leader_election.cc:336] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.234s (SENT)
I20260709 13:04:30.926574  2313 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:30.942960  1722 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:30.933328  2347 rpcz_store.cc:276] 0709 13:04:29.230998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.231264 (+   266us) service_pool.cc:224] Handling call
0709 13:04:30.924196 (+1692932us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400474d54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:30.924334 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3345359104}
I20260709 13:04:30.964093  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:30.964093  1685 leader_election.cc:336] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.372s (SENT)
I20260709 13:04:30.965230  1685 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:30.966164  1537 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:30.967629  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:30.872628  7185 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:30.979962  1640 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:30.901216  7252 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:30.982636  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:30.981887  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:30.995211  1644 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:30.936512  7349 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:30.933251  7283 raft_consensus.cc:697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:31.001415  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 894) took 1791 ms. Trace:
I20260709 13:04:30.947893  7249 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:31.001902  1568 rpcz_store.cc:276] 0709 13:04:29.210276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.320330 (+110054us) service_pool.cc:224] Handling call
0709 13:04:29.320642 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:29.344237 (+ 23595us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.350182 (+  5945us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:29.359576 (+  9394us) log.cc:844] Serialized 26 byte log entry
0709 13:04:29.365035 (+  5459us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:29.365083 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.365163 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:30.795010 (+1429847us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:30.994173 (+199163us) raft_consensus.cc:1685] finished
0709 13:04:30.994210 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:31.001359 (+  7149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19562,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"spinlock_wait_cycles":4351232,"thread_start_us":4968,"threads_started":1,"wal-append.queue_time_us":142276}
I20260709 13:04:30.922676  7235 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:04:30.907905  7321 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:30.959167  7094 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.980564  7277 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:31.004479  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:30.975129  7370 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:30.984176  7362 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:31.013406  7371 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:31.021996  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:30.948043  7299 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:31.034188  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 70165.6us
W20260709 13:04:31.040745  1685 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.429s (SENT)
I20260709 13:04:31.041288  1685 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:31.037175  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2503) took 1999 ms (client timeout 1825 ms). Trace:
W20260709 13:04:31.042250  1720 rpcz_store.cc:269] 0709 13:04:29.038199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.039212 (+  1013us) service_pool.cc:224] Handling call
0709 13:04:31.036990 (+1997778us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b5400248454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:31.037104 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":473456512}
I20260709 13:04:31.046752  1537 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:31.049587  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:31.063606  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:31.034793  6636 consensus_meta.cc:287] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.672s	user 0.011s	sys 0.003s
I20260709 13:04:31.017984  7286 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:30.999990  7252 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:31.022362  7249 raft_consensus.cc:697] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 13 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:31.001832  7152 consensus_queue.cc:1059] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:04:31.095474  6636 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:31.089172  7261 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:31.117630  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:55704 (request call id 2) took 1987 ms. Trace:
I20260709 13:04:31.029584  7323 consensus_queue.cc:1059] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
W20260709 13:04:31.122555  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 81825.2us
I20260709 13:04:31.034090  7355 consensus_queue.cc:1059] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.110s
I20260709 13:04:31.118166  1641 rpcz_store.cc:276] 0709 13:04:29.219725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.289607 (+ 69882us) service_pool.cc:224] Handling call
0709 13:04:30.758853 (+1469246us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b54002790a0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e02520 00007f8248e7f652 00007f824de4d60a 00007f82473d37d4 00007f82473d3762 00007f82473d3691 00007f82473d360c 00007f82473d28c2 00007f82471c048c 00007f82471bf8c6
0709 13:04:30.946227 (+187374us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54002790a0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e005e7 00007f8248e7fc68 00007f8248e7f84a 00007f824de4d60a 00007f82473d37d4 00007f82473d3762 00007f82473d3691 00007f82473d360c 00007f82473d28c2 00007f82471c048c
0709 13:04:31.117399 (+171172us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400097254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e7f853 00007f824de4d60a 00007f82473d37d4 00007f82473d3762 00007f82473d3691 00007f82473d360c 00007f82473d28c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:31.117521 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":533207680}
I20260709 13:04:31.035996  7094 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:31.104364  7067 consensus_meta.cc:287] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.533s	user 0.004s	sys 0.008s
I20260709 13:04:31.126271  7361 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:31.128961  2341 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:31.133726  1714 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:31.141891  7280 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:31.142812  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.189992  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 143277us
I20260709 13:04:31.191421  1537 leader_election.cc:400] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16.
I20260709 13:04:31.191892  1537 leader_election.cc:403] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:31.095865  6995 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:04:31.109910  6974 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:31.157804  7304 consensus_meta.cc:287] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.538s	user 0.014s	sys 0.002s
I20260709 13:04:31.150517  7362 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:30.953944  1466 catalog_manager.cc:5697] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 8 to 11, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 11 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:31.211957  7259 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:31.261478  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 70268us
I20260709 13:04:31.217288  7293 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.115978  7308 consensus_queue.cc:1059] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.140s
I20260709 13:04:31.184137  7067 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:31.255384  7356 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16.
I20260709 13:04:31.275192  1537 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:31.273252  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 492834us
I20260709 13:04:31.167279  7283 consensus_queue.cc:237] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:31.285912  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:31.282771  1641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:55704
I20260709 13:04:31.286631  1644 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:04:31.287201  1641 raft_consensus.cc:606] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:31.287554  1641 raft_consensus.cc:614] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:31.288273  1644 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:04:31.288775  1646 raft_consensus.cc:1240] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:04:31.291026  6636 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:31.252198  7377 consensus_queue.cc:1059] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:04:31.291992  7185 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:31.291536  1466 catalog_manager.cc:5697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:04:31.292533  7377 raft_consensus.cc:3055] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:31.293004  7377 raft_consensus.cc:740] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:31.231395  7304 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:31.290439  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:31.295487  1644 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 14: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:31.303934  1645 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:31.310614  2348 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:31.310452  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:31.312405  1643 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:31.315013  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:31.315538  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:31.316161  1645 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:31.323940  1722 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:31.325924  1640 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: [13.9-15.10]
I20260709 13:04:31.328471  1645 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:04:31.329691  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:31.294090  7294 raft_consensus.cc:2764] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:04:31.343581  1719 raft_consensus.cc:2393] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 13: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:31.274655  7293 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:31.333513  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:31.353886  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:31.342604  1644 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:31.291402  7094 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:31.346320  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:31.357905  1645 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:31.358228  1611 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:31.343595  7321 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:31.359647  7373 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:04:31.359532  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:31.360773  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:31.361539  1565 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17.
I20260709 13:04:31.362596  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:31.362900  6616 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:31.365840  6936 consensus_queue.cc:1059] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:31.369268  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:31.370016  1639 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:31.334339  7278 consensus_meta.cc:287] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.684s	user 0.003s	sys 0.009s
I20260709 13:04:31.322921  7262 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:31.311795  7299 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:31.375264  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:31.384286  2348 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:04:31.376354  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:31.385704  1640 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:31.387063  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:31.363180  1565 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:31.402259  1643 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
W20260709 13:04:31.404246  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 46097.8us
I20260709 13:04:31.406572  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:31.407521  2312 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:31.409451  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:31.417587  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:31.420341  1569 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:31.382004  7321 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.382129  7161 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:31.409471  7378 log.cc:927] Time spent T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.310s	user 0.002s	sys 0.000s
I20260709 13:04:31.424784  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:31.436434  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:31.411187  7075 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.000s	sys 0.022s
W20260709 13:04:31.439061  7075 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.001s	sys 0.023s
I20260709 13:04:31.405726  7252 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:31.419675  7339 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.701s	user 0.013s	sys 0.003s
I20260709 13:04:31.405845  7094 consensus_queue.cc:1059] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:04:31.416025  7281 raft_consensus.cc:3055] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:31.401667  7290 consensus_queue.cc:1059] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
W20260709 13:04:31.441033  7365 consensus_meta.cc:287] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.596s	user 0.012s	sys 0.000s
I20260709 13:04:31.398231  7278 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:31.375329  1571 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.007s	sys 0.010s
W20260709 13:04:31.487010  1571 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.007s	sys 0.010s
W20260709 13:04:31.459008  7380 log.cc:927] Time spent T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:04:31.293318  1466 catalog_manager.cc:5697] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 11 to 13, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 13 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:31.406633  1611 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:31.491034  1566 raft_consensus.cc:2393] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 10: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:31.502724  1713 raft_consensus.cc:1240] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:04:31.500339  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:31.501686  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 378060us
W20260709 13:04:31.464929  7285 consensus_meta.cc:287] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.718s	user 0.014s	sys 0.000s
I20260709 13:04:31.507951  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:31.467222  7281 raft_consensus.cc:740] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:31.516702  1566 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:31.518081  1685 leader_election.cc:336] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.778s (SENT)
I20260709 13:04:31.518573  1685 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:31.520437  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:31.433441  7321 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:31.440166  7161 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:31.321694  7377 consensus_queue.cc:260] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:31.521327  2348 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:31.528388  1685 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.838s (SENT)
W20260709 13:04:31.531900  1685 leader_election.cc:336] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.894s (SENT)
I20260709 13:04:31.532603  1685 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:31.534168  1685 leader_election.cc:336] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.920s (SENT)
I20260709 13:04:31.534698  1685 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:31.536464  1685 leader_election.cc:336] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.922s (SENT)
I20260709 13:04:31.536911  1685 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:31.539016  7258 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:31.539968  7258 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:31.507754  7361 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:04:31.544993  1685 leader_election.cc:336] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.013s (SENT)
I20260709 13:04:31.549078  1685 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:31.468679  7339 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:31.280913  7249 consensus_queue.cc:237] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:31.546872  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:31.564185  1565 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:31.560982  1714 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:04:31.565886  1714 pending_rounds.cc:85] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 7
I20260709 13:04:31.567154  1714 pending_rounds.cc:107] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:04:31.567427  1714 raft_consensus.cc:2889] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:31.580873  2348 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:04:31.535662  7290 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:31.543200  7371 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:31.548959  7361 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:31.623799  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:31.627948  1565 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:31.628891  1466 catalog_manager.cc:5697] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 8 to 11, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 11 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:31.631161  1466 catalog_manager.cc:5697] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:04:31.634254  7262 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:31.634745  7390 log.cc:927] Time spent T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:04:31.642428  7389 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:04:31.644282  2340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 728) took 2308 ms. Trace:
I20260709 13:04:31.562521  7067 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:31.647330  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2499) took 2593 ms. Trace:
I20260709 13:04:31.647805  1714 rpcz_store.cc:276] 0709 13:04:29.031757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.048192 (+ 16435us) service_pool.cc:224] Handling call
0709 13:04:29.048728 (+   536us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.133402 (+2084674us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b5800767cd0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248de10aa 00007f8248e63447 00007f8248e7bfe7 00007f8248e7d7de 00007f8248e87b49 00007f8248e88416 00007f8248e84d8f 00007f8248e8483e 00007f824de49419 00007f82473c0bd4
0709 13:04:31.624498 (+491096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.624839 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4063658240}
I20260709 13:04:31.644814  2340 rpcz_store.cc:276] 0709 13:04:29.335474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.376261 (+ 40787us) service_pool.cc:224] Handling call
0709 13:04:29.402479 (+ 26218us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:29.421937 (+ 19458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.421982 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:29.465658 (+ 43676us) log.cc:844] Serialized 26 byte log entry
0709 13:04:30.932905 (+1467247us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b1800395e60. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a250d1 00007f8248e40a0f
0709 13:04:31.433742 (+500837us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:31.433790 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.433877 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:31.643837 (+209960us) raft_consensus.cc:1685] finished
0709 13:04:31.643862 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:31.644184 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95983,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":16571,"spinlock_wait_cycles":4583680,"thread_start_us":493643,"threads_started":1,"wal-append.queue_time_us":2064345}
I20260709 13:04:31.645699  6616 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:04:31.642428  7388 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:04:31.646384  7260 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:04:31.646690  7361 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:31.667400  7376 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.354s	user 0.001s	sys 0.000s
W20260709 13:04:31.713408  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.729516  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.730060  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.730429  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.735642  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:31.632974  1466 catalog_manager.cc:5697] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 7 to 11, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 11 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:31.713253  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 350514us
I20260709 13:04:31.649442  1566 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
I20260709 13:04:31.577875  7281 consensus_queue.cc:260] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:31.740749  7262 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:31.759727  6616 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:31.715726  7323 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.784818  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:31.732110  7293 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:31.827925  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 420239us
W20260709 13:04:31.794742  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 387987us
W20260709 13:04:31.669592  7203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.209s	user 0.000s	sys 0.022s
I20260709 13:04:31.776300  1639 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:04:31.802500  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:31.650323  7374 log.cc:927] Time spent T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.356s	user 0.001s	sys 0.000s
W20260709 13:04:31.788935  7388 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:04:31.802397  1612 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:31.756549  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 211492us
W20260709 13:04:31.825671  7363 consensus_meta.cc:287] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.041s	user 0.011s	sys 0.005s
I20260709 13:04:31.831861  1566 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 13: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:31.837486  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:31.839286  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:31.840981  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:31.835112  1685 leader_election.cc:336] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.022s (SENT)
I20260709 13:04:31.731559  7371 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:31.845535  1685 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:31.830855  7203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.374s	user 0.001s	sys 0.022s
I20260709 13:04:31.853924  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:31.855209  1565 raft_consensus.cc:2368] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12.
I20260709 13:04:31.850338  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:31.860842  1685 leader_election.cc:336] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.100s (SENT)
I20260709 13:04:31.861441  1685 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:31.830991  7278 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:31.837173  7281 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:31.872344  2340 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:31.866957  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.879776  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:31.836068  6616 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.826691  7262 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:31.909020  1537 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:31.857355  7203 raft_consensus.cc:697] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:31.997769  7203 consensus_queue.cc:237] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:31.870054  7363 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:31.922955  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:32.001394  7280 consensus_meta.cc:287] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.790s	user 0.008s	sys 0.006s
W20260709 13:04:31.900425  7300 consensus_meta.cc:287] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.033s	user 0.016s	sys 0.002s
W20260709 13:04:32.006557  7394 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 13:04:31.864491  7399 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:31.949829  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2511) took 1306 ms. Trace:
I20260709 13:04:31.896796  7294 consensus_queue.cc:1059] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:04:32.008069  7294 raft_consensus.cc:3055] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:32.008510  7294 raft_consensus.cc:740] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:32.009506  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 100531us
I20260709 13:04:32.009388  7294 consensus_queue.cc:260] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:32.010425  1537 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:32.011238  7285 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:32.012096  7285 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:31.853870  7094 raft_consensus.cc:3055] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:04:32.015404  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.010752  7294 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:31.914299  7417 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:32.018399  7417 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.019261  7417 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:32.022449  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:32.023012  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:32.016919  7094 raft_consensus.cc:740] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:04:31.948768  7386 log.cc:927] Time spent T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.325s	user 0.000s	sys 0.001s
I20260709 13:04:31.927141  7161 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:31.958850  7384 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.332s	user 0.000s	sys 0.002s
I20260709 13:04:32.023335  1713 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:32.042191  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:32.042393  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 240002us
I20260709 13:04:32.042985  1713 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:32.043447  1612 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:31.979570  1566 raft_consensus.cc:2410] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:04:31.959827  7339 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:31.986868  7392 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.249s	user 0.000s	sys 0.002s
I20260709 13:04:31.913295  1466 catalog_manager.cc:5697] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 7 to 12, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 12 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:31.924089  7278 consensus_queue.cc:1059] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:04:32.060106  7278 raft_consensus.cc:3055] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:32.060587  7278 raft_consensus.cc:740] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:32.060283  7406 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:32.006747  1567 raft_consensus.cc:2410] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:04:32.014976  7361 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:32.016198  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.916164  7321 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:31.832469  7323 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:32.028652  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:32.064529  7361 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:32.061192  7409 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:32.065270  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:32.066000  1721 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:32.065657  7409 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:32.066694  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2516) took 1191 ms. Trace:
I20260709 13:04:32.068751  7409 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:31.907121  7075 consensus_queue.cc:1059] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:04:32.007395  1718 rpcz_store.cc:276] 0709 13:04:30.643912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.645652 (+  1740us) service_pool.cc:224] Handling call
0709 13:04:30.646107 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:30.657704 (+ 11597us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:30.934888 (+277184us) log.cc:844] Serialized 14 byte log entry
0709 13:04:31.384724 (+449836us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:31.410183 (+ 25459us) log.cc:844] Serialized 26 byte log entry
0709 13:04:31.410389 (+   206us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:31.410441 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.410535 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:31.949567 (+539032us) raft_consensus.cc:1685] finished
0709 13:04:31.949593 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:31.949762 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10553,"prepare.queue_time_us":630977,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":36847,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":12046,"thread_start_us":652597,"threads_started":2,"wal-append.queue_time_us":548802}
I20260709 13:04:32.070688  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:32.057072  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:32.059860  7161 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:32.067515  1572 rpcz_store.cc:276] 0709 13:04:30.875756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.878497 (+  2741us) service_pool.cc:224] Handling call
0709 13:04:30.908068 (+ 29571us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:30.908446 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:30.908475 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:30.914183 (+  5708us) log.cc:844] Serialized 26 byte log entry
0709 13:04:31.139351 (+225168us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b480036ffd8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ddaeaa1 00007f824dda5b37 00007f824dda59ca 00007f824de9d26f 00007f824de9cf37 00007f824deb155c 00007f824deb1508 00007f824deb1488 00007f824deb1443 00007f824deb06ec 00007f824cddaa96
0709 13:04:31.452385 (+313034us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:31.535044 (+ 82659us) log.cc:844] Serialized 14 byte log entry
0709 13:04:31.543685 (+  8641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.543801 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:32.066369 (+522568us) raft_consensus.cc:1685] finished
0709 13:04:32.066470 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.066625 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81199,"raft.queue_time_us":15824,"raft.run_cpu_time_us":2922,"raft.run_wall_time_us":235286,"spinlock_wait_cycles":4288256,"thread_start_us":462253,"threads_started":1,"wal-append.queue_time_us":706875}
I20260709 13:04:32.043936  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:32.077970  1713 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:32.071372  1718 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:32.071969  1722 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:31.995848  7262 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:32.061287  7406 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:32.090026  1721 raft_consensus.cc:2410] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:04:32.023759  6616 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:32.053316  7417 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:32.075233  1466 catalog_manager.cc:5697] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 4 to 9, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:32.096060  6616 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:32.094606  7339 consensus_queue.cc:1048] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:32.095925  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:32.100487  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 57133.7us
I20260709 13:04:32.061685  7278 consensus_queue.cc:260] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:32.101472  1612 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:32.101814  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:32.102659  1721 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
I20260709 13:04:32.103621  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:32.118361  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:32.121313  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.124347  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:32.124734  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:32.125671  1718 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:32.125790  1721 raft_consensus.cc:2410] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:04:32.096194  1466 catalog_manager.cc:5697] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 10 to 11, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 11 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:04:32.127485  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:32.124508  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:32.132656  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:32.133522  1566 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:32.098021  7406 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:32.124176  6616 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:32.134402  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:32.201825  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 253591us
I20260709 13:04:32.135169  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:32.133569  1565 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:32.142732  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:32.181525  1685 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:32.206676  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2525) took 1184 ms. Trace:
I20260709 13:04:32.207130  1572 raft_consensus.cc:2410] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:04:32.210012  1721 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:32.211584  1721 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:04:32.212082  1718 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:32.212355  1722 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
W20260709 13:04:32.143680  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.135324  1612 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:32.216220  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 80959.6us
I20260709 13:04:32.219327  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:32.221082  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:32.221787  1721 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:04:32.223609  1721 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:32.225437  1566 raft_consensus.cc:2368] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 16. Current term is 17.
I20260709 13:04:32.236093  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:32.168407  1537 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:32.238497  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 70205.2us
I20260709 13:04:32.237263  1565 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:32.240353  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:32.242290  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:32.243351  1565 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 22: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:32.244029  2340 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:32.241223  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:32.246174  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:32.174886  7420 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:04:32.249382  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:32.145056  6936 raft_consensus.cc:2764] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:04:32.256130  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:32.247722  1537 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:32.257248  1722 raft_consensus.cc:3055] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:32.260350  1722 raft_consensus.cc:740] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:32.186842  7278 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:32.216033  7280 raft_consensus.cc:2764] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:04:32.261916  1716 raft_consensus.cc:1275] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:32.267781  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:32.262418  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.206167  7361 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.261425  1537 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:32.186617  6616 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:32.244087  7399 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:32.247059  7075 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.261372  1722 consensus_queue.cc:260] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:32.268437  1565 raft_consensus.cc:2376] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14. Re-sending same reply.
I20260709 13:04:32.254678  7300 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:32.265883  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:32.189524  7321 consensus_queue.cc:1059] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
W20260709 13:04:32.269145  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.267170  7285 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:04:32.263867  7258 consensus_meta.cc:287] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.723s	user 0.015s	sys 0.000s
I20260709 13:04:32.143532  7417 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.118s
I20260709 13:04:32.280514  1722 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:32.277396  1716 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:32.280990  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:32.266825  7400 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:32.273347  6616 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:32.285470  1565 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:32.284232  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:32.302124  1566 raft_consensus.cc:2393] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 16: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:32.144918  7094 consensus_queue.cc:260] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:32.279170  7262 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:32.279101  7300 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.304922  1722 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:32.289554  1537 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:32.290908  7283 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.011s	sys 0.011s
I20260709 13:04:32.313686  1566 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:32.312261  1537 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:32.322070  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 899) took 1068 ms. Trace:
W20260709 13:04:32.282567  7351 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.005s	sys 0.016s
I20260709 13:04:32.286356  7321 consensus_queue.cc:1243] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer 066ee0109aba4465ba17fa65e14ef504 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:04:32.278681  7411 consensus_queue.cc:1059] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:04:32.322568  1566 rpcz_store.cc:276] 0709 13:04:31.253828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.309868 (+1056040us) service_pool.cc:224] Handling call
0709 13:04:32.310284 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:32.321619 (+ 11335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.321995 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:32.307427  7245 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.867s	user 0.019s	sys 0.035s
I20260709 13:04:32.327510  1565 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
W20260709 13:04:32.288473  7368 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.022s	sys 0.014s
I20260709 13:04:32.281062  7075 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:32.313452  7300 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:32.328967  7431 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:32.290097  7258 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:32.287124  7400 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.271212  7361 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:32.582002  7249 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:32.582468  7152 raft_consensus.cc:3055] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:32.585714  1721 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:32.586035  7152 raft_consensus.cc:740] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:32.324734  7415 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.328318  7203 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:32.309182  7262 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:04:32.326751  7351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.005s	sys 0.016s
W20260709 13:04:32.355054  7245 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.019s	sys 0.035s
I20260709 13:04:32.326246  7424 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:32.576498  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:32.558009  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 318705us
I20260709 13:04:32.605257  1612 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:04:32.366273  7368 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.926s	user 0.023s	sys 0.014s
W20260709 13:04:32.476207  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.577515  1565 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:32.285905  7365 consensus_queue.cc:1059] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:04:32.289738  7295 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.046s
W20260709 13:04:32.580720  1685 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.824s (SENT) [suppressed 31 similar messages]
I20260709 13:04:32.336786  7094 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:32.549615  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 233982us
I20260709 13:04:32.287376  7252 consensus_queue.cc:1059] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 13:04:32.584642  7235 consensus_queue.cc:1059] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:04:32.313163  7283 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.012s	sys 0.012s
W20260709 13:04:32.588160  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.588410  1566 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:04:32.593456  2312 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:04:32.598749  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:32.599371  7422 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.321s	user 0.000s	sys 0.011s
I20260709 13:04:32.652901  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 903) took 1333 ms. Trace:
W20260709 13:04:32.599318  7423 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.317s	user 0.007s	sys 0.005s
I20260709 13:04:32.585217  7400 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:32.608251  7258 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:32.689277  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.585865  7249 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:32.669862  1713 consensus_meta.cc:287] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.591s	user 0.014s	sys 0.006s
I20260709 13:04:32.690560  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 902) took 1377 ms. Trace:
W20260709 13:04:32.689337  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.600749  7262 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:32.692099  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2516) took 1827 ms. Trace:
I20260709 13:04:32.703552  1648 rpcz_store.cc:276] 0709 13:04:30.864984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.867027 (+  2043us) service_pool.cc:224] Handling call
0709 13:04:30.867351 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:30.867797 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:31.290316 (+422519us) log.cc:844] Serialized 14 byte log entry
0709 13:04:32.053579 (+763263us) log.cc:844] Serialized 14 byte log entry
0709 13:04:32.053916 (+   337us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:32.060015 (+  6099us) log.cc:844] Serialized 26 byte log entry
0709 13:04:32.060247 (+   232us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:32.060290 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.060377 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:32.691860 (+631483us) raft_consensus.cc:1685] finished
0709 13:04:32.691880 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.692031 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":139174,"prepare.queue_time_us":594678,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":10996,"raft.queue_time_us":154367,"raft.run_cpu_time_us":429,"raft.run_wall_time_us":11307,"thread_start_us":875617,"threads_started":2,"wal-append.queue_time_us":665998}
I20260709 13:04:32.656225  7411 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:32.695851  1565 rpcz_store.cc:276] 0709 13:04:31.319951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.366716 (+1046765us) service_pool.cc:224] Handling call
0709 13:04:32.577318 (+210602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:32.652582 (+ 75264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.652817 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:32.609247  1612 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:04:32.648774  7419 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.435s	user 0.000s	sys 0.012s
I20260709 13:04:32.706152  7441 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:04:32.706153  7277 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
W20260709 13:04:32.706642  7425 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:04:32.701733  1566 rpcz_store.cc:276] 0709 13:04:31.313210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.366138 (+1052928us) service_pool.cc:224] Handling call
0709 13:04:32.577620 (+211482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:32.690022 (+112402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.690451 (+   429us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:32.709126  1713 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:04:32.760294  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 905) took 1423 ms. Trace:
I20260709 13:04:32.762666  1566 rpcz_store.cc:276] 0709 13:04:31.336997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.759049 (+1422052us) service_pool.cc:224] Handling call
0709 13:04:32.759527 (+   478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:32.759794 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:32.759836 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:32.759900 (+    64us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:32.759949 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.760026 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.760204 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:32.765259  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2526) took 1424 ms. Trace:
I20260709 13:04:32.595309  7285 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.105s
I20260709 13:04:32.598496  7415 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:32.722214  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 140091us
I20260709 13:04:32.687165  7361 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:32.694905  7260 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:32.755323  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 174508us
I20260709 13:04:32.756587  7411 raft_consensus.cc:697] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:32.655812  7417 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.702159  7152 consensus_queue.cc:260] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:32.782728  7417 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:32.761933  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2905) took 1131 ms. Trace:
I20260709 13:04:32.783730  7249 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:32.782637  7232 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:32.784021  1713 rpcz_store.cc:276] 0709 13:04:31.630849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.043665 (+412816us) service_pool.cc:224] Handling call
0709 13:04:32.761830 (+718165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:32.784868  7232 raft_consensus.cc:697] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:32.786513  7232 consensus_queue.cc:237] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:32.742851  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 904) took 1411 ms. Trace:
I20260709 13:04:32.709609  7428 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.766332  1566 rpcz_store.cc:276] 0709 13:04:31.341057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.764139 (+1423082us) service_pool.cc:224] Handling call
0709 13:04:32.764612 (+   473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:32.764851 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:32.764876 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:32.764927 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:32.764979 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.765059 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.765201 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:32.789501  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.788344  7428 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:32.790318  7433 log.cc:927] Time spent T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 13:04:32.774211  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 180784us
I20260709 13:04:32.791894  7283 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.792800  7428 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:32.792486  7283 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:32.784843  7455 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:32.794095  7455 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:32.793910  1565 rpcz_store.cc:276] 0709 13:04:31.331494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.741784 (+1410290us) service_pool.cc:224] Handling call
0709 13:04:32.742209 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:32.742425 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:32.742462 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:32.742528 (+    66us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:32.742574 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.742641 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.742779 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:32.797081  7455 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:32.797489  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:32.775096  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 119824us
W20260709 13:04:32.798347  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:32.744951  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 135729us
I20260709 13:04:32.787037  7417 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:32.791283  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:32.801180  2344 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:32.649482  7424 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:32.802680  7455 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:32.802913  7412 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:32.805966  2348 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:32.799564  1537 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:32.764648  7439 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:32.763345  7429 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:32.813078  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:32.813544  7429 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:32.812757  7404 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:32.814791  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:32.812762  7455 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:32.815933  7429 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:32.805272  7424 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:32.810842  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:32.810192  7295 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:32.783648  7152 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:32.710568  7258 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:32.819648  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:32.831310  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.834391  1565 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:04:32.784237  7411 consensus_queue.cc:237] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:32.935187  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.802206  7283 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:32.944053  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:32.808650  7235 raft_consensus.cc:3055] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:04:32.942016  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 55702.7us
I20260709 13:04:32.941440  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:32.818673  7277 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:32.758107  7441 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:32.958168  1644 raft_consensus.cc:2393] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 10: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:32.828554  7439 raft_consensus.cc:697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:32.828382  7455 raft_consensus.cc:697] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:32.798319  7260 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:32.984594  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.801687  7440 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.172s
I20260709 13:04:32.959307  7203 raft_consensus.cc:697] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:32.967588  7406 consensus_meta.cc:287] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.777s	user 0.013s	sys 0.004s
I20260709 13:04:32.814723  7404 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:32.948063  7238 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.082s
W20260709 13:04:32.966377  7300 consensus_meta.cc:287] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.562s	user 0.008s	sys 0.008s
I20260709 13:04:32.944247  6995 consensus_queue.cc:1059] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.074s
I20260709 13:04:32.938750  7295 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:32.947935  7235 raft_consensus.cc:740] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:33.033435  1645 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:32.952545  7283 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:32.972558  7294 consensus_meta.cc:287] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.923s	user 0.020s	sys 0.003s
W20260709 13:04:33.004882  7278 consensus_meta.cc:287] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.738s	user 0.016s	sys 0.000s
I20260709 13:04:32.964936  7431 raft_consensus.cc:697] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:33.187014  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:33.004354  6616 consensus_meta.cc:287] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.695s	user 0.007s	sys 0.007s
I20260709 13:04:33.258543  6616 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.206086  1720 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:04:33.181615  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2511) took 2322 ms. Trace:
W20260709 13:04:33.201978  1719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2507) took 4126 ms (client timeout 2485 ms). Trace:
I20260709 13:04:33.260432  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:33.260713  1719 rpcz_store.cc:269] 0709 13:04:29.075721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.299840 (+224119us) service_pool.cc:224] Handling call
0709 13:04:33.201735 (+3901895us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b5400407c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:33.201882 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4675444992}
I20260709 13:04:33.261312  2343 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 19: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
W20260709 13:04:33.198011  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.262122  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.086338  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.250643  1572 raft_consensus.cc:2410] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:33.257007  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:33.262643  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.265689  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.161783  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2515) took 2287 ms. Trace:
I20260709 13:04:33.259719  2346 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:33.279100  1564 rpcz_store.cc:276] 0709 13:04:30.873927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.874725 (+   798us) service_pool.cc:224] Handling call
0709 13:04:30.875130 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:30.905489 (+ 30359us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:31.283452 (+377963us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b1800393e20. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d12c0a 00007f824f452219 00007f8245d9aa40 00007f8245d954e4 00007f8245d95455 00007f824c87d3c7 00007f8248e8f9a7 00007f8248e9f0bf 00007f8248e9f038
0709 13:04:31.547283 (+263831us) log.cc:844] Serialized 14 byte log entry
0709 13:04:32.306644 (+759361us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:32.315630 (+  8986us) log.cc:844] Serialized 26 byte log entry
0709 13:04:32.325025 (+  9395us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:32.325155 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.325381 (+   226us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:33.135706 (+810325us) raft_consensus.cc:1685] finished
0709 13:04:33.161397 (+ 25691us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.161683 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125925,"prepare.queue_time_us":427131,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":76697,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":15113,"spinlock_wait_cycles":8291072,"thread_start_us":906672,"threads_started":3,"wal-append.queue_time_us":467635}
W20260709 13:04:33.280823  1567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2517) took 2401 ms (client timeout 3035 ms). Trace:
I20260709 13:04:33.277355  7260 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:33.281409  1567 rpcz_store.cc:269] 0709 13:04:30.879065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.880448 (+  1383us) service_pool.cc:224] Handling call
0709 13:04:33.280564 (+2400116us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54002b9154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:33.280744 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2563921280}
I20260709 13:04:33.261211  6616 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:33.282117  7260 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:33.283087  1567 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:33.283851  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2561) took 1549 ms. Trace:
I20260709 13:04:33.284252  1567 rpcz_store.cc:276] 0709 13:04:31.734531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.282537 (+1548006us) service_pool.cc:224] Handling call
0709 13:04:33.282903 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.283568 (+   665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.283785 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:33.190737  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:33.037304  7437 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.224s	user 0.000s	sys 0.000s
I20260709 13:04:33.272831  7249 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.287153  7249 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:33.290369  7067 raft_consensus.cc:3055] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:33.290800  7067 raft_consensus.cc:740] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:33.291620  7067 consensus_queue.cc:260] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.292637  7067 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:33.190215  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:33.295075  1642 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:33.011776  7075 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:33.056341  7440 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.289328  7308 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:33.205672  7401 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.217s	user 0.003s	sys 0.000s
I20260709 13:04:33.271651  1713 raft_consensus.cc:1275] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:33.309348  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:33.310436  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:33.311364  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:33.312254  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:33.272416  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:33.272732  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:33.057215  7453 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.275334  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:33.313513  1719 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:33.282720  7260 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:33.260146  1646 rpcz_store.cc:276] 0709 13:04:30.859337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.860210 (+   873us) service_pool.cc:224] Handling call
0709 13:04:30.860554 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:31.289125 (+428571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.181278 (+1892153us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b54001f3454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:33.181551 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":983998208}
I20260709 13:04:33.136436  7417 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.318969  2340 raft_consensus.cc:2368] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10.
I20260709 13:04:33.296532  1567 raft_consensus.cc:1240] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:04:33.047466  7415 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:33.039350  7406 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.191411  7404 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:33.301075  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2520) took 2415 ms. Trace:
I20260709 13:04:33.050787  7441 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:33.269112  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:32.968506  7277 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:33.304126  2314 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.007s	sys 0.019s
I20260709 13:04:33.275974  7258 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:33.323938  1713 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:04:33.331645  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.333666  2312 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.062s	sys 0.165s
W20260709 13:04:33.351747  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.367072  2312 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.063s	sys 0.165s
W20260709 13:04:33.368614  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.341566  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.298813  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2900) took 1733 ms. Trace:
I20260709 13:04:33.352391  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2566) took 1559 ms. Trace:
I20260709 13:04:33.297876  7249 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:33.363260  2314 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.008s	sys 0.019s
I20260709 13:04:33.227947  7300 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.370720  1567 rpcz_store.cc:276] 0709 13:04:31.793446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.285105 (+1491659us) service_pool.cc:224] Handling call
0709 13:04:33.285521 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:33.343493 (+ 57972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.352318 (+  8825us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:33.349187  7252 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.007s	sys 0.015s
W20260709 13:04:33.368562  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 566333us
I20260709 13:04:33.376173  1642 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19.
I20260709 13:04:33.370545  1721 rpcz_store.cc:276] 0709 13:04:31.565220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.224535 (+659315us) service_pool.cc:224] Handling call
0709 13:04:32.224876 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:32.586217 (+361341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.294756 (+708539us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b54003903d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:33.298707 (+  3951us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":828972928}
W20260709 13:04:33.314354  7412 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.007s	sys 0.013s
I20260709 13:04:33.048177  7203 consensus_queue.cc:237] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.319223  7308 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.381435  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:33.353037  7453 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.362164  2347 raft_consensus.cc:2376] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16. Re-sending same reply.
I20260709 13:04:33.308800  7365 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:33.262573  7283 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:33.378647  1639 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 12.
I20260709 13:04:33.413322  1567 raft_consensus.cc:1275] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:33.478693  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2568) took 1655 ms. Trace:
I20260709 13:04:33.479512  1567 rpcz_store.cc:276] 0709 13:04:31.822821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.393444 (+1570623us) service_pool.cc:224] Handling call
0709 13:04:33.393928 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.477831 (+ 83903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.478574 (+   743us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:33.455193  7300 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:33.480983  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:33.403687  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.395951  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:33.415539  2346 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:33.198415  7455 consensus_queue.cc:237] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.425153  7295 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:33.355088  7417 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.427759  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2523) took 2468 ms. Trace:
I20260709 13:04:33.429598  7436 raft_consensus.cc:3055] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:04:33.495571  1537 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.021s	sys 0.030s
W20260709 13:04:33.438737  7450 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:04:33.440553  7277 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:33.309500  7440 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.036787  7439 consensus_queue.cc:237] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.354835  7415 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:33.450914  7258 consensus_queue.cc:1059] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:33.451844  7406 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:33.413398  7252 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.008s	sys 0.015s
I20260709 13:04:33.450812  1686 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:33.422518  1716 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:04:33.394635  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:33.467180  1719 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:33.514424  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:33.358793  1570 rpcz_store.cc:276] 0709 13:04:30.885497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.910131 (+ 24634us) service_pool.cc:224] Handling call
0709 13:04:30.910465 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.910902 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:31.682068 (+771166us) log.cc:844] Serialized 14 byte log entry
0709 13:04:33.289901 (+1607833us) log.cc:844] Serialized 14 byte log entry
0709 13:04:33.300550 (+ 10649us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:33.300638 (+    88us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:33.300713 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.300820 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.300999 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3233792,"prepare.queue_time_us":1762270,"prepare.run_cpu_time_us":1560,"prepare.run_wall_time_us":4816,"raft.queue_time_us":11113,"raft.run_cpu_time_us":369,"raft.run_wall_time_us":22194,"thread_start_us":797558,"threads_started":2,"wal-append.queue_time_us":578706}
W20260709 13:04:33.475396  7262 consensus_meta.cc:287] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.755s	user 0.011s	sys 0.001s
W20260709 13:04:33.490798  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 476606us
I20260709 13:04:33.429720  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3004) took 2131 ms. Trace:
I20260709 13:04:33.381197  7431 consensus_queue.cc:237] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.392558  7235 consensus_queue.cc:260] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:33.496630  1537 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.021s	sys 0.030s
I20260709 13:04:33.498350  1567 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:33.524097  1647 rpcz_store.cc:276] 0709 13:04:31.298524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.298683 (+   159us) service_pool.cc:224] Handling call
0709 13:04:31.318609 (+ 19926us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:31.319093 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:31.319133 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:31.355852 (+ 36719us) log.cc:844] Serialized 26 byte log entry
0709 13:04:31.727588 (+371736us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:31.727640 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.727731 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:33.205005 (+1477274us) spinlock_profiling.cc:243] Waited 3.48 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:33.429273 (+224268us) raft_consensus.cc:1685] finished
0709 13:04:33.429349 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.429617 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"spinlock_wait_cycles":8012032,"thread_start_us":305815,"threads_started":1,"wal-append.queue_time_us":542613}
I20260709 13:04:33.549880  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2569) took 1726 ms. Trace:
I20260709 13:04:33.550309  1567 rpcz_store.cc:276] 0709 13:04:31.823147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.497463 (+1674316us) service_pool.cc:224] Handling call
0709 13:04:33.498058 (+   595us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.549493 (+ 51435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.549776 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:33.499819  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.491171  1573 rpcz_store.cc:276] 0709 13:04:30.959623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.959864 (+   241us) service_pool.cc:224] Handling call
0709 13:04:30.960191 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:30.982122 (+ 21931us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:31.789068 (+806946us) log.cc:844] Serialized 14 byte log entry
0709 13:04:33.290418 (+1501350us) log.cc:844] Serialized 14 byte log entry
0709 13:04:33.300645 (+ 10227us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:33.306666 (+  6021us) log.cc:844] Serialized 26 byte log entry
0709 13:04:33.306841 (+   175us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:33.306886 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.306983 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:33.427454 (+120471us) raft_consensus.cc:1685] finished
0709 13:04:33.427481 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.427679 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1501343,"prepare.queue_time_us":3293417,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":83,"raft.queue_time_us":37333,"raft.run_cpu_time_us":604,"raft.run_wall_time_us":69090,"thread_start_us":742388,"threads_started":2,"wal-append.queue_time_us":523148}
I20260709 13:04:33.554431  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 909) took 1339 ms. Trace:
I20260709 13:04:33.554953  1573 rpcz_store.cc:276] 0709 13:04:32.215482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.553264 (+1337782us) service_pool.cc:224] Handling call
0709 13:04:33.553727 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.553995 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:33.554035 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:33.554099 (+    64us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:33.554146 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.554219 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.554353 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:33.490123  7400 consensus_meta.cc:287] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.775s	user 0.007s	sys 0.004s
I20260709 13:04:33.511011  1713 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:33.499176  7436 raft_consensus.cc:740] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:33.513520  2340 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
W20260709 13:04:33.518170  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 67463.8us
I20260709 13:04:33.560943  1686 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:33.516378  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 907) took 1680 ms. Trace:
I20260709 13:04:33.486093  7365 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.562069  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 908) took 1725 ms. Trace:
I20260709 13:04:33.557173  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 911) took 1276 ms. Trace:
I20260709 13:04:33.561789  1570 rpcz_store.cc:276] 0709 13:04:31.836454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.515548 (+1679094us) service_pool.cc:224] Handling call
0709 13:04:33.515876 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.516054 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:33.516085 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:33.516124 (+    39us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:33.516158 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.516210 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.516319 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:33.547789  7260 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:33.546729  7258 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:33.568166  1573 rpcz_store.cc:276] 0709 13:04:32.280369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.556119 (+1275750us) service_pool.cc:224] Handling call
0709 13:04:33.556539 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.556740 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:33.556771 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:33.556815 (+    44us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:33.556855 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.556932 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.557099 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:33.512074  7415 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:33.589802  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:33.590490  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:33.565572  1567 rpcz_store.cc:276] 0709 13:04:31.837155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.560886 (+1723731us) service_pool.cc:224] Handling call
0709 13:04:33.561312 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.561567 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:33.561604 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:33.561709 (+   105us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:33.561779 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.561853 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.561994 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:33.592480  1573 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:33.597043  1564 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:04:33.545938  7262 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.578310  7258 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.579717  7464 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:33.565860  7400 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.602473  7235 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:33.609315  7258 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:33.624459  7411 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.625048  7411 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.596834  7260 consensus_queue.cc:1059] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
W20260709 13:04:33.628262  1612 connection.cc:256] RPC call timeout handler was delayed by -1.9933477980000021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2485s for a response
W20260709 13:04:33.632640  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.635752  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:33.636317  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:33.636852  1648 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:33.637173  1642 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:33.637457  1646 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:33.636485  1639 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:33.621482  7464 raft_consensus.cc:697] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:04:33.643157  1685 connection.cc:256] RPC call timeout handler was delayed by -1.0096674260000214s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1839s for a response
I20260709 13:04:33.648801  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:33.656909  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:33.657902  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 1.82675e+06us
I20260709 13:04:33.657963  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:33.658970  1645 raft_consensus.cc:2410] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:33.660524  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:33.662840  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:33.663174  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:33.663688  2348 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:33.666172  1611 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:33.667368  6987 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:33.667768  7262 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:33.669520  7479 raft_consensus.cc:3055] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:33.670106  7479 raft_consensus.cc:740] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:04:33.672665  7467 log.cc:927] Time spent T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:04:33.671178  7479 consensus_queue.cc:260] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:33.674151  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:33.675170  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:33.678154  1611 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:33.681936  1643 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:33.682268  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:33.678120  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:33.684653  1646 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:04:33.686725  6987 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:33.687443  7308 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:33.689284  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.696198  7411 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:33.696502  7400 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:33.696736  7249 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:33.807212  7249 raft_consensus.cc:697] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:33.696532  6987 raft_consensus.cc:697] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:33.810918  6987 consensus_queue.cc:237] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:33.697396  7462 log.cc:927] Time spent T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
W20260709 13:04:33.698482  7476 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:04:33.699560  7473 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
W20260709 13:04:33.700448  7152 consensus_meta.cc:287] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.761s	user 0.010s	sys 0.002s
I20260709 13:04:32.207173  1568 rpcz_store.cc:276] 0709 13:04:31.022879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.023187 (+   308us) service_pool.cc:224] Handling call
0709 13:04:31.023634 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.023852 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:31.863979 (+840127us) log.cc:844] Serialized 14 byte log entry
0709 13:04:32.206316 (+342337us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:32.206374 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:32.206439 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.206508 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.206624 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":398204,"raft.queue_time_us":112,"thread_start_us":738479,"threads_started":2}
I20260709 13:04:33.819761  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:33.617337  7436 consensus_queue.cc:260] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.652434  7417 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:33.675012  7283 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:33.705578  7365 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:33.833491  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:33.834354  1715 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:04:33.722625  7415 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:33.723382  7440 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:33.725040  1572 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2571) took 1894 ms (client timeout 2048 ms). Trace:
W20260709 13:04:33.837226  1572 rpcz_store.cc:269] 0709 13:04:31.830214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.255634 (+425420us) service_pool.cc:224] Handling call
0709 13:04:33.724761 (+1469127us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b5400406ad4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:33.724965 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2271082752}
I20260709 13:04:33.839334  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:33.727877  7323 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:33.728350  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:33.846777  1718 raft_consensus.cc:3055] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:33.847306  1718 raft_consensus.cc:740] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:33.848342  1718 consensus_queue.cc:260] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.734614  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2557) took 2112 ms. Trace:
I20260709 13:04:33.850888  1565 rpcz_store.cc:276] 0709 13:04:31.622108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.933411 (+1311303us) service_pool.cc:224] Handling call
0709 13:04:32.933900 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:32.934195 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:33.292976 (+358781us) log.cc:844] Serialized 14 byte log entry
0709 13:04:33.734203 (+441227us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:33.734263 (+    60us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:33.734322 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.734413 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.734549 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":594176,"prepare.queue_time_us":363644,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":231773,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":70,"thread_start_us":209320,"threads_started":1,"wal-append.queue_time_us":320159}
W20260709 13:04:33.736619  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.854486  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.854979  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.743669  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:33.857244  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:33.858028  1567 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:33.858657  1713 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:33.859364  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:33.859935  1713 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:33.750483  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:33.758353  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:33.860960  1719 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:04:33.861203  1573 raft_consensus.cc:2410] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:04:33.862195  1719 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:33.862285  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:33.862854  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.863590  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:33.749841  7132 consensus_queue.cc:1059] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.034s
I20260709 13:04:33.719357  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:33.767051  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.867703  7377 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:33.753659  7304 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:33.735246  7465 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:33.739179  7464 consensus_queue.cc:237] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:33.853299  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2559) took 2230 ms. Trace:
I20260709 13:04:33.738346  7367 raft_consensus.cc:3055] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:33.683069  1640 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:33.787194  7442 raft_consensus.cc:3055] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:33.871829  7367 raft_consensus.cc:740] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:33.873481  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 906) took 2465 ms. Trace:
I20260709 13:04:33.873948  1566 rpcz_store.cc:276] 0709 13:04:31.408397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.790454 (+1382057us) service_pool.cc:224] Handling call
0709 13:04:32.797772 (+  7318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.019115 (+221343us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:33.294173 (+275058us) log.cc:844] Serialized 14 byte log entry
0709 13:04:33.791514 (+497341us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:33.791575 (+    61us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:33.791625 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.873003 (+ 81378us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54000f69d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:33.873257 (+   254us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.873413 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":559012,"prepare.queue_time_us":286036,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":70,"raft.queue_time_us":284436,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"spinlock_wait_cycles":507612032,"thread_start_us":214298,"threads_started":1,"wal-append.queue_time_us":314489}
W20260709 13:04:33.875922  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.871418  7442 raft_consensus.cc:740] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:33.732503  7245 raft_consensus.cc:3055] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:04:33.876848  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.788168  7479 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:33.789392  1643 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:33.879602  2313 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:33.878918  1564 rpcz_store.cc:276] 0709 13:04:31.677707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.281724 (+1604017us) service_pool.cc:224] Handling call
0709 13:04:33.427802 (+146078us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:33.597993 (+170191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.852232 (+254239us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b54001b6554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:33.852993 (+   761us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54001b6550. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:33.853207 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":722458752}
I20260709 13:04:33.879765  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2921) took 1069 ms. Trace:
I20260709 13:04:33.683719  7412 consensus_queue.cc:1048] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:04:33.790102  7203 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.790542  7262 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:33.790652  1685 leader_election.cc:400] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 16. Current term is 17.
I20260709 13:04:33.738615  7347 raft_consensus.cc:3055] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:33.776226  7275 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:33.678377  6636 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.792882  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2511) took 2926 ms. Trace:
I20260709 13:04:33.798167  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:33.804381  7280 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:33.806241  7468 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:33.808198  7249 consensus_queue.cc:237] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:33.815090  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2922) took 1004 ms. Trace:
I20260709 13:04:33.817907  2344 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
W20260709 13:04:33.804001  7067 consensus_meta.cc:287] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.511s	user 0.016s	sys 0.001s
I20260709 13:04:33.828428  1721 raft_consensus.cc:2410] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 13:04:33.817059  7094 consensus_meta.cc:287] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.129s	user 0.014s	sys 0.002s
I20260709 13:04:33.829396  1538 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:33.841291  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:33.835474  7478 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.134s	user 0.000s	sys 0.005s
W20260709 13:04:33.830549  7387 log.cc:927] Time spent T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 2.293s	user 0.007s	sys 0.004s
W20260709 13:04:33.830704  7481 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.066s	user 0.001s	sys 0.002s
I20260709 13:04:33.809818  7452 raft_consensus.cc:3055] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:33.842609  7283 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:33.844203  7436 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:33.832713  7474 log.cc:927] Time spent T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.086s	user 0.001s	sys 0.001s
I20260709 13:04:33.845610  7409 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.732884  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2891) took 2317 ms. Trace:
I20260709 13:04:33.869904  7465 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.877930  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:33.877955  7453 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:33.878275  7277 raft_consensus.cc:3055] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:33.678157  7406 consensus_queue.cc:1059] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:04:33.732118  7368 raft_consensus.cc:3055] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:33.881001  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:33.953118  7440 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:33.853606  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:33.882647  2340 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:34.021287  1565 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 13: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:33.876204  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:34.022014  1566 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:34.022343  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:34.022606  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 324183us
I20260709 13:04:34.022961  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:34.024148  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:33.885143  1718 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:34.024952  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:33.880674  1716 rpcz_store.cc:276] 0709 13:04:32.809878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.810412 (+   534us) service_pool.cc:224] Handling call
0709 13:04:33.879534 (+1069122us) spinlock_profiling.cc:243] Waited 607 ms on lock 0x7b54002c6ad4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:33.879692 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1396259072}
I20260709 13:04:33.893797  7304 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:04:33.883718  7482 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.107s	user 0.004s	sys 0.001s
W20260709 13:04:33.894784  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.892706  6995 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:33.879129  7367 consensus_queue.cc:260] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.887154  7245 raft_consensus.cc:740] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:33.899488  1685 leader_election.cc:403] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 13:04:34.038434  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 247868us
I20260709 13:04:34.039712  1685 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:33.892581  7426 log.cc:927] Time spent T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 1.243s	user 0.005s	sys 0.000s
I20260709 13:04:33.910218  1722 rpcz_store.cc:276] 0709 13:04:32.810219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.810358 (+   139us) service_pool.cc:224] Handling call
0709 13:04:32.810684 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:32.816698 (+  6014us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:32.816732 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:32.817214 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 13:04:33.314908 (+497694us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:33.314970 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.315085 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:33.814643 (+499558us) raft_consensus.cc:1685] finished
0709 13:04:33.814669 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.815008 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5601,"raft.queue_time_us":5742,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"thread_start_us":497245,"threads_started":1,"wal-append.queue_time_us":682152}
I20260709 13:04:33.902071  7203 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.942458  1537 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:04:34.044806  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 102373us
I20260709 13:04:33.830725  7258 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:33.902009  1571 rpcz_store.cc:276] 0709 13:04:30.866207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.866307 (+   100us) service_pool.cc:224] Handling call
0709 13:04:30.866614 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:30.867027 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:30.867057 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:30.867505 (+   448us) log.cc:844] Serialized 26 byte log entry
0709 13:04:31.487529 (+620024us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:33.200859 (+1713330us) spinlock_profiling.cc:243] Waited 7.7 ms on lock 0x7b1800306dc0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d12c0a 00007f824f452219 00007f8245d9aa40 00007f8245d954e4 00007f8245d95455 00007f824c87d3c7 00007f8248e8f9a7 00007f8248e9f0bf 00007f8248e9f038
0709 13:04:33.499587 (+298728us) log.cc:844] Serialized 14 byte log entry
0709 13:04:33.791978 (+292391us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.792394 (+   416us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:33.792458 (+    64us) raft_consensus.cc:1685] finished
0709 13:04:33.792559 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.792803 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":179589,"prepare.queue_time_us":1715606,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":191979,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":10235,"spinlock_wait_cycles":17713536,"thread_start_us":1038825,"threads_started":3,"wal-append.queue_time_us":820549,"wal-append.run_cpu_time_us":4703,"wal-append.run_wall_time_us":1603949}
I20260709 13:04:34.045929  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:34.046195  1722 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:04:34.046538  1571 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:33.899952  7347 raft_consensus.cc:740] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:33.949820  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 120447us
I20260709 13:04:33.899544  7262 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:34.035862  6995 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.986054  1720 rpcz_store.cc:276] 0709 13:04:31.415029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.470649 (+ 55620us) service_pool.cc:224] Handling call
0709 13:04:33.205843 (+1735194us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:33.259713 (+ 53870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.732574 (+472861us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b5400407c50. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:33.732794 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3987462912}
W20260709 13:04:33.987553  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 322845us
W20260709 13:04:33.994218  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.985216  7283 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.906206  6636 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:34.016911  1564 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:33.879765  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2522) took 2957 ms. Trace:
I20260709 13:04:34.077844  1569 rpcz_store.cc:276] 0709 13:04:30.922639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.923091 (+   452us) service_pool.cc:224] Handling call
0709 13:04:30.923458 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.428246 (+504788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.879533 (+2451287us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b54002baa54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:33.879705 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1141867264}
I20260709 13:04:33.954305  7452 raft_consensus.cc:740] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:33.884466  1573 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:33.911089  7275 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:33.910657  7280 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:34.036720  7304 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:34.027050  1565 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 22: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:34.034924  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:34.036134  1611 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:34.041302  1685 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:34.294826  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 253479us
I20260709 13:04:34.296535  1685 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:34.016423  7368 raft_consensus.cc:740] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:33.927548  7468 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:34.047324  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:33.963974  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.061213  1538 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:34.059269  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:33.997382  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.303318  1565 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:04:34.006192  7277 raft_consensus.cc:740] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:34.066346  1612 leader_election.cc:400] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10.
I20260709 13:04:34.006721  7409 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:34.031062  1568 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:34.151240  1718 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:34.063602  7367 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:34.072593  6995 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:34.063268  7453 consensus_queue.cc:1059] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.139s
I20260709 13:04:34.098079  7294 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.270210  1567 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:34.206415  7245 consensus_queue.cc:260] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:34.296598  7152 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:34.031030  7323 consensus_queue.cc:1059] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:04:34.083132  7132 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:04:34.062320  7400 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:04:34.088477  7406 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:34.355594  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2534) took 1078 ms. Trace:
W20260709 13:04:34.272536  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.272876  1573 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:34.059729  7440 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:34.363582  1573 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:34.095155  1715 raft_consensus.cc:2393] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 13: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:34.082315  7428 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:34.082245  7067 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:34.280611  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.282548  1564 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 13:04:34.368743  1564 pending_rounds.cc:85] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 8
I20260709 13:04:34.369076  1564 pending_rounds.cc:107] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:04:34.369335  1564 raft_consensus.cc:2889] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:34.347823  7431 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:34.289821  1717 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:34.290422  1714 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:34.308959  1612 leader_election.cc:403] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:04:34.311484  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:34.312170  1565 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:04:34.317497  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2914) took 2275 ms. Trace:
I20260709 13:04:34.321039  1571 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 16: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:34.320014  7417 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:34.298134  7448 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.297749  7441 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:34.343631  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 282418us
I20260709 13:04:34.382784  1538 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:34.345701  7452 consensus_queue.cc:260] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:34.270792  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:34.333391  7360 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:04:34.328392  7295 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.048322  7415 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:34.364923  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:34.440407  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:34.441170  1722 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:04:34.441855  7488 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:04:34.352528  7245 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:34.048473  7321 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 16. Current term is 17.
I20260709 13:04:34.339795  7460 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:34.487300  7460 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:34.289096  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:34.440240  7417 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:34.374030  1718 rpcz_store.cc:276] 0709 13:04:32.042088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.254029 (+211941us) service_pool.cc:224] Handling call
0709 13:04:32.274096 (+ 20067us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.846643 (+1572547us) spinlock_profiling.cc:243] Waited 60.1 ms on lock 0x7b54001160a0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248dfcdc0 00007f8248e871c0 00007f8248e88231 00007f8248e84d8f 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c
0709 13:04:34.317189 (+470546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.317449 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":138118016}
I20260709 13:04:34.354151  7323 consensus_queue.cc:1243] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer cf2f71592be8465fb18e36e15d0b2f89 log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:04:34.482787  7152 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:34.376474  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 65064.8us
I20260709 13:04:34.379522  1573 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:04:34.492198  7152 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:34.340480  7294 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:34.428314  1571 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:34.372411  7258 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:34.371992  7262 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:34.481033  1722 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:34.375448  7440 raft_consensus.cc:697] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:34.490404  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 201300us
W20260709 13:04:34.370970  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.371963  7067 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:34.492707  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:34.368414  7075 raft_consensus.cc:3055] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:34.426817  7455 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:34.371747  7428 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:34.434099  7465 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:34.426954  7441 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:34.557317  1714 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:34.507087  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:34.443818  7203 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:34.511674  1718 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 14: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:34.423242  7453 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 13:04:34.367095  7368 consensus_queue.cc:260] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:34.491287  7460 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:34.555832  1685 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:34.528940  7455 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.563648  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2533) took 1286 ms. Trace:
I20260709 13:04:34.554268  7275 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:34.555959  7262 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:34.431200  7448 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:34.533435  7452 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:34.516129  7075 raft_consensus.cc:740] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:34.443651  7295 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:34.578224  1685 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:34.579922  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:34.555348  7490 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.111s	user 0.001s	sys 0.002s
W20260709 13:04:34.567633  7377 consensus_meta.cc:287] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.699s	user 0.008s	sys 0.014s
I20260709 13:04:34.583542  7455 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:34.600687  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.579922  1719 rpcz_store.cc:276] 0709 13:04:33.277016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.863086 (+586070us) service_pool.cc:224] Handling call
0709 13:04:33.863424 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:33.888644 (+ 25220us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:33.896837 (+  8193us) log.cc:844] Serialized 14 byte log entry
0709 13:04:34.283775 (+386938us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:34.284416 (+   641us) log.cc:844] Serialized 26 byte log entry
0709 13:04:34.284793 (+   377us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:34.284987 (+   194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.285089 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:34.563395 (+278306us) raft_consensus.cc:1685] finished
0709 13:04:34.563417 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:34.563578 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38893,"prepare.queue_time_us":7988,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":36054,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":16962,"thread_start_us":318937,"threads_started":1,"wal-append.queue_time_us":474704}
I20260709 13:04:34.614369  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:34.588428  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 159957us
I20260709 13:04:34.541862  7417 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:04:34.562937  7283 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:34.606123  2340 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:04:34.609378  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.609571  1685 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:34.610241  7275 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.527218  7323 consensus_queue.cc:1059] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:04:34.580963  7415 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:34.612810  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:34.619050  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:34.603067  7280 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:34.623996  1685 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:34.606230  7368 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:34.603719  7262 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:34.613371  7463 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.627578  2344 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:34.627749  1722 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:34.608613  7258 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:34.631969  1538 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:34.356510  1713 rpcz_store.cc:276] 0709 13:04:33.277308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.893124 (+615816us) service_pool.cc:224] Handling call
0709 13:04:33.893450 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.898700 (+  5250us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:33.899304 (+   604us) log.cc:844] Serialized 14 byte log entry
0709 13:04:34.355046 (+455742us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:34.355126 (+    80us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:34.355192 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.355291 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:34.355504 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3776,"prepare.queue_time_us":125,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":4506,"thread_start_us":450599,"threads_started":1}
I20260709 13:04:34.636839  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:34.637920  1567 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:34.616106  7409 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:34.617769  7304 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:34.641091  1719 raft_consensus.cc:2410] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:34.620980  7377 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:34.642529  1685 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:34.643252  1538 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:34.643093  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 135341us
I20260709 13:04:34.626624  7283 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.654790  1685 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:34.652694  1717 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:34.654618  1537 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:34.655220  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:34.657028  1722 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:34.662199  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:34.662075  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 50239.3us
I20260709 13:04:34.664759  1685 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:34.634371  7262 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:34.667680  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.667955  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 914) took 1231 ms. Trace:
W20260709 13:04:34.652339  7487 log.cc:927] Time spent T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.229s	user 0.000s	sys 0.002s
I20260709 13:04:34.669873  2344 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:34.675236  1570 rpcz_store.cc:276] 0709 13:04:33.436864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.567110 (+130246us) service_pool.cc:224] Handling call
0709 13:04:33.567609 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:33.794132 (+226523us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:33.794173 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:33.795115 (+   942us) log.cc:844] Serialized 26 byte log entry
0709 13:04:34.072186 (+277071us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:34.072235 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.072338 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:34.667658 (+595320us) raft_consensus.cc:1685] finished
0709 13:04:34.667683 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:34.667880 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":212950,"raft.queue_time_us":223906,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"thread_start_us":276637,"threads_started":1,"wal-append.queue_time_us":560259}
I20260709 13:04:34.652460  7465 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:34.626097  7275 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:34.650035  7377 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.691602  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:34.679498  1686 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:04:34.694655  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.648638  7502 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:34.662344  7304 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:34.657403  7067 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:34.679006  7479 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.788s	user 0.016s	sys 0.000s
I20260709 13:04:34.639554  7453 consensus_queue.cc:1059] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:04:34.658679  7203 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:04:34.632086  7440 consensus_queue.cc:237] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:34.668984  7323 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:34.690429  7498 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
W20260709 13:04:34.696348  7491 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260709 13:04:34.696671  7492 log.cc:927] Time spent T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 13:04:34.694077  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:34.704419  1719 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 13: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:04:34.708417  1685 connection.cc:256] RPC call timeout handler was delayed by -1.0649258600000167s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1595s for a response
W20260709 13:04:34.754408  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.674891  7280 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:34.704136  7377 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:34.714797  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:34.645977  7463 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:34.760905  1713 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
W20260709 13:04:34.736012  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 92816.1us
I20260709 13:04:34.661468  7283 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:34.738595  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 59164.9us
W20260709 13:04:34.750396  7487 log.cc:927] Time spent T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:04:34.741340  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2537) took 1078 ms. Trace:
I20260709 13:04:34.772289  1715 rpcz_store.cc:276] 0709 13:04:33.662660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.841021 (+178361us) service_pool.cc:224] Handling call
0709 13:04:34.741085 (+900064us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b5400248e54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:34.741262 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347801984}
I20260709 13:04:34.738379  7504 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:34.683341  7465 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:34.634528  7075 consensus_queue.cc:260] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:34.650274  7258 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:34.762671  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:34.785363  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.704051  7501 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.070s
W20260709 13:04:34.788959  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49446.7us
I20260709 13:04:34.765952  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:34.685472  7499 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:04:34.769424  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:34.704460  7294 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:34.759828  7493 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.249s	user 0.000s	sys 0.002s
I20260709 13:04:34.737190  7505 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:34.746315  1722 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:34.798106  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:34.799072  1722 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:04:34.799638  1719 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:34.757306  7479 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:34.786441  1685 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:34.774711  7323 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.737687  7304 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.689383  6995 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:34.695372  7442 consensus_queue.cc:260] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:34.823583  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 572) took 1387 ms. Trace:
I20260709 13:04:34.741219  7458 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:04:34.775223  7441 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:34.824190  1721 rpcz_store.cc:276] 0709 13:04:33.436345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.941794 (+505449us) service_pool.cc:224] Handling call
0709 13:04:33.942122 (+   328us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:33.942538 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:33.942569 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:33.943115 (+   546us) log.cc:844] Serialized 52 byte log entry
0709 13:04:34.361056 (+417941us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:34.361105 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.361177 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:34.823289 (+462112us) raft_consensus.cc:1685] finished
0709 13:04:34.823319 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:34.823508 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6361,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":22408,"thread_start_us":417539,"threads_started":1,"wal-append.queue_time_us":657571}
I20260709 13:04:34.797991  7505 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.766512  7245 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:04:34.782569  7465 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:34.810921  1719 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:34.847388  1718 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:34.848819  1718 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:04:34.839351  7479 raft_consensus.cc:697] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:34.854686  7479 consensus_queue.cc:237] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:34.855911  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3018) took 1226 ms. Trace:
W20260709 13:04:34.859808  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 823309us
I20260709 13:04:34.784462  7509 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:34.815582  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:34.873709  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 59057.6us
I20260709 13:04:34.830657  7417 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:34.822460  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:34.813778  7075 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:34.829447  1722 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:04:34.808627  7262 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:34.813251  7094 raft_consensus.cc:3055] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:34.840533  7295 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:34.841531  7504 raft_consensus.cc:697] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:34.846436  7245 consensus_queue.cc:1243] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:04:34.850898  7439 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:34.782709  7500 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.172s	user 0.000s	sys 0.002s
I20260709 13:04:34.884091  7321 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:34.883008  7417 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.822137  6995 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:34.875097  1685 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:34.883606  1570 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:34.888686  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 822028us
I20260709 13:04:34.817085  7304 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:34.840090  7442 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:34.851169  7275 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:34.850538  7277 consensus_queue.cc:260] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:34.897703  7455 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:34.884358  7436 consensus_meta.cc:287] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.897s	user 0.015s	sys 0.000s
I20260709 13:04:34.891793  7094 raft_consensus.cc:740] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:34.839174  7505 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:34.823233  7323 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:34.916164  7417 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:34.943365  7323 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:34.872819  7509 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.943990  7509 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:34.899255  7367 consensus_meta.cc:287] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.570s	user 0.012s	sys 0.000s
I20260709 13:04:34.906626  7262 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:34.950197  2314 leader_election.cc:336] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.554s (SENT)
I20260709 13:04:34.950860  2314 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:34.952549  7455 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:34.907233  7504 consensus_queue.cc:237] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:34.954041  7094 consensus_queue.cc:260] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:34.955555  7094 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:34.856402  1647 rpcz_store.cc:276] 0709 13:04:33.629054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.629299 (+   245us) service_pool.cc:224] Handling call
0709 13:04:33.661122 (+ 31823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.661324 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:33.739891 (+ 78567us) log.cc:844] Serialized 14 byte log entry
0709 13:04:34.237421 (+497530us) log.cc:844] Serialized 14 byte log entry
0709 13:04:34.843504 (+606083us) spinlock_profiling.cc:243] Waited 6.24 ms on lock 0x7b18003b05c0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d12c0a 00007f824f452219 00007f824cd5958f 00007f8245d98571 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd
0709 13:04:34.855513 (+ 12009us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:34.855579 (+    66us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:34.855630 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.855716 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:34.855846 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":660024,"prepare.queue_time_us":228332,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":107,"raft.queue_time_us":11089,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":5663,"spinlock_wait_cycles":14355328,"thread_start_us":345997,"threads_started":1,"wal-append.queue_time_us":619009}
I20260709 13:04:34.906677  7258 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:34.907195  7439 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:34.887787  7249 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:34.920368  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:34.979825  7490 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:04:34.899717  7067 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 9. Current term is 10.
W20260709 13:04:34.937187  1685 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.595s (SENT)
I20260709 13:04:34.937776  7501 consensus_queue.cc:1059] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:04:34.938767  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet eca30b43c26f420c965ee25f475581a0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:04:34.931034  7502 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:04:34.864190  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.984486  7361 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:34.954836  1718 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:34.917725  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:34.991640  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 130334us
I20260709 13:04:34.995826  1685 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:34.978816  7275 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.017s
W20260709 13:04:34.996649  1456 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 58027.1us
W20260709 13:04:35.005257  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 68061.6us
W20260709 13:04:35.006454  1685 leader_election.cc:336] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.799s (SENT)
I20260709 13:04:35.006958  1685 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:35.013433  7509 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.014267  7323 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:04:35.016258  2314 leader_election.cc:336] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.623s (SENT)
I20260709 13:04:35.016844  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:35.021870  7133 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:35.005152  7258 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:35.024753  7505 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:35.027110  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:35.027688  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.008260  7502 raft_consensus.cc:697] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:35.029582  7502 consensus_queue.cc:237] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.030481  7152 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:35.034108  1572 raft_consensus.cc:2410] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:04:35.035769  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2580) took 1296 ms. Trace:
W20260709 13:04:35.035910  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.036108  1570 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:04:35.036185  1572 rpcz_store.cc:276] 0709 13:04:33.739294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.877557 (+138263us) service_pool.cc:224] Handling call
0709 13:04:35.035569 (+1158012us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400305a54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:35.035706 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2407335168}
I20260709 13:04:35.039072  7448 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:35.040249  7439 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:35.049932  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:35.058382  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 140736us
I20260709 13:04:35.062345  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.066181  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.067106  1644 raft_consensus.cc:2393] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 10: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:35.069813  1715 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 14: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:35.071082  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.072103  1644 raft_consensus.cc:2410] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:04:35.076076  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:35.076962  1640 raft_consensus.cc:2376] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19. Re-sending same reply.
I20260709 13:04:35.081470  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:35.058252  7152 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.057878  7067 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.083629  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:35.083716  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:35.086069  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:35.089365  1643 raft_consensus.cc:2376] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 12. Re-sending same reply.
I20260709 13:04:35.086138  1647 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:04:35.064877  6636 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:35.121531  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:35.068497  7439 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.119385  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:35.125836  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:35.067878  7436 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:35.123021  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:35.067106  7509 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:35.145112  7351 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.145944  7351 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:35.148886  7351 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:35.122730  7411 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:35.122982  7412 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:35.126684  7524 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:35.161688  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.153416  7321 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:35.157477  7411 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.158015  7412 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.158860  7524 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:35.154799  7525 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:04:35.127166  7249 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:35.168511  7428 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:35.141074  7439 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.119903  7067 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.092330  7152 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.268537  1639 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:35.269179  1644 raft_consensus.cc:1275] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:35.160768  6636 consensus_queue.cc:1059] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
W20260709 13:04:35.178861  7527 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:04:35.183440  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:35.273742  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.124246  6987 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:35.084712  7486 consensus_queue.cc:1059] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 13:04:35.186769  1570 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:35.149362  7094 consensus_queue.cc:1048] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.115s
I20260709 13:04:35.136898  7436 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.126618  7523 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:35.267325  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:35.330646  1647 raft_consensus.cc:2368] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 16. Current term is 17.
I20260709 13:04:35.331149  1640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:55704
I20260709 13:04:35.331892  1640 raft_consensus.cc:606] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:35.332706  7523 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.272392  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:35.333346  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.335145  1648 raft_consensus.cc:2393] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 13: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:35.274727  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.334129  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:35.336238  1643 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:35.336305  1645 raft_consensus.cc:2410] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:04:35.275420  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:35.279610  1644 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:04:35.282882  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.339277  1639 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:35.331907  1646 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:35.161854  7417 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:35.311960  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.319157  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:35.352492  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.352918  1572 raft_consensus.cc:2410] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:04:35.144639  7304 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:35.185828  1687 leader_election.cc:336] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.515s (SENT)
I20260709 13:04:35.280494  1571 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:35.358525  1687 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:35.359654  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 173815us
I20260709 13:04:35.331764  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.348198  1645 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:04:35.338508  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.326898  7360 raft_consensus.cc:3055] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:35.355306  1640 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:35.370039  1644 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:35.271391  7406 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:35.370358  2314 leader_election.cc:336] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.985s (SENT)
I20260709 13:04:35.346309  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:35.317948  7351 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:35.422402  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.430982  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.262046  7377 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.437180  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:35.335443  7436 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:35.401590  2314 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:35.454906  1643 raft_consensus.cc:2410] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:04:35.400949  7360 raft_consensus.cc:740] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:35.458189  7304 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:35.263556  7249 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.333308  7465 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.433727  7258 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:35.333808  6987 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:35.483402  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 396338us
I20260709 13:04:35.483482  7505 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:35.401597  7486 raft_consensus.cc:3055] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:35.485003  7486 raft_consensus.cc:740] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:35.485759  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:35.486912  7404 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:35.486431  7486 consensus_queue.cc:260] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.487396  1612 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:35.488354  7404 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:35.488389  7486 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:35.488965  1612 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:04:35.433676  7527 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:04:35.490006  7404 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:35.490499  7404 raft_consensus.cc:697] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:04:35.465672  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 95281.8us
W20260709 13:04:35.492070  2314 leader_election.cc:336] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.022s (SENT)
I20260709 13:04:35.492811  2314 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:35.492237  7404 consensus_queue.cc:237] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.482785  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.490437  1612 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:35.494590  1642 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:35.496469  7132 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:35.448982  1572 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:35.499279  1572 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:04:35.471820  7238 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:35.506610  7531 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:04:35.510771  7417 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:35.512079  7411 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:35.512908  7411 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:35.513324  7411 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.513734  7411 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:35.518378  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.519225  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.519758  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.444269  7351 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.475394  7533 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:35.430001  7406 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.520571  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:35.529479  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:35.537750  1641 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:35.520107  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.527902  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.520722  1646 raft_consensus.cc:2410] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:04:35.539640  1645 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 11.
I20260709 13:04:35.539965  1648 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:04:35.540400  1643 raft_consensus.cc:2368] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 16. Current term is 17.
W20260709 13:04:35.540953  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:35.533334  1567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2583) took 1793 ms (client timeout 1799 ms). Trace:
W20260709 13:04:35.542115  1567 rpcz_store.cc:269] 0709 13:04:33.740129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.859058 (+118929us) service_pool.cc:224] Handling call
0709 13:04:35.533124 (+1674066us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54002233d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:35.533265 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":546554624}
I20260709 13:04:35.531224  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2536) took 1868 ms. Trace:
I20260709 13:04:35.544839  1716 rpcz_store.cc:276] 0709 13:04:33.662360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.177479 (+515119us) service_pool.cc:224] Handling call
0709 13:04:34.295844 (+118365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:34.333515 (+ 37671us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:34.333881 (+   366us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:34.333962 (+    81us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:34.334021 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.530944 (+1196923us) spinlock_profiling.cc:243] Waited 34.5 ms on lock 0x7b5400247a54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:35.531000 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:35.531157 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79348096}
W20260709 13:04:35.532488  1568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2579) took 1793 ms (client timeout 1595 ms). Trace:
W20260709 13:04:35.546562  1568 rpcz_store.cc:269] 0709 13:04:33.738988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.819449 (+ 80461us) service_pool.cc:224] Handling call
0709 13:04:35.532239 (+1712790us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b5400306954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:35.532403 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":483732352}
W20260709 13:04:35.526365  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.548012  7347 consensus_queue.cc:260] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:35.553720  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.554344  1642 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:35.554772  7347 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:35.555753  1642 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:35.556980  7523 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:35.557803  7412 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:35.524752  7468 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:35.558753  7360 consensus_queue.cc:260] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.559893  7360 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:35.551674  1687 leader_election.cc:336] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.823s (SENT)
W20260709 13:04:35.577201  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.532063  7152 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:35.530107  7465 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:35.530478  7415 raft_consensus.cc:3055] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:35.527562  7249 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:35.522699  7524 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:35.579473  7453 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.815s	user 0.080s	sys 0.099s
I20260709 13:04:35.582201  7415 raft_consensus.cc:740] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:04:35.613029  7453 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.081s	sys 0.100s
I20260709 13:04:35.549700  7533 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:35.691797  7539 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:04:35.691991  7406 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:35.693712  7415 consensus_queue.cc:260] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.694974  7415 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:35.695909  7463 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:35.680364  7439 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:35.713153  7495 log.cc:927] Time spent T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 1.109s	user 0.003s	sys 0.013s
W20260709 13:04:35.715060  7497 log.cc:927] Time spent T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.112s	user 0.000s	sys 0.017s
I20260709 13:04:35.718267  7533 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:35.718649  7377 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:35.720956  7508 raft_consensus.cc:3055] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:35.723901  7508 raft_consensus.cc:740] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:35.724980  7496 log.cc:927] Time spent T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Append to log took a long time: real 1.123s	user 0.001s	sys 0.015s
I20260709 13:04:35.724926  7508 consensus_queue.cc:260] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.726387  7409 raft_consensus.cc:993] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: : Instructing follower 066ee0109aba4465ba17fa65e14ef504 to start an election
I20260709 13:04:35.727001  7411 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.727574  7533 raft_consensus.cc:1081] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Signalling peer 066ee0109aba4465ba17fa65e14ef504 to start an election
I20260709 13:04:35.729228  7411 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:35.780784  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.854723  7347 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.855903  7347 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.858178  7152 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.858764  7321 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.859156  7152 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:35.860194  7321 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:35.790738  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 300051us
W20260709 13:04:35.792568  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.577898  1641 raft_consensus.cc:2410] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:04:35.854785  1566 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:35.795771  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.798063  1644 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 16: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:35.798620  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:35.799366  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 919) took 1850 ms. Trace:
I20260709 13:04:35.799845  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.804034  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:35.811604  1687 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.765s (SENT)
W20260709 13:04:35.812836  1714 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2535) took 2150 ms (client timeout 1577 ms). Trace:
I20260709 13:04:35.815999  1721 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: []
W20260709 13:04:35.818778  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 321542us
I20260709 13:04:35.823536  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:35.825125  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.824949  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:35.828786  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2942) took 1488 ms. Trace:
I20260709 13:04:35.835461  6636 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.837030  7075 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.835284  6987 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:35.839222  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.709400  7436 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:35.910347  1645 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 13: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:35.910737  1643 raft_consensus.cc:2393] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 13: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:35.849478  7351 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:35.906605  2340 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:35.906627  1640 raft_consensus.cc:2393] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:04:35.901125  1714 rpcz_store.cc:269] 0709 13:04:33.662091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.719055 (+ 56964us) service_pool.cc:224] Handling call
0709 13:04:35.812196 (+2093141us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b540014ffd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:35.812514 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":974909696}
I20260709 13:04:35.907266  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.908142  7075 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:35.902238  1718 rpcz_store.cc:276] 0709 13:04:34.340012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.849519 (+509507us) service_pool.cc:224] Handling call
0709 13:04:34.866193 (+ 16674us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:35.003081 (+136888us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.828087 (+825006us) spinlock_profiling.cc:243] Waited 88.2 ms on lock 0x7b540002e9d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:35.828558 (+   471us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":202815872}
I20260709 13:04:35.912102  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:35.912201  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:35.910995  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.726398  7508 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:35.915894  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.916236  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.918396  1647 raft_consensus.cc:2393] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 13: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:35.919459  7321 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:35.863222  7262 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.900235  1564 rpcz_store.cc:276] 0709 13:04:33.949214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.427125 (+477911us) service_pool.cc:224] Handling call
0709 13:04:34.478323 (+ 51198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:34.498011 (+ 19688us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:34.498069 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:34.498130 (+    61us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:34.498175 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.798525 (+1300350us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b5400013e54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:35.798694 (+   169us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:35.799065 (+   371us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44493184}
I20260709 13:04:35.912732  7152 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:35.940042  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:35.946381  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:35.950294  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 58260.6us
I20260709 13:04:35.948334  1646 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:35.952919  1648 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:35.957779  1645 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:35.971294  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:35.973887  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:35.974946  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.976517  7075 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:35.977308  2340 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:35.977499  1642 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:35.979182  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:35.980324  1642 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:35.982546  7452 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.983928  7452 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:35.985906  1686 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:35.987864  7152 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:35.988467  7152 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.988901  7152 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:35.990216  2340 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:04:35.990764  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:35.991572  2344 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 22.
W20260709 13:04:35.994484  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.973038  1646 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:36.006668  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:35.959517  6636 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:36.008154  1714 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
W20260709 13:04:36.009438  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 198118us
W20260709 13:04:36.010456  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.009999  7415 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:04:36.011812  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:36.012320  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.007290  7347 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:36.013450  1687 leader_election.cc:336] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.887s (SENT)
I20260709 13:04:36.023279  1646 raft_consensus.cc:1240] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:04:36.030931  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:36.035073  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:36.039144  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.001348  7452 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:36.045792  7295 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:04:36.048439  1717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2538) took 2385 ms (client timeout 1639 ms). Trace:
I20260709 13:04:36.051635  7436 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:36.053894  1719 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:36.063827  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2931) took 1770 ms. Trace:
I20260709 13:04:36.033181  1687 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:36.073938  1721 rpcz_store.cc:276] 0709 13:04:34.295129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.836086 (+540957us) service_pool.cc:224] Handling call
0709 13:04:35.815041 (+978955us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:35.907853 (+ 92812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.063478 (+155625us) spinlock_profiling.cc:243] Waited 943 ms on lock 0x7b540014ffd0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:36.063718 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2168763136}
I20260709 13:04:36.013945  7262 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:36.089594  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:04:36.069931  1717 rpcz_store.cc:269] 0709 13:04:33.662871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.797811 (+134940us) service_pool.cc:224] Handling call
0709 13:04:36.047948 (+2250137us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b5400209c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:36.048186 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":887032704}
I20260709 13:04:36.097070  1648 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:36.101462  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 88086.8us
I20260709 13:04:36.101843  7295 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:36.116788  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:36.117410  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 225578us
I20260709 13:04:36.125969  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:36.127615  2347 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:04:36.131351  7415 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.132535  7415 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:36.133781  7360 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:36.135476  7245 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.135876  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:36.136354  7245 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:36.136850  1567 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 22.
I20260709 13:04:36.138633  6636 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:36.139973  7415 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:36.142135  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:36.142426  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:36.148305  1639 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:36.144793  1568 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:36.145382  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:36.151193  2340 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:04:36.156983  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:36.158591  6636 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:04:36.159188  1566 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2590) took 2340 ms (client timeout 2518 ms). Trace:
I20260709 13:04:36.159412  6636 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.160087  6636 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:36.125348  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:36.159742  1566 rpcz_store.cc:269] 0709 13:04:33.818579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.034515 (+215936us) service_pool.cc:224] Handling call
0709 13:04:36.158882 (+2124367us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b5400011b54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:36.159091 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3586272640}
I20260709 13:04:36.145531  1612 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:36.166266  7262 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:36.175848  7067 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:36.182914  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 64449.4us
W20260709 13:04:36.190188  1571 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.002s	sys 0.010s
I20260709 13:04:36.203755  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:36.204435  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 59151.8us
W20260709 13:04:36.211704  1571 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.003s	sys 0.011s
I20260709 13:04:36.223134  7541 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:36.210264  7262 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:36.234715  6636 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:36.238752  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:36.259589  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:36.262454  7550 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.095s	user 0.000s	sys 0.002s
I20260709 13:04:36.229377  7360 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:04:36.266301  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2586) took 2525 ms. Trace:
I20260709 13:04:36.266779  1569 rpcz_store.cc:276] 0709 13:04:33.740869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.095808 (+354939us) service_pool.cc:224] Handling call
0709 13:04:34.275530 (+179722us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:34.297840 (+ 22310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:34.297885 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:34.308187 (+ 10302us) log.cc:844] Serialized 26 byte log entry
0709 13:04:35.787568 (+1479381us) spinlock_profiling.cc:243] Waited 2.62 ms on lock 0x7b180024f580. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a250d1 00007f8248e40a0f
0709 13:04:36.050081 (+262513us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:36.050131 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.050293 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:36.265753 (+215460us) raft_consensus.cc:1685] finished
0709 13:04:36.265777 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:36.266088 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20964,"raft.queue_time_us":17274,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":18008,"spinlock_wait_cycles":6016384,"thread_start_us":231484,"threads_started":1,"wal-append.queue_time_us":1834516}
I20260709 13:04:36.270740  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:36.271886  1718 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 22.
I20260709 13:04:36.273962  7295 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:36.211226  7415 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:04:36.279034  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2934) took 1979 ms. Trace:
I20260709 13:04:36.279606  1719 rpcz_store.cc:276] 0709 13:04:34.299519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.847625 (+548106us) service_pool.cc:224] Handling call
0709 13:04:36.053056 (+1205431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:36.088421 (+ 35365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.278301 (+189880us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400209c50. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:36.278918 (+   617us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2770027776}
I20260709 13:04:36.281163  1648 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:36.284478  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:36.285467  1567 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:36.287153  7277 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:36.289904  6636 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:36.290246  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:36.292514  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2540) took 1016 ms. Trace:
I20260709 13:04:36.293128  1647 rpcz_store.cc:276] 0709 13:04:35.276262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.331391 (+ 55129us) service_pool.cc:224] Handling call
0709 13:04:36.292128 (+960737us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b54002dc154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:36.292407 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1261737728}
I20260709 13:04:36.295853  1612 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:36.257136  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 54029.7us
I20260709 13:04:36.300172  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:36.303105  2344 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:36.303066  1645 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:36.304230  1640 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:04:36.310020  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:36.309352  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:36.315510  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:36.310587  1687 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:36.319025  7436 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:36.319906  7436 raft_consensus.cc:697] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 13 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:36.320410  1567 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:36.313432  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:36.329401  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 47973us
I20260709 13:04:36.329844  1719 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:36.333752  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:36.316648  7360 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.334740  1714 raft_consensus.cc:1275] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:36.342803  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 924) took 1458 ms. Trace:
I20260709 13:04:36.343011  7295 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.343381  2313 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:36.327862  7245 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:36.343778  7295 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:36.344468  7283 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:36.344146  7545 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.327s	user 0.000s	sys 0.006s
W20260709 13:04:36.342851  1538 connection.cc:256] RPC call timeout handler was delayed by -1.1119638210000176s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
I20260709 13:04:36.345839  1566 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
W20260709 13:04:36.346223  1538 connection.cc:256] RPC call timeout handler was delayed by -1.0633638209999958s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1559s for a response
I20260709 13:04:36.346505  7295 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:36.346704  1538 connection.cc:256] RPC call timeout handler was delayed by -1.0561638209999842s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
I20260709 13:04:36.346941  7502 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:36.347548  7502 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.348336  7502 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:36.350903  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.343392  1565 rpcz_store.cc:276] 0709 13:04:34.863388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.864465 (+  1077us) service_pool.cc:224] Handling call
0709 13:04:34.864967 (+   502us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:34.865412 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:34.865457 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:35.777522 (+912065us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b2c00000658. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824777ea11 00007f824778e372 00007f8248ec0a01 00007f8248e8543a 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2
0709 13:04:35.779235 (+  1713us) log.cc:844] Serialized 26 byte log entry
0709 13:04:36.243255 (+464020us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:36.243287 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.253511 (+ 10224us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:36.321597 (+ 68086us) raft_consensus.cc:1685] finished
0709 13:04:36.321620 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:36.321984 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4751,"raft.queue_time_us":480,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":10892,"spinlock_wait_cycles":24018304,"thread_start_us":463304,"threads_started":1,"wal-append.queue_time_us":538776}
W20260709 13:04:36.351481  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.342002  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:36.356127  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 48149.7us
I20260709 13:04:36.314453  7152 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:36.366884  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.354179  1612 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:36.368692  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 46531.8us
I20260709 13:04:36.369462  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:36.341480  6636 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:04:36.382092  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:36.382431  1537 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:36.361805  7452 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:36.390336  1567 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:36.321440  7436 consensus_queue.cc:237] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:36.407100  1721 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 12: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:36.408838  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:36.409533  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:36.410874  1570 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:36.412688  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:36.414168  1573 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:36.414525  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:36.394467  7452 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.418182  7452 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:36.422137  7360 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:36.427040  1716 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:04:36.428211  7360 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:36.436028  1569 raft_consensus.cc:2393] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:36.428925  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:36.434330  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:36.378392  7351 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:36.446820  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:36.448333  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:36.461623  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 107341us
I20260709 13:04:36.455018  1565 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:36.468612  1721 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 15: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:36.475436  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:36.454674  7295 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.476513  1567 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:36.482874  7152 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:36.487424  1611 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:36.490188  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:36.476835  7295 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:36.468886  1687 leader_election.cc:336] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.062s (SENT)
I20260709 13:04:36.491309  1611 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:36.459501  7351 raft_consensus.cc:697] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:36.500679  1714 raft_consensus.cc:2410] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 13:04:36.503050  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:36.503526  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.490209  7245 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:36.467314  7067 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.504540  7245 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.505122  7245 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:36.505154  7067 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:36.508167  7067 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:36.509138  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:36.510006  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:36.511561  1567 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:40656
I20260709 13:04:36.512446  1567 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:36.512893  1567 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:36.517313  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2542) took 1175 ms. Trace:
I20260709 13:04:36.526836  7067 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:36.532333  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:36.492856  7321 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.547578  7361 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:36.550132  7094 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:36.563050  7351 consensus_queue.cc:237] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:36.569766  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 85600.1us
I20260709 13:04:36.548573  7361 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:36.553592  1687 leader_election.cc:336] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.141s (SENT)
I20260709 13:04:36.580224  2344 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
W20260709 13:04:36.582966  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 82357.5us
I20260709 13:04:36.589146  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:36.526158  1571 rpcz_store.cc:276] 0709 13:04:35.342062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.401525 (+ 59463us) service_pool.cc:224] Handling call
0709 13:04:35.417917 (+ 16392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:35.418151 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:36.253073 (+834922us) log.cc:844] Serialized 14 byte log entry
0709 13:04:36.462278 (+209205us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:36.474339 (+ 12061us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:36.474594 (+   255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.474826 (+   232us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:36.517227 (+ 42401us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3293,"prepare.queue_time_us":16937,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":12451,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":24942,"thread_start_us":984016,"threads_started":2}
I20260709 13:04:36.600080  1687 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:36.592795  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:36.604113  1719 raft_consensus.cc:2393] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 13: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:04:36.604417  7557 log.cc:927] Time spent T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:04:36.582422  7321 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:36.614291  6636 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:36.625910  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:36.616032  1611 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:36.595674  7152 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:04:36.634572  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.639031  1567 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:36.643446  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:36.647199  2346 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:36.648247  1537 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:36.652942  7415 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.641978  7152 raft_consensus.cc:697] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 20 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:36.653759  7360 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:04:36.672780  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 119219us
W20260709 13:04:36.668465  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 70227us
I20260709 13:04:36.661303  7094 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:36.687605  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 175151us
I20260709 13:04:36.690665  1567 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:36.693893  7508 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:36.694440  7508 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:36.694841  7508 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:36.702099  1538 leader_election.cc:336] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.693s (SENT)
I20260709 13:04:36.706910  1719 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:36.702416  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:36.708724  2344 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:36.700410  7067 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.717949  7560 consensus_queue.cc:1059] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:36.719770  7560 raft_consensus.cc:3055] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:36.720232  7560 raft_consensus.cc:740] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:36.726657  7560 consensus_queue.cc:260] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:36.728977  7560 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:36.730193  7152 consensus_queue.cc:237] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:36.738699  2346 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:36.725175  7245 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:36.692673  7415 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:36.770535  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 187006us
I20260709 13:04:36.704357  7360 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:36.737360  7295 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:36.783038  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.790158  6636 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:36.791206  6636 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:36.791659  7360 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:36.793053  7559 log.cc:927] Time spent T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:04:36.794246  1537 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:36.767740  7067 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:36.804687  7067 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:36.806938  6636 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:36.828944  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:36.836901  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 138549us
W20260709 13:04:36.847527  7277 consensus_meta.cc:287] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.560s	user 0.018s	sys 0.000s
I20260709 13:04:36.848938  1720 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:36.850796  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:36.853884  7377 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.854918  1569 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16.
W20260709 13:04:36.849018  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 67170.8us
I20260709 13:04:36.868825  7361 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:36.869266  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 580) took 3430 ms. Trace:
I20260709 13:04:36.869782  1720 rpcz_store.cc:276] 0709 13:04:33.438735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.064402 (+625667us) service_pool.cc:224] Handling call
0709 13:04:34.064879 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:36.849811 (+2784932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.868917 (+ 19106us) spinlock_profiling.cc:243] Waited 2.78 s on lock 0x7b5400247f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:36.869178 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6397926144}
I20260709 13:04:36.870191  1537 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:36.868929  7556 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.322s	user 0.001s	sys 0.002s
I20260709 13:04:36.826372  7295 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:36.886497  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 100343us
W20260709 13:04:36.903858  1538 leader_election.cc:336] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.732s (SENT)
I20260709 13:04:36.889026  7277 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.904572  1538 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:36.896530  7415 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:04:36.901620  7245 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:36.923768  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 53660.1us
W20260709 13:04:36.926770  7558 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.317s	user 0.002s	sys 0.000s
I20260709 13:04:36.927425  7321 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:36.936316  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 46179.6us
I20260709 13:04:36.937207  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:36.937959  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:36.941017  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:36.936388  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:36.948899  7245 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.949579  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:36.950531  2348 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:04:36.950737  1717 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:36.950865  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:36.952797  1720 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:36.953305  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:36.943544  7502 consensus_meta.cc:287] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.594s	user 0.010s	sys 0.006s
I20260709 13:04:36.854477  7377 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:36.954499  7555 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.425s	user 0.004s	sys 0.000s
I20260709 13:04:36.959151  1716 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:36.959276  7295 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:36.960011  7295 raft_consensus.cc:697] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:36.963084  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:36.943710  7277 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:36.990177  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:36.982959  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:36.998788  1721 raft_consensus.cc:2393] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:36.961055  7295 consensus_queue.cc:237] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:37.003993  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.008090  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:37.005383  1714 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
W20260709 13:04:37.011592  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 68823.9us
I20260709 13:04:36.943995  6995 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 13:04:36.949621  7245 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:36.947230  7436 consensus_queue.cc:1059] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:04:37.027846  1642 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:36.985019  7351 consensus_queue.cc:1059] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
W20260709 13:04:37.025485  1538 leader_election.cc:336] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.862s (SENT)
I20260709 13:04:36.985245  7094 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:04:37.030027  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:36.962224  7502 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:37.031005  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 69371.8us
I20260709 13:04:37.031208  2314 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:36.989689  6987 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:04:37.017335  7152 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:37.044092  7452 consensus_meta.cc:287] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.625s	user 0.013s	sys 0.004s
I20260709 13:04:37.044756  7452 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:37.047412  7245 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:37.049031  7502 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:37.050246  1722 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 18: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:37.051159  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:37.054131  1718 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:37.054210  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:37.064697  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:37.065331  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.054601  1716 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:37.055503  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:37.066607  1715 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 12: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:37.060015  1611 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:37.059568  1686 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:37.067910  6636 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:37.068429  6636 raft_consensus.cc:697] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:37.062678  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.069576  1642 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:37.071534  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:37.072258  1721 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:37.075402  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:37.075695  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:37.076248  1647 raft_consensus.cc:2376] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10. Re-sending same reply.
I20260709 13:04:37.076329  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:37.078043  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.057258  7152 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:37.086149  7377 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:37.037106  7520 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:04:37.092039  6636 consensus_queue.cc:237] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:37.094130  1720 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:37.096848  7569 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:37.097280  7277 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:37.098676  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:37.099730  1719 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:37.102252  7568 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:37.104055  7562 log.cc:927] Time spent T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:04:37.081276  7152 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:37.106572  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 82454.4us
I20260709 13:04:37.120458  7520 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:37.121608  1714 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:37.121943  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:37.125584  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:37.126092  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 91521.1us
I20260709 13:04:37.128769  6987 raft_consensus.cc:2764] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:04:37.139753  7185 raft_consensus.cc:3055] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:37.140254  7185 raft_consensus.cc:740] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:37.142076  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:37.152438  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 132564us
W20260709 13:04:37.153446  1538 leader_election.cc:336] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.889s (SENT)
I20260709 13:04:37.153914  1538 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:37.042941  7508 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:37.175571  7508 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:37.141158  7185 consensus_queue.cc:260] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:37.181914  7245 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:37.182772  1640 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:37.183414  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.183537  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.184131  1642 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:37.184384  1648 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:37.185658  1643 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:37.186110  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.186425  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.191928  7452 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:37.192479  1645 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
W20260709 13:04:37.193962  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 57854.6us
I20260709 13:04:37.197428  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:37.198323  1714 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:37.198447  7415 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.199563  7415 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:37.186527  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:37.203361  1647 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:37.204110  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.182258  1611 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:37.219933  1719 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:37.227744  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:37.228689  1567 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:37.223124  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 137552us
I20260709 13:04:37.236912  1648 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:37.225476  6987 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:37.245597  1537 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:37.245431  7508 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:04:37.240363  7560 consensus_meta.cc:287] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.506s	user 0.008s	sys 0.008s
I20260709 13:04:37.235637  1611 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:37.252139  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.255736  7576 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:37.262348  7576 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.262920  7576 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:37.248118  6987 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.266115  6987 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:37.268877  7185 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:37.281611  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 58816.3us
I20260709 13:04:37.284139  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:37.286239  1648 raft_consensus.cc:1275] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:37.282864  1611 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:37.288997  7560 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.289799  7560 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:37.291898  1714 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 23.
I20260709 13:04:37.293452  2313 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:37.252674  7415 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:37.310719  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.315574  7573 raft_consensus.cc:2764] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:04:37.301586  7575 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:37.318358  7575 raft_consensus.cc:697] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:37.319357  7575 consensus_queue.cc:237] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:37.319900  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.320652  1648 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:37.308668  7578 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:37.322680  7578 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.325688  7578 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:37.325955  7436 raft_consensus.cc:3055] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:37.326393  7436 raft_consensus.cc:740] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:37.327474  7436 consensus_queue.cc:260] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:37.328864  7436 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:37.307734  7245 consensus_queue.cc:1059] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:04:37.330925  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.331679  1648 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:37.336655  1611 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:37.337921  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:37.357828  1640 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:37.358767  1647 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:37.361778  7508 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.362581  7508 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:37.365423  7508 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:37.377758  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.372728  7560 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:37.366101  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.390197  1567 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 23.
I20260709 13:04:37.390570  1645 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:37.369230  1643 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 14: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:04:37.401862  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.402781  7573 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:37.403380  7573 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:37.403805  7573 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:37.405779  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 94173.3us
I20260709 13:04:37.406848  1611 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:37.407824  7572 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:37.408278  7572 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.408627  7572 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:37.371222  7579 raft_consensus.cc:3055] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:37.410085  7579 raft_consensus.cc:740] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:37.412199  7579 consensus_queue.cc:260] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:37.414227  7579 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:37.416201  7094 raft_consensus.cc:993] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b: : Instructing follower 066ee0109aba4465ba17fa65e14ef504 to start an election
I20260709 13:04:37.416765  7094 raft_consensus.cc:1081] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Signalling peer 066ee0109aba4465ba17fa65e14ef504 to start an election
I20260709 13:04:37.431612  1640 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:37.435075  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:37.435843  1647 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:37.439985  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.441939  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:37.442545  1645 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:37.443890  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.445093  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.445679  1641 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:37.446257  1642 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 13 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:37.446892  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.447239  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:37.447464  1641 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:04:37.447796  1642 raft_consensus.cc:2376] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12. Re-sending same reply.
I20260709 13:04:37.445497  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.451469  1644 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
W20260709 13:04:37.469092  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 66947.3us
I20260709 13:04:37.467433  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:37.473502  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.486691  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.522342  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.472330  1641 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:37.504884  2347 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:04:37.513126  1567 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:04:37.472091  7508 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.365350  6987 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:37.521359  1647 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:37.521895  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:37.527161  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 53732.3us
I20260709 13:04:37.525094  1567 pending_rounds.cc:85] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 8
I20260709 13:04:37.536845  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.538092  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.544159  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.544502  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.546077  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:37.547009  1643 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:37.532608  1567 pending_rounds.cc:107] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:04:37.550058  7245 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.551185  6987 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:37.550196  1567 raft_consensus.cc:2889] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:37.534286  7295 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.550800  7245 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:37.553092  7295 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:37.533890  7508 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:37.555467  7245 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:37.556357  7508 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:37.536688  7067 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.558089  1644 raft_consensus.cc:2410] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:04:37.558311  7067 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:37.560889  7295 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:37.561333  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.562018  1640 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:37.566231  7572 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:37.569269  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:37.570065  1644 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:37.573557  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:37.574194  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:37.574985  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:37.575210  1647 raft_consensus.cc:3055] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:37.575959  1647 raft_consensus.cc:740] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:37.576141  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.576956  1642 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 14: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:37.577844  1537 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:37.581207  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.582345  1640 raft_consensus.cc:2368] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15.
I20260709 13:04:37.578629  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.583540  1642 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:37.584144  7572 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:37.584450  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.585176  1642 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16.
I20260709 13:04:37.586421  7576 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:37.588127  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 52406.4us
I20260709 13:04:37.590826  2347 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:04:37.591107  7245 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:37.591652  7245 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.592023  7245 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:37.593834  7508 consensus_queue.cc:1048] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:37.597363  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:37.586426  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.598979  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:37.598485  7277 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:37.599179  1714 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:37.598111  2344 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 10.
I20260709 13:04:37.601329  7152 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:37.601539  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.602557  1642 raft_consensus.cc:2410] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:04:37.603554  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.604758  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.605329  1642 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:37.587399  1687 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:37.601749  1538 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.091s (SENT) [suppressed 70 similar messages]
I20260709 13:04:37.606359  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.610360  1640 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:37.577221  1647 consensus_queue.cc:260] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:37.607038  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.611910  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:37.611931  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.612792  1639 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:37.610764  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.608906  1645 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:37.614496  2347 raft_consensus.cc:2393] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 13: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:37.560891  7067 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:37.616946  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:37.621243  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:37.621996  1567 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:04:37.623119  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:37.626744  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:37.627097  1567 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:37.627341  1714 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:37.626899  2314 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:37.621176  7295 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:04:37.629098  7565 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:37.623585  7152 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:37.629570  7565 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:37.629942  7152 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.630199  7565 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:37.630451  7152 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:37.633003  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.634876  1714 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:37.635069  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:37.637142  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:37.640420  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.640004  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.642376  7572 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:37.643000  7572 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:37.644802  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:37.645059  7572 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:37.646602  7578 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:37.650010  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:37.650791  1571 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:37.651659  1648 raft_consensus.cc:2368] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16.
I20260709 13:04:37.652828  7578 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:37.654024  1538 leader_election.cc:336] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.208s (SENT)
I20260709 13:04:37.654670  1538 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:37.656229  7579 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:37.657301  1642 raft_consensus.cc:2393] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 16: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:37.656461  1569 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 12: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:37.660953  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:37.662065  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:37.665824  1687 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:37.666982  7277 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:37.667510  7277 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.667901  7277 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:37.679075  1644 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:37.681613  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:37.681952  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:37.682375  1565 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:37.682859  1566 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:37.683574  1571 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:04:37.683956  1571 pending_rounds.cc:85] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 9
I20260709 13:04:37.684294  1571 pending_rounds.cc:107] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:04:37.684551  1571 raft_consensus.cc:2889] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:37.686273  1647 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:04:37.687084  1573 raft_consensus.cc:2410] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:04:37.688045  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:37.682382  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:37.658699  7576 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:37.689567  1568 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
W20260709 13:04:37.691174  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.692631  6987 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:37.693414  1612 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:37.703668  7578 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:37.704589  1612 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:37.705794  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 51819.6us
W20260709 13:04:37.706209  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:37.706522  1538 leader_election.cc:336] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.304s (SENT)
I20260709 13:04:37.707000  7185 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:37.707563  7185 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:37.708022  7185 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:37.708670  7578 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.709254  7578 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:37.709877  7573 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:37.711148  1641 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:37.712218  7573 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:37.712391  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.712535  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.713030  1648 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:37.713392  1640 raft_consensus.cc:2376] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13. Re-sending same reply.
I20260709 13:04:37.713575  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.714462  1643 raft_consensus.cc:2393] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 13: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:37.713438  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:37.669682  1687 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:37.738288  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:37.740528  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.741058  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.741452  1569 raft_consensus.cc:1275] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:37.743258  1644 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 12: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:37.743139  1647 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:37.747308  1566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:40656
I20260709 13:04:37.747995  1566 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:37.748360  1566 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:37.748898  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.743615  7415 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:37.752995  7277 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:37.741410  7576 consensus_queue.cc:1059] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:04:37.754741  7576 raft_consensus.cc:3055] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:37.755100  7576 raft_consensus.cc:740] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:37.755968  7576 consensus_queue.cc:260] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:37.765854  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.769038  1612 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:37.768528  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.766836  1538 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:37.769037  1644 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:04:37.772174  1573 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:37.770401  1643 raft_consensus.cc:2376] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16. Re-sending same reply.
I20260709 13:04:37.760836  1566 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:37.765087  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.748291  7578 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:37.783314  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.783565  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.790668  1644 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 16: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:37.744989  7351 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:37.783754  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 114315us
I20260709 13:04:37.792474  1569 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 14. Current term is 15. Ops: [12.8-14.9]
I20260709 13:04:37.757352  7576 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:37.756350  7277 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:37.796460  1647 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:37.798910  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.776754  7573 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:37.802165  1643 raft_consensus.cc:2410] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:04:37.803987  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.805817  1644 raft_consensus.cc:2410] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:04:37.806427  1538 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:37.764264  7569 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
W20260709 13:04:37.808561  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 62864.3us
I20260709 13:04:37.809373  1612 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:37.811769  1569 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:37.814194  7569 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:37.814823  7569 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.815562  7569 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:37.763285  7568 consensus_queue.cc:1048] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:04:37.817704  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:37.818326  1644 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:37.793452  7579 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:37.823027  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:37.823577  7579 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.824051  7579 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:37.823038  7415 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:37.824163  1573 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:37.825891  7415 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:37.823953  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:37.756074  7152 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:37.823743  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:37.828819  1687 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:37.831092  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:37.831518  1571 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:37.834194  1566 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:37.839195  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:37.843022  7277 raft_consensus.cc:2764] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:04:37.846478  1643 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 14. Current term is 15. Ops: [12.8-14.9]
I20260709 13:04:37.847518  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:37.857050  1573 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 12: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:37.855662  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:37.860626  1571 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:37.861423  1687 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:37.864181  1718 raft_consensus.cc:2393] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:37.870894  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:37.871877  1565 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:37.836926  7351 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:37.872970  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:37.877079  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 69700.6us
W20260709 13:04:37.875365  1685 leader_election.cc:336] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.062s (SENT)
I20260709 13:04:37.764252  7565 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:37.891633  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.892513  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:37.877847  1569 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:37.895782  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:37.897303  1647 raft_consensus.cc:2393] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 13: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:37.886340  1685 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:37.900529  1538 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:37.892951  1643 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
W20260709 13:04:37.902092  7593 log.cc:927] Time spent T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Append to log took a long time: real 0.076s	user 0.002s	sys 0.000s
I20260709 13:04:37.906514  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.907297  1642 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
W20260709 13:04:37.905122  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 95765.9us
W20260709 13:04:37.906634  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 62141.3us
I20260709 13:04:37.910907  6636 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:37.912182  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:37.909684  1644 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:04:37.914044  1564 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:37.910238  7568 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:37.926409  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:37.927347  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.926436  2347 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:37.927398  1571 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:37.917179  7277 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:04:37.935467  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 48064.2us
W20260709 13:04:37.936831  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 61519.4us
I20260709 13:04:37.939265  7277 raft_consensus.cc:697] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:37.940657  7277 consensus_queue.cc:237] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:37.936267  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:37.946395  1538 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:37.925195  7568 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.948906  1569 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:37.953756  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:37.954463  1643 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:37.956007  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 46336.9us
I20260709 13:04:37.931960  7245 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:37.962494  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.950856  7576 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:37.969892  1567 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:37.966938  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:37.950688  7568 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:37.971828  7245 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:37.973599  1538 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:37.975423  1538 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:37.958631  7295 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:37.976317  7576 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:37.976980  7576 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:37.968614  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:37.978652  7295 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:37.979427  7295 raft_consensus.cc:697] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:37.980387  7295 consensus_queue.cc:237] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:37.986661  1612 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:37.989981  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:37.991228  1569 raft_consensus.cc:2410] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 13:04:37.991295  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 79059.6us
W20260709 13:04:37.997226  1685 leader_election.cc:336] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.612s (SENT)
I20260709 13:04:38.003166  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:38.003926  1642 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:38.005234  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:38.010216  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:38.011227  1645 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:38.012454  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:38.013183  1645 raft_consensus.cc:2376] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16. Re-sending same reply.
I20260709 13:04:38.022533  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.023687  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:38.024804  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.025440  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:38.026371  1645 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:37.990017  7245 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:38.023258  1569 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:37.964358  7572 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:04:37.997753  7578 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:38.034564  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:38.035516  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:38.042794  1647 raft_consensus.cc:2368] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12.
I20260709 13:04:38.043164  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:38.043298  1641 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:38.042825  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:38.049877  1640 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 16: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:38.050768  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:38.051875  1642 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:38.053725  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:38.017764  7351 raft_consensus.cc:2764] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:04:38.064554  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.067659  1647 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:04:38.069727  7094 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:38.071848  7185 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:38.074811  7185 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:38.081879  1643 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:38.050217  7152 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:38.044691  1645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:55704
I20260709 13:04:38.160961  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:38.117870  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.164680  1571 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:38.128196  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 152788us
W20260709 13:04:38.102391  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 79898us
W20260709 13:04:38.162653  1685 leader_election.cc:336] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.584s (SENT)
I20260709 13:04:38.166889  1538 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:38.174183  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 110592us
I20260709 13:04:38.146028  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.176805  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:38.111258  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 118251us
W20260709 13:04:38.114709  7598 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:04:38.123596  7590 log.cc:927] Time spent T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 13:04:38.185034  1642 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:04:38.181295  7415 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:38.167932  1645 raft_consensus.cc:606] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:38.187160  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:38.192643  2344 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 21.
I20260709 13:04:38.188202  1564 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
W20260709 13:04:38.189491  2314 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.025s (SENT)
I20260709 13:04:38.193933  2314 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:38.195758  1645 raft_consensus.cc:614] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:38.196585  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:38.196857  1612 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:38.197674  1571 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:38.162708  7565 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:38.197799  1685 leader_election.cc:336] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.715s (SENT)
I20260709 13:04:38.195374  6636 consensus_queue.cc:1059] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:04:38.198493  1686 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:38.210402  6636 raft_consensus.cc:3055] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:38.211568  6636 raft_consensus.cc:740] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:38.061038  7600 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.136s
I20260709 13:04:38.212471  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.206089  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.086970  7578 consensus_queue.cc:1048] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.121s
I20260709 13:04:38.076507  7436 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.120s
I20260709 13:04:38.075591  7568 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:38.223873  7568 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:38.225351  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.226694  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.227577  1538 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:38.227923  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:38.228672  7602 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:38.229010  1538 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:38.212001  7351 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:38.212476  6636 consensus_queue.cc:260] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:38.230480  1612 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:38.212085  7605 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:38.232841  7579 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:38.207301  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 202389us
I20260709 13:04:38.218464  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.209426  1569 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 14: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:38.207273  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.237334  7094 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.206949  1685 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:38.199213  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.183506  7067 consensus_queue.cc:1059] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:04:38.234162  1567 raft_consensus.cc:2393] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 13: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:38.217245  1564 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:38.241350  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.242131  1564 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:38.238801  1571 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:38.230520  7568 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:38.244688  7094 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:38.247792  7568 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:38.249315  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:38.244616  7605 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:38.220196  7607 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:38.204555  7576 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:38.254901  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:38.286983  1570 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:04:38.260272  6636 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:38.264361  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 74903.6us
W20260709 13:04:38.264528  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 58447.1us
I20260709 13:04:38.266077  1566 raft_consensus.cc:2368] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16.
I20260709 13:04:38.267941  1568 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:38.255041  1612 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:38.253533  1565 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
W20260709 13:04:38.270917  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 73108.2us
W20260709 13:04:38.270256  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:38.296593  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.299374  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:38.299999  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:38.300148  1564 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:38.304052  1642 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:04:38.304484  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:38.305358  1643 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:38.305740  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:38.307199  1645 raft_consensus.cc:2410] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:04:38.248539  7568 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:38.270619  7604 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:38.308130  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 53168.3us
I20260709 13:04:38.259253  7607 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:38.309028  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:38.264766  7571 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:38.254420  7185 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:38.255034  7067 raft_consensus.cc:3055] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:04:38.315681  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 86713.7us
I20260709 13:04:38.267737  7579 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:38.267756  7576 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:38.315873  1611 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:38.315312  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:38.319454  1639 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:38.318082  7608 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:38.318181  7578 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:38.332424  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.333149  1567 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:38.334308  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:38.335115  1571 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 12: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:38.335280  1644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:55704
I20260709 13:04:38.335880  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.336592  1571 raft_consensus.cc:2368] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12.
I20260709 13:04:38.337318  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:38.338001  1685 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.659s (SENT)
I20260709 13:04:38.313416  7600 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:38.312099  7607 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:38.351682  1569 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:38.259862  7569 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:38.357421  7094 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:38.358071  1573 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 18: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:38.358834  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.314842  7067 raft_consensus.cc:740] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:38.342875  1643 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:04:38.345996  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:38.349309  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.350209  1639 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:38.308732  7575 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.351887  1714 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:04:38.315063  7185 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:38.353473  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.360177  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.364636  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:38.365794  1567 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:38.376132  1564 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17.
I20260709 13:04:38.375797  1611 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:38.385108  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:38.385815  1566 raft_consensus.cc:2376] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16. Re-sending same reply.
I20260709 13:04:38.385818  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.358531  7569 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:38.388430  1538 leader_election.cc:400] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15.
I20260709 13:04:38.383988  1687 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:38.388805  1538 leader_election.cc:403] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:38.320065  7573 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:38.387357  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.366150  7436 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.368520  1612 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:38.390020  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.390136  1568 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:38.400630  1565 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:38.400987  1564 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:38.367729  1569 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 16: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:38.391867  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.399652  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:38.393052  6636 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:38.409355  7436 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:38.390884  6987 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:38.387216  7575 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:38.414428  1573 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:38.413717  1644 raft_consensus.cc:606] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:38.421128  1644 raft_consensus.cc:614] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:38.421839  1564 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:04:38.423808  1566 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:38.384402  7185 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:38.425374  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 56935.7us
I20260709 13:04:38.426321  1612 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:38.430136  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 79544.4us
I20260709 13:04:38.399467  7571 consensus_queue.cc:1059] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:04:38.434154  2347 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:38.434469  7579 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15.
I20260709 13:04:38.434707  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.435391  1572 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:38.436368  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.436725  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:38.437108  1572 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 21: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:38.437355  1718 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:38.414016  7067 consensus_queue.cc:260] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:38.439453  1714 pending_rounds.cc:85] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 8
I20260709 13:04:38.439584  7067 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:38.439849  1714 pending_rounds.cc:107] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:04:38.440164  1714 raft_consensus.cc:2889] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:38.426369  7573 raft_consensus.cc:697] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 15 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:38.442593  7568 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:38.443154  7573 consensus_queue.cc:237] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:38.443728  1573 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:38.421407  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.447221  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.453547  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.454572  1571 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:38.447424  1570 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:38.457666  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.413450  7436 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:38.463707  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:38.465849  1567 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:04:38.465320  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 47058.7us
I20260709 13:04:38.466645  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.469756  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:38.361588  7600 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:38.473539  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:38.477063  1721 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:38.473810  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:38.474439  1718 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 18: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:38.445619  7568 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:38.460250  7094 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:04:38.483276  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 77190.3us
I20260709 13:04:38.485121  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:38.485476  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:38.486395  1719 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:38.486545  1715 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:04:38.488772  1687 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:38.449098  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.445859  6987 raft_consensus.cc:697] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 15 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:38.491034  6987 consensus_queue.cc:237] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:38.493952  1611 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:38.494519  7575 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:38.500782  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:38.502398  6636 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:38.502878  1718 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:38.506795  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 80520.4us
I20260709 13:04:38.508561  1573 raft_consensus.cc:1275] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:38.511798  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:38.512964  1721 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:38.516909  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:38.519052  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:38.519243  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:38.520324  1719 raft_consensus.cc:2410] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:04:38.523388  6636 consensus_queue.cc:1059] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:38.482721  7578 consensus_queue.cc:1059] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:04:38.517907  2347 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:38.505142  7436 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.534417  1612 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:38.540303  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:04:38.541671  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.542809  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:38.543851  2347 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:38.547612  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:38.549675  7615 log.cc:927] Time spent T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:04:38.551118  7600 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:38.551949  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:38.548700  1721 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:38.555924  1719 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:38.515173  7351 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
W20260709 13:04:38.562315  1537 connection.cc:256] RPC call timeout handler was delayed by -1.0847824790000686s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15100000000000002s for a response
I20260709 13:04:38.569999  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:38.554121  7152 raft_consensus.cc:3055] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:38.548403  7436 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:38.607954  7604 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:04:38.583717  7553 consensus_queue.cc:1059] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.111s
I20260709 13:04:38.682251  7553 raft_consensus.cc:3055] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:38.682713  7553 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:38.683580  7553 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:38.684895  7553 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:38.546806  7094 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:04:38.686427  1570 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 12. Current term is 13. Ops: []
W20260709 13:04:38.590024  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 65514.9us
I20260709 13:04:38.606148  1685 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:38.606460  6636 consensus_queue.cc:1059] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
W20260709 13:04:38.696065  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 127129us
I20260709 13:04:38.698457  1612 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:38.699599  1687 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:38.701954  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:38.705360  1722 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:38.578471   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:38.602531  7605 raft_consensus.cc:2764] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:04:38.720291  1612 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:38.573583  7608 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:38.601814  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:38.726404  1715 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:38.676322  7614 raft_consensus.cc:3055] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:38.739329  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:38.742142  1713 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17.
W20260709 13:04:38.743594  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 44081.2us
I20260709 13:04:38.744364  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:38.686977  7152 raft_consensus.cc:740] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:38.736056  2314 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:38.558101  7575 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.148s
I20260709 13:04:38.586189  7618 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:38.590535  1719 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:04:38.559150  7600 consensus_queue.cc:1059] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.151s
I20260709 13:04:38.704298  7572 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:38.583425  7185 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:38.757287  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 142231us
W20260709 13:04:38.768610  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 226034us
I20260709 13:04:38.743153  7277 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.706965  7351 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
W20260709 13:04:38.779712  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 173553us
I20260709 13:04:38.781999  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:38.751334  7614 raft_consensus.cc:740] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:38.782757  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:38.788740  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 186945us
I20260709 13:04:38.743273  7067 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:04:38.767927  7602 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:38.770107  7608 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:38.786576  7565 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:38.717842  7560 consensus_queue.cc:1059] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:04:38.804381  2344 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:04:38.805338  1717 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:04:38.820679  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:38.835201  1538 leader_election.cc:336] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.560s (SENT)
I20260709 13:04:38.874653  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:38.835603  1611 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:38.875550  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:38.876436  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:38.876878  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:38.820634  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:38.848567  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:38.877867  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:38.856364  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:38.862522  1720 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:38.862782  1537 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:38.780555  7607 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:38.874162  1717 pending_rounds.cc:85] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 8
I20260709 13:04:38.887188  1717 pending_rounds.cc:107] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:04:38.887548  1717 raft_consensus.cc:2889] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:38.880335  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:38.876468  1648 raft_consensus.cc:2376] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12. Re-sending same reply.
W20260709 13:04:38.810177  7623 log.cc:927] Time spent T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:04:38.890842  1713 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:38.818203  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:38.885154  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:38.896103  1640 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:38.897822  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:38.889585  1647 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 13: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:38.902108  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:38.903090  1646 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:38.811463  7599 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:38.918221  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:38.918689  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:38.924636  1641 raft_consensus.cc:2410] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 13:04:38.919574  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 83999.1us
I20260709 13:04:38.918812  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:38.929358  1644 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 14: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:38.923080  1722 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
W20260709 13:04:38.891139  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 135180us
I20260709 13:04:38.809279  7609 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:38.809331  7619 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:38.977193  1719 raft_consensus.cc:2393] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:04:38.977061  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 79403.4us
I20260709 13:04:38.893733  7605 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:38.936084  6636 consensus_queue.cc:1059] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:38.939131  2346 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:38.940039  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 104823us
I20260709 13:04:38.961660  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:38.982097  1713 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
W20260709 13:04:38.968868  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.936129  7602 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:38.926443  7608 raft_consensus.cc:697] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:38.908396  7620 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:38.994943  1647 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: [10.8-12.9]
I20260709 13:04:38.974545  1685 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:38.972242  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 88500.9us
I20260709 13:04:38.880014  7277 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:38.818122  7604 raft_consensus.cc:3055] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:38.978600  1537 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:38.931468  7626 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:38.938519  7568 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:38.980628  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 48766.2us
I20260709 13:04:38.983049  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:38.985136  7625 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:38.909627  7152 consensus_queue.cc:260] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:39.032918  7620 raft_consensus.cc:697] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:38.965898  7597 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:39.041273  7630 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.140s	user 0.000s	sys 0.002s
I20260709 13:04:38.986938  7436 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:38.994910  7619 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.041577  7597 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:39.042310  7619 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:39.043596  7628 log.cc:927] Time spent T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:04:39.044137  7597 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:38.986215  7605 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:39.045996  7605 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:39.047789  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:39.048488  1643 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:39.050397  1645 raft_consensus.cc:1240] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:04:39.005895  7604 raft_consensus.cc:740] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:39.051622  7604 consensus_queue.cc:260] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:39.052248  1639 raft_consensus.cc:1275] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:04:39.052954  1639 pending_rounds.cc:85] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 8
I20260709 13:04:39.053237  1639 pending_rounds.cc:107] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:04:38.989387  7609 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.039229  7638 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:39.039840  7568 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:38.999575  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:39.042110  1717 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:39.051839  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:39.031590  7608 consensus_queue.cc:237] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:39.038357  1722 raft_consensus.cc:1275] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:39.038777  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:39.057083  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 64058.8us
I20260709 13:04:39.076611  1538 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:39.055820  1713 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:39.057161  1717 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: [10.8-12.9]
I20260709 13:04:39.061007  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:39.057826  1645 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 18: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:39.053489  1639 raft_consensus.cc:2889] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:39.064119  1647 raft_consensus.cc:3055] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:39.066085  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:39.080549  1647 raft_consensus.cc:740] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:39.080799  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:39.066787  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:39.080942  1570 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:39.038187  1641 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:04:39.082279  1643 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:39.084803  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:39.085669  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:39.092418  1639 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:39.093087  1647 consensus_queue.cc:260] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:39.083707  1537 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:39.093708  6636 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.094843  6636 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:39.097764  1537 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:39.084980  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 77538.8us
I20260709 13:04:38.978844  1718 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
W20260709 13:04:39.112892  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.996801  1687 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:39.113238  1643 raft_consensus.cc:1275] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:04:39.008391  1685 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:39.123963  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:39.122284  1721 consensus_meta.cc:287] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.002s
I20260709 13:04:39.125949  1721 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:39.053508  7604 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:39.059922  7616 log.cc:927] Time spent T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.459s	user 0.003s	sys 0.000s
I20260709 13:04:39.126579  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:39.159152  1645 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:39.127586  1565 raft_consensus.cc:2393] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 13: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:39.129113  2314 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:39.130060  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:39.131582  1647 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:39.067984  7632 raft_consensus.cc:3055] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:39.128301  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:39.067425  7152 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:39.065966  7569 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:39.083086  7626 raft_consensus.cc:697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:39.069144  7607 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:39.138645  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 62087.5us
I20260709 13:04:39.177430  1538 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:39.068358  7277 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:39.053851  7609 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:39.138742  6636 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:39.202297  6636 raft_consensus.cc:2764] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:04:39.146615  1713 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:39.122555  7568 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:39.127848  7597 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:39.154597  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:39.204519  1719 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 14: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:04:39.155486  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 147104us
I20260709 13:04:39.205626  1685 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:39.163197  1716 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:39.149549  7618 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.215195  1611 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:39.150792  7638 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:39.162194  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:39.138810  7602 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:39.163311  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:39.167443  1720 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:39.148654  2346 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 11.
W20260709 13:04:39.171763  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 54427.8us
I20260709 13:04:39.234476  1612 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:39.174196  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:39.073005  7614 consensus_queue.cc:260] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:39.218422  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:39.235725  1713 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:39.170053  7632 raft_consensus.cc:740] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:39.224463  1647 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:39.244270  1720 raft_consensus.cc:1275] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:04:39.041265  7620 consensus_queue.cc:237] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:39.249128  7626 consensus_queue.cc:237] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:39.163326  7185 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:39.207932  1685 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:39.207216  7568 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.259572  1718 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:39.207617  7597 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:04:39.260702  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 108878us
W20260709 13:04:39.229744  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 232902us
I20260709 13:04:39.175294  7553 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:39.222878  1645 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
W20260709 13:04:39.238099  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 60751.7us
W20260709 13:04:39.249013  7621 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:04:39.228295  6636 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:39.257126  1713 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
W20260709 13:04:39.253526  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45653.8us
I20260709 13:04:39.272888  6636 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.274691  6636 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:39.169374  7572 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:04:39.225137  7436 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:39.271719  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:39.266426  1687 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:39.281245  1720 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 18: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:39.267148  7597 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:39.284662  7652 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.285387  7652 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:39.286799  7641 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.287513  7641 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:39.291399  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:39.292462  1570 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:39.271083  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:39.266706  1538 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:39.231173  7638 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:39.274289  1685 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:39.252967  7625 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:39.237527  7602 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:39.243633  7094 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:39.264293  7568 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:39.260313  7569 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:39.329022  1687 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:39.337244  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 70596.5us
I20260709 13:04:39.338974  1538 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:39.230160  7618 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:39.300231  7652 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:39.332864  7602 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:39.356124  7607 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:39.330303  7625 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.325450  7638 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:39.330278  7642 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:39.397091  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 137330us
I20260709 13:04:39.396889   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:39.316977  7436 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:39.388965  7635 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:39.426625  1467 catalog_manager.cc:5697] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:39.428064  7635 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:39.434010  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:39.433998  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 105007us
I20260709 13:04:39.386183  7625 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:39.397423  7642 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.439206  1719 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:39.397826  7652 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:39.450006  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 175731us
I20260709 13:04:39.451048  1687 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:39.341235  7094 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:39.492406  6636 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:39.473770  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 134823us
I20260709 13:04:39.487656  1685 leader_election.cc:400] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12.
I20260709 13:04:39.507236  1685 leader_election.cc:403] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:04:39.494990  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 44033.7us
I20260709 13:04:39.508443  1687 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:39.498044  7648 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:39.468396  2348 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:04:39.514227  1537 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:39.517619  1538 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:39.525427  1718 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:39.479256  7642 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:39.531029  6636 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:39.498050  7607 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:39.539124  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:39.542330  1719 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:39.482946  7641 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:39.350271  7632 consensus_queue.cc:260] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:39.553617  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 68660.3us
I20260709 13:04:39.554718  1685 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:39.555578  6636 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:39.555984  1646 consensus_meta.cc:287] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.623s	user 0.014s	sys 0.003s
I20260709 13:04:39.556144  6636 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.556517  1646 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:39.556661  6636 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:39.529299  7599 raft_consensus.cc:2764] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:04:39.535298  7569 raft_consensus.cc:697] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:39.534817  7572 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:39.537833  7652 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:39.622344  1685 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:39.544667  7607 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:39.626696  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 118288us
I20260709 13:04:39.644989  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:39.625533  7650 log.cc:927] Time spent T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 13:04:39.643182  1538 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:39.594532  7436 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:39.669749  7664 log.cc:927] Time spent T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
W20260709 13:04:39.670990  7646 log.cc:927] Time spent T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.287s	user 0.002s	sys 0.000s
W20260709 13:04:39.707090  7605 consensus_meta.cc:287] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.661s	user 0.012s	sys 0.000s
I20260709 13:04:39.707669  7605 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:39.643718  7572 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:39.663983  7654 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:39.668774  7651 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12.
I20260709 13:04:39.672029  7656 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:39.650815  1719 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:39.652158  1687 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:39.720450  1716 consensus_meta.cc:287] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.505s	user 0.009s	sys 0.004s
I20260709 13:04:39.720921  1716 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
W20260709 13:04:39.721050  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 68928.1us
I20260709 13:04:39.722268  1687 leader_election.cc:400] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 11 pre-election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 11. Current term is 12.
I20260709 13:04:39.649618  7660 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:39.700752  7094 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:39.700884  7560 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:39.703510  2341 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 21.
I20260709 13:04:39.735496  7560 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:39.738072  7659 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.115s	user 0.000s	sys 0.002s
I20260709 13:04:39.653831  7607 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:39.708560  7670 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:39.742420  7670 raft_consensus.cc:697] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:39.744901  7670 consensus_queue.cc:237] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:39.757033  7669 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:39.757887  7669 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.758351  7669 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:39.763351  7665 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:39.771165  2344 raft_consensus.cc:1240] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:04:39.774922  7653 consensus_queue.cc:1059] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:04:39.778544  7655 log.cc:927] Time spent T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.166s	user 0.000s	sys 0.003s
I20260709 13:04:39.785351  7597 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:39.787948  7597 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:39.788486  7614 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:39.790597  7618 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:39.653869  7652 raft_consensus.cc:697] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:39.718976  6636 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:39.728818  1687 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:39.730139  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 87016us
I20260709 13:04:39.731208  7634 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:39.731832  7654 raft_consensus.cc:697] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:39.727689  7660 raft_consensus.cc:697] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:39.814303  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 85567.1us
I20260709 13:04:39.815662  1687 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:39.820400  1687 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:39.821782  7648 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:39.822623  7619 consensus_meta.cc:287] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.780s	user 0.009s	sys 0.007s
I20260709 13:04:39.823182  7619 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:39.823935  7663 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.178s	user 0.000s	sys 0.000s
I20260709 13:04:39.825353  7619 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:39.730526  7635 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:39.910230  1564 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
W20260709 13:04:39.829274  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.908713  1572 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:39.981788  6636 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:04:39.846309  1537 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:39.942785  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:40.003381  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 48254.2us
W20260709 13:04:39.847630  7604 consensus_meta.cc:287] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.689s	user 0.015s	sys 0.000s
I20260709 13:04:39.982183  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:39.847206  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:04:40.008774  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 162463us
I20260709 13:04:40.015823  2344 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:39.988355  1612 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.055s	sys 0.044s
W20260709 13:04:39.860535  7659 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.121s	user 0.002s	sys 0.000s
I20260709 13:04:40.014873  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:40.024477  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:39.870682  1687 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:40.025185  1572 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:40.021049  1570 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:40.026511  6636 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:04:39.956238  7638 consensus_meta.cc:287] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.532s	user 0.010s	sys 0.002s
I20260709 13:04:40.023833  1566 raft_consensus.cc:1275] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:04:39.835592  7152 consensus_meta.cc:287] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.666s	user 0.012s	sys 0.010s
I20260709 13:04:40.016393  1537 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:40.025833  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 155230us
W20260709 13:04:39.852034  7609 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.615s	user 0.006s	sys 0.008s
W20260709 13:04:40.017789  1612 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.056s	sys 0.045s
W20260709 13:04:40.049366  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 814538us
I20260709 13:04:39.811686  7632 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:39.907198  7675 consensus_queue.cc:1059] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.120s
I20260709 13:04:40.027886  1571 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 13: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:39.898363  7620 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:39.842406  7605 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:39.818058  7656 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:39.953154  7600 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.835016  7560 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:39.924783  7599 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:40.051630  1687 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:39.925349  7277 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:39.860375  7651 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:40.051929  1686 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:40.028116  6636 raft_consensus.cc:697] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:40.060928  1566 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:04:40.076017  1566 pending_rounds.cc:85] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 8
I20260709 13:04:40.076370  1566 pending_rounds.cc:107] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:04:40.076683  1566 raft_consensus.cc:2889] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:39.814356  7568 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:39.958921  7625 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:39.837565  7602 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.073550  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:39.838161  7642 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:39.840130  7648 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.223s
I20260709 13:04:39.841775  7569 consensus_queue.cc:237] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.085789  1687 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:39.826753  1467 catalog_manager.cc:5697] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:40.111948  7642 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:40.057168  7669 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.114087  7625 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:40.116334  1687 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:40.066309  7620 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:40.134836  7682 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:40.139878  7669 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:40.141964  7683 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:40.142469  6636 consensus_queue.cc:237] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.065834  7600 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.072930  7599 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:39.938832  7652 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:40.134737  1570 raft_consensus.cc:2393] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 16: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:40.060886  7152 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:40.080714  7605 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:40.148429  7620 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:40.150934  7648 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:40.174955  7648 raft_consensus.cc:697] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 21 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:39.827335  7654 consensus_queue.cc:237] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.151651  7682 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:40.178277  7682 raft_consensus.cc:697] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:40.047526  7638 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.179495  7682 consensus_queue.cc:237] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.181573  7638 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:40.176992  7648 consensus_queue.cc:237] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:40.182591  7602 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:40.152627  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 100744us
I20260709 13:04:40.154515  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:40.064566  7604 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
W20260709 13:04:40.157755  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 95942.8us
I20260709 13:04:40.187026  1612 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:39.827900  7660 consensus_queue.cc:237] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.072522  7627 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.163002  1568 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16. Ops: []
W20260709 13:04:40.115679  7668 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:04:40.066006  7436 consensus_queue.cc:1048] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.095s
I20260709 13:04:40.063109  6987 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:04:40.080633  7572 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:40.058976  7609 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:40.198769  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:40.205533  7620 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:40.212018  7639 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
W20260709 13:04:40.220537  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 526032us
I20260709 13:04:40.222289  7599 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:40.193315  7152 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:40.225363  1715 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 15: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:40.226737  7568 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:40.186408  7605 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:40.227787  7605 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:40.229036  7665 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:40.081220  7277 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:40.230679  7597 raft_consensus.cc:3055] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:40.231192  7597 raft_consensus.cc:740] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:40.232003  7277 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:40.232311  7597 consensus_queue.cc:260] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:40.233803  7600 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:40.235140  7597 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:40.166265  7651 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:40.224527  7609 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:40.241813  1612 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:40.241770  1572 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:04:40.250689  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:40.251626  1721 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:40.252669  1720 raft_consensus.cc:1275] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:04:40.253285  1720 pending_rounds.cc:85] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 7
I20260709 13:04:40.253525  1720 pending_rounds.cc:107] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:04:40.253870  1720 raft_consensus.cc:2889] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:40.254176  1720 pending_rounds.cc:107] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:04:40.254273  7436 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:40.252776  7568 consensus_queue.cc:1048] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:40.255182  7436 raft_consensus.cc:697] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:40.256031  7568 consensus_queue.cc:1059] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:40.254420  1720 raft_consensus.cc:2889] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:40.257952  1685 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:40.228623  7607 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.250602  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:40.261453  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:40.262672  1718 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
W20260709 13:04:40.263886  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 200281us
I20260709 13:04:40.172071  2344 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:40.272330  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:40.274698  1713 raft_consensus.cc:1275] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:40.274981  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:40.276950  7665 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:40.277125  7568 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:40.277510  1719 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:40.264946  7651 raft_consensus.cc:697] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:40.278434  1538 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:40.226752  7152 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:40.242415  7639 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:40.286363  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 269974us
I20260709 13:04:40.289788  1721 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:40.256505  7436 consensus_queue.cc:237] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:40.315280  1537 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:40.265161  7632 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:04:40.311440  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 53527.8us
I20260709 13:04:40.316494  1685 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:40.306686  1717 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:40.358349  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:40.361322  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:40.361881  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:40.367182  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:40.368273  1641 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:40.368782  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:40.291486  7599 raft_consensus.cc:2764] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:04:40.242105  7627 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.372013  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:40.399111  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:40.369580  1643 raft_consensus.cc:2376] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16. Re-sending same reply.
I20260709 13:04:40.400297  1646 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 18: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:40.400812  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:40.400399  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:40.401526  1643 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:04:40.401712  1644 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:40.402482  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:40.403306  1643 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 20.
I20260709 13:04:40.367794  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:40.370076  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 76910.2us
I20260709 13:04:40.395618  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:40.396306  1685 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:40.407066  1647 raft_consensus.cc:2376] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16. Re-sending same reply.
W20260709 13:04:40.397135  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 247565us
I20260709 13:04:40.411631  7571 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:40.415102  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:40.419904  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:40.430415  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:40.434353  1685 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:40.436621  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:40.441108  1685 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:40.369093  7619 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:40.436251  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:40.446707  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:40.394871  7609 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:40.392395  7152 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.450975  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:40.459950  2344 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:40.460573  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:40.449724  7682 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:40.458313  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:40.462692  2348 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:40.429955  7642 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:40.439090  7665 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.393797  7632 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:40.504365  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 97318.3us
I20260709 13:04:40.452337  7609 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:40.440588  7639 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.518617  1714 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:04:40.448195  7675 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:04:40.510950  1715 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2974) took 1767 ms (client timeout 2241 ms). Trace:
W20260709 13:04:40.529148  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 55955.2us
I20260709 13:04:40.464577  7658 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:40.541374  1715 rpcz_store.cc:269] 0709 13:04:38.743809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.744074 (+   265us) service_pool.cc:224] Handling call
0709 13:04:40.510648 (+1766574us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b540038fed4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:40.510872 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3376530176}
I20260709 13:04:40.550328  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:40.556562  7694 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:40.484413  7152 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:40.557809  7694 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:40.560593  7694 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:40.561182  7152 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:40.562402  1717 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:40.499199  7600 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:40.525257  7651 consensus_queue.cc:237] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.563380  7686 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:40.565151  7632 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:40.522558  7690 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:40.510715  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:40.509680  7578 consensus_queue.cc:1059] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 13:04:40.551915  7639 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.562476  6987 consensus_queue.cc:1048] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:04:40.561317  7694 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:04:40.461122  7619 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.579797  1641 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:40.541328  7609 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:40.591941  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 276513us
I20260709 13:04:40.565775  7687 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:40.512701  7642 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:40.501004  7568 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:04:40.617950  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:40.621181  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 184930us
I20260709 13:04:40.594326  7686 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.665542  7636 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.664085  7688 consensus_queue.cc:1059] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.247s
I20260709 13:04:40.670130  7607 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:40.705845  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 174145us
I20260709 13:04:40.588366  7675 raft_consensus.cc:697] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:40.750541  7675 consensus_queue.cc:237] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.584834  7671 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:40.754067  7671 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:40.545630  7665 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.759953  7351 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.760461  7665 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:40.720675  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 87776.3us
I20260709 13:04:40.720621  7568 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.722321  1537 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:40.722271  1612 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:40.527475  7605 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.565598  7658 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.575022  7573 consensus_queue.cc:1059] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.102s
I20260709 13:04:40.724291  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:40.706422  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 397746us
I20260709 13:04:40.687801  7578 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:40.671223  7436 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:04:40.733610  7672 raft_consensus.cc:3055] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:40.734046  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2979) took 1794 ms. Trace:
I20260709 13:04:40.735375  7642 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:40.682381  7632 consensus_queue.cc:1048] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:04:40.735605  6987 consensus_queue.cc:1048] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:40.735873  7094 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:40.737949  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:40.807394  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:40.807773  2344 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:40.738623  7697 log.cc:927] Time spent T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:04:40.740711  7600 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:40.748580  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2577) took 1968 ms. Trace:
I20260709 13:04:40.823902  1714 rpcz_store.cc:276] 0709 13:04:38.779672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.050259 (+270587us) service_pool.cc:224] Handling call
0709 13:04:40.518347 (+1468088us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:40.535793 (+ 17446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.748350 (+212557us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b540038fed0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:40.748511 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3373705984}
W20260709 13:04:40.779507  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 57282us
W20260709 13:04:40.780954  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 58714.1us
I20260709 13:04:40.825244  1612 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:40.731685  1611 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:40.720292  7687 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:40.793977  1722 rpcz_store.cc:276] 0709 13:04:38.940073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.070533 (+130460us) service_pool.cc:224] Handling call
0709 13:04:39.071140 (+   607us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:39.071591 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.071624 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:39.072202 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 13:04:39.476730 (+404528us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:39.476825 (+    95us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.476922 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:40.732990 (+1256068us) spinlock_profiling.cc:243] Waited 35.2 ms on lock 0x7b2400430178. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245c1e9c4 00007f8245c1cacb 00007f8245dd7dc9 00007f8245dd7a55 00007f8245dd6fb0 00007f8248a4d488 00007f8248a4dbe8 00007f8248a25398 00007f8248a2076a 00007f8248a1fbb6
0709 13:04:40.733630 (+   640us) raft_consensus.cc:1685] finished
0709 13:04:40.733656 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.733961 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1284,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":117,"spinlock_wait_cycles":80942848,"thread_start_us":401587,"threads_started":1,"wal-append.queue_time_us":495004}
I20260709 13:04:40.782778  7641 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:40.785566  7660 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:40.783103  7682 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.799526  7627 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:40.814662  7619 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:40.774724  7636 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:40.818070  2344 raft_consensus.cc:2410] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:40.726243  7686 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:40.825446  1537 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:40.779460  7568 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:40.826427  7600 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:40.760591  7351 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.808308  7672 raft_consensus.cc:740] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:40.793587  7578 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.857206  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:40.835160  7711 raft_consensus.cc:3055] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:40.858098  2344 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:04:40.846980  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:40.863677  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:40.868768  2314 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:40.906575  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 174925us
I20260709 13:04:40.867307  7711 raft_consensus.cc:740] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:40.867731  7641 raft_consensus.cc:697] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:40.913005  7711 consensus_queue.cc:260] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.913336  7641 consensus_queue.cc:237] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.915135  7711 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:40.915779  2341 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:40.905958  7560 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:40.916692  7560 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:40.917114  7560 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:40.900117  7619 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:04:40.918532  7704 consensus_queue.cc:1059] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.144s
W20260709 13:04:40.920357  7703 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:04:40.922024  7706 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:40.922818  7706 raft_consensus.cc:697] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:04:40.923031  7702 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 13:04:40.923830  7706 consensus_queue.cc:237] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.870636  7094 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:04:40.897692  7597 consensus_meta.cc:287] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.662s	user 0.008s	sys 0.007s
I20260709 13:04:40.897497  7553 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:40.950204  2347 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:40.947567  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:40.953783  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 85087.9us
I20260709 13:04:40.955946  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:40.949252  1643 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:40.842392  7682 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:40.955058  1640 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:40.959399  1612 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:40.930394  7569 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:40.960454  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:40.943773  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 118363us
W20260709 13:04:40.944262  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 45216us
I20260709 13:04:40.957084  1646 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 16: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:40.948740  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:40.955722  1538 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:40.965679  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:40.970795  1647 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:40.970402  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:40.970832  1644 raft_consensus.cc:2376] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13. Re-sending same reply.
I20260709 13:04:40.963645  6636 consensus_queue.cc:1059] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:40.964216  1687 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:40.958671  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:40.972304  1640 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:40.976591  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:40.936631  7619 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.944021  6987 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.938313  7553 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:40.983364  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:40.999598  2344 raft_consensus.cc:2393] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:40.984813  1639 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:40.935878  7094 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.992647  1647 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:40.982420  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:40.994728  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:40.995031  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:40.955360  7642 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:04:41.030869  7705 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.221s	user 0.001s	sys 0.000s
I20260709 13:04:41.032310  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:41.002455  1717 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:04:40.955490  7671 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:40.968660  7569 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:41.077692  1717 pending_rounds.cc:85] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 7
I20260709 13:04:41.078426  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:40.954484  7627 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:41.083667  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:41.086551  1717 pending_rounds.cc:107] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:04:41.087074  1642 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:41.087163  1643 raft_consensus.cc:2376] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16. Re-sending same reply.
I20260709 13:04:41.088025  1717 raft_consensus.cc:2889] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:41.086588  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 103553us
I20260709 13:04:41.089547  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:41.025686  7094 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:41.094732  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:41.002209  7553 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:41.098721  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:41.033849  7689 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:04:41.105837  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.099668  1639 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
W20260709 13:04:41.003396  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 47727.7us
W20260709 13:04:41.105541  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 146153us
I20260709 13:04:41.111856  1612 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:41.125555  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.092365  7569 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:41.001930  7619 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.092540  7605 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:40.976341  7694 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.025422  6987 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.090806  7639 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:41.034116  7152 raft_consensus.cc:2764] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:04:41.109678  1538 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:41.190543  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 80998.2us
I20260709 13:04:41.191506  1538 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:41.158207  7718 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:41.143869  7729 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:41.238528  7696 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:41.239284  7696 raft_consensus.cc:697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:41.240288  7696 consensus_queue.cc:237] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.169261  7560 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:41.242220  2343 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 20. Current term is 21. Ops: []
W20260709 13:04:41.242863  7717 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 13:04:41.243199  7560 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:41.237474  7605 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:41.243955  7605 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.244330  7605 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:41.245144  6987 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:41.164465  7636 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:41.170634  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:41.169368  7094 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:41.151015  7708 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:04:40.982496  7672 consensus_queue.cc:260] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:41.094475  7627 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.194295  1538 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:41.198645  1643 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:41.198510  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 234265us
I20260709 13:04:41.233218  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:41.099103  7597 consensus_queue.cc:1059] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:04:41.140792  7689 raft_consensus.cc:697] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:41.236936  7719 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.139s	user 0.001s	sys 0.001s
I20260709 13:04:41.283636  7672 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:41.245776  6987 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.264384  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:41.270419  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:41.274103  1716 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 15: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:41.106467  7642 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.288164  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:41.295321  1719 raft_consensus.cc:2376] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17. Re-sending same reply.
I20260709 13:04:41.300367  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:41.187973  7694 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.301309  1719 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:41.303890  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:41.306427  1647 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:41.306480  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 112223us
I20260709 13:04:41.308101  1538 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:41.309015  1538 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:41.310442  1538 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:41.306643  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:41.330082  1647 raft_consensus.cc:2410] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:04:41.291535  7634 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.298527  7718 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:41.298100  7687 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:41.298285  7654 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:41.340116  1642 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:41.343364  1643 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:41.319139  7597 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:41.293232  7725 consensus_queue.cc:1059] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:04:41.332410  7675 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:41.355557  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:41.319397  7714 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:41.317940  7723 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.357779  7682 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:41.301930  6987 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.358768  7654 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:41.360054  7627 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:41.362936  7694 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:41.369619  7657 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.348382  7634 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.373806  7568 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:41.376560  7605 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.394526  7619 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:41.394802  7652 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.354341  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:41.407711  7597 raft_consensus.cc:697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:41.352710  7687 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.406867  7675 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.430681  7675 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:41.411562  7723 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.432076  7700 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:41.432871  7700 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.419765  7725 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.433961  7725 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.434266  7723 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:41.421366  1687 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:41.397596  1642 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 12.
I20260709 13:04:41.444705  1716 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:41.416081  7152 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.428898  6987 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:41.429711  7687 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.435289  7700 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:41.436156  7725 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:41.438563  7568 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:41.439085  7683 raft_consensus.cc:3055] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:41.440029  7094 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:41.442235  7684 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 1.268s	user 0.009s	sys 0.006s
W20260709 13:04:41.396445  7609 consensus_meta.cc:287] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.674s	user 0.014s	sys 0.003s
I20260709 13:04:41.466832  7152 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.576012  7152 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:41.447376  2313 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:41.463165  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:41.466100  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 384798us
I20260709 13:04:41.542719  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:41.578393  1639 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 15: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:41.543879  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:41.414999  7654 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:41.590464  7731 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:04:41.592056  7654 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:41.474130  1641 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:41.331362  7689 consensus_queue.cc:237] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.594008  7737 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.518687  7642 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:41.467335  7723 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:04:41.493108  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:41.607244  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 3121) took 1242 ms (client timeout 1555 ms). Trace:
I20260709 13:04:41.514065  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:41.596761  7569 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.418198  7714 raft_consensus.cc:697] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:41.522037  7738 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:41.535063  7609 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.535127  7724 raft_consensus.cc:3055] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:04:41.534575  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 65175.6us
I20260709 13:04:41.558244  7688 raft_consensus.cc:3055] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:04:41.567210  7686 consensus_meta.cc:287] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.698s	user 0.015s	sys 0.005s
I20260709 13:04:41.518323  7683 raft_consensus.cc:740] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:41.551862  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:41.579380  1537 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:41.623745  1722 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:41.580422  1642 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:41.418310  7657 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:41.601205  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 153831us
I20260709 13:04:41.609338  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:41.426002  7652 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:41.528577  7725 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:41.607723  1648 rpcz_store.cc:269] 0709 13:04:40.357236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.359682 (+  2446us) service_pool.cc:224] Handling call
0709 13:04:41.543702 (+1184020us) spinlock_profiling.cc:243] Waited 2.94 ms on lock 0x7b5800003098. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de4b0a0 00007f824de4a984 00007f82473c56d4 00007f82473c5662
0709 13:04:41.599637 (+ 55935us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6762880}
I20260709 13:04:41.605592  7737 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.552489  7632 raft_consensus.cc:3055] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:41.633569  7632 raft_consensus.cc:740] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:41.634883  7652 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:41.483561  7675 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:41.636081  7736 log.cc:927] Time spent T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:04:41.635675  7632 consensus_queue.cc:260] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.614396  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:41.637037  7675 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:41.576640  7152 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:41.602087  7636 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:41.626041  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:41.637390  1639 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:41.639055  7714 consensus_queue.cc:237] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.640070  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:41.628175  7724 raft_consensus.cc:740] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:41.613827  1687 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:41.637876  7725 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.639009  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:41.643133  1648 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:41.644383  7725 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:41.631696  1714 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: []
W20260709 13:04:41.633582  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 54722.2us
I20260709 13:04:41.646229  7634 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:41.647459  1537 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:41.635058  7732 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:04:41.641247  7724 consensus_queue.cc:260] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:41.630775  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:41.649781  7553 consensus_meta.cc:287] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.514s	user 0.012s	sys 0.001s
I20260709 13:04:41.648008  7609 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:41.630878  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:41.625748  2313 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:41.633801  7687 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:41.621594  7688 raft_consensus.cc:740] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:41.638878  7683 consensus_queue.cc:260] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.620407  7738 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.650487  1648 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:41.621341  7686 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.533715  7597 consensus_queue.cc:237] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.658975  1722 raft_consensus.cc:2410] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:04:41.659983  1718 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:41.661126  1640 raft_consensus.cc:1275] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:04:41.696537  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 82764.8us
I20260709 13:04:41.662688  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3124) took 1299 ms. Trace:
I20260709 13:04:41.675033  1642 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:04:41.699486  1687 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:41.671833  1647 raft_consensus.cc:1275] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:41.680644  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2592) took 2513 ms. Trace:
I20260709 13:04:41.677749  1719 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:04:41.685407  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.710973  1640 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:41.773020  1647 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:04:41.653942  1537 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:41.690100  1648 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:04:41.640398  7739 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:41.679818  7654 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:04:41.842721  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.667248  7351 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:41.661799  7636 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:41.706645  1568 rpcz_store.cc:276] 0709 13:04:39.166853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.168169 (+  1316us) service_pool.cc:224] Handling call
0709 13:04:39.168531 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.199235 (+1030704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.680391 (+1481156us) spinlock_profiling.cc:243] Waited 974 ms on lock 0x7b54004056d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:41.680579 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2240794880}
I20260709 13:04:41.709504  1687 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:41.676574  7686 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:41.845322  1640 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:04:41.700551  7665 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.023s	sys 0.003s
I20260709 13:04:41.698738  1645 rpcz_store.cc:276] 0709 13:04:40.363069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.363195 (+   126us) service_pool.cc:224] Handling call
0709 13:04:40.414374 (+ 51179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.724892 (+310518us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.724927 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:40.724972 (+    45us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:40.725007 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.662352 (+937345us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b54000a5ad4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:41.662406 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.662578 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":703662976}
W20260709 13:04:41.852097  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 198564us
W20260709 13:04:41.864795  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 155326us
I20260709 13:04:41.772869  7634 consensus_queue.cc:1059] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 13:04:41.865420  7723 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:41.774403  7729 consensus_meta.cc:287] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.536s	user 0.012s	sys 0.000s
I20260709 13:04:41.878211  7729 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:41.880774  7600 consensus_meta.cc:287] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.980s	user 0.010s	sys 0.011s
I20260709 13:04:41.881374  7600 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.658653  7553 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:41.884812  7654 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:41.885596  1537 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:41.886229  7553 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:41.886991  7654 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:41.887269  1537 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:41.887614  7654 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.888096  7654 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:41.888669  1537 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:41.677801  7569 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:41.655687  7605 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:41.889760  7696 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:41.889901  7746 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:41.796598  7609 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:41.891155  7600 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:41.891280  7636 raft_consensus.cc:697] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:41.668921  7738 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:41.679440  7737 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:41.897039  7743 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
I20260709 13:04:41.898424  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:41.871461  1687 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:41.667359  7657 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:41.659163  7724 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:41.899538  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2989) took 2417 ms. Trace:
I20260709 13:04:41.666720  7683 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:41.900035  1720 rpcz_store.cc:276] 0709 13:04:39.481982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.482169 (+   187us) service_pool.cc:224] Handling call
0709 13:04:39.504902 (+ 22733us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.265552 (+760650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.899271 (+1633719us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b54004079d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:41.899474 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1717364096}
I20260709 13:04:41.901026  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:41.876335  7739 raft_consensus.cc:697] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:41.902833  7739 consensus_queue.cc:237] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.899572  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:41.908396  2344 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:41.908839  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2597) took 1047 ms. Trace:
I20260709 13:04:41.909242  1714 rpcz_store.cc:276] 0709 13:04:40.861432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.861607 (+   175us) service_pool.cc:224] Handling call
0709 13:04:40.861920 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:41.645152 (+783232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.908565 (+263413us) spinlock_profiling.cc:243] Waited 769 ms on lock 0x7b54003a39d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:41.908775 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1768968064}
I20260709 13:04:41.909886  7675 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:41.910488  7675 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.898522  7653 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:41.912974  1568 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:04:41.914582  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:41.637755  7152 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.917562  1720 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:41.772543  7690 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:41.920019  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:41.685884  7652 consensus_queue.cc:1048] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.184s
I20260709 13:04:41.921243  1568 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:41.922557  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:41.919955  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:41.875133  7665 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.972s	user 0.028s	sys 0.003s
W20260709 13:04:41.922456  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 189327us
I20260709 13:04:41.924393  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:41.935180  1569 raft_consensus.cc:2410] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 13:04:41.925475  1570 raft_consensus.cc:2393] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 21: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:41.665346  7688 consensus_queue.cc:260] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.932864  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:41.936868  1572 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:41.933908  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:41.946120  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:41.946853  1571 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
W20260709 13:04:41.948195  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 76810.2us
I20260709 13:04:41.893049  7636 consensus_queue.cc:237] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:41.920672  6987 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.923089  7690 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.923328  7609 raft_consensus.cc:697] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:41.963627  1571 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:41.963900  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:41.958527  7724 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:41.997561  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:41.968276  2348 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:04:41.997236  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:41.942301  7738 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.011726  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2604) took 1057 ms. Trace:
I20260709 13:04:42.034921  1612 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:42.012179  1720 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:42.036506  1572 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:42.036463  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:42.032168  1722 rpcz_store.cc:276] 0709 13:04:40.954405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.775197 (+820792us) service_pool.cc:224] Handling call
0709 13:04:41.775577 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:41.827459 (+ 51882us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.827491 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:41.832680 (+  5189us) log.cc:844] Serialized 26 byte log entry
0709 13:04:41.941104 (+108424us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:41.941148 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.941217 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:42.011444 (+ 70227us) raft_consensus.cc:1685] finished
0709 13:04:42.011472 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.011651 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32746,"raft.queue_time_us":22366,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":24549,"thread_start_us":108025,"threads_started":1,"wal-append.queue_time_us":165396}
I20260709 13:04:42.040048  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:42.023368  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.041142  1571 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:42.041517  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.042303  1612 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:42.042582  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.042886  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:41.969444  1687 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:42.043874  1566 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:42.044533  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 75168.7us
I20260709 13:04:41.974330  2344 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:42.036070  7724 raft_consensus.cc:697] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:42.050554  1572 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:42.051121  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 529) took 1562 ms. Trace:
I20260709 13:04:42.051765  1568 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:42.045475  1687 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:42.053203  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:42.053936  1644 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17.
I20260709 13:04:42.062587  1568 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:04:42.051523  2348 rpcz_store.cc:276] 0709 13:04:40.469120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.469349 (+   229us) service_pool.cc:224] Handling call
0709 13:04:41.967700 (+1498351us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b580003a298. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:04:41.967957 (+   257us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:42.031472 (+ 63515us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.031746 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33402752}
I20260709 13:04:42.054028  1687 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:42.067718  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:41.994199  7618 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.082830  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:42.085249  1720 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:04:42.072515  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.078531  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:42.087035  1564 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:42.087523  1573 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:42.082803  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:41.968000  7600 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.996117  7573 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:42.088313  1570 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:04:41.998886  7658 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:42.056831  7706 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:42.111299  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.113622  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.037557  7688 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:41.994949  7634 consensus_queue.cc:1059] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:04:42.123610  1564 raft_consensus.cc:2393] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 13: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:42.124655  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.034436  7669 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:42.051471  7738 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:41.968313  6987 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.085050  7675 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:42.072149  7737 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:42.114153  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.181134  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:42.182227  1567 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:42.186693  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.188081  1570 raft_consensus.cc:2393] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 16: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:41.966120  7690 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:42.061892  7729 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:42.154976  7602 raft_consensus.cc:3055] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:42.128683  7706 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:42.207636  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:42.207970  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.050808  7642 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:42.039565  7746 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:42.125483  1564 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 16: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:41.995572  7626 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.072853  7436 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:42.127954  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.062891  7725 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:42.151798  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:42.061560  7620 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.093s
I20260709 13:04:42.061779  7625 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 13:04:42.071859  7686 consensus_queue.cc:1059] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 13:04:42.195905  7729 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.111862  7692 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:42.151189  7653 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:42.176491  7669 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.204731  7602 raft_consensus.cc:740] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:42.210148  1566 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:42.215278  1564 raft_consensus.cc:1275] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:04:42.247340  1564 pending_rounds.cc:85] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 7
I20260709 13:04:42.247627  1564 pending_rounds.cc:107] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:04:42.247933  1564 raft_consensus.cc:2889] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:42.248354  1564 pending_rounds.cc:107] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:04:42.248622  1564 raft_consensus.cc:2889] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:42.209503  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:42.250275  1721 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 15: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:42.189325  7737 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.219250  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:42.117452  7573 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.100633  7618 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.228576  1565 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:42.097885  7671 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:42.231894  1569 raft_consensus.cc:2410] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:42.114574  7600 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.216403  7706 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:42.365247  7600 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:42.218334  7725 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.215138  7642 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:42.119670  1537 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:42.379019  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 259455us
I20260709 13:04:42.260609  1567 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:42.229269  7572 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.269340  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:42.271618  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:42.280592  7709 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.227485  7436 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.283625  7632 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:42.183137  7609 consensus_queue.cc:237] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.295392  7152 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:42.296824  7749 log.cc:927] Time spent T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
W20260709 13:04:42.300311  7754 log.cc:927] Time spent T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:04:42.301832  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 616) took 1345 ms. Trace:
I20260709 13:04:42.437680  1718 rpcz_store.cc:276] 0709 13:04:40.956178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.844649 (+888471us) service_pool.cc:224] Handling call
0709 13:04:42.284947 (+440298us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b580009db98. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:04:42.285220 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.285469 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.285509 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:42.285569 (+    60us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:42.285630 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.301637 (+ 16007us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.301769 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8155904}
I20260709 13:04:42.267727  7692 raft_consensus.cc:697] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:42.304003  7682 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.309630  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3132) took 1351 ms. Trace:
I20260709 13:04:42.311658  7648 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:42.309342  7690 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:42.257439  7653 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:42.454322  1721 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 18: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:04:42.313571  7699 log.cc:927] Time spent T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 1.531s	user 0.002s	sys 0.001s
I20260709 13:04:42.455112  7653 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:42.455489  7738 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:42.315573  7723 raft_consensus.cc:3055] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:42.456666  7723 raft_consensus.cc:740] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:42.458153  7723 consensus_queue.cc:260] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.459497  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.459692  7723 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:42.460908  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.326776  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:42.462332  1714 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:42.343623  7751 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.478s	user 0.001s	sys 0.000s
I20260709 13:04:42.344895  7578 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:42.346050  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.356081  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2632) took 1405 ms. Trace:
W20260709 13:04:42.359483  2341 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 527) took 1929 ms (client timeout 1953 ms). Trace:
I20260709 13:04:42.043316  1571 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:42.476724  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.220739  7626 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.367297  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2606) took 1224 ms. Trace:
I20260709 13:04:42.368827  1722 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
W20260709 13:04:42.371802  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 112044us
I20260709 13:04:42.151738  7724 consensus_queue.cc:237] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.235985  7729 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:42.394138  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 340012us
I20260709 13:04:41.390749   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:42.340911  7687 consensus_queue.cc:1059] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:42.508412  7687 raft_consensus.cc:3055] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:42.508862  7687 raft_consensus.cc:740] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:42.508946  7618 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:42.256924  7686 raft_consensus.cc:3055] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:42.509902  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:42.510165  7686 raft_consensus.cc:740] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:42.510800  2340 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:42.470135  7648 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:42.511201  7686 consensus_queue.cc:260] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.512316  7686 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:42.513157  1721 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:04:42.419600  7689 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:42.513872  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2609) took 1341 ms. Trace:
I20260709 13:04:42.514273  1721 rpcz_store.cc:276] 0709 13:04:41.172454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.512600 (+1340146us) service_pool.cc:224] Handling call
0709 13:04:42.512927 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:42.513603 (+   676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.513820 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.419137  1467 catalog_manager.cc:5697] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 5 to 8, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 8 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:42.516297  7715 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 1.619s	user 0.002s	sys 0.004s
I20260709 13:04:42.517243  2343 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:04:42.424228  7597 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:42.517899  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2595) took 2225 ms. Trace:
I20260709 13:04:42.518330  7597 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.518236  1713 rpcz_store.cc:276] 0709 13:04:40.292533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.567569 (+275036us) service_pool.cc:224] Handling call
0709 13:04:40.568012 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:40.568506 (+   494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.568542 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:40.569128 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 13:04:40.710913 (+141785us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:40.789141 (+ 78228us) log.cc:844] Serialized 14 byte log entry
0709 13:04:40.789366 (+   225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.789450 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:42.401292 (+1611842us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:42.517658 (+116366us) raft_consensus.cc:1685] finished
0709 13:04:42.517750 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.517849 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":131894,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":10552,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":5676,"spinlock_wait_cycles":296119680,"thread_start_us":216753,"threads_started":2,"wal-append.queue_time_us":259807}
I20260709 13:04:42.520272  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:42.427698  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:42.392784  7725 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:42.429960  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.414260  7709 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.407858  7572 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.564899  7572 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:42.432173  1564 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:42.566026  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:42.566862  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.568185  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.569029  1564 raft_consensus.cc:2410] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 19 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:04:42.433116  1566 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:42.433951  1573 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:04:42.570596  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:42.255154  7737 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:42.302685  7625 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.443537  2344 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:42.392652  7642 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:42.398371  7720 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:42.445789  1642 rpcz_store.cc:276] 0709 13:04:40.958519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.639934 (+681415us) service_pool.cc:224] Handling call
0709 13:04:41.640281 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:41.700226 (+ 59945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.309143 (+608917us) spinlock_profiling.cc:243] Waited 34.4 ms on lock 0x7b54003566d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:42.309455 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79158528}
I20260709 13:04:42.478188  1571 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 14: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:42.483596  1714 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:42.504578  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:42.584800  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:42.574054  2313 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:42.505256  2346 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:42.575767  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:42.506919  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:42.566855  2343 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:42.494905  1716 rpcz_store.cc:276] 0709 13:04:41.142828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.892611 (+749783us) service_pool.cc:224] Handling call
0709 13:04:41.892934 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:41.943225 (+ 50291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.943256 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:41.943310 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:41.943347 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.366936 (+423589us) spinlock_profiling.cc:243] Waited 50 ms on lock 0x7b5400213c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:42.367021 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.367209 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115108608}
I20260709 13:04:42.514001  7689 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:42.421006  7714 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:42.520413  7597 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:42.425442  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:42.601325  1569 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:42.521413  7690 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:42.419953  7569 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:42.522142  7692 consensus_queue.cc:237] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.523587  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 613) took 1581 ms. Trace:
I20260709 13:04:42.524818  7706 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:42.509938  7687 consensus_queue.cc:260] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.464811  7602 consensus_queue.cc:260] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.464013  7682 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.527772  1611 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:42.528319  7626 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:42.529953  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 619) took 1109 ms. Trace:
I20260709 13:04:42.635380  1722 rpcz_store.cc:276] 0709 13:04:41.420625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.528961 (+1108336us) service_pool.cc:224] Handling call
0709 13:04:42.529340 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.529573 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.529600 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:42.529675 (+    75us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:42.529717 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.529778 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.529890 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:42.636227  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 108533us
I20260709 13:04:42.637729  1642 raft_consensus.cc:1275] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:04:42.391551  7671 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:42.561461  7578 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:42.562495  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:42.582672  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:42.513339  7648 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:42.643443  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:42.583568  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:42.644551  7709 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:42.644811  1640 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:42.591903  7625 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:42.647188  7625 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:42.615756  7597 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:04:42.641880  7572 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:42.583948  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.592206  7675 consensus_queue.cc:1048] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:04:42.571970  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.594033  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.527803  7436 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:42.595595  1641 raft_consensus.cc:1240] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:42.596014  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.662041  1571 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:04:42.663307  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.596515  1570 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:42.597352  1716 raft_consensus.cc:1240] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:04:42.597302  1713 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:42.587595  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.535032  2314 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:42.665431  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:42.665848  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 130845us
I20260709 13:04:42.602561  1565 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:42.604626  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:42.529970  7669 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:42.601034  7651 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:42.494153  2341 rpcz_store.cc:269] 0709 13:04:40.430411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.433368 (+  2957us) service_pool.cc:224] Handling call
0709 13:04:42.359061 (+1925693us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b5400600ed4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:42.359327 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":710704000}
I20260709 13:04:42.625768  7687 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:42.595221  7571 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:42.583225  7152 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
W20260709 13:04:42.630651  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.616698  1719 rpcz_store.cc:276] 0709 13:04:40.942173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.708979 (+766806us) service_pool.cc:224] Handling call
0709 13:04:41.709487 (+   508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:41.710012 (+   525us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.710060 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:41.832695 (+122635us) log.cc:844] Serialized 26 byte log entry
0709 13:04:41.865713 (+ 33018us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:04:42.121090 (+255377us) log.cc:844] Serialized 14 byte log entry
0709 13:04:42.121357 (+   267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.121470 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:42.342760 (+221290us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:42.523229 (+180469us) raft_consensus.cc:1685] finished
0709 13:04:42.523340 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.523497 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":372833,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":67,"raft.queue_time_us":177365,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":37085,"spinlock_wait_cycles":5720448,"thread_start_us":232271,"threads_started":2,"wal-append.queue_time_us":32518}
I20260709 13:04:42.571125  7738 consensus_queue.cc:1059] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:04:42.473502  1646 rpcz_store.cc:276] 0709 13:04:40.950550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.392709 (+442159us) service_pool.cc:224] Handling call
0709 13:04:41.393252 (+   543us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:41.395792 (+  2540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.395825 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:41.406938 (+ 11113us) log.cc:844] Serialized 26 byte log entry
0709 13:04:41.610459 (+203521us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:42.354674 (+744215us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b1800075520. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f824c87d3c7 00007f8248e8f9a7 00007f8248e9f0bf
0709 13:04:42.355026 (+   352us) log.cc:844] Serialized 14 byte log entry
0709 13:04:42.355472 (+   446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.355592 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:42.355648 (+    56us) raft_consensus.cc:1685] finished
0709 13:04:42.355783 (+   135us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.355974 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1799,"prepare.queue_time_us":18210,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":41,"raft.queue_time_us":6264,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":1671,"spinlock_wait_cycles":64636160,"thread_start_us":357329,"threads_started":2,"wal-append.queue_time_us":375102}
I20260709 13:04:42.638235  1611 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:42.639577  1642 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:42.640771  1612 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:42.640861  1644 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:42.617617  7685 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.015s	sys 0.024s
I20260709 13:04:42.582015  7600 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:04:42.585462  1718 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:42.652168  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:42.583664  7653 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:04:42.657200  1567 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:04:42.657998  1647 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 12. Current term is 13. Ops: []
W20260709 13:04:42.652251  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.659641  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.667834  1643 raft_consensus.cc:2410] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:42.641307  7689 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:42.727972  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 89823us
I20260709 13:04:42.776273  1611 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:42.605885  7714 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:42.729067  1569 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:42.671239  1640 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:04:42.671399  1645 raft_consensus.cc:1275] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:04:42.672272  7745 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:04:42.734992  1685 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.505s (SENT) [suppressed 22 similar messages]
W20260709 13:04:42.746810  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 62230.4us
W20260709 13:04:42.749786  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.618238  7569 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:42.747539  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.757073  6636 raft_consensus.cc:3055] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:42.633431  7665 consensus_queue.cc:1059] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:04:42.680898  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.678552  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:42.688793  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 48187us
I20260709 13:04:42.678015  7560 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:42.746802  7688 consensus_meta.cc:287] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.623s	user 0.012s	sys 0.000s
I20260709 13:04:42.746858  6987 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:42.753026  7436 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:42.755102  7709 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:42.757838  7607 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:42.760229  7738 consensus_queue.cc:1243] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:42.683703  7651 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.784384  1720 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:42.795511  6636 raft_consensus.cc:740] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:42.802357  1565 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 18: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:04:42.810292  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 329710us
W20260709 13:04:42.822333  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.874063  6636 consensus_queue.cc:260] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.880985  6636 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:42.880038  1644 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:42.831852  1568 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:04:42.882318  1568 pending_rounds.cc:85] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 11
I20260709 13:04:42.882562  1568 pending_rounds.cc:107] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:04:42.882788  1568 raft_consensus.cc:2889] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:42.823920  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.884768  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:42.886220  1719 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:42.890259  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:42.894228  1714 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:04:42.895062  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:42.902390  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:42.907646  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:42.894066  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:42.914985  1719 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:42.918663  1722 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:42.920645  1537 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:42.922894  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:42.930073  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:42.864880  1716 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:42.865549  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 294168us
I20260709 13:04:42.896601  7763 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:42.872022  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.790477  1611 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:42.808693  7560 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:42.935556  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 145124us
I20260709 13:04:42.800540  7658 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:42.937000  7658 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:42.889211  7738 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.867326  7573 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:42.830561  7607 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.935714  7560 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:42.938032  7738 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:42.941846  7682 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:42.942373  7760 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:04:42.942973  7569 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:42.943693  7569 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:42.944171  7569 raft_consensus.cc:697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:42.945576  1572 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:42.946368  7653 consensus_queue.cc:1059] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:42.947134  1572 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:04:42.947527  1572 pending_rounds.cc:85] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 8
I20260709 13:04:42.947742  1572 pending_rounds.cc:107] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:04:42.947981  1572 raft_consensus.cc:2889] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:42.819978  7572 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.128s
I20260709 13:04:42.808501  7625 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.121s
I20260709 13:04:42.950069  7671 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:42.907508  7764 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.951030  7764 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.952127  7560 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:42.844996  1718 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
W20260709 13:04:42.953837  7757 log.cc:927] Time spent T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.327s	user 0.000s	sys 0.002s
I20260709 13:04:42.901074  7706 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:42.955452  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3133) took 1996 ms. Trace:
I20260709 13:04:42.953505  7764 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:42.956698  7763 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.945092  7569 consensus_queue.cc:237] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.959982  7763 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:42.811883  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.819876  7665 consensus_queue.cc:1243] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:04:42.964562  7573 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:42.790524  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.839138  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.966439  7651 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:42.966849  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.967430  1567 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:42.968573  1570 raft_consensus.cc:1275] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:42.955936  1639 rpcz_store.cc:276] 0709 13:04:40.958680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.640990 (+682310us) service_pool.cc:224] Handling call
0709 13:04:41.641320 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:41.918824 (+277504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.918863 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:41.919541 (+   678us) log.cc:844] Serialized 26 byte log entry
0709 13:04:42.119912 (+200371us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:42.119953 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.120016 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:42.340424 (+220408us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b48002619d8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ddaeaa1 00007f824dda5b37 00007f824dda59ca 00007f824de9d26f 00007f824de9cf37 00007f824deb155c 00007f824deb1508 00007f824deb1488 00007f824deb1443 00007f824deb06ec 00007f824cddaa96
0709 13:04:42.955136 (+614712us) raft_consensus.cc:1685] finished
0709 13:04:42.955161 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.955387 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":277102,"raft.queue_time_us":277265,"raft.run_cpu_time_us":4107,"raft.run_wall_time_us":419841,"spinlock_wait_cycles":5422464,"thread_start_us":199488,"threads_started":1,"wal-append.queue_time_us":584391}
I20260709 13:04:42.970824  7725 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.819622  7436 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:42.973673  7436 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:42.973773  1572 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:42.977727  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:42.978554  1572 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:42.982681  7688 raft_consensus.cc:3055] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:42.983311  7688 raft_consensus.cc:740] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:42.984196  7688 consensus_queue.cc:260] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:42.923408  7689 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:04:42.986048  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:42.986776  1722 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:42.972327  7560 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:42.987715  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:42.957628  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 3003) took 1044 ms. Trace:
I20260709 13:04:42.798115  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.990084  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2602) took 2061 ms. Trace:
I20260709 13:04:42.991216  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.990473  1717 rpcz_store.cc:276] 0709 13:04:40.928903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.689025 (+760122us) service_pool.cc:224] Handling call
0709 13:04:41.689432 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:41.689852 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:41.941419 (+251567us) log.cc:844] Serialized 14 byte log entry
0709 13:04:42.407880 (+466461us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:42.416214 (+  8334us) log.cc:844] Serialized 26 byte log entry
0709 13:04:42.416594 (+   380us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:42.416667 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.416994 (+   327us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:42.989855 (+572861us) raft_consensus.cc:1685] finished
0709 13:04:42.989882 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.990021 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7342,"prepare.queue_time_us":489086,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":7950,"raft.run_cpu_time_us":376,"raft.run_wall_time_us":106782,"thread_start_us":501532,"threads_started":2,"wal-append.queue_time_us":694282}
I20260709 13:04:42.992537  7738 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:42.992183  1571 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:42.993172  7625 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:42.994280  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.994107  7625 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:42.995731  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.996615  1571 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 16: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:42.989524  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:42.998059  7706 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:42.998257  1722 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:42.998720  7706 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:42.999610  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:43.000420  1722 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:43.001765  7706 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:43.002207  1718 rpcz_store.cc:276] 0709 13:04:41.913692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.449797 (+536105us) service_pool.cc:224] Handling call
0709 13:04:42.957551 (+507754us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:43.003304  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.997483  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:42.997251  7625 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:43.008898  1571 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:42.817026  7762 consensus_queue.cc:1059] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.137s
I20260709 13:04:43.015897  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:43.016953  1640 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 18: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:43.017903  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:43.019892  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:43.020615  1572 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:04:43.021807  1640 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 12.
I20260709 13:04:42.924746  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:43.012267  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:43.012475  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:43.013106  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:43.015007  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:43.023699  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:43.020851  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:43.022972  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:42.801801  7745 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.015364  1646 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
W20260709 13:04:43.024823  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 63232.8us
I20260709 13:04:43.028148  1645 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 16: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:43.023567  1642 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:43.028424  1537 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:43.023511  1714 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:43.033492  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:42.993475  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:43.035085  1717 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 16: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:43.041910  1537 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:43.012435  7720 raft_consensus.cc:697] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:43.046489  1611 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:43.069514  1537 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:43.045395  1714 raft_consensus.cc:3055] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:43.073089  1714 raft_consensus.cc:740] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:04:43.025909  7723 consensus_meta.cc:287] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.565s	user 0.015s	sys 0.001s
W20260709 13:04:43.072625  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 337410us
I20260709 13:04:43.056516  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 2997) took 1161 ms. Trace:
I20260709 13:04:43.072654  1611 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:43.083153  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 54931.4us
I20260709 13:04:43.076098  1714 consensus_queue.cc:260] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.086898  7706 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:43.087548  1714 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:43.025763  7675 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:43.031800  7688 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:43.043999  7654 consensus_meta.cc:287] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.155s	user 0.011s	sys 0.006s
W20260709 13:04:43.089771  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.043952  7761 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.084636  1721 rpcz_store.cc:276] 0709 13:04:41.895273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.530738 (+635465us) service_pool.cc:224] Handling call
0709 13:04:42.531173 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.531499 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:42.590811 (+ 59312us) log.cc:844] Serialized 14 byte log entry
0709 13:04:43.020326 (+429515us) log.cc:844] Serialized 14 byte log entry
0709 13:04:43.055772 (+ 35446us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:43.055949 (+   177us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:43.056034 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.056132 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:43.056299 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34888,"prepare.queue_time_us":61310,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":1866,"raft.queue_time_us":124981,"raft.run_cpu_time_us":393,"raft.run_wall_time_us":19444,"spinlock_wait_cycles":45221120,"thread_start_us":336523,"threads_started":1}
I20260709 13:04:43.001358  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:43.115042  1722 raft_consensus.cc:2410] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 17 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:04:43.081449  1641 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:04:43.011171  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:43.044183  7686 consensus_meta.cc:287] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.531s	user 0.009s	sys 0.005s
W20260709 13:04:43.102979  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 66437.8us
I20260709 13:04:43.120756  1718 raft_consensus.cc:2393] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 16: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:43.102972  1611 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:43.108841  1685 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:43.085191  1537 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:43.122763  1611 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:43.088806  7766 log.cc:927] Time spent T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:04:43.118103  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:43.130654  1722 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:04:43.133177  2314 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:43.044665  7689 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:43.116261  1612 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:43.123713  1687 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:43.107255  7573 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:04:43.100679  7675 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:04:43.151474  7729 consensus_meta.cc:287] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.579s	user 0.014s	sys 0.001s
I20260709 13:04:43.152405  7764 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:43.125995  7658 consensus_queue.cc:1059] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:04:43.153143  7764 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.153666  7675 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:43.112419  7709 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:04:43.152472  7729 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.158084  7777 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:43.158952  1644 raft_consensus.cc:2393] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 13: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:04:43.160832  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:43.135301  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 50292.8us
I20260709 13:04:43.163106  1537 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:43.163923  7729 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:43.161383  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:43.152786  7686 consensus_queue.cc:1059] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:04:43.165462  1644 raft_consensus.cc:2410] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 17 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 13:04:43.146368  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.146103  7720 consensus_queue.cc:237] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:43.134810  7626 consensus_queue.cc:1048] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:04:43.085157  7687 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:43.171670  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:43.134766  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2596) took 2829 ms. Trace:
I20260709 13:04:43.151160  1717 raft_consensus.cc:1275] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:43.155332  1687 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:43.142657  1685 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:43.160600  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:43.138481  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:43.173414  1715 rpcz_store.cc:276] 0709 13:04:40.305063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.584311 (+279248us) service_pool.cc:224] Handling call
0709 13:04:40.660692 (+ 76381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:40.662376 (+  1684us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.662411 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:42.272921 (+1610510us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b1800173da0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d662a6 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824777ea11 00007f824778e372 00007f8248ec0a01 00007f8248e8543a
0709 13:04:42.273506 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 13:04:42.672132 (+398626us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:42.728495 (+ 56363us) log.cc:844] Serialized 14 byte log entry
0709 13:04:42.756312 (+ 27817us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.756438 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:42.999672 (+243234us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b1800173da0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d98522 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:04:43.134519 (+134847us) raft_consensus.cc:1685] finished
0709 13:04:43.134547 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:43.134703 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68728,"prepare.queue_time_us":56115,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":4171,"raft.queue_time_us":41416,"raft.run_cpu_time_us":353,"raft.run_wall_time_us":43275,"spinlock_wait_cycles":49334656,"thread_start_us":398219,"threads_started":1,"wal-append.queue_time_us":650860}
I20260709 13:04:43.175477  1721 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16. Ops: [14.11-15.12]
I20260709 13:04:43.176165  1716 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:43.143999  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:43.178475  1639 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:43.137131  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 2993) took 1243 ms. Trace:
I20260709 13:04:43.180415  1720 rpcz_store.cc:276] 0709 13:04:41.894129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.271203 (+377074us) service_pool.cc:224] Handling call
0709 13:04:43.136929 (+865726us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b5400211954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:43.137057 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":816460032}
I20260709 13:04:43.151242  7640 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:43.136953  7714 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:43.187597  7636 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:43.104962  7634 consensus_queue.cc:1059] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 13:04:43.116572  7654 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.167518  1537 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:43.188962  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:43.199743  1641 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:43.200107  1714 raft_consensus.cc:1275] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 13:04:43.189042  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:43.201681  1718 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:04:43.194588  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:43.196463  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 80250.6us
I20260709 13:04:43.203744  1612 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:43.205030  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:43.132339  7725 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:43.188179  7636 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.202143  7560 consensus_queue.cc:1059] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:04:43.206732  7636 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:43.207384  7642 consensus_meta.cc:287] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.606s	user 0.013s	sys 0.000s
I20260709 13:04:43.207875  7642 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.202303  7723 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:43.208904  7723 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:43.209307  7723 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:43.189769  7597 consensus_queue.cc:1059] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:04:43.212059  7597 raft_consensus.cc:3055] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:43.212457  7597 raft_consensus.cc:740] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:43.204447  7669 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:43.125799  7685 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:43.216888  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:43.112840  7772 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:43.218578  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2685) took 1152 ms. Trace:
I20260709 13:04:43.219018  1568 rpcz_store.cc:276] 0709 13:04:42.066345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.066622 (+   277us) service_pool.cc:224] Handling call
0709 13:04:42.066979 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.905542 (+838563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.218270 (+312728us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b54002baa54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:43.218510 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1756984704}
I20260709 13:04:43.153666  7764 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:43.224886  7654 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:43.226080  1565 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:04:43.213322  7597 consensus_queue.cc:260] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.228449  7597 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:43.230564  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:43.231889  7687 raft_consensus.cc:697] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:43.233366  7687 consensus_queue.cc:237] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.239233  7714 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:43.134564  6987 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:43.135366  7652 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:43.196483  7572 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:43.190181  7709 raft_consensus.cc:3055] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:43.246034  7709 raft_consensus.cc:740] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:43.140808  7578 raft_consensus.cc:3055] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:43.246951  7709 consensus_queue.cc:260] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.247597  7578 raft_consensus.cc:740] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:43.198041  1647 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
W20260709 13:04:43.198233  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 55657.5us
W20260709 13:04:43.250110  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.250237  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:43.221504  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:43.252357  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 48709.6us
I20260709 13:04:43.252576  1643 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:43.253257  1612 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:43.255606  1612 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:43.257331  1646 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:04:43.258152  1612 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:43.259845  1717 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:43.260609  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:43.264333  1685 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:43.200547  1714 pending_rounds.cc:85] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 10
I20260709 13:04:43.143529  7738 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:43.278113  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:43.171196  7626 consensus_queue.cc:1243] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer 066ee0109aba4465ba17fa65e14ef504 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:04:43.277252  1716 raft_consensus.cc:1275] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:43.279327  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:43.275781  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:43.279013  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:43.254457  7737 consensus_meta.cc:287] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.626s	user 0.010s	sys 0.000s
I20260709 13:04:43.281739  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:43.282325  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:43.282451  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:43.270882  1714 pending_rounds.cc:107] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:04:43.311504  1714 raft_consensus.cc:2889] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:43.285321  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:43.287576  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:43.266438  1612 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:43.171200  7775 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:43.305173  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:43.315557  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:43.320802  1565 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 18: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:04:43.316131  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 373584us
I20260709 13:04:43.316825  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:43.263695  7573 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:43.319396  1716 raft_consensus.cc:1275] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:43.264108  6987 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:43.323964  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 57602us
I20260709 13:04:43.266652  7685 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:43.125056  7761 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.278646  7763 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:43.266376  7572 raft_consensus.cc:2764] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:04:43.328619  1538 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:43.270787  7706 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:04:43.334158  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 55052.4us
I20260709 13:04:43.354118  1685 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:43.330507  1612 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:43.288292  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:43.248575  7578 consensus_queue.cc:260] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:43.289499  7634 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:43.276729  7714 raft_consensus.cc:697] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:43.284363  7773 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.349936  1538 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:43.333380  7692 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:43.350867  1722 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:43.285537  7739 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:43.331597  7648 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 13:04:43.358927  1612 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:43.365815  7634 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:43.343839  7689 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.043s
W20260709 13:04:43.366050  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.354746  1643 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:04:43.366751  1537 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:43.320189  7775 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:43.408667  7775 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:43.410035  7714 consensus_queue.cc:237] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.413659  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:43.414067  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:43.414511  2345 raft_consensus.cc:2393] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 14: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:43.414939  7745 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:43.415680  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:43.331331  7729 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:43.416882  7729 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.417299  7729 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:43.417853  7654 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:43.418498  7654 raft_consensus.cc:697] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:43.419023  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:43.419776  2344 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 21.
I20260709 13:04:43.419560  7654 consensus_queue.cc:237] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.382575  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:43.409623  1716 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:04:43.349722  7688 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:43.380574  7642 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:43.391114  1612 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:04:43.441727  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 50728.4us
I20260709 13:04:43.442729  1612 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:43.284104  7652 raft_consensus.cc:697] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:43.399453  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:43.445212  1612 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:43.363179  1685 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:43.370306  7724 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:04:43.407023  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:04:43.421792  7779 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:04:43.358749  7569 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:04:43.414857  2347 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:43.423120  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:43.325747  7737 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.424546  1716 raft_consensus.cc:1275] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:04:43.405215  7706 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:04:43.424821  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 293020us
I20260709 13:04:43.386528  7692 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:04:43.418516  7759 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.602s	user 0.005s	sys 0.000s
I20260709 13:04:43.376711  7746 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:04:43.377295  7773 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.403261  7634 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:43.477274  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 114097us
I20260709 13:04:43.486852  1722 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:43.465299  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 412098us
I20260709 13:04:43.509187  7569 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:43.511927  7762 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:43.512791  7761 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:43.515246  1720 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:43.387916  7436 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:43.517598  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:43.521327  1685 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:43.491901  7692 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:43.475629  7572 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:43.487824  7688 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.486737  7658 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:43.526333  1719 raft_consensus.cc:2393] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:43.394243  7739 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:43.488751  7626 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:43.526544  1718 raft_consensus.cc:1275] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:04:43.528631  2312 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:43.532627  7573 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:43.493381  7745 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:43.493049  7706 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:43.535153  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 168418us
I20260709 13:04:43.560951  7761 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:43.540535  1573 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:43.446467  7772 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:04:43.542268  1685 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:43.498042  7777 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:43.504582  7642 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:04:43.529556  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:43.536392  7560 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:04:43.565810  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3008) took 1049 ms. Trace:
I20260709 13:04:43.551020  7572 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.537250  7762 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:43.536835  1720 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:04:43.582942  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:43.522171  7675 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:43.556162  7658 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:43.592217  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 50029.2us
I20260709 13:04:43.586112  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:43.516655  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 142412us
I20260709 13:04:43.613027  1565 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:04:43.600044  1713 rpcz_store.cc:276] 0709 13:04:42.516231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.668303 (+152072us) service_pool.cc:224] Handling call
0709 13:04:42.668652 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:42.669032 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.669061 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:42.669598 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 13:04:43.110898 (+441300us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:43.110937 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.111000 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:43.565563 (+454563us) raft_consensus.cc:1685] finished
0709 13:04:43.565585 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:43.565745 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5456,"raft.queue_time_us":146,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":57002,"thread_start_us":349136,"threads_started":1,"wal-append.queue_time_us":609492}
I20260709 13:04:43.613399  1685 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:43.593420  7787 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:43.579852  7706 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:43.604632  1718 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:43.620540  1568 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:04:43.559758  7688 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.615306  1685 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:43.577905  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 89093.3us
I20260709 13:04:43.605423  7572 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:43.591255  7764 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.602473  7761 raft_consensus.cc:697] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:43.621819  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:43.628319  1611 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:43.636580  6636 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:43.637481  1685 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:43.615842  7658 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:43.611595  7797 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:43.599680  7636 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:43.651218  1646 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:43.621459  7763 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:43.643627  6636 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:43.648188  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:43.648674  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:43.652045  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:43.659056  1717 raft_consensus.cc:1240] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:43.663317  7685 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:43.599058  7652 consensus_queue.cc:237] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:43.666589  1643 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:43.666316  1645 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:04:43.618857  7714 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:43.671643  6987 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.672233  7685 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:43.633422  7787 raft_consensus.cc:697] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:43.687493  2346 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:43.662200  7797 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:43.695616  7793 log.cc:927] Time spent T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:04:43.739918  7578 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:43.689080  7796 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:04:43.698616  7737 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:43.680788  6987 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.743765  7675 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:43.744087  6987 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:43.669672  7794 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.058s	user 0.001s	sys 0.001s
I20260709 13:04:43.738631  7651 raft_consensus.cc:3055] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:43.745072  7651 raft_consensus.cc:740] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:43.703312  7772 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:43.650434  7777 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:04:43.695765  7787 consensus_queue.cc:237] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:43.678643  7642 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.659816  7746 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:04:43.663343  7803 consensus_queue.cc:1048] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:04:43.765683  7651 consensus_queue.cc:260] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:43.628515  7669 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:04:43.715667  7788 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:43.716678  7602 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:43.717921  7782 log.cc:927] Time spent T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:04:43.718827  7762 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:43.720603  7723 consensus_meta.cc:287] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.511s	user 0.008s	sys 0.001s
I20260709 13:04:43.720992  7569 consensus_queue.cc:1059] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:43.721889  7784 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:43.723644  7625 raft_consensus.cc:3055] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:43.689388  7720 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:43.784006  7625 raft_consensus.cc:740] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:04:43.784525  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.715699  7685 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:43.730695  7761 consensus_queue.cc:237] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.784938  7625 consensus_queue.cc:260] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.786568  7625 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:43.734131  7739 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:43.734939  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.735711  7764 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:43.736835  7688 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:43.738623  7714 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:43.725162  7775 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.744242  7797 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:43.794554  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 166309us
I20260709 13:04:43.795023  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:43.663475  7687 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:43.736444  1467 catalog_manager.cc:5697] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:43.750397  1722 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:04:43.740841  7572 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.767580  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:43.769098  7651 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:43.772619  7597 consensus_meta.cc:287] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.544s	user 0.012s	sys 0.002s
I20260709 13:04:43.781767  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2619) took 1342 ms. Trace:
I20260709 13:04:43.804014  1569 rpcz_store.cc:276] 0709 13:04:42.439256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.825247 (+385991us) service_pool.cc:224] Handling call
0709 13:04:42.850902 (+ 25655us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:42.860628 (+  9726us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.860673 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:42.872212 (+ 11539us) log.cc:844] Serialized 26 byte log entry
0709 13:04:43.346188 (+473976us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:43.346228 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.346313 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:43.781399 (+435086us) raft_consensus.cc:1685] finished
0709 13:04:43.781432 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:43.781683 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10591,"raft.queue_time_us":9868,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":3002,"thread_start_us":421998,"threads_started":1,"wal-append.queue_time_us":588854}
I20260709 13:04:43.727161  7436 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:43.774605  7642 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.798834  7739 raft_consensus.cc:3055] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:43.799373  7729 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.800159  7775 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:43.767599  7636 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:43.795459  1611 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:43.803568  1640 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:43.810760  1612 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:43.814124  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:43.815303  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:43.797556  6636 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:43.834367  1722 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:04:43.839033  7762 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:43.841755  7686 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:43.843532  7686 consensus_queue.cc:1048] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:43.806910  7685 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:43.806821  7724 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:43.828536  7572 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:43.849342  7642 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:43.849622  7653 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.850558  7653 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.854501  1687 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:43.838416  7804 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:43.857297  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:43.859229  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:43.860663  1643 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:04:43.865046  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:43.866173  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:43.866797  1570 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:43.867452  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:43.868456  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:43.869405  1567 raft_consensus.cc:1275] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:04:43.869550  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:43.870136  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.870643  1640 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
I20260709 13:04:43.868652  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:43.810067  7739 raft_consensus.cc:740] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:43.873138  7729 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:43.874006  7739 consensus_queue.cc:260] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.875284  7739 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:43.792685  7723 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:43.877945  7669 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:43.859855  7634 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:43.879452  7723 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:43.880829  7634 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:43.860040  7692 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:43.884856  7692 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:43.853049  7653 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:43.885602  1571 raft_consensus.cc:2368] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 13. Current term is 14.
I20260709 13:04:43.885948  1720 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:43.886667  1572 pending_rounds.cc:85] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 8
I20260709 13:04:43.887089  1572 pending_rounds.cc:107] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:04:43.887336  1572 raft_consensus.cc:2889] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:43.849996  7642 consensus_queue.cc:1059] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:04:43.886715  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:43.889549  1571 raft_consensus.cc:2410] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:04:43.890751  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:43.894013  7636 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.895295  7636 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:43.882411  7626 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:43.889310  7808 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:43.900390  1687 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:43.909267  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2615) took 1021 ms. Trace:
I20260709 13:04:43.910189  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:43.910442  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:43.910838  1714 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:43.911943  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:43.909771  1719 rpcz_store.cc:276] 0709 13:04:42.887452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.921939 (+ 34487us) service_pool.cc:224] Handling call
0709 13:04:43.909004 (+987065us) spinlock_profiling.cc:243] Waited 597 ms on lock 0x7b5400407c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:43.909198 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1374163200}
I20260709 13:04:43.912241  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:43.913805  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:43.915352  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:43.917238  1687 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:43.920759  7607 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:43.921502  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:43.924906  1571 raft_consensus.cc:2410] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:43.920802  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:43.926256  1612 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:43.807447  7714 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:43.929986  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:43.930884  1640 raft_consensus.cc:1275] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:43.931202  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:43.936429  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:43.931998  1645 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:43.933125  1685 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:43.933519  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:43.938359  1722 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:43.928997  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:43.934013  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:43.938673  1687 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:43.939487  1642 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:43.942397  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:43.943450  1640 raft_consensus.cc:2368] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14.
I20260709 13:04:43.944494  1611 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:43.942533  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:43.945709  1645 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:43.958241  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:43.958947  1565 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:43.959860  1687 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:43.961805  1685 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:43.908670  7808 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:43.992574  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 66412.6us
W20260709 13:04:43.992792  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 48389.9us
I20260709 13:04:43.974318  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:43.993906  1611 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:43.908783  7773 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:43.921729  7692 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:43.998152  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:43.998899  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:43.998270  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:44.000543  1643 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:44.002763  1611 leader_election.cc:400] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 13. Current term is 14.
I20260709 13:04:44.003325  1611 leader_election.cc:403] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:04:44.000125  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:44.007763  1713 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:44.012403  6636 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:04:44.013743  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 51996us
I20260709 13:04:44.015100  1685 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:44.015830  1645 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:44.010499  7686 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:44.018900  7761 consensus_queue.cc:1048] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:44.020862  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:43.988821  7669 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:44.002533  7788 consensus_queue.cc:1059] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:04:43.966012  7651 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:44.022807  7651 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.023227  7651 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:44.024662  7714 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:44.024863  7725 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:44.027426  7636 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:44.006598  7626 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:44.004127  7808 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:44.030339  7799 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.346s	user 0.000s	sys 0.002s
I20260709 13:04:44.004323  7634 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:44.032224  7634 raft_consensus.cc:697] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:43.976661  7607 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:44.042702  1685 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:44.030429  7723 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.042122  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:44.049809  1642 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:44.050127  1565 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:44.052212  6636 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:44.052594  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49953.5us
I20260709 13:04:44.052098  1685 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:44.044586  7607 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:44.078233  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.033207  7634 consensus_queue.cc:237] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:44.085765  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.058079  7642 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:44.087705  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:44.096446  1642 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:04:44.054488  7685 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 13. Current term is 14.
I20260709 13:04:44.061853  7669 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:44.095386  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:44.098044  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:44.105495  1639 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:04:44.106232  1722 raft_consensus.cc:2410] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 18 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:44.055604  7761 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:44.134691  1565 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:44.127023  1687 leader_election.cc:400] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14.
I20260709 13:04:44.139194  1687 leader_election.cc:403] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
W20260709 13:04:44.139725  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 48615.5us
I20260709 13:04:44.139905  7648 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:44.140686  1687 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:44.055478  7706 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:44.077857  7738 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:44.141544  1687 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:44.146171  1687 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:44.087306  7607 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:44.146466  6636 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:44.150329  6636 raft_consensus.cc:697] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:44.067442  7723 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:44.106874  7578 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.174665  6636 consensus_queue.cc:237] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:44.176004  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.162631  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:44.170122  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.192160  1565 raft_consensus.cc:2393] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:44.204619  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:44.209957  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 48993us
I20260709 13:04:44.099926  7658 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:44.214912  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:44.216611  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:44.217346  1565 raft_consensus.cc:2368] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14.
W20260709 13:04:44.196602  7816 log.cc:927] Time spent T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:04:44.221503  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:44.171721  7767 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:44.250128  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:44.253288  1565 raft_consensus.cc:1275] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:44.194309  7578 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:44.219463  7763 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:04:44.252954  7725 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:44.304075  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:44.304430  7817 log.cc:927] Time spent T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 13:04:44.304636  7625 consensus_meta.cc:287] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.517s	user 0.015s	sys 0.005s
W20260709 13:04:44.306411  7819 log.cc:927] Time spent T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
W20260709 13:04:44.300112  7791 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.078s	user 0.000s	sys 0.009s
I20260709 13:04:44.362949  1568 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:44.326639  7762 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:44.351840  7689 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:44.480777  7687 consensus_meta.cc:287] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.678s	user 0.014s	sys 0.002s
I20260709 13:04:44.367267  7739 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:04:44.555244  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:44.581681  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:44.484762  7825 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.125s	user 0.001s	sys 0.001s
I20260709 13:04:44.511897  7687 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:44.595148  7795 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:44.615507  7795 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:44.620352  7823 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:04:44.623344  1648 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:44.512640  7714 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:44.592558  7767 raft_consensus.cc:697] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:44.544695  7813 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:44.602803  7706 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:44.492719  7689 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:44.667958  7651 consensus_meta.cc:287] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.644s	user 0.012s	sys 0.000s
W20260709 13:04:44.642444  7815 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.498s	user 0.002s	sys 0.008s
I20260709 13:04:44.595747  7739 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:44.639760  7821 log.cc:927] Time spent T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.357s	user 0.000s	sys 0.002s
W20260709 13:04:44.697129  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:44.699339  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:44.700338  1640 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.018s	sys 0.066s
I20260709 13:04:44.706575  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3016) took 1451 ms. Trace:
W20260709 13:04:44.707295  1640 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.018s	sys 0.066s
I20260709 13:04:44.707484  1714 rpcz_store.cc:276] 0709 13:04:43.255308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.312995 (+ 57687us) service_pool.cc:224] Handling call
0709 13:04:43.910632 (+597637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:43.911302 (+   670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.701452 (+790150us) spinlock_profiling.cc:243] Waited 597 ms on lock 0x7b5400407c50. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:44.706447 (+  4995us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1372798208}
I20260709 13:04:44.668620  7651 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:44.670110  7822 log.cc:927] Time spent T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.418s	user 0.001s	sys 0.001s
W20260709 13:04:44.725765  7825 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:04:44.756696  1716 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
W20260709 13:04:44.760802  7781 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 1.243s	user 0.009s	sys 0.000s
I20260709 13:04:44.734105  7658 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:44.636940  7714 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:44.764961  7828 log.cc:927] Time spent T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:04:44.765789  1566 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:04:44.765965  7824 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:04:44.734814  7737 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.725427  7834 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:44.773003  1568 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:04:44.773743  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2623) took 2317 ms. Trace:
W20260709 13:04:44.787390  1647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2648) took 1775 ms (client timeout 2046 ms). Trace:
W20260709 13:04:44.792095  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.800132  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55890 (request call id 577) took 1454 ms. Trace:
I20260709 13:04:44.777525  7709 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:44.810746  7826 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:04:44.801136  7739 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:44.820039  1467 catalog_manager.cc:5697] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 4 to 6, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 6 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:44.776798  7767 consensus_queue.cc:237] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:44.656644  7795 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:44.835116  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 195963us
W20260709 13:04:44.806651  7821 log.cc:927] Time spent T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:04:44.795347  7578 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:44.769562  7648 consensus_meta.cc:287] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.629s	user 0.011s	sys 0.000s
I20260709 13:04:44.797361  7723 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:44.787865  7689 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:44.847134  1572 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.003s	sys 0.013s
W20260709 13:04:44.859088  1647 rpcz_store.cc:269] 0709 13:04:43.012147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.012271 (+   124us) service_pool.cc:224] Handling call
0709 13:04:44.787058 (+1774787us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54001ff9d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:44.787277 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":252542208}
W20260709 13:04:44.896250  1572 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.005s	sys 0.013s
I20260709 13:04:44.861387  1568 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:04:44.836349  7808 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.806s	user 0.013s	sys 0.003s
I20260709 13:04:44.858284  1573 rpcz_store.cc:276] 0709 13:04:42.456687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.908968 (+452281us) service_pool.cc:224] Handling call
0709 13:04:42.909257 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:43.589041 (+679784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.773408 (+1184367us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b5400012054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:44.773672 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1450587520}
I20260709 13:04:44.857437  2346 rpcz_store.cc:276] 0709 13:04:43.346014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.346196 (+   182us) service_pool.cc:224] Handling call
0709 13:04:43.350218 (+  4022us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:43.688061 (+337843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.799727 (+1111666us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b54006d0454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:44.800036 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":774828800}
I20260709 13:04:44.651278  7813 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:44.891474  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.894295  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2653) took 1036 ms. Trace:
I20260709 13:04:44.901919  1569 rpcz_store.cc:276] 0709 13:04:43.857532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.857683 (+   151us) service_pool.cc:224] Handling call
0709 13:04:43.876866 (+ 19183us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:43.901397 (+ 24531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:43.901444 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:43.906476 (+  5032us) log.cc:844] Serialized 26 byte log entry
0709 13:04:44.086663 (+180187us) spinlock_profiling.cc:243] Waited 1.42 ms on lock 0x7b1800482500. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a250d1 00007f8248e40a0f
0709 13:04:44.137552 (+ 50889us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:44.137660 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.137750 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:44.874250 (+736500us) raft_consensus.cc:1685] finished
0709 13:04:44.874281 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:44.893311 (+ 19030us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24834,"raft.queue_time_us":29253,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":12176,"spinlock_wait_cycles":3258752,"thread_start_us":50395,"threads_started":1,"wal-append.queue_time_us":287689}
I20260709 13:04:44.864553  7737 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:44.912714  7737 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:44.914188  7692 consensus_meta.cc:287] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.919s	user 0.009s	sys 0.012s
I20260709 13:04:44.914762  7692 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:44.902750  7648 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14.
I20260709 13:04:44.916289  7763 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.916950  7763 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:44.924261  7795 consensus_queue.cc:1059] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:44.925269  7795 raft_consensus.cc:3055] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:44.925580  7795 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:44.872071  7834 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:44.929730  7689 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.935607  7714 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:44.935827  7689 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:44.881973  7739 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.938036  7739 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:44.940943  7739 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:44.943964  7813 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:44.903982  7808 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:44.951442  7763 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:44.955649  7837 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.145s	user 0.000s	sys 0.002s
I20260709 13:04:44.957502  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:44.958680  1715 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:44.960003  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:44.961139  2346 raft_consensus.cc:2410] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 13:04:44.962093  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2629) took 2014 ms. Trace:
I20260709 13:04:44.962172  2340 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:04:44.961200  1722 raft_consensus.cc:1275] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:04:44.960289  7607 consensus_meta.cc:287] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.763s	user 0.012s	sys 0.002s
I20260709 13:04:44.966536  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2620) took 2523 ms. Trace:
I20260709 13:04:44.969393  1564 rpcz_store.cc:276] 0709 13:04:42.443164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.826864 (+383700us) service_pool.cc:224] Handling call
0709 13:04:42.827392 (+   528us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:42.828112 (+   720us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.828176 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:42.828889 (+   713us) log.cc:844] Serialized 26 byte log entry
0709 13:04:43.285536 (+456647us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:43.285582 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.285688 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:44.759860 (+1474172us) spinlock_profiling.cc:243] Waited 5.4 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:44.929846 (+169986us) raft_consensus.cc:1685] finished
0709 13:04:44.929875 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:44.966458 (+ 36583us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":219,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":4918,"spinlock_wait_cycles":12409088,"thread_start_us":421495,"threads_started":1,"wal-append.queue_time_us":608189}
I20260709 13:04:44.973101  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:44.973891  1715 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:44.975184  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:44.962543  1721 rpcz_store.cc:276] 0709 13:04:42.948096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.176596 (+228500us) service_pool.cc:224] Handling call
0709 13:04:43.177132 (+   536us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:43.178088 (+   956us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:43.725607 (+547519us) spinlock_profiling.cc:243] Waited 58.3 ms on lock 0x7b18002a7a20. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f824c87d3c7 00007f8248e8f9a7 00007f8248e9f0bf 00007f8248e9f038
0709 13:04:43.917167 (+191560us) log.cc:844] Serialized 14 byte log entry
0709 13:04:44.060895 (+143728us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:44.072309 (+ 11414us) log.cc:844] Serialized 26 byte log entry
0709 13:04:44.072471 (+   162us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:44.072516 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.072605 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:44.961737 (+889132us) raft_consensus.cc:1685] finished
0709 13:04:44.961852 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:44.962004 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":228889,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":16039,"raft.run_cpu_time_us":390,"raft.run_wall_time_us":13157,"spinlock_wait_cycles":134048640,"thread_start_us":281530,"threads_started":2,"wal-append.queue_time_us":266783}
I20260709 13:04:44.967402  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:44.968251  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:44.928067  7834 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:44.989111  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.926656  7795 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:44.990541  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:44.993080  1714 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
W20260709 13:04:44.998003  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 328315us
I20260709 13:04:44.968596  7808 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:44.972090  7706 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.002697  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:45.004590  7706 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.006971  7706 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:44.977110  7607 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.010353  7795 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:45.012156  7607 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:45.014006  7839 consensus_queue.cc:1059] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:45.015100  7839 raft_consensus.cc:3055] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:45.015475  7839 raft_consensus.cc:740] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:45.017869  1719 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:04:45.020677  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:45.021528  1721 raft_consensus.cc:2393] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:45.016419  7839 consensus_queue.cc:260] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:45.022682  7839 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:45.035424  2314 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:45.038750  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:45.041337  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:45.044623  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.048753  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:45.050648  7706 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.052233  1714 raft_consensus.cc:2410] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:04:45.055578  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:45.053491  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:45.063681  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:45.051265  7706 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.044977  7689 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:45.073436  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.070415  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:45.077346  1722 raft_consensus.cc:1275] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:04:45.078415  1722 pending_rounds.cc:85] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 7
I20260709 13:04:45.079597  1722 pending_rounds.cc:107] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:04:45.080564  1722 raft_consensus.cc:2889] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:45.111562  1612 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:45.114241  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.117882  1721 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:45.121563  1612 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:45.130455  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3023) took 1810 ms. Trace:
I20260709 13:04:45.131178  1717 rpcz_store.cc:276] 0709 13:04:43.319726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.542709 (+222983us) service_pool.cc:224] Handling call
0709 13:04:43.543094 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:43.669304 (+126210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.130026 (+1460722us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b5400248954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:45.130351 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":236834048}
I20260709 13:04:45.106940  7671 raft_consensus.cc:3055] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:45.144883  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:45.146075  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:45.123883  7578 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:45.124421  7689 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.128008  7844 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.149034  7671 raft_consensus.cc:740] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:04:45.155766  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 120371us
W20260709 13:04:45.168686  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.156148  7642 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:45.179299  7724 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:45.179682  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2639) took 2674 ms. Trace:
I20260709 13:04:45.205920  1567 rpcz_store.cc:276] 0709 13:04:42.505955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.968524 (+462569us) service_pool.cc:224] Handling call
0709 13:04:42.968997 (+   473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:43.869827 (+900830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.179380 (+1309553us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b54002242d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:45.179609 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2068822912}
I20260709 13:04:45.180523  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3034) took 1249 ms. Trace:
I20260709 13:04:45.207255  1719 rpcz_store.cc:276] 0709 13:04:43.931293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.005251 (+ 73958us) service_pool.cc:224] Handling call
0709 13:04:44.028589 (+ 23338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:45.018446 (+989857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.180234 (+161788us) spinlock_profiling.cc:243] Waited 988 ms on lock 0x7b5400249354. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:45.180459 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2273310592}
I20260709 13:04:45.157341  7845 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.205937  7724 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:45.212809  7783 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 1.708s	user 0.006s	sys 0.005s
I20260709 13:04:45.184441  1714 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:45.214708  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2645) took 1402 ms. Trace:
I20260709 13:04:45.185114  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:45.215112  1713 rpcz_store.cc:276] 0709 13:04:43.812581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.825219 (+ 12638us) service_pool.cc:224] Handling call
0709 13:04:43.825701 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:44.011959 (+186258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.214478 (+1202519us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b54002c60d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:45.214646 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":399444224}
I20260709 13:04:45.215844  1717 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:04:45.216591  7725 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:45.183101  7636 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:45.217753  7636 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:45.196408  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.221280  7767 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:45.222553  7636 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:45.159533  7689 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:45.223191  7833 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.427s	user 0.003s	sys 0.000s
I20260709 13:04:45.224656  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2714) took 1310 ms. Trace:
I20260709 13:04:45.224920  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3037) took 1289 ms. Trace:
I20260709 13:04:45.186753  7840 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.204681  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:45.225200  1565 rpcz_store.cc:276] 0709 13:04:43.913890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.254161 (+340271us) service_pool.cc:224] Handling call
0709 13:04:44.254448 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:44.255011 (+   563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:44.255040 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:44.255608 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 13:04:44.537974 (+282366us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:44.538032 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.538139 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:45.222575 (+684436us) spinlock_profiling.cc:243] Waited 8.85 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:45.224202 (+  1627us) raft_consensus.cc:1685] finished
0709 13:04:45.224236 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:45.224572 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12991,"raft.queue_time_us":11160,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":3360,"spinlock_wait_cycles":20364672,"thread_start_us":236941,"threads_started":1,"wal-append.queue_time_us":443281}
I20260709 13:04:45.164758  7844 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.228983  7844 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:45.231909  1467 catalog_manager.cc:5697] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:45.233160  7804 raft_consensus.cc:3055] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:04:45.235165  7842 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:04:45.237013  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2716) took 1322 ms. Trace:
I20260709 13:04:45.201870  7795 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.237565  1572 rpcz_store.cc:276] 0709 13:04:43.914452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.308427 (+393975us) service_pool.cc:224] Handling call
0709 13:04:44.308850 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:44.309108 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:44.309138 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:44.309717 (+   579us) log.cc:844] Serialized 26 byte log entry
0709 13:04:44.896687 (+586970us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:44.896731 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.896800 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:45.234379 (+337579us) spinlock_profiling.cc:243] Waited 31.3 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:45.236577 (+  2198us) raft_consensus.cc:1685] finished
0709 13:04:45.236613 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:45.236938 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71961344,"thread_start_us":586535,"threads_started":1,"wal-append.queue_time_us":753670}
I20260709 13:04:45.238647  7795 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.241240  7795 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:45.222141  7767 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:45.287103  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2666) took 1195 ms. Trace:
I20260709 13:04:45.294315  1566 rpcz_store.cc:276] 0709 13:04:44.091807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.852328 (+760521us) service_pool.cc:224] Handling call
0709 13:04:44.879307 (+ 26979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:44.879675 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:45.107540 (+227865us) log.cc:844] Serialized 14 byte log entry
0709 13:04:45.286683 (+179143us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:45.286748 (+    65us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:45.286814 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.286888 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:45.287038 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":295378,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":463,"thread_start_us":380122,"threads_started":2,"wal-append.queue_time_us":145999}
I20260709 13:04:45.226334  7840 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:45.217173  7725 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:45.210070  7845 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.202966  7671 consensus_queue.cc:260] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:45.616184  7714 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:45.616602  7671 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:45.617103  7714 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:45.618265  7845 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:45.619408  7572 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:45.619601  7714 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:45.619992  7572 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.204829  7706 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:45.245982  7804 raft_consensus.cc:740] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:45.246578  1716 rpcz_store.cc:276] 0709 13:04:43.935089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.134666 (+199577us) service_pool.cc:224] Handling call
0709 13:04:44.135192 (+   526us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:44.830798 (+695606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.224657 (+393859us) spinlock_profiling.cc:243] Waited 621 ms on lock 0x7b54003903d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:45.224856 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1427619712}
I20260709 13:04:45.225314  7689 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:45.260430  1720 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:45.277699  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:45.622810  7804 consensus_queue.cc:260] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.273388  1685 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:45.281555  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:45.154685  1612 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:45.624794  2347 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:45.625432  7745 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:45.626117  7706 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:45.622756  7572 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:45.624820  7804 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:45.625020  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 351594us
I20260709 13:04:45.628151  1685 leader_election.cc:400] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14.
I20260709 13:04:45.630918  1685 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:45.632716  7838 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:45.638131  7607 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:45.633124  2347 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 22.
I20260709 13:04:45.633230  7838 raft_consensus.cc:697] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:04:45.601928  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 151732us
I20260709 13:04:45.642124  7689 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.642709  7689 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:45.643738  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:45.423363  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 241296us
I20260709 13:04:45.292574  7652 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.236335  1467 catalog_manager.cc:5697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:45.402266  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.645499  7689 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:45.646621  7838 consensus_queue.cc:237] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:45.443941  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 274510us
I20260709 13:04:45.605898  2343 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:45.651206  2314 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:45.652216  7839 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:45.652822  7839 raft_consensus.cc:697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:45.653820  7839 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:45.655758  7607 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:45.605300  7642 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.292783  7436 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.621840  7578 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:45.625490  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 470735us
I20260709 13:04:45.150121  7846 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:45.254849  1466 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38666 (request call id 216) took 33506 ms (client timeout 15000 ms). Trace:
I20260709 13:04:45.631583  7745 consensus_queue.cc:1048] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:04:45.646027  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.703586  1612 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:45.663180  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:45.719714  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:45.711889  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 60772.3us
I20260709 13:04:45.722189  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:45.722921  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:45.725656  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:45.727378  1572 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 17. Current term is 18. Ops: [16.10-17.11]
I20260709 13:04:45.657156  7840 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:45.661829  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:45.660451  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:45.758388  2344 raft_consensus.cc:2410] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
W20260709 13:04:45.615507  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:45.712124  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 49589.8us
I20260709 13:04:45.730175  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:45.731114  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:45.730625  1715 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:45.734607  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:45.737484  1569 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:45.739383  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:45.759802  1565 raft_consensus.cc:2410] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:04:45.760954  2342 raft_consensus.cc:2410] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:04:45.659636  7652 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.759743  2313 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:45.742471  7745 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:45.717095  7572 consensus_queue.cc:1048] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
W20260709 13:04:45.744778  7786 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 2.242s	user 0.016s	sys 0.013s
I20260709 13:04:45.659797  7642 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.753768  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:45.756577  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2720) took 1840 ms. Trace:
W20260709 13:04:45.601495  7709 consensus_meta.cc:287] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.728s	user 0.003s	sys 0.016s
I20260709 13:04:45.761616  1572 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:04:45.761739  1648 raft_consensus.cc:1275] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:45.713752  7846 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:45.716180  7847 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.759589  1719 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:45.773921  1571 rpcz_store.cc:276] 0709 13:04:43.915741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.779176 (+863435us) service_pool.cc:224] Handling call
0709 13:04:44.779612 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:45.064630 (+285018us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:45.064688 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:45.110412 (+ 45724us) log.cc:844] Serialized 26 byte log entry
0709 13:04:45.613858 (+503446us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:45.613901 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.613967 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:45.756324 (+142357us) raft_consensus.cc:1685] finished
0709 13:04:45.756344 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:45.756507 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108358,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":31562,"thread_start_us":553148,"threads_started":2,"wal-append.queue_time_us":586930}
I20260709 13:04:45.715631  7578 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.771477  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:45.747738  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:45.776861  1721 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:04:45.777503  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:45.780836  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:45.728971  7706 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:45.782665  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:45.779940  2349 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:45.781376  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:45.782258  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:45.784754  1564 raft_consensus.cc:2410] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:04:45.784760  1643 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:45.787813  1567 raft_consensus.cc:1240] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: []
W20260709 13:04:45.702257  1466 rpcz_store.cc:269] 0709 13:04:11.749169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.755349 (+  6180us) service_pool.cc:224] Handling call
0709 13:04:45.254531 (+33499182us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:33.622076 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:33.697077 (+ 75001us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:33.697135 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:04:33.697154 (+    19us) tablet.cc:662] Decoding operations
0709 13:04:33.814019 (+116865us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:33.814160 (+   141us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:33.814188 (+    28us) tablet.cc:685] Acquiring locks for 32 operations
0709 13:04:33.896398 (+ 82210us) tablet.cc:701] Row locks acquired
0709 13:04:33.903094 (+  6696us) write_op.cc:260] PREPARE: finished
0709 13:04:33.926157 (+ 23063us) write_op.cc:270] Start()
0709 13:04:33.933189 (+  7032us) write_op.cc:276] Timestamp: P: 1783602273926054 usec, L: 0
0709 13:04:33.933370 (+   181us) op_driver.cc:348] REPLICATION: starting
0709 13:04:33.934390 (+  1020us) log.cc:844] Serialized 12653 byte log entry
0709 13:04:35.722694 (+1788304us) spinlock_profiling.cc:243] Waited 6.27 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:36.150415 (+427721us) op_driver.cc:464] REPLICATION: finished
0709 13:04:36.401196 (+250781us) write_op.cc:301] APPLY: starting
0709 13:04:36.401375 (+   179us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:36.987816 (+586441us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:37.003233 (+ 15417us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:37.685880 (+682647us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:37.686409 (+   529us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=64,key_file_lookups=64,delta_file_lookups=64,mrs_lookups=0
0709 13:04:37.686482 (+    73us) write_op.cc:312] APPLY: finished
0709 13:04:37.687251 (+   769us) log.cc:844] Serialized 276 byte log entry
0709 13:04:37.781561 (+ 94310us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:37.808028 (+ 26467us) write_op.cc:454] Released schema lock
0709 13:04:37.812808 (+  4780us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:37.813392 (+   584us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:37.813433 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":828421,"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":238542,"apply.run_cpu_time_us":78948,"apply.run_wall_time_us":1412212,"cfile_cache_hit":130,"cfile_cache_hit_bytes":137951,"num_ops":32,"prepare.queue_time_us":386373,"prepare.run_cpu_time_us":81832,"prepare.run_wall_time_us":767536,"raft.queue_time_us":365543,"raft.run_cpu_time_us":14338,"raft.run_wall_time_us":196919,"replication_time_us":2216615,"spinlock_wait_cycles":14410624,"thread_start_us":1276882,"threads_started":5,"wal-append.queue_time_us":770588,"wal-append.run_cpu_time_us":74168,"wal-append.run_wall_time_us":3769377}]]}
I20260709 13:04:45.699317  7636 consensus_queue.cc:1059] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 13:04:45.711918  7436 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.792753  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:45.793058  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:45.763026  7795 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:45.767167  7840 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.800998  7572 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.803454  7840 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:45.805874  7840 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:45.765389  7804 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:45.806833  7804 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:45.808504  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:45.808947  7804 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:45.797175  7709 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:45.814296  7709 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.732898  7850 consensus_queue.cc:1059] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 13:04:45.818454  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3030) took 1891 ms. Trace:
I20260709 13:04:45.778139  7714 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:04:45.818930  7724 consensus_meta.cc:287] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.608s	user 0.018s	sys 0.000s
I20260709 13:04:45.819332  7714 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.819525  7724 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.819015  1720 rpcz_store.cc:276] 0709 13:04:43.926972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.927563 (+   591us) service_pool.cc:224] Handling call
0709 13:04:43.928027 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:45.623421 (+1695394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.818168 (+194747us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400172854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:45.818378 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3061717376}
I20260709 13:04:45.823055  7725 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.825261  1566 raft_consensus.cc:1275] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:45.803589  7572 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.827055  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:45.829085  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 115022us
I20260709 13:04:45.814777  7767 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:45.789848  7789 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:45.779981  7846 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:45.836025  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:45.836463  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2678) took 1741 ms (client timeout 1667 ms). Trace:
I20260709 13:04:45.836817  1645 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:45.784320  7847 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.843767  1640 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:04:45.836895  1643 rpcz_store.cc:269] 0709 13:04:44.094809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.097758 (+  2949us) service_pool.cc:224] Handling call
0709 13:04:45.836260 (+1738502us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b54002ab7d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:45.836381 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3859007744}
W20260709 13:04:45.832155  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 174827us
I20260709 13:04:45.789943  7658 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.832065  7634 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.783941  7845 consensus_queue.cc:1059] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:04:45.834151  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:45.829766  7709 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:45.834352  7671 raft_consensus.cc:3055] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:45.788545  7642 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:45.821555  7714 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:45.889067  1648 raft_consensus.cc:2410] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 16 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 13:04:45.893221  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 133627us
I20260709 13:04:45.832916  7436 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:45.888652  7671 raft_consensus.cc:740] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:45.896456  7671 consensus_queue.cc:260] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:45.898698  7671 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:45.841200  7652 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:45.905107  7436 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:45.905226  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:45.906159  7436 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.906695  7725 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:45.834334  7850 raft_consensus.cc:3055] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:45.835808  7789 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.828562  7795 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:45.925501  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:45.927627  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:45.929232  1648 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:45.928398  1640 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:04:45.933319  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:45.935282  1720 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:04:45.938242  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:45.939210  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 322976us
I20260709 13:04:45.914279  7845 raft_consensus.cc:3055] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:45.937423  2313 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:45.912766  7651 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:45.913828  7850 raft_consensus.cc:740] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:45.922629  7789 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:45.886165  7658 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.953167  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:45.928406  7724 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:45.954455  1648 raft_consensus.cc:2393] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:45.949585  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:45.955657  1643 raft_consensus.cc:2410] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:45.958164  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:45.959108  1642 raft_consensus.cc:2376] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15. Re-sending same reply.
I20260709 13:04:45.888495  7634 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:45.961669  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:45.961877  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:45.962575  1644 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:45.964560  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:45.965068  1641 raft_consensus.cc:2393] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 13: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:45.931962  7838 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:45.967051  1640 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 22.
I20260709 13:04:45.946573  7845 raft_consensus.cc:740] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:45.952261  7636 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:04:45.964215  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 180639us
W20260709 13:04:45.969148  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 250473us
I20260709 13:04:45.994431  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:45.995450  1640 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:45.996722  1645 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:04:45.989228  1537 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:46.004501  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:46.005159  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:46.021884  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:46.021958  1640 raft_consensus.cc:2376] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14. Re-sending same reply.
I20260709 13:04:46.023718  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:46.024670  1640 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
W20260709 13:04:46.011605  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.013399  1647 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:04:46.026912  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:46.028157  1640 raft_consensus.cc:2410] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:04:46.013800  1644 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:46.011690  1648 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:46.041604  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:46.044369  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:46.046793  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:45.989077  7636 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.047484  2346 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:04:46.045240  2340 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:45.977172  7738 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:46.010668  7723 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.002471  7706 raft_consensus.cc:697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:46.039608  7652 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:46.036204  7657 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:46.062659  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:45.976219  7838 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.044407  7436 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:46.054533  7738 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:46.052628  7636 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:46.083176  2343 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:04:46.083473  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:46.083822  2343 pending_rounds.cc:85] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Aborting all ops after (but not including) 10
I20260709 13:04:46.084733  2343 pending_rounds.cc:107] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:04:46.085168  2343 raft_consensus.cc:2889] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:46.084324  2342 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:46.090569  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:46.068485  7652 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:46.073606  7573 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.105479  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:46.095062  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:46.106616  1720 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:46.098598  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:46.094024  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:46.109377  1716 raft_consensus.cc:2410] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:04:46.078243  7724 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.072552  7572 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:46.108253  1719 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:04:46.120290  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:46.088600  7847 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:46.039846  7850 consensus_queue.cc:260] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:46.123735  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:46.124478  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:46.089745  7861 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:46.067269  7723 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.103920  7652 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:46.079270  7657 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:46.102135  7866 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:46.066318  7845 consensus_queue.cc:260] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.145309  7795 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:46.134094  7865 consensus_queue.cc:1059] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:04:46.121167  7724 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.162823  7845 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:46.164495  7724 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:46.134402  7861 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.179797  7861 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:46.231101  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.235831  1715 raft_consensus.cc:2410] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:04:46.285300  7880 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.287125  7880 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:46.288833  7578 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:46.121434  7573 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:46.294454  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:46.268716  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 222099us
I20260709 13:04:46.327657  7657 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:46.359776  7845 consensus_queue.cc:1059] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:04:46.364634  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 315416us
W20260709 13:04:46.365820  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 323549us
I20260709 13:04:46.372246  7573 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:46.327723  7879 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:46.369716  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 178141us
W20260709 13:04:46.397872  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.398313  7723 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:46.373515  7838 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:46.398903  7651 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:46.400210  7651 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:46.402043  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.403568  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:46.412834  7847 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:46.414435  7847 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:46.417290  1686 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:46.421404  7745 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:46.422827  7578 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.423740  7578 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.426012  1686 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:46.439198  7634 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:46.452432  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:46.455745  7883 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.149s	user 0.002s	sys 0.000s
I20260709 13:04:46.413421  7879 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.459496  7861 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:46.460414  7861 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.460857  7879 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:46.462051  7893 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.462944  7893 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.464500  7880 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:46.466058  7724 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:46.466964  7724 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.467514  7578 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:46.468721  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.128706  7706 consensus_queue.cc:237] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.469889  7607 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:46.470290  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:46.470815  7607 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.472790  1569 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 13:04:46.473196  1569 pending_rounds.cc:85] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 12
I20260709 13:04:46.471232  1566 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 21.
I20260709 13:04:46.473433  1569 pending_rounds.cc:107] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:04:46.480708  1569 raft_consensus.cc:2889] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:46.465396  7893 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:46.500391  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.504870  7651 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:46.473577  7607 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:46.509835  7880 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.510675  7880 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.512887  7636 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:46.478762  7658 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:46.479904  7724 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:46.517405  1571 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:46.516245  7864 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:04:46.514689  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 104396us
W20260709 13:04:46.528615  7878 log.cc:927] Time spent T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.074s	user 0.002s	sys 0.000s
I20260709 13:04:46.529999  7845 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.513510  7880 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:46.523046  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 111924us
I20260709 13:04:46.529881  7864 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.530829  7845 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:46.542759  7642 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:04:46.543540  7886 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.227s	user 0.002s	sys 0.000s
I20260709 13:04:46.545884  1611 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:46.546135  7885 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.128s	user 0.003s	sys 0.000s
I20260709 13:04:46.548156  7642 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:46.553144  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:46.554247  1715 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:46.554025  7578 consensus_queue.cc:1059] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 14.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:04:46.555132  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 65561.5us
I20260709 13:04:46.556001  7578 raft_consensus.cc:3055] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:46.556576  7578 raft_consensus.cc:740] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:46.522931  7893 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:46.562777  7578 consensus_queue.cc:260] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:46.563338  7846 consensus_meta.cc:287] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.637s	user 0.005s	sys 0.013s
I20260709 13:04:46.564544  7578 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:46.564065  7846 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:46.565811  1612 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:46.568012  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 53747.2us
I20260709 13:04:46.574164  7436 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:46.569443  1612 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:46.584944  2341 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:46.585554  7894 log.cc:927] Time spent T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:04:46.588059  7861 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:46.588680  2341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 801) took 1387 ms. Trace:
I20260709 13:04:46.590065  7689 raft_consensus.cc:3055] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:46.589182  2341 rpcz_store.cc:276] 0709 13:04:45.200955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.202910 (+  1955us) service_pool.cc:224] Handling call
0709 13:04:45.203361 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:46.585915 (+1382554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.588226 (+  2311us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b5400651b54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:46.588520 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3157179136}
I20260709 13:04:46.591192  1715 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:04:46.592360  7689 raft_consensus.cc:740] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:46.594573  2312 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:46.596062  7862 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:46.596849  7862 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:46.597378  7862 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:46.595644  2313 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:46.589417  7845 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:46.602417  7787 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:46.602797  7689 consensus_queue.cc:260] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.604300  7865 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:46.604660  7689 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:46.604933  7865 raft_consensus.cc:697] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:46.606175  7865 consensus_queue.cc:237] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:46.607128  7846 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:46.591095  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 170726us
I20260709 13:04:46.612435  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:46.579936  7636 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:46.590128  7864 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:46.631697  7852 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:46.644567  7850 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:46.632647  7436 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:46.647231  7671 consensus_meta.cc:287] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.731s	user 0.013s	sys 0.011s
I20260709 13:04:46.658959  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:04:46.674467  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 105165us
I20260709 13:04:46.678658  7893 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:46.680262  7893 raft_consensus.cc:697] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 22 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:04:46.682143  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 53100.5us
W20260709 13:04:46.682468  7738 consensus_meta.cc:287] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.590s	user 0.014s	sys 0.004s
I20260709 13:04:46.683881  7893 consensus_queue.cc:237] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:46.687420  1537 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:46.676836  1612 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:46.700932  7706 consensus_queue.cc:1048] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:04:46.707194  7896 log.cc:927] Time spent T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:04:46.712611  7738 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:46.722185  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 62194us
I20260709 13:04:46.723621  1537 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:46.724998  7897 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:46.725752  7897 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:46.726163  7866 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:46.726229  7897 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:46.726984  7866 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.727485  7866 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:46.730453  7864 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:04:46.730657  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 54004.5us
I20260709 13:04:46.731099  7864 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.731680  7864 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:46.732133  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:46.733358  7871 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.734417  7871 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:46.743188  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.750416  1718 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:46.737557  7897 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.752067  7738 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:46.742674  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:46.745600  1686 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:46.762081  7895 consensus_queue.cc:1048] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.098s
W20260709 13:04:46.762801  7789 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.799s	user 0.014s	sys 0.003s
I20260709 13:04:46.765014  1566 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:04:46.764640  7789 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:46.766345  2345 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 16. Ops: []
I20260709 13:04:46.766907  7642 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:46.767714  7642 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:46.768355  7642 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:46.768666  7714 consensus_queue.cc:1059] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:46.770028  7714 consensus_queue.cc:1243] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:04:46.770828  7714 consensus_queue.cc:1059] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:46.772269  7882 consensus_queue.cc:1059] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:46.773551  7882 raft_consensus.cc:3055] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:46.774019  7689 raft_consensus.cc:3055] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:46.774154  7882 raft_consensus.cc:740] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:46.774649  7689 raft_consensus.cc:740] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:46.775957  7882 consensus_queue.cc:260] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.768016  2345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 543) took 1438 ms. Trace:
I20260709 13:04:46.772267  7714 raft_consensus.cc:3055] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:46.778817  7714 raft_consensus.cc:740] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:46.780182  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:46.781188  1642 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:04:46.780354  1639 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:46.778587  2345 rpcz_store.cc:276] 0709 13:04:45.329750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.618849 (+289099us) service_pool.cc:224] Handling call
0709 13:04:46.038215 (+419366us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b580003a298. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:04:46.043195 (+  4980us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:46.766862 (+723667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.767692 (+   830us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b54003f84d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:46.767943 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1760024704}
I20260709 13:04:46.783576  1639 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:46.777014  7689 consensus_queue.cc:260] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.785394  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:46.787942  1643 raft_consensus.cc:1240] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:46.787245  7714 consensus_queue.cc:260] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.788904  1642 raft_consensus.cc:1240] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:04:46.789671  7714 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:46.790311  1643 raft_consensus.cc:1240] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:04:46.793630  1644 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:46.795359  1648 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:04:46.787974  7871 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:46.797088  1537 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:46.687397  7652 consensus_meta.cc:287] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.542s	user 0.014s	sys 0.003s
W20260709 13:04:46.801138  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:46.801267  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.811048  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:46.797796  7895 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:04:46.805150  7861 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:46.819965  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:46.820330  1639 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:46.826639   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:46.823122  7689 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:46.827057  1566 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:46.834776  7897 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:46.845189  1643 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:04:46.845281  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:46.846396  1646 raft_consensus.cc:2410] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
W20260709 13:04:46.846776  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 101283us
I20260709 13:04:46.845808  1643 pending_rounds.cc:85] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 11
I20260709 13:04:46.848855  1643 pending_rounds.cc:107] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:04:46.849130  1643 raft_consensus.cc:2889] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:46.808601  7652 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:46.853291  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.813119  7738 consensus_queue.cc:1048] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:04:46.853374  1642 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:46.851822  1566 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:46.861292  1686 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:46.853797  1537 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:46.862161  7882 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:46.866029  7871 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:46.872255  7871 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:46.872859  7871 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:46.876915  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:46.885825  7861 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:46.889307  7850 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.889667  7864 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:46.890398  7850 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:46.894006  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.901165  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:46.901998  7738 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:46.905786  1643 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:46.906040  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:46.907255  1644 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 14: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:46.897446  7850 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:46.909453  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:46.911520  7862 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:46.912352  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:46.913724  1566 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:46.912744  7866 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.916604  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:46.917919  1646 raft_consensus.cc:2410] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:04:46.918391  7897 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.919183  7897 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.919390  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:46.920691  1571 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
I20260709 13:04:46.921702  7905 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:46.922652  7905 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.922411  7873 consensus_queue.cc:1059] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:04:46.924477  7905 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:46.926324  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:46.927770  1644 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:46.876015  7879 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:04:46.927902  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 47615.4us
I20260709 13:04:46.922358  7652 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:46.935936  7642 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:46.942265  1639 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: [14.8-16.9]
I20260709 13:04:46.944486  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:46.931541  7879 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.932716  7895 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:04:46.961074  1571 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:04:46.961244  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:46.966730  1572 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:46.936962  7738 raft_consensus.cc:3055] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:46.968773  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:46.973737  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 101125us
I20260709 13:04:46.945426  7866 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:46.957006  7897 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:46.966226  7879 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:46.989097  1644 raft_consensus.cc:2410] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:04:46.989948  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:46.970827  7738 raft_consensus.cc:740] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:46.989228  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.002998  7861 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:47.003245  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.004166  7861 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:47.009771  7893 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:04:47.010505  7908 log.cc:927] Time spent T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Append to log took a long time: real 0.148s	user 0.000s	sys 0.002s
I20260709 13:04:47.010903  7893 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:46.976063  7789 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:46.998363  7910 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.165s	user 0.002s	sys 0.000s
I20260709 13:04:46.999684  1687 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:47.023181  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.025477  1639 raft_consensus.cc:1275] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:47.027854  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:47.027484  7738 consensus_queue.cc:260] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:47.032495  7738 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:47.034466  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.035349  1644 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:04:47.036845  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.038333  7642 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:47.038625  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.045158  2345 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:46.989485  7873 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:47.027504  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.087277  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.088253  1571 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:47.022305  7706 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.094148  7706 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:47.096999  7706 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:47.099256  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.102626  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:47.105710  1570 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:47.106630  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.107542  1641 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:47.109766  1573 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:04:47.112118  7873 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:47.114742  1644 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:47.115032  7873 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:47.115744  7873 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:47.121625  7578 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.123238  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.160485  1641 raft_consensus.cc:2410] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:04:47.123665  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.123783  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:47.125676  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.138756  7861 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:47.153908  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.146932  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.139376  7893 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:47.140025  7864 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:47.140686  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.168107  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.170673  1567 raft_consensus.cc:2368] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18.
I20260709 13:04:47.120577  1646 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:47.147713  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.172112  1643 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:47.156153  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.023289  7862 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 22 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:47.150043  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:47.145423  7911 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:04:47.141842  1572 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 22.
I20260709 13:04:47.147724  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.180341  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.185135  1566 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16.
I20260709 13:04:47.176294  7861 consensus_queue.cc:1059] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.011s
I20260709 13:04:47.182382  2312 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:47.188315  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:47.156129  7642 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.192278  1642 raft_consensus.cc:2410] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:04:47.193981  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.196593  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.198377  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.201795  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.197409  1646 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:04:47.204391  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.152462  7905 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:47.209441  7905 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:47.218254  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.194713  1639 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:04:47.220307  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.220829  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.221226  1639 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 14: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:47.223259  1569 raft_consensus.cc:2410] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:04:47.225235  1567 raft_consensus.cc:2410] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:04:47.221948  7862 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:47.226331  1640 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:47.235531  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.233724  7789 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.236469  1647 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:04:47.238628  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.239972  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.241983  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.242779  1646 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:47.243818  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.244725  1643 raft_consensus.cc:2410] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 17 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:04:47.244517  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.245597  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.238163  7789 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:47.245988  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.246495  1643 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:47.247612  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.249346  1644 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:47.247699  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.251649  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:47.252494  1639 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:04:47.254174  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:47.254877  1647 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:47.245767  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:47.258128  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.259850  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.256356  1643 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:04:47.265482  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.268173  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:47.269160  1646 raft_consensus.cc:2376] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15. Re-sending same reply.
I20260709 13:04:47.270831  1641 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:47.272413  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.224413  7642 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:47.266497  1567 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:47.282116  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.283812  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:47.285683  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.285784  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.283950  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.287132  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.287606  1644 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:47.287909  1640 raft_consensus.cc:2393] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 16: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:47.288540  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:47.289268  1644 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 14: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:47.290331  1643 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:47.285950  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.199770  7578 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:47.294018  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:47.248606  7862 raft_consensus.cc:697] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:04:47.299813  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.254582  7789 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:47.303308  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.306265  7862 consensus_queue.cc:237] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:47.309985  7864 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.310694  7864 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:47.312140  7871 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:47.312989  7864 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:47.314006  1642 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:04:47.314389  1642 pending_rounds.cc:85] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 8
I20260709 13:04:47.314625  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.315155  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.315810  1644 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:47.316133  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:47.314954  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.316759  1648 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:47.314684  1642 pending_rounds.cc:107] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:04:47.317472  1642 raft_consensus.cc:2889] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:47.318291  2345 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:04:47.316948  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:47.318974  7714 consensus_meta.cc:287] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.517s	user 0.010s	sys 0.002s
I20260709 13:04:47.320752  7873 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:47.321501  7873 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:47.321619  1639 raft_consensus.cc:2376] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 22. Re-sending same reply.
I20260709 13:04:47.324955  1611 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:47.303030  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 322516us
I20260709 13:04:47.326040  7879 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:47.326449  1687 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:47.327164  7864 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:47.327744  7864 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.328593  7879 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:47.328642  7864 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:47.331705  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.333698  1564 raft_consensus.cc:2410] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:04:47.336428  1646 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:47.340113  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:47.345424  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.347666  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.348845  1566 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 16: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
W20260709 13:04:47.355720  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.356626  1611 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:47.360736  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.365892  1567 raft_consensus.cc:2393] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 16: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:47.376984  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:47.378269  1572 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:47.379220  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.380179  1611 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:47.374910  1568 raft_consensus.cc:1275] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:47.381345  7893 raft_consensus.cc:3055] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:47.385980  7893 raft_consensus.cc:740] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:04:47.385299  1538 leader_election.cc:336] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.612s (SENT)
I20260709 13:04:47.386726  1538 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:47.387372  7893 consensus_queue.cc:260] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:47.388145  7879 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:47.395757  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.396416  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.389276  7893 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:47.401556  1647 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
W20260709 13:04:47.411659  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 85338.3us
W20260709 13:04:47.412058  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 44975.5us
I20260709 13:04:47.415138  1566 raft_consensus.cc:2410] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:04:47.413985  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.421203  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:47.414304  7436 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:47.430068  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.394905  7861 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:04:47.428822  1611 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:47.431680  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 111191us
I20260709 13:04:47.432475  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.434569  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.436586  7917 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:47.430538  7436 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:47.424228  7871 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:47.441587  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.442433  7436 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:47.446774  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:04:47.450003  1538 leader_election.cc:336] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.668s (SENT)
I20260709 13:04:47.454650  1569 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:04:47.455740  1538 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:47.464090  7917 raft_consensus.cc:697] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:47.464267  7882 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:47.465915  1571 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16.
I20260709 13:04:47.465596  7917 consensus_queue.cc:237] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:47.471364  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:47.474059  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 109764us
I20260709 13:04:47.474841  1564 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 13:04:47.475486  1564 pending_rounds.cc:85] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 6
I20260709 13:04:47.475715  1564 pending_rounds.cc:107] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:04:47.475962  1564 raft_consensus.cc:2889] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:47.476570  1564 pending_rounds.cc:107] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:04:47.476895  1564 raft_consensus.cc:2889] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:47.476945  2314 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:47.480331  2314 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:47.482471  7846 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:47.483366  7846 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:47.486143  7642 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:47.487339  7642 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:47.487895  7642 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:47.488332  7642 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:47.496903  2312 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:47.451297  7738 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.450856  7578 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:47.506321  2343 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
W20260709 13:04:47.508687  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 84767.4us
I20260709 13:04:47.458942  7873 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:47.509680  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:47.513828  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.519083  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.521068  1570 raft_consensus.cc:2368] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18.
I20260709 13:04:47.525859  7738 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:47.508148  7846 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.526965  7846 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:47.528185  7738 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:47.528973  1564 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:47.525836  7862 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:47.529528  7846 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:47.530753  7850 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.531386  7850 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:47.535809  7850 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:47.537806  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:47.531304  7862 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.538643  1718 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:47.538736  7862 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:47.536676  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:47.541441  1721 raft_consensus.cc:2376] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17. Re-sending same reply.
I20260709 13:04:47.530427  1611 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:47.549227  7578 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:47.549839  7578 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:47.550282  7578 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:47.559401  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.560807  1564 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
W20260709 13:04:47.565897  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 111413us
I20260709 13:04:47.567476  1687 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:47.568439  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.569269  1569 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:47.580466  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.587144  1648 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
W20260709 13:04:47.582010  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 140893us
I20260709 13:04:47.590993  1611 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:47.611742  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.612834  1570 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:04:47.620066  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.631284  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:47.631616  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:47.630718  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.633092  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:47.645484  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:47.649889  7879 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.661164  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:47.662212  2345 raft_consensus.cc:2393] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 13: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:47.663762  1570 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
I20260709 13:04:47.667604  1572 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:47.669994  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:47.671818  1564 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: []
W20260709 13:04:47.676934  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 109525us
W20260709 13:04:47.678776  1538 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.897s (SENT)
I20260709 13:04:47.661201  7879 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:47.686617  7929 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:47.687191  7929 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.687542  7929 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:47.651819  7926 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:47.689505  7926 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:47.640801  7861 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:47.693557  7861 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:47.694172  7861 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:47.695407  1687 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:47.695577  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:47.696646  1687 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:47.697475  7873 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:47.699573  7873 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:47.700099  7873 raft_consensus.cc:697] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:47.603566  7578 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:47.713229  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.676584  7864 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:47.710421  1572 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:47.736141  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:47.737035  2345 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 16: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:47.736758  1719 raft_consensus.cc:1240] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:47.739277  7436 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:47.718050  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 134881us
I20260709 13:04:47.723913  1715 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:04:47.750304  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:47.750728  7926 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:47.751896  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:47.752799  1538 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.890s (SENT) [suppressed 31 similar messages]
I20260709 13:04:47.753921  7738 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:47.754745  7738 raft_consensus.cc:697] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:47.755971  7738 consensus_queue.cc:237] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:47.756752  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.752830  1721 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:47.760313  7936 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.761039  7936 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:47.764297  7436 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:47.750619  1719 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:47.771988  2312 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:47.773530  7787 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:47.774111  7787 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.774660  7787 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:47.775949  7879 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:47.779712  1687 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:47.782116  7926 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:47.782819  7926 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.701578  7873 consensus_queue.cc:237] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:47.786912  1687 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:47.786326  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:47.788666  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:47.789548  2348 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:47.789623  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:47.790910  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.791121  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:47.792466  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:47.813867  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:47.827536  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 71763.9us
I20260709 13:04:47.828997  1722 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:47.783272  7926 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:47.839068  1537 connection.cc:256] RPC call timeout handler was delayed by -1.0186661779999895s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
I20260709 13:04:47.828454  7436 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:04:47.896320  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:47.899942  1718 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:04:47.860823  7578 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:47.847488  7879 consensus_queue.cc:1059] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:04:47.893064  1569 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:47.851902  7928 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:04:47.924398  1537 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:47.926847  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:47.940799  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 146736us
I20260709 13:04:47.933086  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:47.939035  1717 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:47.901504  7714 consensus_queue.cc:1059] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 13:04:48.048336  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 219191us
I20260709 13:04:47.912832  7642 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:48.062897  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:48.063123  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:48.064538  1686 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:48.067314  1718 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:04:47.942763  7917 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:48.073773  2342 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:47.937368  7936 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:48.046075  7879 raft_consensus.cc:3055] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:47.935082  7436 consensus_queue.cc:1059] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.122s
I20260709 13:04:47.933596  7882 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.125s
W20260709 13:04:48.059185  7862 consensus_meta.cc:287] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.517s	user 0.012s	sys 0.002s
W20260709 13:04:48.087755  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.086297  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:48.094980  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:48.099615  1718 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 15: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:48.073774  7928 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:04:48.115289  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 264596us
I20260709 13:04:48.110739  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:48.079380  7578 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:48.126981  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 210532us
I20260709 13:04:48.089049  7879 raft_consensus.cc:740] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:48.128259  1717 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:48.145265  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:48.147761  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:48.141219  1685 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:48.152490  1642 raft_consensus.cc:1240] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:48.143935  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:48.166002  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:48.172158  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:48.172799  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:48.173135  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:48.124822  7706 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:48.173698  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:48.176522  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:48.188233  1646 raft_consensus.cc:2393] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 13: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:48.103909  7862 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:48.211310  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.129151  7929 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:48.181993  7952 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:48.169039  7436 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:48.149554  7578 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:48.250823  1646 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:48.215159  7928 raft_consensus.cc:697] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:48.264420  1641 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 18: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:48.218647  7950 raft_consensus.cc:3055] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:04:48.290131  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.275352  7714 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:48.283571  7706 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:48.301330  1569 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:04:48.266391  7940 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.357s	user 0.000s	sys 0.002s
W20260709 13:04:48.308946  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.305383  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:48.303722  1687 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:48.315915  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:48.294303  7955 raft_consensus.cc:3055] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:48.326035  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:48.331733  1569 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:48.290402  7950 raft_consensus.cc:740] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:48.330340  1639 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: []
W20260709 13:04:48.313757  7933 log.cc:927] Time spent T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.404s	user 0.000s	sys 0.002s
I20260709 13:04:48.331341  7955 raft_consensus.cc:740] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:48.307425  7706 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:48.308123  7882 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.345888  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:48.349344  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:48.361990  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 195645us
W20260709 13:04:48.344511  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 203309us
I20260709 13:04:48.307763  7436 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:48.359436  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:48.362841  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:48.371398  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:48.331931  7893 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:48.383137  1717 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 22.
W20260709 13:04:48.364917  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 210385us
I20260709 13:04:48.386308  1569 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 18: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:48.387771  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:48.290931  7879 consensus_queue.cc:260] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:48.368628  7939 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.303s	user 0.000s	sys 0.002s
I20260709 13:04:48.393497  1715 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:48.367861  7947 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:04:48.406973  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.411418  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:48.392190  7966 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:48.413708  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 124176us
I20260709 13:04:48.366019  7882 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:48.408780  7864 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:48.436339  7873 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:48.466310  7879 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:48.404886  7893 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:48.348060  7955 consensus_queue.cc:260] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:48.448410  7947 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:48.541810  7959 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
W20260709 13:04:48.562310  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.715672  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:48.468693  7950 consensus_queue.cc:260] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:48.592988  7953 log.cc:927] Time spent T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.340s	user 0.000s	sys 0.002s
W20260709 13:04:48.622357  7960 log.cc:927] Time spent T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 13:04:48.647892  1717 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 18
W20260709 13:04:48.710059  7787 consensus_meta.cc:287] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.935s	user 0.016s	sys 0.006s
I20260709 13:04:48.738260  7787 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:48.739290  7917 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:48.739954  7917 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:48.701447  7882 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:48.742264  7882 consensus_queue.cc:1059] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:48.743646  7955 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:48.697005  7873 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.753129  7873 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:48.698176  7958 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.056s
W20260709 13:04:48.650314  7926 consensus_meta.cc:287] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.763s	user 0.014s	sys 0.003s
I20260709 13:04:48.754487  7926 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:48.756927  7926 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:48.412094  7949 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.757791  7949 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:48.758978  7936 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.759976  7936 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:48.763217  7972 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:48.763933  7972 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:48.764891  7950 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:48.650314  2342 consensus_meta.cc:287] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.567s	user 0.011s	sys 0.002s
I20260709 13:04:48.651813  1646 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:48.769496  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:48.770246  1643 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 22.
I20260709 13:04:48.770780  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:48.771063  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:48.771571  1643 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:48.771657  7893 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:48.772159  1646 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:48.652537  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:48.758266  1713 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:48.773576  1566 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:48.654601  7865 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:48.684644  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:48.410416  7928 consensus_queue.cc:237] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:48.679431  1685 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:48.693173  1639 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:04:48.702559  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:48.702790  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:48.692317  1721 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.841s	user 0.004s	sys 0.018s
I20260709 13:04:48.783777  1644 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:48.783823  1721 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.935s	user 0.004s	sys 0.019s
I20260709 13:04:48.710088  1611 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:48.592393  7954 log.cc:927] Time spent T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:04:48.715673  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:48.648864  7966 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:48.730373  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:48.808698  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 98675.4us
I20260709 13:04:48.689109  7738 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.649863  1719 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:48.751670  1612 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:48.746938  7952 raft_consensus.cc:3055] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:48.774432  2342 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
W20260709 13:04:48.711989  7861 consensus_meta.cc:287] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.017s	user 0.017s	sys 0.004s
I20260709 13:04:48.790508  1717 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
W20260709 13:04:48.804713  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 500843us
I20260709 13:04:48.804495  1567 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:48.810355  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 152218us
I20260709 13:04:48.781006  7578 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:48.819212  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.804142  7862 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:48.843191  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 163845us
W20260709 13:04:48.839774  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:48.849472  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 438253us
I20260709 13:04:48.809367  7977 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.846452  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:48.860976  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:48.824287  7958 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:48.861526  7893 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:48.824271  7738 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:48.825896  7959 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:04:48.817607  7952 raft_consensus.cc:740] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:48.880592  1646 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:04:48.891377  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:48.898123  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 63404.8us
I20260709 13:04:48.898922  1644 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:04:48.890098  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:48.904140  1643 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16.
I20260709 13:04:48.834964  7957 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:48.847019  7947 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:48.817348  7969 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:04:48.934952  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 205727us
I20260709 13:04:48.905704  7578 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:04:48.939669  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 74151.4us
I20260709 13:04:48.940002  7964 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:04:48.941073  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 96951.3us
W20260709 13:04:48.934996  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 71281.3us
I20260709 13:04:48.826104  7861 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:48.961071  1566 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:04:48.971277  1715 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:04:48.992614  1685 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:49.006294  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:49.013530  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:49.014016  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.947719  7983 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:49.016079  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:48.946570  7969 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.025377  1715 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:49.027673  1612 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:49.028554  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.007331  1720 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:49.015808  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:48.906031  7958 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:49.017266  7972 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:49.036003  7962 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.068449  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:48.965816  7957 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:48.901980  7977 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:49.030031  7969 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:49.080101  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 87554us
W20260709 13:04:49.081014  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 68363.7us
W20260709 13:04:49.069309  7706 consensus_meta.cc:287] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.702s	user 0.015s	sys 0.000s
I20260709 13:04:49.083295  1714 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17.
I20260709 13:04:49.093149  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:49.094992  7738 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:49.099151  7981 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:49.101758  7948 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.102761  7948 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:49.107537  7968 consensus_queue.cc:1059] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
W20260709 13:04:49.110250  7961 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.653s	user 0.004s	sys 0.000s
I20260709 13:04:49.112779  7706 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:49.115957  7706 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:49.118762  7738 raft_consensus.cc:3055] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:49.119339  7738 raft_consensus.cc:740] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:49.119886  7642 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:49.030874  7952 consensus_queue.cc:260] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:49.093997  7964 consensus_queue.cc:1059] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:04:49.123209  7964 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:49.124202  7964 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:49.054581  7967 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:04:49.126807  7962 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:49.126822  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:49.128419  1714 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:49.066192  7947 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:49.130257  7966 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:49.131593  7990 consensus_queue.cc:1059] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:04:49.135367  1713 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2681) took 1341 ms (client timeout 1655 ms). Trace:
I20260709 13:04:49.075673  7936 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:49.137502  7941 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.879s	user 0.004s	sys 0.001s
I20260709 13:04:49.138063  7978 raft_consensus.cc:3055] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:49.175521  7978 raft_consensus.cc:740] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:04:49.142573  7714 consensus_meta.cc:287] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.842s	user 0.010s	sys 0.006s
I20260709 13:04:49.176877  2343 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 817) took 1568 ms. Trace:
I20260709 13:04:49.176678  7714 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:49.121074  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:49.143926  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:49.182334  2346 raft_consensus.cc:2393] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 19: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:49.183281  1571 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:49.144320  7917 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:49.145865  7987 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:04:49.148602  7436 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:49.129457  7962 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:49.152966  7787 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:49.155464  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:49.193737  1566 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:49.155871  7958 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:49.157902  7983 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.177588  2343 rpcz_store.cc:276] 0709 13:04:47.608772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.619320 (+ 10548us) service_pool.cc:224] Handling call
0709 13:04:47.644628 (+ 25308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:47.645096 (+   468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.645131 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:47.691872 (+ 46741us) log.cc:844] Serialized 26 byte log entry
0709 13:04:48.045495 (+353623us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:48.045537 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.045604 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:49.136497 (+1090893us) spinlock_profiling.cc:243] Waited 4.34 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:49.176477 (+ 39980us) raft_consensus.cc:1685] finished
0709 13:04:49.176506 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:49.176799 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1627,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":8588,"spinlock_wait_cycles":9976704,"thread_start_us":272453,"threads_started":1,"wal-append.queue_time_us":432152}
I20260709 13:04:49.159113  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3065) took 1375 ms. Trace:
W20260709 13:04:49.159956  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.135471  1717 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
I20260709 13:04:49.089532  1685 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:49.163341  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 147390us
I20260709 13:04:49.167047  1611 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:49.162565  1713 rpcz_store.cc:269] 0709 13:04:47.795184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.830427 (+ 35243us) service_pool.cc:224] Handling call
0709 13:04:49.135027 (+1304600us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b5400210f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:49.135273 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1519419264}
I20260709 13:04:49.120857  7738 consensus_queue.cc:260] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:49.208623  1714 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:04:49.169162  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:49.174103  7990 raft_consensus.cc:3055] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:49.143119  7949 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:49.179997  1567 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:04:49.180368  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2679) took 1325 ms. Trace:
I20260709 13:04:49.215870  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 647) took 1005 ms. Trace:
I20260709 13:04:49.218026  1568 raft_consensus.cc:2410] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:04:49.197846  1570 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:49.204862  1719 rpcz_store.cc:276] 0709 13:04:47.783699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.786014 (+  2315us) service_pool.cc:224] Handling call
0709 13:04:47.786357 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:48.813431 (+1027074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.157439 (+344008us) spinlock_profiling.cc:243] Waited 5.79 ms on lock 0x7b540011c4a0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248dfcdc0 00007f8248e88a02 00007f8248e84dd6 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2
0709 13:04:49.158724 (+  1285us) spinlock_profiling.cc:243] Waited 766 ms on lock 0x7b5400299fd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:49.158977 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1774505600}
I20260709 13:04:49.211539  1715 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16.
W20260709 13:04:49.213694  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 46719.8us
I20260709 13:04:49.214082  1612 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:04:49.143764  7944 log.cc:927] Time spent T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.968s	user 0.011s	sys 0.000s
I20260709 13:04:49.217742  1720 rpcz_store.cc:276] 0709 13:04:47.789968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.790687 (+   719us) service_pool.cc:224] Handling call
0709 13:04:49.115236 (+1324549us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400209c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:49.115413 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2726240000}
I20260709 13:04:49.218178  1714 rpcz_store.cc:276] 0709 13:04:48.211075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.203017 (+991942us) service_pool.cc:224] Handling call
0709 13:04:49.203494 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.209195 (+  5701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.215785 (+  6590us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:49.223636  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 644) took 1116 ms. Trace:
W20260709 13:04:49.235422  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 145930us
I20260709 13:04:49.240923  1715 rpcz_store.cc:276] 0709 13:04:48.120085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.005917 (+885832us) service_pool.cc:224] Handling call
0709 13:04:49.223544 (+217627us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:49.216794  7990 raft_consensus.cc:740] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:49.246616  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 317148us
I20260709 13:04:49.249612  1714 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:04:49.255406  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.213192  7642 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.235150  7937 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:04:49.261077  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 190486us
I20260709 13:04:49.259989  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3066) took 1475 ms. Trace:
I20260709 13:04:49.260066  1611 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:49.268325  1566 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19.
W20260709 13:04:49.275713  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 61735.1us
I20260709 13:04:49.245013  7862 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:49.262097  7706 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:04:49.309659  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 121934us
I20260709 13:04:49.213541  7983 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:49.314623  1685 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:49.270020  7994 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:49.277421  7947 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:49.251701  7955 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:04:49.267618  1721 rpcz_store.cc:276] 0709 13:04:47.784096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.793056 (+  8960us) service_pool.cc:224] Handling call
0709 13:04:47.793339 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.826657 (+ 33318us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:48.784655 (+957998us) log.cc:844] Serialized 14 byte log entry
0709 13:04:49.259269 (+474614us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:49.259364 (+    95us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:49.259439 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.259670 (+   231us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:49.259871 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5291,"prepare.queue_time_us":317534,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":4702,"thread_start_us":1372672,"threads_started":2}
I20260709 13:04:49.327952  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:49.328238  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:49.275188  7871 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:49.288757  7873 consensus_meta.cc:287] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.002s
I20260709 13:04:49.270854  7642 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:49.245049  7978 consensus_queue.cc:260] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:49.334313  7879 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.494344  7879 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:49.389668  7977 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:49.389231  7957 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:49.360384  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2731) took 1181 ms. Trace:
W20260709 13:04:49.494398  7950 consensus_meta.cc:287] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.728s	user 0.014s	sys 0.001s
W20260709 13:04:49.484984  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:49.324924  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 54189.7us
I20260709 13:04:49.383176  7948 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:49.394270  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 652) took 1124 ms. Trace:
I20260709 13:04:49.404511  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:49.312050  7706 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:49.343586  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:49.333464  1537 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:49.559151  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 225733us
I20260709 13:04:49.559994  1537 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:49.358894  7964 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:49.561960  7966 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:49.562446  7966 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:49.562832  7966 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:49.563206  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:49.334390  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:49.567260  1641 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:04:49.568957  1645 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:49.550074  1640 rpcz_store.cc:276] 0709 13:04:48.169687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.170423 (+   736us) service_pool.cc:224] Handling call
0709 13:04:48.190894 (+ 20471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:48.401419 (+210525us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:48.401474 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:48.457238 (+ 55764us) log.cc:844] Serialized 26 byte log entry
0709 13:04:48.956117 (+498879us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:48.956182 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.956491 (+   309us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:49.351144 (+394653us) raft_consensus.cc:1685] finished
0709 13:04:49.351169 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:49.351363 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40910,"raft.queue_time_us":246582,"raft.run_cpu_time_us":505,"raft.run_wall_time_us":34949,"thread_start_us":321855,"threads_started":2,"wal-append.queue_time_us":310718}
I20260709 13:04:49.570544  1648 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:49.571722  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:49.572540  1643 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:49.451900  1570 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
W20260709 13:04:49.351773  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 44549.8us
I20260709 13:04:49.312734  7862 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:49.325712  7937 raft_consensus.cc:697] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:04:49.360226  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:49.506817  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 179934us
I20260709 13:04:49.469050  1571 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:49.397481  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3069) took 1550 ms. Trace:
I20260709 13:04:49.526876  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2708) took 1078 ms. Trace:
W20260709 13:04:49.393189  7991 log.cc:927] Time spent T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.302s	user 0.003s	sys 0.000s
I20260709 13:04:49.332804  7990 consensus_queue.cc:260] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:49.373618  7994 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:49.528759  7873 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:49.585850  1722 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:04:49.394953  8004 consensus_queue.cc:1059] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.099s
I20260709 13:04:49.383620  7999 consensus_queue.cc:1048] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.115s
I20260709 13:04:49.487365  7998 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:49.587087  2312 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:49.358780  2341 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 12. Current term is 13. Ops: [11.10-12.11]
I20260709 13:04:49.334265  7714 consensus_queue.cc:1059] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.161s
I20260709 13:04:49.595516  7937 consensus_queue.cc:237] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:49.596240  7714 raft_consensus.cc:3055] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:49.597008  7714 raft_consensus.cc:740] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:49.536497  1714 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:49.597554  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:49.571905  7969 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:49.537062  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 222426us
I20260709 13:04:49.600986  1685 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:49.598311  1640 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:49.602596  1685 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:49.287196  7995 consensus_queue.cc:1059] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.242s
I20260709 13:04:49.603598  7951 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:49.604874  7951 raft_consensus.cc:697] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 19 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:49.315074  7578 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.606351  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:49.526530  7983 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:49.609081  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.539306  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:49.351094  1568 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: []
W20260709 13:04:49.542208  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 177308us
W20260709 13:04:49.610158  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.359568  7917 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:49.543095  7929 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:49.545673  7947 raft_consensus.cc:697] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:49.527967  7981 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:49.606441  7951 consensus_queue.cc:237] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:49.554198  1566 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:49.553277  1717 rpcz_store.cc:276] 0709 13:04:48.270238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.839725 (+569487us) service_pool.cc:224] Handling call
0709 13:04:49.394047 (+554322us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54000230d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:49.394200 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":494078720}
I20260709 13:04:49.561376  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:49.577979  1646 raft_consensus.cc:1275] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 13:04:49.620362  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 59060.5us
I20260709 13:04:49.578153  7706 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:49.621246  8009 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:49.615845  7994 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:49.622560  7947 consensus_queue.cc:237] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:49.578576  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:49.582706  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:49.586817  7998 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:49.535569  7978 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:49.582083  1569 rpcz_store.cc:276] 0709 13:04:48.395550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.395722 (+   172us) service_pool.cc:224] Handling call
0709 13:04:48.396160 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:48.400330 (+  4170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:48.400377 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:48.412430 (+ 12053us) log.cc:844] Serialized 26 byte log entry
0709 13:04:49.147260 (+734830us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b18000fe140. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a250d1 00007f8248e40a0f
0709 13:04:49.310597 (+163337us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:49.310667 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.313434 (+  2767us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:49.438877 (+125443us) raft_consensus.cc:1685] finished
0709 13:04:49.438908 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:49.473253 (+ 34345us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":63235,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":23795,"spinlock_wait_cycles":4521728,"thread_start_us":159506,"threads_started":1,"wal-append.queue_time_us":899850}
I20260709 13:04:49.591827  1611 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:49.593621  7873 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:49.594466  7862 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:49.607295  2345 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:49.435498  7861 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:49.609872  1685 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:49.604637  7995 raft_consensus.cc:3055] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:49.615842  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:49.603644  7952 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:49.608289  7642 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:49.581305  1716 rpcz_store.cc:276] 0709 13:04:47.785300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.786634 (+  1334us) service_pool.cc:224] Handling call
0709 13:04:47.787366 (+   732us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:47.789518 (+  2152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.789560 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:47.790183 (+   623us) log.cc:844] Serialized 26 byte log entry
0709 13:04:47.937750 (+147567us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:47.937820 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.938117 (+   297us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:49.143400 (+1205283us) spinlock_profiling.cc:243] Waited 68.7 ms on lock 0x7b24003d9148. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245c1e9c4 00007f8245c1cacb 00007f8245dd7dc9 00007f8245dd7a55 00007f8245dd709d 00007f8248a0910e 00007f8248a2088c 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221
0709 13:04:49.330229 (+186829us) raft_consensus.cc:1685] finished
0709 13:04:49.330262 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:49.335280 (+  5018us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1548,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":137,"spinlock_wait_cycles":157932288,"thread_start_us":146763,"threads_started":1,"wal-append.queue_time_us":303873}
W20260709 13:04:49.625798  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.584189  7955 raft_consensus.cc:2764] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 13:04:49.607008  7578 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:49.644826  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:49.625276  1645 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
W20260709 13:04:49.659039  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 75233.5us
I20260709 13:04:49.674878  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:49.693687  1714 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:49.696954  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 118460us
W20260709 13:04:49.698886  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 89065.2us
I20260709 13:04:49.700182  1685 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:49.700518  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:49.696619  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:49.677487  1643 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
W20260709 13:04:49.677968  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 52562us
I20260709 13:04:49.643980  1565 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:49.601538  7969 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:49.704545  1537 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:49.689889  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:49.705514  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:49.706840  1572 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:04:49.706426  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:49.703714  1717 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:49.715195  1645 raft_consensus.cc:2368] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16.
I20260709 13:04:49.725147  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:49.726163  1641 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:49.733978  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:49.708747  1720 raft_consensus.cc:2376] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16. Re-sending same reply.
I20260709 13:04:49.711825  1537 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:49.737093  1537 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:49.731791  1611 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:49.706426  1713 raft_consensus.cc:2376] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 22. Re-sending same reply.
I20260709 13:04:49.740083  1538 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:49.684669  7950 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:49.750717  1644 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:49.685472  7995 raft_consensus.cc:740] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:49.743677  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 1046) took 1039 ms. Trace:
I20260709 13:04:49.744282  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:49.758471  1640 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:49.757979  1570 rpcz_store.cc:276] 0709 13:04:48.696246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.696428 (+   182us) service_pool.cc:224] Handling call
0709 13:04:49.735453 (+1039025us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b5400013954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:49.735574 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1106237184}
I20260709 13:04:49.762264  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:49.743462  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:49.634116  7958 consensus_queue.cc:1048] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:04:49.765036  1567 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:49.707551  7917 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:49.733458  7948 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:49.760239  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 87634.3us
W20260709 13:04:49.785562  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.740304  7967 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:49.772480  1611 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:49.764219  7941 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 13:04:49.805524  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 48094.6us
I20260709 13:04:49.756629  7955 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:04:49.819387  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 82338.3us
W20260709 13:04:49.820142  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 80126.1us
I20260709 13:04:49.820824  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:49.823261  1645 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:49.814145  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 77932.3us
I20260709 13:04:49.734683  7999 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:04:49.761798  7929 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:49.833279  1717 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
I20260709 13:04:49.783258  7958 consensus_queue.cc:1243] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer 066ee0109aba4465ba17fa65e14ef504 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:04:49.778203  7950 raft_consensus.cc:697] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:49.799634  7970 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.850562  8018 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:49.822837  7955 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:49.858744  1685 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:49.823350  7957 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:04:49.845006  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 85553.5us
I20260709 13:04:49.781550  7981 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:04:49.763446  7968 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 13:04:49.874990  1567 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
W20260709 13:04:49.899611  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 75073us
W20260709 13:04:49.903735  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 63731.8us
I20260709 13:04:49.837754  7929 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:49.781840  7994 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.070s
I20260709 13:04:49.774127  7714 consensus_queue.cc:260] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:49.909317  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 62449.3us
I20260709 13:04:49.918161  2313 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:04:49.912144  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 86969.5us
I20260709 13:04:49.856791  7970 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:49.852807  7955 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:49.928822  1538 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:49.866870  7957 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.924818  1537 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:49.910292  8010 log.cc:927] Time spent T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:04:49.932893  1686 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:49.905539  7879 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:49.927006  7861 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:49.815423  1713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:49.965775  1713 raft_consensus.cc:606] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:49.958698  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:49.958952  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 54681.2us
I20260709 13:04:49.914294  7929 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:49.929697  7980 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:49.895015  7862 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:04:49.933467  8005 raft_consensus.cc:3055] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:49.966928  1687 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:49.937702  7957 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:49.926404  7996 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:49.979657  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 50891.1us
W20260709 13:04:49.991073  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 51631.2us
I20260709 13:04:49.956859  7967 raft_consensus.cc:697] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:50.003247  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:49.993911  8007 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.255s	user 0.000s	sys 0.002s
I20260709 13:04:50.005743  7998 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:50.007113  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:49.968390  7861 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:50.008080  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3078) took 1780 ms. Trace:
W20260709 13:04:50.008179  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 77842.4us
I20260709 13:04:49.974470  7965 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:49.977721  7951 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:49.974077  7879 raft_consensus.cc:2764] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:04:49.974292  7917 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:50.020457  7861 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:50.018834  1719 rpcz_store.cc:276] 0709 13:04:48.227611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.219357 (+991746us) service_pool.cc:224] Handling call
0709 13:04:49.250987 (+ 31630us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:49.258968 (+  7981us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:49.258990 (+    22us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:49.275271 (+ 16281us) log.cc:844] Serialized 52 byte log entry
0709 13:04:49.529382 (+254111us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:49.548568 (+ 19186us) log.cc:844] Serialized 14 byte log entry
0709 13:04:49.549668 (+  1100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.549760 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:50.007798 (+458038us) raft_consensus.cc:1685] finished
0709 13:04:50.007822 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:50.007995 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8410,"prepare.queue_time_us":22323,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":51,"raft.queue_time_us":62250,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":10725,"spinlock_wait_cycles":22243072,"thread_start_us":192413,"threads_started":1,"wal-append.queue_time_us":260224}
I20260709 13:04:49.902901  7995 consensus_queue.cc:260] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.039233  1612 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:49.964658  7928 consensus_queue.cc:1059] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:04:49.977813  8005 raft_consensus.cc:740] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:50.036130  1687 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:50.031349  7966 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.022500  7981 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:50.020452  8009 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:49.981565  7999 consensus_queue.cc:1059] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:04:50.049875  2312 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:04:50.032864  8008 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.294s	user 0.002s	sys 0.000s
W20260709 13:04:50.052837  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 45799.4us
I20260709 13:04:49.978982  7950 consensus_queue.cc:237] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.042884  7951 raft_consensus.cc:697] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:50.044682  7998 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:50.056077  2312 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:50.041515  7965 raft_consensus.cc:697] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:50.070978  1569 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:50.052347  7981 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.073004  8029 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:50.078246  7967 consensus_queue.cc:237] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:50.096503  8013 log.cc:927] Time spent T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:04:50.071228  1687 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:50.086792  1714 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19.
I20260709 13:04:50.072292  8014 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.106s
I20260709 13:04:50.091933  8028 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:50.094542  7738 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:50.104426  8031 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:50.105576  8002 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.106400  1687 leader_election.cc:400] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16.
I20260709 13:04:50.106829  1687 leader_election.cc:403] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:04:50.109896  7928 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:50.100595  7981 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:50.111567  8037 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:50.045476  7996 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:04:50.106437  8002 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:50.118960  1641 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:04:50.122925  1538 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:50.123364  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 67331.8us
I20260709 13:04:50.104012  8028 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.104502  8016 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:50.105137  8031 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:50.081073  7998 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.139636  8030 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.140409  8030 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:50.140529  8005 consensus_queue.cc:260] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.134647  8028 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:50.142333  8026 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:50.143432  8030 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:50.094579  7706 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.136849  7928 raft_consensus.cc:697] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:50.146384  8036 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:50.147157  8036 raft_consensus.cc:697] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:50.146901  7928 consensus_queue.cc:237] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.148705  7998 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:50.149916  7706 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:50.150347  7965 consensus_queue.cc:237] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.137079  8031 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:50.153889  7982 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.028s	user 0.005s	sys 0.012s
I20260709 13:04:50.149149  8016 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:50.155661  8005 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:50.155874  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2734) took 1859 ms. Trace:
I20260709 13:04:50.156519  7970 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:50.159338  7951 consensus_queue.cc:237] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:50.148216  8036 consensus_queue.cc:237] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:50.167269  7966 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:50.099638  7995 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:50.169838  8023 log.cc:927] Time spent T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.080s	user 0.000s	sys 0.002s
W20260709 13:04:50.174018  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 82963.1us
I20260709 13:04:50.175433  1538 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:50.178826  1713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:50.179592  1713 raft_consensus.cc:606] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:50.179994  1713 raft_consensus.cc:614] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:50.132514  1645 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
W20260709 13:04:50.195678  8020 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.388s	user 0.004s	sys 0.000s
I20260709 13:04:50.156524  1642 rpcz_store.cc:276] 0709 13:04:48.296130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.296876 (+   746us) service_pool.cc:224] Handling call
0709 13:04:48.297243 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:48.655348 (+358105us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:48.655425 (+    77us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:48.656520 (+  1095us) log.cc:844] Serialized 26 byte log entry
0709 13:04:48.821668 (+165148us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:48.821722 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.821809 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:49.965176 (+1143367us) spinlock_profiling.cc:243] Waited 61.2 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:50.155309 (+190133us) raft_consensus.cc:1685] finished
0709 13:04:50.155351 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:50.155793 (+   442us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19756,"raft.queue_time_us":395184,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":28704,"spinlock_wait_cycles":140831488,"thread_start_us":365297,"threads_started":2,"wal-append.queue_time_us":414047}
I20260709 13:04:50.200078  1538 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:50.196828  7929 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:50.185395  7787 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.000s	sys 0.014s
I20260709 13:04:50.209045  1538 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:50.215417  8002 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:50.219182  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:50.208089  7787 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.000s	sys 0.014s
I20260709 13:04:50.224071  7929 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:50.226601  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3071) took 2330 ms. Trace:
I20260709 13:04:50.227263  1722 rpcz_store.cc:276] 0709 13:04:47.896270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.901026 (+  4756us) service_pool.cc:224] Handling call
0709 13:04:48.689510 (+788484us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b580009db98. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:04:48.689800 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:49.586322 (+896522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.226199 (+639877us) spinlock_profiling.cc:243] Waited 895 ms on lock 0x7b540002dfd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:50.226480 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2061818624}
I20260709 13:04:50.229946  7966 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:50.230548  7966 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.230984  7966 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:50.233815  8029 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:50.234309  8029 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.234676  8029 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:50.238596  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.229190  7948 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:50.254568  8026 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:50.259132  1722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:50.260003  1722 raft_consensus.cc:606] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:50.260432  1722 raft_consensus.cc:614] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:04:50.253386  7978 consensus_meta.cc:287] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.606s	user 0.010s	sys 0.003s
W20260709 13:04:50.275547  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 416709us
I20260709 13:04:50.291107  1685 leader_election.cc:400] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18.
I20260709 13:04:50.291597  1685 leader_election.cc:403] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:04:50.291891  7962 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.852s	user 0.006s	sys 0.021s
I20260709 13:04:50.305513  1685 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:50.296725  2346 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.015s	sys 0.015s
W20260709 13:04:50.312022  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:50.312358  2346 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.017s	sys 0.015s
W20260709 13:04:50.313045  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:50.314960  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 113254us
I20260709 13:04:50.316174  1538 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:50.319876  1538 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:50.322464  1538 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:50.330292  7995 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:50.331101  7995 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:50.331470  7995 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:50.335855  8009 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:50.339959  7968 raft_consensus.cc:3055] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:50.340407  7968 raft_consensus.cc:740] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:50.342623  1567 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:50.343724  7882 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:50.344647  7882 raft_consensus.cc:697] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:50.345841  7882 consensus_queue.cc:237] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.348541  8043 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:50.352705  1722 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:50.302775  7978 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18.
I20260709 13:04:50.359223  1722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:50.366086  1645 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:50.376978  1722 raft_consensus.cc:606] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
W20260709 13:04:50.379608  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.377758  1722 raft_consensus.cc:614] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:50.343433  1538 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:50.387588  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:50.313482  7962 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.885s	user 0.007s	sys 0.022s
I20260709 13:04:50.392442  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:50.393392  1571 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:50.386400  8009 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:50.396234  7861 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.341364  7968 consensus_queue.cc:260] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.397974  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:50.398624  7952 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.710s	user 0.006s	sys 0.009s
I20260709 13:04:50.398761  1572 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:50.400383  7957 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.367197  8025 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
W20260709 13:04:50.403990  8001 log.cc:927] Time spent T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 1.045s	user 0.002s	sys 0.009s
I20260709 13:04:50.404656  7578 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:50.404793  7998 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.405767  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3074) took 2297 ms. Trace:
I20260709 13:04:50.406219  1718 rpcz_store.cc:276] 0709 13:04:48.108388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.108555 (+   167us) service_pool.cc:224] Handling call
0709 13:04:48.152122 (+ 43567us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:49.150790 (+998668us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b18003354a0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f8248e7d060 00007f8248e7dbfb 00007f8248e8864b
0709 13:04:49.151013 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:49.151051 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:49.151709 (+   658us) log.cc:844] Serialized 26 byte log entry
0709 13:04:49.274828 (+123119us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:49.274889 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.275019 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:50.370368 (+1095349us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:50.405416 (+ 35048us) raft_consensus.cc:1685] finished
0709 13:04:50.405489 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:50.405686 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":79239,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":9366,"spinlock_wait_cycles":36754048,"thread_start_us":224094,"threads_started":2,"wal-append.queue_time_us":182189}
I20260709 13:04:50.407665  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:50.408833  1648 raft_consensus.cc:2368] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16.
I20260709 13:04:50.371543  8028 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:50.411379  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.411487  1714 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:04:50.382318  1687 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:50.419819  7861 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:50.420795  7990 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:50.422665  7998 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:50.410948  8028 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:50.353545  7714 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:50.428755  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 85425.1us
I20260709 13:04:50.398859  7968 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:50.360543  7955 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.443457  1570 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:50.445237  7955 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:50.448618  1538 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:50.452302  8037 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16.
I20260709 13:04:50.454035  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:50.359980  7981 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:50.454308  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:50.457728  7981 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:50.460491  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:50.461227  1713 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:50.466570  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:50.467496  1718 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 14: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:50.472726  1641 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:50.490942  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:50.518074  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:50.519505  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.523027  7957 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:50.525502  7948 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:50.525815  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 220330us
I20260709 13:04:50.527006  1685 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:50.527758  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:50.528043  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:50.528541  2341 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:50.531052  2346 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:04:50.535756  7995 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:50.557294  1571 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:50.559728  1713 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:50.561446  7995 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:50.493552  8016 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:50.591765  7965 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.568058  7966 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:50.670136  2314 leader_election.cc:336] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.601s (SENT)
I20260709 13:04:50.680176  2314 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:50.698938  2341 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
W20260709 13:04:50.698828  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 247351us
W20260709 13:04:50.698648  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 250048us
I20260709 13:04:50.702977  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:50.707803  7966 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:50.711678  8009 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:04:50.713610  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 186631us
I20260709 13:04:50.717592  1685 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:50.719189  7977 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:50.718813  8029 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.713845  7871 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.714552  8028 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:50.715024  7865 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:50.723771  7865 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:50.724519  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 54431.7us
I20260709 13:04:50.719717  7977 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.726166  7977 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:50.721102  7871 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:50.730341  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 698596us
I20260709 13:04:50.732002  7871 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
W20260709 13:04:50.715145  1467 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47518 (request call id 98) took 56368 ms (client timeout 15000 ms). Trace:
W20260709 13:04:50.733284  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 350951us
I20260709 13:04:50.734421  1687 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:50.737915  7970 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:50.715822  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 185619us
I20260709 13:04:50.738662  7789 consensus_queue.cc:1059] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:50.740123  7951 raft_consensus.cc:3055] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:50.740478  1686 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:50.742413  8005 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:50.742125  1470 catalog_manager.cc:5697] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 12 to 13, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 13 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:04:50.743384  7964 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:50.738922  7970 consensus_queue.cc:1059] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:50.718876  1685 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:50.744437  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 270386us
I20260709 13:04:50.751159  1611 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:50.744678  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2747) took 1042 ms. Trace:
I20260709 13:04:50.737838  8037 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:04:50.746692  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.738034  1612 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:50.752326  1646 rpcz_store.cc:276] 0709 13:04:49.702355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.702603 (+   248us) service_pool.cc:224] Handling call
0709 13:04:49.743330 (+ 40727us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.743579 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:49.930784 (+187205us) log.cc:844] Serialized 14 byte log entry
0709 13:04:50.470890 (+540106us) log.cc:844] Serialized 14 byte log entry
0709 13:04:50.744342 (+273452us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:50.744390 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:50.744453 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.744517 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:50.744622 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":539393,"prepare.queue_time_us":238458,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":85,"raft.queue_time_us":545718,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":75,"thread_start_us":404312,"threads_started":2,"wal-append.queue_time_us":323971}
I20260709 13:04:50.765671  1470 catalog_manager.cc:5697] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 11 to 12, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 12 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:04:50.740135  7958 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:04:50.773440  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 54605us
I20260709 13:04:50.777277  1685 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:50.740545  7951 raft_consensus.cc:740] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:50.715289  8016 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:50.779961  1714 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:04:50.782850  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:50.788247  2314 leader_election.cc:336] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.609s (SENT)
I20260709 13:04:50.788807  2314 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:50.789775  1714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:50.790727  1714 raft_consensus.cc:606] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:50.792316  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:50.764845  8004 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:50.771720  7578 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:50.770891  7967 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:50.820683  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:50.808466  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.698159  7965 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.787855  1470 catalog_manager.cc:5697] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:50.801854  1537 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:50.829017  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 73563.2us
I20260709 13:04:50.773770  7964 raft_consensus.cc:697] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:50.777405  8037 raft_consensus.cc:697] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:50.790745  7955 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.797295  7787 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:50.849273  1685 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:50.794926  7981 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:50.841207  1611 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:50.850005  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 112014us
I20260709 13:04:50.858826  7962 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:50.859899  8051 log.cc:927] Time spent T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:04:50.860357  7962 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:50.858172  1612 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:50.798565  8029 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:50.797322  8036 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.044s
W20260709 13:04:50.862895  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 61046.2us
I20260709 13:04:50.863261  7952 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:50.859027  8034 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.826220  7958 raft_consensus.cc:697] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:50.872524  1537 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:50.877164  1685 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:50.867563  7578 raft_consensus.cc:697] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:50.840775  7967 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:50.879765  7578 consensus_queue.cc:237] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.880296  8024 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:50.880951  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:50.841063  7871 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
W20260709 13:04:50.855655  8031 consensus_meta.cc:287] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.702s	user 0.015s	sys 0.000s
I20260709 13:04:50.880463  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:50.854669  7966 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.857290  1470 catalog_manager.cc:5697] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 11 to 14, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 14 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:50.852213  7950 raft_consensus.cc:2764] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:04:50.890760  1611 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:50.898660  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.881345  8024 raft_consensus.cc:697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:50.903736  2314 leader_election.cc:336] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.768s (SENT)
I20260709 13:04:50.904663  2314 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:50.877388  7969 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.895588  1470 catalog_manager.cc:5697] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 13 to 20. New cstate: current_term: 20 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.852116  7995 consensus_queue.cc:1059] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:04:50.853194  7955 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.916815  1612 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:50.918238  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:50.885223  7642 raft_consensus.cc:2764] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:04:50.922083  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:50.942019  2313 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:50.922775  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:50.923444  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:50.946394  1641 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:50.946659  1687 leader_election.cc:400] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16.
I20260709 13:04:50.946964  1687 leader_election.cc:403] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:50.906276  7957 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:50.896572  7998 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:50.921545  7970 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:50.733237  1467 rpcz_store.cc:269] 0709 13:03:54.347000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.347522 (+   522us) service_pool.cc:224] Handling call
0709 13:03:54.815836 (+468314us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b54002b0054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248def090 00007f8248e8df3a 00007f824ec441cc 00007f824ed4237e 00007f824d2032a4 00007f824d203232 00007f824d203161 00007f824d2030dc 00007f824d202392 00007f82471c048c 00007f82471bf8c6
0709 13:04:50.714832 (+55898996us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:46.170751 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:46.171437 (+   686us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:46.171491 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:04:46.171508 (+    17us) tablet.cc:662] Decoding operations
0709 13:04:46.172393 (+   885us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:46.172491 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:46.172519 (+    28us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:04:46.173220 (+   701us) tablet.cc:701] Row locks acquired
0709 13:04:46.173278 (+    58us) write_op.cc:260] PREPARE: finished
0709 13:04:46.173452 (+   174us) write_op.cc:270] Start()
0709 13:04:46.173601 (+   149us) write_op.cc:276] Timestamp: P: 1783602286173383 usec, L: 0
0709 13:04:46.173619 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:04:46.359295 (+185676us) log.cc:844] Serialized 2961 byte log entry
0709 13:04:46.648382 (+289087us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8245d7b0ac 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b
0709 13:04:47.243362 (+594980us) op_driver.cc:464] REPLICATION: finished
0709 13:04:47.499934 (+256572us) write_op.cc:301] APPLY: starting
0709 13:04:47.500087 (+   153us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:47.521983 (+ 21896us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:47.522024 (+    41us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:47.565958 (+ 43934us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:47.566441 (+   483us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 13:04:47.566545 (+   104us) write_op.cc:312] APPLY: finished
0709 13:04:47.567070 (+   525us) log.cc:844] Serialized 73 byte log entry
0709 13:04:47.604963 (+ 37893us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:47.658728 (+ 53765us) write_op.cc:454] Released schema lock
0709 13:04:47.683369 (+ 24641us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:47.684429 (+  1060us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:47.684482 (+    53us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5626479,"reported_tablets":20,"spinlock_wait_cycles":26262784,"child_traces":[["op",{"apply.queue_time_us":247079,"apply.run_cpu_time_us":16268,"apply.run_wall_time_us":184710,"cfile_cache_hit":30,"cfile_cache_hit_bytes":33401,"num_ops":7,"prepare.queue_time_us":335617,"prepare.run_cpu_time_us":30503,"prepare.run_wall_time_us":647958,"raft.queue_time_us":230709,"raft.run_cpu_time_us":18154,"raft.run_wall_time_us":178007,"replication_time_us":1069518,"spinlock_wait_cycles":36838784,"thread_start_us":1164672,"threads_started":4,"wal-append.queue_time_us":493271,"wal-append.run_cpu_time_us":68302,"wal-append.run_wall_time_us":1848738}]]}
I20260709 13:04:50.910301  7436 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:50.927121   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:50.927335  7958 consensus_queue.cc:237] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.928454  1470 catalog_manager.cc:5697] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 9 to 10, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 10 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
W20260709 13:04:50.934677  8058 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:04:50.870885  7951 consensus_queue.cc:260] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:50.901557  7929 consensus_queue.cc:1059] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:04:50.888384  8034 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.937569  7995 raft_consensus.cc:3055] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:50.896713  8031 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:50.890442  7981 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:50.943104  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:51.006990  1714 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
I20260709 13:04:50.943230  1612 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:50.924135  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:51.008663  1643 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:50.945392  1640 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:50.945881  1646 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:50.956789  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:50.901342  7947 consensus_queue.cc:1059] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:04:50.964769  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:50.918124  7865 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:50.960987   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:50.977213  1570 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:50.987851  1719 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:04:50.989080  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:50.950430  7937 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:50.907691  7966 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.994720  1566 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:04:51.025244  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 82088.1us
I20260709 13:04:51.025916  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:51.014047  2313 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:04:50.962023  8036 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:50.957980  7957 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.978933  7436 raft_consensus.cc:697] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:50.961381  8028 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:04:51.010305  1565 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:51.014137  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.920486  7969 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:51.020790  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:51.050345  7965 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:51.026765  1566 raft_consensus.cc:2368] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16.
I20260709 13:04:51.027860  7986 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:51.030223  1568 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:51.031059  1612 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:51.004899  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:50.952211  7873 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.053s
W20260709 13:04:51.078567  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 47585.4us
I20260709 13:04:51.079695  1612 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:51.038666  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:51.078830  1470 catalog_manager.cc:5697] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 9 to 14, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 14 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:04:50.992733  7995 raft_consensus.cc:740] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:51.095541  7995 consensus_queue.cc:260] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:51.004933  7951 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:51.052785  1571 raft_consensus.cc:2410] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:04:51.022744  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:50.998909  7970 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:51.070554  1646 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:51.110821  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:51.076236  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:51.112545  1566 pending_rounds.cc:85] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 6
I20260709 13:04:51.112962  1566 pending_rounds.cc:107] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:04:51.113250  1566 raft_consensus.cc:2889] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:50.968504  7861 consensus_queue.cc:1059] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:04:51.105497  1564 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 13.
I20260709 13:04:51.035060  1687 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:51.024704  7642 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.914800  7964 consensus_queue.cc:237] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:51.073309  7967 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:50.959228  8037 consensus_queue.cc:237] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:50.996812  1641 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:51.030212  7981 raft_consensus.cc:697] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:51.040760  7957 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:51.039495  7865 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:04:51.042105  1538 connection.cc:256] RPC call timeout handler was delayed by -2.1417044209999858s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1655s for a response
I20260709 13:04:51.114984  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:51.114871  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:51.199187  1538 connection.cc:256] RPC call timeout handler was delayed by -2.1058895030000144s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1816s for a response
W20260709 13:04:51.095292  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b82d3b49001741eda872554a7ecb2db3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:04:51.123584  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 88598.6us
I20260709 13:04:51.077527  7986 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:51.111992  1470 catalog_manager.cc:5697] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 12 to 16, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 16 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:51.125810  1570 raft_consensus.cc:1275] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:04:51.205437  1570 pending_rounds.cc:85] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 8
I20260709 13:04:51.205739  1570 pending_rounds.cc:107] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:04:51.205981  1570 raft_consensus.cc:2889] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:51.097549  7995 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:51.112735  7968 consensus_meta.cc:287] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.675s	user 0.009s	sys 0.005s
I20260709 13:04:51.117465  7980 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:51.118485  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:51.199707  1538 connection.cc:256] RPC call timeout handler was delayed by -2.0177510320000351s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1525s for a response
W20260709 13:04:51.220626  1538 connection.cc:256] RPC call timeout handler was delayed by -1.5969942560000163s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1522s for a response
I20260709 13:04:51.120401  7937 raft_consensus.cc:2764] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:04:51.202164  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:51.226488  1641 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:04:51.202399  1687 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:51.204424  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:51.204689  1456 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 109585us
I20260709 13:04:51.122699  8045 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:51.047516  8024 consensus_queue.cc:237] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:51.130919  7861 consensus_queue.cc:1243] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:04:51.229770  1564 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:51.158663  7962 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:51.221236  1538 connection.cc:256] RPC call timeout handler was delayed by -1.5155248040000515s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18660000000000002s for a response
W20260709 13:04:51.242653  1538 connection.cc:256] RPC call timeout handler was delayed by -1.3598538170000438s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2383s for a response
W20260709 13:04:51.243465  1538 connection.cc:256] RPC call timeout handler was delayed by -1.2550430860000006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2197s for a response
W20260709 13:04:51.246071  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.208510  7986 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:51.210347  7926 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:04:51.251174  2314 leader_election.cc:336] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.148s (SENT)
I20260709 13:04:51.234556  1687 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:51.254159  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:51.276074  1572 raft_consensus.cc:2410] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
W20260709 13:04:51.255757  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.258399  1570 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:51.259099  1470 catalog_manager.cc:5697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 11 to 17, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 17 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:51.264525  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:51.267249  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:51.267956  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:51.281472  1717 raft_consensus.cc:2393] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 14: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:51.198793  7865 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:51.124457  7642 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:51.283790  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:51.278884  1722 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:51.288321  1564 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:51.287508  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:51.289283  1568 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:51.250458  7968 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:51.239162  8009 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.298954  1685 leader_election.cc:400] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16.
I20260709 13:04:51.304180  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:51.304888  1685 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:51.306063  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 71608.6us
I20260709 13:04:51.287541  1470 catalog_manager.cc:5697] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 11 to 12, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 12 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
W20260709 13:04:51.310774  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.307082  1687 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:51.320607  1470 catalog_manager.cc:5697] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 14 to 15, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 15 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:04:51.285259  7926 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.291424  7865 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:51.229962  7949 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:04:51.334625  1570 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 13:04:51.349841  1685 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:51.210080  7436 consensus_queue.cc:237] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:51.300336  8064 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.336820  7926 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:51.289326  7873 consensus_queue.cc:1059] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:04:51.325158  1470 catalog_manager.cc:5697] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 9 to 12, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 12 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
W20260709 13:04:51.290126  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 210485us
W20260709 13:04:51.414572  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 78486us
I20260709 13:04:51.336683  7948 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:51.391963  7871 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:51.332666  7955 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:51.398854  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:51.336450  7968 raft_consensus.cc:697] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:51.418200  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:51.336898  7975 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:04:51.421600  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.353564  7861 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:51.326306  7962 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:04:51.428130  1570 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:51.428977  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:51.426692  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 77287.1us
I20260709 13:04:51.331661  8009 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:51.399251  8059 log.cc:927] Time spent T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.347s	user 0.000s	sys 0.001s
W20260709 13:04:51.417282  7882 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.006s	sys 0.008s
I20260709 13:04:51.401314  8066 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.467871  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:51.481098  1470 catalog_manager.cc:5697] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 17 to 20. New cstate: current_term: 20 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:04:51.430277  8026 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.421720  7948 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.436117  7966 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:51.467092  8034 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:51.483332  7969 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:51.480098  7871 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:51.467391  8031 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:51.505515  7882 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.006s	sys 0.008s
I20260709 13:04:51.481274  7975 raft_consensus.cc:697] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:04:51.515143  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 267091us
I20260709 13:04:51.528594  1470 catalog_manager.cc:5697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 12 to 15, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 15 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:51.422529  8064 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:51.417682  7981 consensus_queue.cc:237] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:51.534524  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 116378us
W20260709 13:04:51.539851  1538 leader_election.cc:336] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.655s (SENT)
I20260709 13:04:51.547459  1538 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:51.526681  8057 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.407s	user 0.002s	sys 0.000s
I20260709 13:04:51.549204  1685 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:51.551281  1572 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:04:51.526926  8060 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.396s	user 0.001s	sys 0.002s
I20260709 13:04:51.552481  1470 catalog_manager.cc:5697] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index 6 to 9, term changed from 10 to 13, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New cstate: current_term: 13 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:04:51.522066  8077 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:04:51.557330  1538 leader_election.cc:336] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.866s (SENT)
I20260709 13:04:51.559919  1685 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:51.578536  1643 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:51.523702  7948 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:51.588580  1685 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:51.538699  8034 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:51.587453  1538 leader_election.cc:336] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.197s (SENT)
I20260709 13:04:51.610868  1538 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:51.587703  1565 consensus_meta.cc:287] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.012s
W20260709 13:04:51.609316  8076 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.182s	user 0.002s	sys 0.000s
I20260709 13:04:51.610369  1572 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:51.610258  1570 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 12.
I20260709 13:04:51.611737  1565 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:51.591382  1470 catalog_manager.cc:5697] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:04:51.553562  7970 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:51.615608  1640 consensus_meta.cc:287] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.606s	user 0.013s	sys 0.015s
I20260709 13:04:51.556206  7966 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:51.518725  8026 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:51.624099  1640 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
W20260709 13:04:51.571091  8063 log.cc:927] Time spent T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.353s	user 0.000s	sys 0.001s
I20260709 13:04:51.627735  1685 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:51.631170  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 202228us
I20260709 13:04:51.647059  1470 catalog_manager.cc:5697] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 11 to 14, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 14 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:04:51.649299  1612 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:51.562412  7873 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:51.652813  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:51.591384  7871 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:51.618677  8061 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.402s	user 0.000s	sys 0.002s
I20260709 13:04:51.519636  8066 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:51.678838  8065 log.cc:927] Time spent T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.379s	user 0.001s	sys 0.001s
I20260709 13:04:51.681191  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2758) took 1247 ms. Trace:
I20260709 13:04:51.681740  1644 rpcz_store.cc:276] 0709 13:04:50.437402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.449702 (+ 12300us) service_pool.cc:224] Handling call
0709 13:04:50.450177 (+   475us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:50.759447 (+309270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.759484 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:50.771003 (+ 11519us) log.cc:844] Serialized 52 byte log entry
0709 13:04:51.072375 (+301372us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:51.072421 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.072494 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:51.680799 (+608305us) raft_consensus.cc:1685] finished
0709 13:04:51.680837 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.681099 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":355783,"raft.queue_time_us":331489,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":3475,"thread_start_us":233084,"threads_started":1,"wal-append.queue_time_us":414836}
W20260709 13:04:51.659734  8059 log.cc:927] Time spent T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:04:51.634629  7962 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.692636  1470 catalog_manager.cc:5697] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 13 to 15, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 15 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:04:51.695163  1572 raft_consensus.cc:1275] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:04:51.676404  8071 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.345s	user 0.001s	sys 0.000s
I20260709 13:04:51.719874  1572 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:51.723109  7706 raft_consensus.cc:3055] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:51.710460  7975 consensus_queue.cc:237] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:51.723855  7706 raft_consensus.cc:740] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:51.724751  1470 catalog_manager.cc:5697] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:04:51.725907  8090 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:51.726604  1470 catalog_manager.cc:5697] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 9 to 14, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 14 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:51.727624  8090 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:04:51.728003  8070 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 13:04:51.624943  8034 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:51.667668  8077 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:51.691583  8060 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:04:51.711983  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 60477.2us
I20260709 13:04:51.691422  7873 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:51.728392  1470 catalog_manager.cc:5697] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:51.732100  2314 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.338s (SENT)
I20260709 13:04:51.591516  7968 consensus_queue.cc:237] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:51.770426  1612 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:51.778198  1639 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
W20260709 13:04:51.684005  8074 log.cc:927] Time spent T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:04:51.773375  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:51.784969  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 1064) took 1393 ms. Trace:
I20260709 13:04:51.705071  8075 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:51.788492  1570 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:04:51.791307  1470 catalog_manager.cc:5697] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:51.793382  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 205882us
I20260709 13:04:51.785730  1567 rpcz_store.cc:276] 0709 13:04:50.391400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.391613 (+   213us) service_pool.cc:224] Handling call
0709 13:04:50.392722 (+  1109us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.430933 (+ 38211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.431005 (+    72us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:50.514395 (+ 83390us) log.cc:844] Serialized 26 byte log entry
0709 13:04:50.951736 (+437341us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:50.951805 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.951912 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:51.784479 (+832567us) raft_consensus.cc:1685] finished
0709 13:04:51.784512 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.784860 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":146661,"raft.queue_time_us":39988,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":7903,"thread_start_us":377753,"threads_started":1,"wal-append.queue_time_us":521060}
I20260709 13:04:51.731814  8067 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:51.794696  1538 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 3.220s (SENT)
I20260709 13:04:51.895361  1538 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:51.746804  7966 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:51.765786  7871 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:51.910501  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 115777us
W20260709 13:04:51.911347  1538 leader_election.cc:336] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 3.228s (SENT)
I20260709 13:04:51.807462  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2757) took 1379 ms. Trace:
I20260709 13:04:51.912231  1648 rpcz_store.cc:276] 0709 13:04:50.428187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.429476 (+  1289us) service_pool.cc:224] Handling call
0709 13:04:50.436668 (+  7192us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.686673 (+250005us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.686723 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:50.710090 (+ 23367us) log.cc:844] Serialized 26 byte log entry
0709 13:04:50.868948 (+158858us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:50.902484 (+ 33536us) log.cc:844] Serialized 14 byte log entry
0709 13:04:50.902865 (+   381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.902950 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:51.726294 (+823344us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b18004c9780. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d98522 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:04:51.807049 (+ 80755us) raft_consensus.cc:1685] finished
0709 13:04:51.807170 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.807369 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":254315,"prepare.queue_time_us":33243,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":278413,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":6799,"spinlock_wait_cycles":44002048,"thread_start_us":158166,"threads_started":1,"wal-append.queue_time_us":341287}
I20260709 13:04:51.869102  8086 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.876526  1567 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:04:51.904687  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 134535us
I20260709 13:04:51.911782  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:51.910164  8080 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
W20260709 13:04:51.980633  1568 consensus_meta.cc:287] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.688s	user 0.022s	sys 0.003s
I20260709 13:04:51.923393  8075 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:51.716379  7962 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:51.988369  1568 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:51.990260  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:51.990671  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:51.985689  1713 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:51.993121  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:51.999758  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.994803  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:52.001111  2345 raft_consensus.cc:2410] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:51.961385  8005 raft_consensus.cc:3055] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:52.004923  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:52.005937  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:52.006858  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:52.007491  8098 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:52.008551  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:52.009078  8088 consensus_queue.cc:1059] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.299s
W20260709 13:04:52.010797  8091 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:04:52.011255  8016 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:52.050158  1719 raft_consensus.cc:2410] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
W20260709 13:04:52.017046  7986 consensus_meta.cc:287] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.729s	user 0.002s	sys 0.015s
I20260709 13:04:52.021759  7873 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:52.021804  8083 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:04:52.023784  8078 log.cc:927] Time spent T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.399s	user 0.007s	sys 0.006s
I20260709 13:04:52.024027  8096 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:04:52.025146  8068 log.cc:927] Time spent T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.394s	user 0.000s	sys 0.003s
I20260709 13:04:52.042770  1722 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 18: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:52.044174  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:51.979372  7966 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:52.065984  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:52.066453  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:51.920379  8067 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:51.952788  7871 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:52.052246  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:52.030866  7977 consensus_meta.cc:287] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.304s	user 0.018s	sys 0.000s
I20260709 13:04:52.055397  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:52.057614  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:52.045289  7978 raft_consensus.cc:993] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: : Instructing follower 066ee0109aba4465ba17fa65e14ef504 to start an election
I20260709 13:04:52.057878  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:52.034385  8075 raft_consensus.cc:697] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 13 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:52.102133  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:52.032817  8005 raft_consensus.cc:740] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:52.066831  1717 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:04:52.105264  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:52.106494  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:52.106853  7962 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:52.107967  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:52.108229  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2782) took 1091 ms. Trace:
I20260709 13:04:52.109175  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:52.068240  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:52.048651  8009 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:52.089598  2345 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:51.969229  8086 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:52.094142  8079 log.cc:927] Time spent T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.311s	user 0.000s	sys 0.001s
I20260709 13:04:52.095965  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:52.096505  8028 consensus_meta.cc:287] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.044s	user 0.022s	sys 0.003s
I20260709 13:04:52.097098  8026 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:52.108927  1566 rpcz_store.cc:276] 0709 13:04:51.017068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.111968 (+ 94900us) service_pool.cc:224] Handling call
0709 13:04:51.112325 (+   357us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:51.118504 (+  6179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:51.118532 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:51.119090 (+   558us) log.cc:844] Serialized 52 byte log entry
0709 13:04:51.419034 (+299944us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:51.579584 (+160550us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.650685 (+ 71101us) log.cc:844] Serialized 14 byte log entry
0709 13:04:52.019354 (+368669us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b1800377500. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d98522 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:04:52.022877 (+  3523us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:52.106949 (+ 84072us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b1800377500. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d662a6 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b
0709 13:04:52.107612 (+   663us) spinlock_profiling.cc:243] Waited 3.69 ms on lock 0x7b1800377500. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287
0709 13:04:52.107802 (+   190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.107917 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:52.107972 (+    55us) raft_consensus.cc:1685] finished
0709 13:04:52.107992 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:52.108160 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54968,"prepare.queue_time_us":435318,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":5843,"raft.queue_time_us":267342,"raft.run_cpu_time_us":532,"raft.run_wall_time_us":99078,"spinlock_wait_cycles":86668416,"thread_start_us":429839,"threads_started":3,"wal-append.queue_time_us":377369}
I20260709 13:04:52.100658  7978 raft_consensus.cc:1081] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Signalling peer 066ee0109aba4465ba17fa65e14ef504 to start an election
I20260709 13:04:52.048960  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:52.110083  8084 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.361s	user 0.002s	sys 0.000s
I20260709 13:04:52.112298  1716 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:52.119786  1537 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:52.124576  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:52.070912  8088 consensus_queue.cc:1243] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:04:52.151924  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:52.063122  7873 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:52.101881  8075 consensus_queue.cc:237] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:52.137368  2348 raft_consensus.cc:2368] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15.
I20260709 13:04:52.069130  8098 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:52.165946  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:52.105005  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:52.140501  1720 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:52.166880  1722 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:04:52.169510  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:52.147692  8090 consensus_queue.cc:1059] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:52.147682  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:52.149103  8086 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:52.058924  7986 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:52.149461  7962 consensus_queue.cc:1048] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:52.150125  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:52.177824  1648 raft_consensus.cc:2410] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:04:52.150135  8094 consensus_queue.cc:1059] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:52.151435  7871 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:52.151547  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:52.152807  1717 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:52.153803  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:52.506445  1647 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 18: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:52.506866  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2768) took 1175 ms. Trace:
I20260709 13:04:52.155190  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:52.507664  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:52.158595  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:52.509443  1641 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 18: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:52.510425  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:52.507263  1646 rpcz_store.cc:276] 0709 13:04:51.331620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.332028 (+   408us) service_pool.cc:224] Handling call
0709 13:04:51.353740 (+ 21712us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:51.400844 (+ 47104us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:51.400890 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:51.432227 (+ 31337us) log.cc:844] Serialized 52 byte log entry
0709 13:04:51.647209 (+214982us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:51.647257 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.647336 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:52.227165 (+579829us) raft_consensus.cc:1685] finished
0709 13:04:52.506611 (+279446us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:52.506805 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1492,"raft.queue_time_us":16140,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":53123,"thread_start_us":127989,"threads_started":1,"wal-append.queue_time_us":266381}
I20260709 13:04:52.511722  1641 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:52.162340   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:52.057266  8045 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 13:04:52.145619  1537 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:04:52.146476  7957 consensus_meta.cc:287] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.937s	user 0.024s	sys 0.000s
I20260709 13:04:52.141968  8028 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16.
I20260709 13:04:52.521078  1714 raft_consensus.cc:1240] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:52.520685  7957 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:52.523603  7957 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:52.229736  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 318299us
I20260709 13:04:52.361702  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:52.524673  1722 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:52.174813  7948 consensus_meta.cc:287] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.565s	user 0.010s	sys 0.004s
I20260709 13:04:52.513517  1641 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:04:52.526036  7948 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:52.475574  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 325351us
I20260709 13:04:52.527191  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:52.527613  7865 consensus_meta.cc:287] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 1.167s	user 0.007s	sys 0.007s
I20260709 13:04:52.527904  1641 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:52.528172  7865 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:52.528743  7948 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:52.187600  7926 consensus_meta.cc:287] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.753s	user 0.010s	sys 0.006s
I20260709 13:04:52.182340  8005 consensus_queue.cc:260] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:52.529801  7926 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:52.530581  8005 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:52.532040  7926 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:04:52.533185  1715 raft_consensus.cc:2368] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16.
W20260709 13:04:52.314508  8037 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.015s	sys 0.014s
I20260709 13:04:52.178565  8098 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:52.220472  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:52.113528  7977 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:52.536880  1721 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
I20260709 13:04:52.539196  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:52.540702  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:52.536494  8037 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.015s	sys 0.015s
I20260709 13:04:52.488862  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:52.489869  7873 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:52.490110  8017 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:52.491494  8103 log.cc:927] Time spent T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.058s	user 0.000s	sys 0.002s
W20260709 13:04:52.492139  7951 consensus_meta.cc:287] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 1.376s	user 0.007s	sys 0.019s
I20260709 13:04:52.549260  1641 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:04:52.493700  7986 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:52.087047  8083 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:52.493969  7995 consensus_meta.cc:287] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.271s	user 0.016s	sys 0.009s
I20260709 13:04:52.172617  8101 consensus_queue.cc:1059] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.279s
I20260709 13:04:52.163905  8026 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:52.560115  8098 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:52.560748  8026 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.561146  1573 raft_consensus.cc:2410] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:04:52.561299  8026 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:52.499110  1648 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:52.562752  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:52.563350  1648 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:52.503266  8104 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.136s	user 0.010s	sys 0.000s
I20260709 13:04:52.509248  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:52.517369  1716 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:04:52.568034  1647 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:52.569224  1647 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:04:52.137884  8077 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.569976  8077 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:52.179320  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:52.575970  1642 raft_consensus.cc:1240] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:52.534401  2341 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:04:52.546660  1644 raft_consensus.cc:2410] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:04:52.547359  1713 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 18: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:52.548578  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:52.577993  8016 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.578812  8016 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:52.545856  8034 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:52.550216  1717 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:52.493216  8067 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:52.560232  8083 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:52.582275  7865 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:52.554030  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:52.555711  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:52.545972  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:52.573280  8077 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:52.591619  1747 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:38281 (5 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:38281 timed out after 15.000s (SENT)
I20260709 13:04:52.596010  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:52.597062  1640 raft_consensus.cc:2393] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 14: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:52.597644  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:52.598309  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:52.598703  1642 raft_consensus.cc:2368] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 14. Current term is 15.
I20260709 13:04:52.599579  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:52.601497  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:52.602931  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:52.603469  8004 raft_consensus.cc:3055] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:52.603868  1642 raft_consensus.cc:2368] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 15. Current term is 16.
I20260709 13:04:52.604135  8004 raft_consensus.cc:740] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:52.606024  7948 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.626691  8097 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:52.627415  8097 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:52.608592  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:52.553290  8017 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:52.628657  1641 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:52.629877  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:52.629982  8097 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:52.610921  8062 log.cc:927] Time spent T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.383s	user 0.005s	sys 0.004s
I20260709 13:04:52.630788  1641 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 18: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:52.609403  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2791) took 1348 ms. Trace:
W20260709 13:04:52.614559  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 468856us
W20260709 13:04:52.631693  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.618010  8102 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:52.633838  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2766) took 1703 ms. Trace:
I20260709 13:04:52.634562  1648 raft_consensus.cc:2410] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:04:52.633762  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:52.619747  2345 raft_consensus.cc:1240] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:52.636461  1640 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:04:52.624436  7871 raft_consensus.cc:3055] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:52.625604  8096 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.627053  7948 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:52.611985  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2765) took 1656 ms. Trace:
I20260709 13:04:52.637810  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:52.610148  1722 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:04:52.639732  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:52.654990  1537 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:52.639434  1538 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:52.634495  1639 rpcz_store.cc:276] 0709 13:04:50.929887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.933725 (+  3838us) service_pool.cc:224] Handling call
0709 13:04:50.934088 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:51.781236 (+847148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.633441 (+852205us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b5400098154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:52.633681 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1934735744}
I20260709 13:04:52.645370  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:52.659153  1639 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:52.659718  1644 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:04:52.649724  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:52.660986  1648 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:04:52.640149  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:52.632332  1572 rpcz_store.cc:276] 0709 13:04:51.261372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.739776 (+478404us) service_pool.cc:224] Handling call
0709 13:04:51.740097 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:51.965150 (+225053us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:51.999204 (+ 34054us) log.cc:844] Serialized 14 byte log entry
0709 13:04:52.172842 (+173638us) log.cc:844] Serialized 14 byte log entry
0709 13:04:52.607078 (+434236us) spinlock_profiling.cc:243] Waited 4.94 ms on lock 0x7b18000f2f20. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d12c0a 00007f824f452219 00007f8245d9aa40 00007f8245d954e4 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287 00007f8248ea0208
0709 13:04:52.607239 (+   161us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:52.607292 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:52.607356 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.609141 (+  1785us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b5400405bd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:52.609185 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:52.609333 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":262650,"prepare.queue_time_us":39422,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":440513,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":44546,"spinlock_wait_cycles":540471168,"thread_start_us":172266,"threads_started":1,"wal-append.queue_time_us":539202}
I20260709 13:04:52.663087  1640 raft_consensus.cc:2376] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16. Re-sending same reply.
I20260709 13:04:52.664566  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:52.662340  1567 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
W20260709 13:04:52.721014  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 66147us
I20260709 13:04:52.721342  1567 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 13:04:52.722218  1537 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:52.654354  1571 rpcz_store.cc:276] 0709 13:04:50.955179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.956450 (+  1271us) service_pool.cc:224] Handling call
0709 13:04:52.611736 (+1655286us) spinlock_profiling.cc:243] Waited 42.4 ms on lock 0x7b54001b6554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:52.611896 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97520896}
I20260709 13:04:52.669340  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:52.724826  1571 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:52.672133  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:52.726676  1568 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:52.676816  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:52.728256  1722 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:52.719260  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:52.766340  1572 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:04:52.632802  8102 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:52.777014  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 109551us
I20260709 13:04:52.779296  1538 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:52.783404  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 61278.2us
I20260709 13:04:52.832653  1722 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:52.777132  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:52.644441  7871 raft_consensus.cc:740] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:52.673194  8098 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:52.860778  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:52.862576  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 83402.9us
I20260709 13:04:52.864461  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:52.866976  1722 raft_consensus.cc:2410] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:04:52.873898  1612 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:52.875584  1612 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:52.842128  8029 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:52.836753  8083 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:52.864956  1538 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:52.840657  8110 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:52.654289  8097 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.200s
I20260709 13:04:52.836357  7966 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:52.836949  7962 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:52.834442  8102 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:52.837064  1566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e"
dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:40670
I20260709 13:04:52.856069  7970 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.778298  8024 consensus_queue.cc:1059] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.111s
W20260709 13:04:52.989319  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 124639us
I20260709 13:04:52.991330  1566 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:52.993113  1572 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:52.666074  8096 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:52.994197  1569 raft_consensus.cc:3055] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:53.000519  1538 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:53.001852  1566 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:52.649718  8004 consensus_queue.cc:260] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:52.902372  8029 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.001091  1569 raft_consensus.cc:740] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:52.900386  8090 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:52.981503  8017 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:52.991148  7966 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.994472  7969 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:52.993184  8098 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:52.900628  7986 consensus_queue.cc:1059] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.114s
I20260709 13:04:53.031989  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2755) took 2630 ms. Trace:
I20260709 13:04:53.032266  1564 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:04:53.008180  8077 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:53.027819  1569 consensus_queue.cc:260] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:53.050652  1569 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:53.021332  8029 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:53.033437  1538 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:53.056576  8088 raft_consensus.cc:3055] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:53.000488  8110 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:53.058784  7977 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:53.005442  7970 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:53.036134  8005 consensus_meta.cc:287] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.505s	user 0.006s	sys 0.006s
I20260709 13:04:53.038650  1645 rpcz_store.cc:276] 0709 13:04:50.388613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.388860 (+   247us) service_pool.cc:224] Handling call
0709 13:04:50.389309 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.687671 (+298362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.687843 (+   172us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:50.689236 (+  1393us) log.cc:844] Serialized 26 byte log entry
0709 13:04:50.990565 (+301329us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:50.990620 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.990693 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:52.610257 (+1619564us) spinlock_profiling.cc:243] Waited 3.51 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:53.009860 (+399603us) raft_consensus.cc:1685] finished
0709 13:04:53.009903 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.018653 (+  8750us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":388007,"raft.queue_time_us":282584,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"spinlock_wait_cycles":8073472,"thread_start_us":249954,"threads_started":1,"wal-append.queue_time_us":368807}
I20260709 13:04:53.031266  7966 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:53.028873  8018 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:52.998344  7962 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:53.061002  7948 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:53.097358  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 64054.5us
I20260709 13:04:53.077320  1566 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:53.104912  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 229310us
I20260709 13:04:53.124390  1612 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:53.126169  2312 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.829s (SENT) [suppressed 17 similar messages]
I20260709 13:04:53.038650  8004 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:53.128317  1612 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:53.132921  1612 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:53.065482  8096 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:53.139372  1569 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:04:53.140667  1538 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:53.144568  1612 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:53.083377  8088 raft_consensus.cc:740] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:53.147279  1566 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:53.127243  8026 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.565s	user 0.005s	sys 0.007s
I20260709 13:04:53.104714  7969 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:53.187786  7969 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:53.134349  8077 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:53.154320  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.148676  1612 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:53.198055  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 49426.1us
I20260709 13:04:53.200214  1612 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:53.152215  1538 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:53.153401  8096 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:53.104591  7714 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:53.146102  7986 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:53.034380  7871 consensus_queue.cc:260] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:53.159821  2341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55888 (request call id 835) took 1429 ms. Trace:
I20260709 13:04:53.160995  7970 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:53.162968  7947 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:53.166941  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2690) took 1175 ms. Trace:
I20260709 13:04:53.139446  8116 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:04:53.169021  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.038901  8083 consensus_queue.cc:1059] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.096s
I20260709 13:04:53.157464  8064 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:53.178447  8097 consensus_queue.cc:1059] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:53.164368  7706 consensus_queue.cc:260] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:53.180179  8088 consensus_queue.cc:260] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:53.150856  8018 raft_consensus.cc:697] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:53.202394  7962 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:53.039724  8090 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:53.270535  8083 raft_consensus.cc:3055] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:53.271090  8083 raft_consensus.cc:740] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:53.271981  8083 consensus_queue.cc:260] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:53.273204  8090 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:53.221278  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2703) took 1121 ms. Trace:
I20260709 13:04:53.226636  2341 rpcz_store.cc:276] 0709 13:04:51.730591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.730918 (+   327us) service_pool.cc:224] Handling call
0709 13:04:53.159234 (+1428316us) spinlock_profiling.cc:243] Waited 753 ms on lock 0x7b5400164754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:53.159436 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1733026560}
I20260709 13:04:53.230660  1715 rpcz_store.cc:276] 0709 13:04:51.991538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.991722 (+   184us) service_pool.cc:224] Handling call
0709 13:04:53.166535 (+1174813us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b5400210554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:53.166824 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1080628096}
I20260709 13:04:53.210361  8077 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:53.205366  8110 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:53.222829  7990 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:53.190544  8026 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:53.254575  7970 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:53.253473  7871 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:53.284621  7929 raft_consensus.cc:3055] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:04:53.211493  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 59349.9us
I20260709 13:04:53.324378  1538 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:53.274266  8097 raft_consensus.cc:3055] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:53.325976  8097 raft_consensus.cc:740] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:53.327937  7972 raft_consensus.cc:3055] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:53.328543  7972 raft_consensus.cc:740] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:53.329615  7972 consensus_queue.cc:260] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:53.331264  7972 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:53.275214  7706 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:53.280686  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:53.283787  8034 raft_consensus.cc:3055] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:53.286562  1718 rpcz_store.cc:276] 0709 13:04:52.099611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.580091 (+480480us) service_pool.cc:224] Handling call
0709 13:04:52.580573 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:52.580841 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:52.899127 (+318286us) log.cc:844] Serialized 14 byte log entry
0709 13:04:53.220678 (+321551us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:53.220758 (+    80us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:53.220894 (+   136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.220994 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.221196 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":426728,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":79,"raft.queue_time_us":2454,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":6641,"thread_start_us":534017,"threads_started":2}
I20260709 13:04:53.339905  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:53.302976  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 73310.4us
W20260709 13:04:53.303150  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 177000us
I20260709 13:04:53.303857  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:53.316080  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:53.267005  7969 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:53.309194  8090 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:04:53.286738  7952 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.077s	sys 0.100s
I20260709 13:04:53.291213  8088 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:53.349984  2342 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:53.339859  7990 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:53.362553  7990 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:53.347654  8098 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.366334  8098 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:53.339134  7955 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:53.323119  7929 raft_consensus.cc:740] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:53.318444  8083 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:53.349467  7952 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.077s	sys 0.100s
I20260709 13:04:53.386314  1612 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:53.390034  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:53.393428  1715 raft_consensus.cc:3055] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:53.394047  1715 raft_consensus.cc:740] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:53.339365  8034 raft_consensus.cc:740] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:53.400527  1573 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2779) took 2395 ms (client timeout 2941 ms). Trace:
I20260709 13:04:53.405077  1612 leader_election.cc:400] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16.
I20260709 13:04:53.405722  1612 leader_election.cc:403] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 13:04:53.401271  1573 rpcz_store.cc:269] 0709 13:04:50.993885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.091914 (+ 98029us) service_pool.cc:224] Handling call
0709 13:04:53.389115 (+2297201us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400014854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:53.389377 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3129491712}
I20260709 13:04:53.408245  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:53.407379  1715 consensus_queue.cc:260] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:53.416405  1715 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:53.418354  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2712) took 1314 ms. Trace:
I20260709 13:04:53.418919  1722 rpcz_store.cc:276] 0709 13:04:52.103705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.945547 (+841842us) service_pool.cc:224] Handling call
0709 13:04:52.946157 (+   610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:52.946495 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:53.127287 (+180792us) log.cc:844] Serialized 14 byte log entry
0709 13:04:53.417872 (+290585us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:53.417925 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:53.418050 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.418115 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.418278 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":122071,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":41,"thread_start_us":421188,"threads_started":2}
I20260709 13:04:53.394635  7970 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.424161  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:53.429776  7929 consensus_queue.cc:260] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:53.404150  7990 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:53.436342  7966 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:53.413446  7947 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:53.451014  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:53.460461  1538 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:53.490455  1538 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:53.467507  8034 consensus_queue.cc:260] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:53.327350  8097 consensus_queue.cc:260] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:53.475945  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:53.371052  8090 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:53.486564  1567 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.017s	sys 0.041s
W20260709 13:04:53.499282  1685 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.027s	sys 0.035s
I20260709 13:04:53.469856  8116 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:53.500114  1722 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
W20260709 13:04:53.501405  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:53.506753  1685 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.027s	sys 0.036s
I20260709 13:04:53.508853  1715 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
W20260709 13:04:53.509521  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:53.500782  8099 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.509s	user 0.005s	sys 0.013s
W20260709 13:04:53.520025  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:53.528981  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:53.531301  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:53.533140  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.501407  7929 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:53.543292  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.499815  7969 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:53.559720  7969 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:53.508558  8064 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:53.570025  1719 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:53.545208  7962 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:53.572377  7962 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:53.575002  7962 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:53.575265  7966 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:53.576956  8026 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:53.579504  8090 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:53.570322  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.503921  7990 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:53.504081  7986 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.586215  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:53.586438  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:53.587744  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:53.691185  1566 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:53.478858  7970 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:53.631789  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:53.692888  1572 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:04:53.573251  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:53.673933  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.416195  8018 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:53.514751  7947 raft_consensus.cc:697] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:04:53.696342  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.709028  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:53.713176  1718 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:04:53.586699  8066 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:53.714493  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:53.607970  7990 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:53.559758  8120 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.237s	user 0.001s	sys 0.001s
I20260709 13:04:53.714493  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:53.580859  8102 consensus_meta.cc:287] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.567s	user 0.015s	sys 0.000s
I20260709 13:04:53.546406  8097 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:53.723402  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:53.715703  7714 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:53.570179  8034 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:53.607419  8026 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:53.715626  7990 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:53.736919  8102 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:53.738687   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:53.619097  7972 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:53.710057  8125 log.cc:927] Time spent T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:04:53.711256  8064 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:53.831852  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:53.847234  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:53.736842  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 655) took 1122 ms. Trace:
I20260709 13:04:53.850512  1639 rpcz_store.cc:276] 0709 13:04:52.613935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.660805 (+ 46870us) service_pool.cc:224] Handling call
0709 13:04:52.661277 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:52.844306 (+183029us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:52.844357 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:52.859743 (+ 15386us) log.cc:844] Serialized 26 byte log entry
0709 13:04:53.375088 (+515345us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:53.731839 (+356751us) log.cc:844] Serialized 14 byte log entry
0709 13:04:53.736216 (+  4377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.736360 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:53.736453 (+    93us) raft_consensus.cc:1685] finished
0709 13:04:53.736466 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.736732 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3832,"raft.queue_time_us":132945,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":8076,"thread_start_us":841990,"threads_started":2,"wal-append.queue_time_us":663477}
I20260709 13:04:53.740213  8017 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.848829  1718 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:04:53.802595  8066 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.718518  7986 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:53.873462  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2714) took 1712 ms. Trace:
W20260709 13:04:53.843623  8120 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:04:53.827344  7955 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.817481  7966 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:04:53.932353  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.941902  1719 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:53.942973  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2713) took 1839 ms. Trace:
I20260709 13:04:53.943464  1719 rpcz_store.cc:276] 0709 13:04:52.103964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.871764 (+1767800us) service_pool.cc:224] Handling call
0709 13:04:53.941411 (+ 69647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.942632 (+  1221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.942899 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.798940  7714 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:54.008862  1611 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.002s	sys 0.009s
W20260709 13:04:54.009404  1611 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.002s	sys 0.009s
W20260709 13:04:54.009820  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 590125us
I20260709 13:04:53.820659  8026 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:53.834923  7972 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:53.862581  8064 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:53.927484  8096 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:54.111125  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:54.111572  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:54.003453  7952 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.001s	sys 0.009s
W20260709 13:04:54.130345  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.909950  1718 rpcz_store.cc:276] 0709 13:04:52.161658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.872162 (+1710504us) service_pool.cc:224] Handling call
0709 13:04:53.872644 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.872845 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:53.873018 (+   173us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:53.873081 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:53.873147 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.873228 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.873383 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.124119  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:54.132411  1572 raft_consensus.cc:2393] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:53.895538  8017 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:54.139671  1538 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.522s (SENT)
I20260709 13:04:54.030006  7706 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:54.144783  1538 leader_election.cc:336] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.525s (SENT)
I20260709 13:04:54.144177  1566 raft_consensus.cc:1275] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:04:53.950222  8066 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:54.074683  7962 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.147723  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:53.915325  7947 consensus_queue.cc:237] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:54.000130  7966 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:54.160306  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:54.130587  7952 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.001s	sys 0.010s
I20260709 13:04:54.056875  7967 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:54.154121  1572 raft_consensus.cc:2410] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:04:53.933772  7955 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:54.154697  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:35480 (request call id 27) took 1621 ms. Trace:
W20260709 13:04:54.162890  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45639.5us
I20260709 13:04:54.169971  1719 rpcz_store.cc:276] 0709 13:04:52.528284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.026816 (+1498532us) service_pool.cc:224] Handling call
0709 13:04:54.149176 (+122360us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.170140  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:54.177230  8098 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:54.151760  7873 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.174408  1564 raft_consensus.cc:2376] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 12. Re-sending same reply.
I20260709 13:04:54.184480  1612 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:54.153745  8004 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:54.188088  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:54.156898  8090 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:04:54.175644  8029 consensus_meta.cc:287] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.072s	user 0.015s	sys 0.003s
W20260709 13:04:54.180464  8088 consensus_meta.cc:287] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.798s	user 0.019s	sys 0.000s
I20260709 13:04:54.143450  8096 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:54.165151  7969 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:54.192042  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:54.193773  1566 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:54.186012  8131 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 13:04:54.155485  7706 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:54.179297  8031 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:04:54.160104  7962 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:54.192299  8136 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.221172  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:54.222132  1719 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:54.191613  8138 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:54.216034  8098 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:54.179350  7967 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:54.216058  8137 consensus_queue.cc:1059] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 13:04:54.190102  7873 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:54.235948  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 51571.1us
I20260709 13:04:54.238581  1718 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:04:54.241816  1612 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:54.242254  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 70356us
I20260709 13:04:54.243135  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3097) took 2976 ms. Trace:
I20260709 13:04:54.194293  8004 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:54.244596  1714 rpcz_store.cc:276] 0709 13:04:51.267180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.267809 (+   629us) service_pool.cc:224] Handling call
0709 13:04:51.268103 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:52.521582 (+1253479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.242905 (+1721323us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400247a54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:54.243075 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2880125184}
I20260709 13:04:54.243923  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:54.245739  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:54.246335  1714 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:04:54.216684  8088 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:54.221251  7970 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:54.199416  8090 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:54.203784  8029 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:54.250664  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:54.264060  1714 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:54.259933  1566 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:54.233270  8142 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:54.269958  8128 log.cc:927] Time spent T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.299s	user 0.002s	sys 0.000s
I20260709 13:04:54.276436  8017 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:54.276872  7714 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:54.278255  8134 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:04:54.286085  1719 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
W20260709 13:04:54.292816  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 237756us
I20260709 13:04:54.262010  7986 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:54.236855  8098 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:54.264734  8144 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:54.229058  8136 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:54.320924  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 72908.8us
I20260709 13:04:54.326228  1719 raft_consensus.cc:1275] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:04:54.334758  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2717) took 1718 ms. Trace:
I20260709 13:04:54.335239  1719 rpcz_store.cc:276] 0709 13:04:52.616198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.325402 (+1709204us) service_pool.cc:224] Handling call
0709 13:04:54.325822 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.334300 (+  8478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.334673 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.262207  8088 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:54.240270  8138 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:54.311954  8026 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:54.339897  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 96020.1us
I20260709 13:04:54.347150  1612 leader_election.cc:400] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15.
I20260709 13:04:54.347568  1612 leader_election.cc:403] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:54.335952  8064 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:54.323791  8147 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:54.364454  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3109) took 1170 ms. Trace:
I20260709 13:04:54.333109  7986 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:54.330660  7969 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:54.361335  8152 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.332712  7972 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:54.363235  1714 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:54.379820  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2802) took 1727 ms. Trace:
I20260709 13:04:54.382731  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3110) took 1164 ms. Trace:
I20260709 13:04:54.384718  1714 rpcz_store.cc:276] 0709 13:04:53.218376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.381502 (+1163126us) service_pool.cc:224] Handling call
0709 13:04:54.381997 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.382315 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.382351 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.382397 (+    46us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:54.382436 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.382496 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.382656 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.365154  1719 rpcz_store.cc:276] 0709 13:04:53.196405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.340836 (+1144431us) service_pool.cc:224] Handling call
0709 13:04:54.363491 (+ 22655us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.363723 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.363763 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.364059 (+   296us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:54.364109 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.364192 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.364370 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.373445  7706 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:54.374861  7967 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:54.320307  8142 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:54.378180  7955 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:54.389261  1719 raft_consensus.cc:1275] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:04:54.378582  8004 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:54.383635  1567 rpcz_store.cc:276] 0709 13:04:52.652817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.722687 (+ 69870us) service_pool.cc:224] Handling call
0709 13:04:52.723065 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:52.787227 (+ 64162us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:52.787276 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:52.837029 (+ 49753us) log.cc:844] Serialized 26 byte log entry
0709 13:04:53.505821 (+668792us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:53.505901 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.506014 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:54.379573 (+873559us) raft_consensus.cc:1685] finished
0709 13:04:54.379599 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.379752 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63524,"raft.queue_time_us":63651,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":11105,"thread_start_us":524592,"threads_started":1,"wal-append.queue_time_us":863036}
I20260709 13:04:54.356005  7966 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:54.405821  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 87589us
I20260709 13:04:54.405880  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3116) took 1147 ms. Trace:
I20260709 13:04:54.368085  8026 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:54.409348  1719 rpcz_store.cc:276] 0709 13:04:53.258635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.388463 (+1129828us) service_pool.cc:224] Handling call
0709 13:04:54.388853 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.405445 (+ 16592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.405792 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.411970  1685 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:54.382669  8066 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:54.402863  8083 consensus_meta.cc:287] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.999s	user 0.016s	sys 0.000s
I20260709 13:04:54.411979  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3117) took 1151 ms. Trace:
I20260709 13:04:54.408099  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:54.415580  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 68559.5us
I20260709 13:04:54.337174  8144 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:54.392643  8088 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:54.419416  1719 rpcz_store.cc:276] 0709 13:04:53.261016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.410501 (+1149485us) service_pool.cc:224] Handling call
0709 13:04:54.410921 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.411293 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.411336 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.411401 (+    65us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:54.411568 (+   167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.411668 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.411882 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.426321  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3121) took 1057 ms. Trace:
I20260709 13:04:54.426738  1719 rpcz_store.cc:276] 0709 13:04:53.369255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.422164 (+1052909us) service_pool.cc:224] Handling call
0709 13:04:54.425827 (+  3663us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.426013 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.426036 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.426074 (+    38us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:54.426109 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.426156 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.426265 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.427377  1572 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:04:54.419847  1567 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:04:54.418960  8026 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:54.435150  8148 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:54.421490  8096 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:54.436604  8088 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:04:54.435892  7990 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.616s	user 0.014s	sys 0.000s
I20260709 13:04:54.465301  1685 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:54.465741  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 45390.9us
I20260709 13:04:54.417790  8153 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15.
I20260709 13:04:54.393926  8152 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:54.438165  7962 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:54.475651  8152 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:54.470230  8148 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:54.484196  8144 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:54.489936  7990 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:54.491573  7873 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:54.463737  8098 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:54.495373  8026 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:54.498184  8138 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:54.498824  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.494804  1685 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:54.521580  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:54.522205  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:54.522481  1645 raft_consensus.cc:2376] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18. Re-sending same reply.
I20260709 13:04:54.522886  1644 raft_consensus.cc:2376] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16. Re-sending same reply.
I20260709 13:04:54.523598  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.524874  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.472512  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.526459  1645 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:54.526695  1639 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:04:54.527427  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.528071  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.472720  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.523641  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.529313  1640 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:54.472240  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:54.530474  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.530949  1567 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 18: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:54.531446  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:54.531494  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.533787  1639 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:04:54.470073  8088 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:54.502082  8034 consensus_meta.cc:287] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.686s	user 0.012s	sys 0.003s
I20260709 13:04:54.536029  1645 raft_consensus.cc:1275] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:04:54.539077  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 44327.9us
I20260709 13:04:54.552500  1685 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:54.553200  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.537568  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:54.554050  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:54.542150  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 180648us
I20260709 13:04:54.543889  7990 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:54.544035  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.544838  1568 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: []
W20260709 13:04:54.545921  1721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2693) took 2553 ms (client timeout 1976 ms). Trace:
I20260709 13:04:54.549630  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:54.555799  1721 rpcz_store.cc:269] 0709 13:04:51.992543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.992670 (+   127us) service_pool.cc:224] Handling call
0709 13:04:54.545627 (+2552957us) spinlock_profiling.cc:243] Waited 67.1 ms on lock 0x7b54002990d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:54.545809 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154331392}
I20260709 13:04:54.556368  1643 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:54.556820  1567 raft_consensus.cc:2393] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 14: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:54.547078  8016 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:54.554984  1648 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:54.544866  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:54.558599  1569 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:54.495257  7962 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:54.549007  8162 consensus_queue.cc:1059] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.090s
I20260709 13:04:54.562177  8146 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:54.569024  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2734) took 3324 ms. Trace:
I20260709 13:04:54.562244  1685 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:54.569540  1646 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 18: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:54.570050  1570 rpcz_store.cc:276] 0709 13:04:51.227986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.634805 (+406819us) service_pool.cc:224] Handling call
0709 13:04:51.635213 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:51.788991 (+153778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.552269 (+2763278us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54001b4754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:54.552455 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":351106432}
W20260709 13:04:54.571017  2314 connection.cc:256] RPC call timeout handler was delayed by -2.350783272000001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19330000000000003s for a response
I20260709 13:04:54.571516  1685 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:54.569083  1611 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:54.530555  8029 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:54.536196  8153 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.573320  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:54.540644  8144 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:54.550192  7873 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:54.578874  1611 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:54.550863  8088 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:54.562775  8004 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:04:54.462692  8160 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.150s
I20260709 13:04:54.579572  7969 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:54.585662  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3100) took 2537 ms. Trace:
I20260709 13:04:54.592152  1717 rpcz_store.cc:276] 0709 13:04:52.048557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.545159 (+496602us) service_pool.cc:224] Handling call
0709 13:04:52.548076 (+  2917us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:52.582043 (+ 33967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.585307 (+2003264us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b5400211e54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:54.585556 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4465536}
I20260709 13:04:54.594671  1568 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:54.591039  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:54.588850  1611 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:54.596540  1646 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:54.604085  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.610519  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.610971  1648 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:04:54.611308  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:54.612488  1644 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:54.613006  1612 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:54.609441  8162 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:04:54.586297  7873 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.617339  7873 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:54.617660  8142 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:54.622288  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2694) took 2627 ms (client timeout 2517 ms). Trace:
W20260709 13:04:54.622786  1720 rpcz_store.cc:269] 0709 13:04:51.994510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.995390 (+   880us) service_pool.cc:224] Handling call
0709 13:04:54.622072 (+2626682us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b5400298bd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:54.622207 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":248596352}
I20260709 13:04:54.581802  8147 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:04:54.624433  7642 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:54.624972  8152 raft_consensus.cc:3055] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:54.584450  8153 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:54.586862  8164 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:54.625953  8164 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.626374  8164 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:54.626507  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:54.627130  7861 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:54.627529  7861 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:54.627879  7861 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:54.629736  8029 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.630426  8029 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:54.631114  8153 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:54.632701  8029 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:54.633987  1639 raft_consensus.cc:1275] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:04:54.634874  8116 consensus_meta.cc:287] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.094s	user 0.014s	sys 0.004s
I20260709 13:04:54.588477  8148 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:54.584954  8159 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:54.589056  7969 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:54.590664  7962 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.625386  8152 raft_consensus.cc:740] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:04:54.637247  7929 consensus_meta.cc:287] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.989s	user 0.014s	sys 0.001s
I20260709 13:04:54.654968  8152 consensus_queue.cc:260] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:54.656282  8121 log.cc:927] Time spent T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 1.327s	user 0.013s	sys 0.019s
I20260709 13:04:54.656541  8152 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:54.635293  8116 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16.
I20260709 13:04:54.652307  8159 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.658452  8159 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:54.656436  7642 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:54.640949  8097 consensus_meta.cc:287] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.839s	user 0.006s	sys 0.008s
I20260709 13:04:54.586094  8144 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:54.663827  8144 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:54.665858  7969 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:54.667277  7969 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:54.669901  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:54.669597  7962 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:54.670876  1648 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:54.636425  8090 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:54.601404  8160 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:54.673853  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:54.674865  1573 raft_consensus.cc:2393] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 14: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:54.675896  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:54.676906  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.679368  7962 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:54.683099  1646 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:54.693499  1642 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:54.679433  8088 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:54.676509  1573 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:54.699929  8139 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.701102  8139 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:54.701195  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:54.695611  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:54.705906  1573 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:54.700731  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:54.708751  1644 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:04:54.699921  8137 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:54.697009  1640 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:54.704324  8153 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:04:54.740021  7962 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:04:54.743898  8139 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:54.742813  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:54.745626  8164 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:54.746304  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:54.747001  1573 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:54.747952  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:54.748528  8164 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:54.748696  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:54.749892  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 43881.5us
W20260709 13:04:54.740568  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 43574.8us
I20260709 13:04:54.748533  1573 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:54.730026  1648 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:54.732661  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:54.733423  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.734198  1643 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:04:54.736100  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:54.737041  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:54.755366  1644 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:54.754835  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:54.797295  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:54.809602  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 57539.1us
I20260709 13:04:54.809826  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3235) took 2290 ms. Trace:
I20260709 13:04:54.810801  1612 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:04:54.818385  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 130293us
I20260709 13:04:54.823246  1719 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:54.834649  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:54.835951  1573 raft_consensus.cc:2393] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:54.810607  1647 rpcz_store.cc:276] 0709 13:04:52.519785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.569962 (+ 50177us) service_pool.cc:224] Handling call
0709 13:04:52.570618 (+   656us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:52.570840 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:52.996239 (+425399us) log.cc:844] Serialized 14 byte log entry
0709 13:04:53.149131 (+152892us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.683535 (+1534404us) spinlock_profiling.cc:243] Waited 35.3 ms on lock 0x7b18002b36c0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287 00007f8248ea0208
0709 13:04:54.740578 (+ 57043us) spinlock_profiling.cc:243] Waited 33.3 ms on lock 0x7b18002b36c0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:04:54.741158 (+   580us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.809398 (+ 68240us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.809456 (+    58us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:54.809525 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.809592 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.809762 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79004,"prepare.queue_time_us":568333,"raft.queue_time_us":42613,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":12336,"spinlock_wait_cycles":157850240,"thread_start_us":613162,"threads_started":3,"wal-append.queue_time_us":253648}
I20260709 13:04:54.841296  1612 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:54.834224  7642 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:04:54.850298  1644 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:04:54.851480  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.851768  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:54.853605  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 281902us
I20260709 13:04:54.858206  8148 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.858820  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.859120  8148 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:54.860265  2314 connection.cc:256] RPC call timeout handler was delayed by -1.4245954750000465s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1829s for a response
I20260709 13:04:54.860447  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.743146  8137 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:54.860718  2314 connection.cc:256] RPC call timeout handler was delayed by -1.336216721000028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17490000000000003s for a response
I20260709 13:04:54.749508  8146 raft_consensus.cc:697] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:54.825935  7970 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:54.866417  7929 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:54.865008  1647 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:54.850426  8116 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:54.877887  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.894104  1646 raft_consensus.cc:2393] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 15: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:54.883265  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.903867  1647 raft_consensus.cc:2410] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 19 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:04:54.887096  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.905130  1641 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:54.908417  8016 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:04:54.921914  8090 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.923697  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:54.883162  8116 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:54.937628  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 364238us
I20260709 13:04:54.938203  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.938977  1685 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:54.875265  8088 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:54.941124  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:54.880831  7929 raft_consensus.cc:697] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 19 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:54.922039  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:54.884410  7970 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:54.929728  8148 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:54.922698  8090 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:54.945780  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:54.952521  1641 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 18: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:54.865010  8159 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:54.869566  7861 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:54.929221  7873 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:54.956133  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 136067us
I20260709 13:04:54.962136  1646 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:54.880738  8146 consensus_queue.cc:237] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:55.027410  1716 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.009s	sys 0.008s
I20260709 13:04:54.996537  1647 raft_consensus.cc:1275] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:04:54.990293  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.964905  1573 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 13.
I20260709 13:04:55.014461  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3111) took 1791 ms. Trace:
I20260709 13:04:55.065142  1714 rpcz_store.cc:276] 0709 13:04:53.222749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.386839 (+1164090us) service_pool.cc:224] Handling call
0709 13:04:54.387349 (+   510us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:54.429518 (+ 42169us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:54.443525 (+ 14007us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.941850 (+498325us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:54.942574 (+   724us) log.cc:844] Serialized 52 byte log entry
0709 13:04:54.942775 (+   201us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:54.943190 (+   415us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.943371 (+   181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.943466 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:55.014206 (+ 70740us) raft_consensus.cc:1685] finished
0709 13:04:55.014227 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.014391 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61508,"prepare.queue_time_us":30071,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":31035,"raft.queue_time_us":43430,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":8935,"spinlock_wait_cycles":6002432,"thread_start_us":344499,"threads_started":1,"wal-append.queue_time_us":459292}
I20260709 13:04:55.067564  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:54.942262  8116 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:54.968751  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b82d3b49001741eda872554a7ecb2db3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:04:55.069372  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:55.077941  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:55.075028  7962 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.078836  2347 raft_consensus.cc:2410] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:04:55.078948  7962 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:55.081868  1456 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 113151us
I20260709 13:04:55.083896  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:55.085832  1717 raft_consensus.cc:1240] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:55.086287  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2709) took 2984 ms. Trace:
I20260709 13:04:55.088289  8110 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.088564  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:55.088970  8110 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:55.092643  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2810) took 1459 ms. Trace:
I20260709 13:04:55.092782  8110 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:54.960203  7929 consensus_queue.cc:237] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:55.095243  1713 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2697) took 3046 ms (client timeout 2071 ms). Trace:
W20260709 13:04:55.095672  1713 rpcz_store.cc:269] 0709 13:04:52.048525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.048658 (+   133us) service_pool.cc:224] Handling call
0709 13:04:55.095013 (+3046355us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b54002995d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:55.095167 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":928945536}
I20260709 13:04:55.096154  7990 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:55.096841  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:55.096736  7990 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:55.098206  1721 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:04:55.069025  8161 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.097599  1713 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:04:55.071810  8174 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:55.100075  8161 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:55.100678  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:55.100502  8174 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:55.100863  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:55.102173  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:55.075734  7981 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.102895  1713 raft_consensus.cc:2393] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:55.089489  1573 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:55.100032  7990 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:55.103293  7981 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:55.104647  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:55.093233  1571 rpcz_store.cc:276] 0709 13:04:53.633427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.633564 (+   137us) service_pool.cc:224] Handling call
0709 13:04:53.633995 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.634444 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:53.732072 (+ 97628us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.589743 (+857671us) spinlock_profiling.cc:243] Waited 3.26 ms on lock 0x7b1800048460. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a2523a 00007f8248e8fa2a
0709 13:04:54.741764 (+152021us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.871088 (+129324us) log.cc:844] Serialized 14 byte log entry
0709 13:04:55.090954 (+219866us) log.cc:844] Serialized 14 byte log entry
0709 13:04:55.092143 (+  1189us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:55.092196 (+    53us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:55.092259 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.092446 (+   187us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.092573 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":234058,"prepare.queue_time_us":228327,"prepare.run_cpu_time_us":178,"prepare.run_wall_time_us":182,"raft.queue_time_us":5582,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":99,"spinlock_wait_cycles":7503616,"thread_start_us":151558,"threads_started":1,"wal-append.queue_time_us":1005415}
I20260709 13:04:55.105947  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:55.010181  8148 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.977396  8088 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:55.108958  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:55.112192  1572 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:55.113175  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:55.020658  8160 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:55.029742  8152 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.068792  8137 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:55.110800  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:55.118933  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:55.120030  1538 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.816s (SENT)
I20260709 13:04:55.120617  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:55.086695  1716 rpcz_store.cc:276] 0709 13:04:52.102435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.583507 (+481072us) service_pool.cc:224] Handling call
0709 13:04:52.584057 (+   550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:52.584350 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:52.783195 (+198845us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.174101 (+1390906us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b1800352c00. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a2523a
0709 13:04:54.297990 (+123889us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.502155 (+204165us) log.cc:844] Serialized 14 byte log entry
0709 13:04:55.059969 (+557814us) log.cc:844] Serialized 14 byte log entry
0709 13:04:55.069199 (+  9230us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:55.069263 (+    64us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:55.069709 (+   446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.086081 (+ 16372us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.086226 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8890,"prepare.queue_time_us":601379,"prepare.run_cpu_time_us":207,"prepare.run_wall_time_us":3762,"raft.queue_time_us":172963,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":16413,"spinlock_wait_cycles":31537408,"thread_start_us":1301132,"threads_started":7,"wal-append.queue_time_us":518684,"wal-append.run_cpu_time_us":1532,"wal-append.run_wall_time_us":1001515}
I20260709 13:04:55.122574  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:55.122694  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.123386  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:55.123703  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.124382  1639 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:55.125876  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:55.126430  1646 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:55.126907  1538 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:55.127737  1713 raft_consensus.cc:2393] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:55.123713  1642 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:04:55.130332  1641 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:55.013173  7969 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:55.108392  8139 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.137414  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:55.146486  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:55.146961  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:55.139662  8029 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:55.141544  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:55.142248  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:55.145378  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:55.155164  1571 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:04:55.147055  1572 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:55.153196  1713 raft_consensus.cc:2368] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 18. Current term is 19.
I20260709 13:04:55.155390  1642 raft_consensus.cc:2368] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 18. Current term is 19.
I20260709 13:04:55.148010  1639 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:04:55.160552  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.161243  1639 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:55.162174  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.162835  1639 raft_consensus.cc:2368] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 18. Current term is 19.
I20260709 13:04:55.157821  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.165213  2313 leader_election.cc:400] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 18. Current term is 19.
I20260709 13:04:55.165700  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.166416  1639 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 22.
I20260709 13:04:55.165907  2313 leader_election.cc:403] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:04:55.167572  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:55.168183  1639 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:55.165617  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:55.170030  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.172252  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:55.175239  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:55.145793  8160 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.156626  8139 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:55.188187  8160 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:55.190676  8139 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:55.191233  8160 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:55.193580  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:55.194383  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:55.195247  1572 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:55.194672  2314 leader_election.cc:400] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 18. Current term is 19.
I20260709 13:04:55.194550  1573 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:04:55.120950  8152 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:55.115456  8148 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:55.196970  2314 leader_election.cc:400] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 pre-election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 18. Current term is 19.
W20260709 13:04:55.246367  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 49508.6us
I20260709 13:04:55.247072  2314 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:55.247453  8148 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:55.185560  1646 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:04:55.248924  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.249544  1646 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:55.250243  8031 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:04:55.251192  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.251277  8031 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:55.251960  8031 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:55.198055  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.187136  1612 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:55.212769  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:55.254228  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 67238.2us
I20260709 13:04:55.213953  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:55.219503  1641 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:04:55.255125  1572 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:04:55.231357  1571 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:04:55.256922  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:55.233934  8090 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:55.238865  8159 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:55.239975  7873 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:55.243160  7969 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:55.245431  8152 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:55.250402  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.250851  1714 raft_consensus.cc:2393] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 14: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:55.198530  1647 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:55.157023  7962 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:55.227285  1639 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:04:55.274844  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.275525  1639 raft_consensus.cc:2376] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18. Re-sending same reply.
I20260709 13:04:55.276552  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.277217  1639 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:55.259871  1571 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 22.
I20260709 13:04:55.260149  1612 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:55.164462  8029 raft_consensus.cc:697] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:55.284528  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:55.285461  1571 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 23.
I20260709 13:04:55.285887  8029 consensus_queue.cc:237] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:55.286901  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:55.289239  1571 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:55.290146  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2748) took 1175 ms. Trace:
I20260709 13:04:55.279738  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.281829  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:55.297016  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.256608  8143 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:55.283485  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:55.290917  1571 rpcz_store.cc:276] 0709 13:04:54.114504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.288662 (+1174158us) service_pool.cc:224] Handling call
0709 13:04:55.289032 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:55.289877 (+   845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.290088 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:55.297574  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.264326  8160 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:55.299219  1639 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:55.304525  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:55.306396  1713 raft_consensus.cc:2410] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 19 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:04:55.199553  8016 consensus_queue.cc:1059] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.084s
I20260709 13:04:55.301504  1572 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
W20260709 13:04:55.301759  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.311569  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:55.313519  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.314343  1639 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16.
I20260709 13:04:55.315558  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.317657  1639 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:04:55.318660  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.319782  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.319264  1639 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 9.
I20260709 13:04:55.320463  1642 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:55.321270  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.321339  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:55.322119  1639 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:04:55.323285  8137 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:55.323786  8137 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.324023  7861 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:55.324281  8137 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:55.326774  2314 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:55.323675  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:55.329268  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.329777  8017 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:55.330075  1713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0df07602233941598bee7b19607b2e3b"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:55.330294  8017 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:55.330526  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.330909  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2754) took 1202 ms. Trace:
I20260709 13:04:55.331725  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:55.333009  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.330752  8017 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:55.302942  8152 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:55.338934  2314 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:55.293241  8116 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:55.331342  1572 rpcz_store.cc:276] 0709 13:04:54.128804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.317229 (+1188425us) service_pool.cc:224] Handling call
0709 13:04:55.330429 (+ 13200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:55.330600 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:55.330624 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:55.330660 (+    36us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:55.330692 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.330743 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.330854 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:55.330696  1713 raft_consensus.cc:606] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:55.302421  8160 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.339836  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.354430  1639 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:55.356093  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.357399  1645 raft_consensus.cc:2393] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:55.341445  1713 raft_consensus.cc:614] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:55.359653  1572 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:04:55.359431  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.360750  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 1080) took 1010 ms. Trace:
I20260709 13:04:55.361084  1639 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:55.361275  1572 rpcz_store.cc:276] 0709 13:04:54.350449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.358817 (+1008368us) service_pool.cc:224] Handling call
0709 13:04:55.359325 (+   508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:55.360381 (+  1056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.360671 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:55.367985  8116 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:55.340821  8016 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:55.370299  8161 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:55.374162  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.375530  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.376260  1642 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:55.377010  8031 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:55.379592  1715 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:04:55.381155  1642 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:04:55.382560  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:55.383332  1642 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:55.383621  1571 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:04:55.339751  8174 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:55.385277  1537 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:55.362911  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.386631  1639 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:55.387578  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:55.388381  1639 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:04:55.389354  1537 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:04:55.395033  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2751) took 1279 ms. Trace:
I20260709 13:04:55.395702  1571 rpcz_store.cc:276] 0709 13:04:54.115400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.316498 (+1201098us) service_pool.cc:224] Handling call
0709 13:04:55.340020 (+ 23522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:55.384238 (+ 44218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.394930 (+ 10692us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:55.358839  8160 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:55.391269  1537 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:04:55.433959  2312 leader_election.cc:336] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.829s (SENT)
I20260709 13:04:55.434736  2312 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:55.408785  8139 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:55.404285  8116 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.409030  8097 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:55.429539  8136 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:55.407933  8016 consensus_queue.cc:1059] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
W20260709 13:04:55.446601  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 319715us
I20260709 13:04:55.448939  1538 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:55.454531  1687 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:55.469466  1715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:55.446491  8116 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:55.470515  1715 raft_consensus.cc:606] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:55.470901  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46982 (request call id 1079) took 1360 ms. Trace:
I20260709 13:04:55.471554  1564 rpcz_store.cc:276] 0709 13:04:54.110482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.110660 (+   178us) service_pool.cc:224] Handling call
0709 13:04:55.470682 (+1360022us) spinlock_profiling.cc:243] Waited 51.4 ms on lock 0x7b5400305f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:55.470832 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118331136}
I20260709 13:04:55.472506  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:55.474521  1568 raft_consensus.cc:1240] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:55.476377  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2746) took 1362 ms. Trace:
I20260709 13:04:55.441093  8139 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.476775  1568 rpcz_store.cc:276] 0709 13:04:54.113933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.596288 (+482355us) service_pool.cc:224] Handling call
0709 13:04:55.474316 (+878028us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:55.475061 (+   745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.476134 (+  1073us) spinlock_profiling.cc:243] Waited 827 ms on lock 0x7b5400305f50. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:55.476306 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1902612864}
I20260709 13:04:55.477403  8139 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:55.477923  8137 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:55.480692  8137 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:55.444440  8031 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:55.471010  1715 raft_consensus.cc:614] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:55.499042  1687 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:55.503829  1687 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:55.481419  8137 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:55.412708  8017 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:55.513170  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 384734us
I20260709 13:04:55.515906  1716 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:04:55.522193  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:55.523066  1643 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:55.485486  7861 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:55.472563  8116 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:55.535835  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2716) took 2921 ms. Trace:
I20260709 13:04:55.536401  1718 rpcz_store.cc:276] 0709 13:04:52.614753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.142886 (+1528133us) service_pool.cc:224] Handling call
0709 13:04:54.143204 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:54.241740 (+ 98536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.535532 (+1293792us) spinlock_profiling.cc:243] Waited 92.6 ms on lock 0x7b5400248454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:55.535769 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":212934400}
I20260709 13:04:55.550832  1720 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:55.534711  1687 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:55.513947  8174 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:55.513065  7978 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:55.485140  7968 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:04:55.564965  1687 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:55.638387  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:55.659133  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:55.727619  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 162691us
I20260709 13:04:55.527386  7962 consensus_queue.cc:1048] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:04:55.526260  7873 consensus_queue.cc:1048] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:04:55.770864  1643 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17.
I20260709 13:04:55.730432  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:55.772506  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.561172  7861 raft_consensus.cc:697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:55.554445  8159 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.186s
W20260709 13:04:55.818862  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 90272.2us
I20260709 13:04:55.820577  1687 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:55.735340  7948 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.400s	user 0.092s	sys 0.154s
I20260709 13:04:55.727977  7978 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:55.768399  8148 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:55.831148  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:55.847033  1641 raft_consensus.cc:2376] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15. Re-sending same reply.
W20260709 13:04:55.835842  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 407212us
W20260709 13:04:55.788471  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 354438us
I20260709 13:04:55.848368  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:55.849303  1568 raft_consensus.cc:2376] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18. Re-sending same reply.
W20260709 13:04:55.843863  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:55.846285  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 338006us
I20260709 13:04:55.774729  8185 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:04:55.827455  7948 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.493s	user 0.092s	sys 0.155s
I20260709 13:04:55.875020  1568 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:04:55.876116  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2765) took 1049 ms. Trace:
I20260709 13:04:55.877046  1568 rpcz_store.cc:276] 0709 13:04:54.826804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.850568 (+1023764us) service_pool.cc:224] Handling call
0709 13:04:55.874460 (+ 23892us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:55.875748 (+  1288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.876046 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:55.845108  7978 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.819164  8137 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:04:55.845980  8148 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:55.884801  1642 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.002s
I20260709 13:04:55.919281  1642 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 23.
I20260709 13:04:55.855928  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:55.905841  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 464507us
W20260709 13:04:55.913261  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 464289us
I20260709 13:04:55.829051  8090 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:04:55.923059  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2794) took 3832 ms. Trace:
I20260709 13:04:55.924124  1565 rpcz_store.cc:276] 0709 13:04:52.090644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.090813 (+   169us) service_pool.cc:224] Handling call
0709 13:04:53.391578 (+1300765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.392044 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:53.392084 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:53.392141 (+    57us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:53.392196 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.392273 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.922824 (+2530551us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400014850. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:55.922980 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":799,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":17083,"spinlock_wait_cycles":2989432576}
I20260709 13:04:55.930160  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:55.930876  1565 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:55.931977  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:55.932797  1565 raft_consensus.cc:2410] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 17 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 6.
I20260709 13:04:55.933521  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:55.934063  1565 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:55.893488  7978 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:55.943218  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2726) took 1086 ms. Trace:
I20260709 13:04:55.922235  8148 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:55.945066  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:55.941658  8143 consensus_meta.cc:287] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.638s	user 0.013s	sys 0.000s
I20260709 13:04:55.985788  2312 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:55.966570  1714 rpcz_store.cc:276] 0709 13:04:54.857199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.067286 (+210087us) service_pool.cc:224] Handling call
0709 13:04:55.942902 (+875616us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54002c60d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:55.943125 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":419142400}
I20260709 13:04:55.973627  1567 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:55.922173  7861 consensus_queue.cc:237] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:55.978334  8145 raft_consensus.cc:3055] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:55.927680  8017 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:55.936129  7962 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:55.986129  1565 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:04:56.008620  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:56.052127  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:56.053287  1565 raft_consensus.cc:2410] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:04:56.064489  1567 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:04:55.991022  8143 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 18. Current term is 19.
I20260709 13:04:55.991746  8145 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:55.995450  8189 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:04:56.079391  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 229988us
I20260709 13:04:56.074723  8176 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:56.052836  7962 raft_consensus.cc:697] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:56.082180  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2769) took 1205 ms. Trace:
I20260709 13:04:56.104130  1565 rpcz_store.cc:276] 0709 13:04:54.876907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.081245 (+1204338us) service_pool.cc:224] Handling call
0709 13:04:56.081589 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:56.081827 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:56.081854 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:56.081897 (+    43us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:56.081938 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.082001 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.082115 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:56.117565  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 1085) took 1188 ms. Trace:
I20260709 13:04:56.124691  1565 rpcz_store.cc:276] 0709 13:04:54.929346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.116542 (+1187196us) service_pool.cc:224] Handling call
0709 13:04:56.116936 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:56.117138 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:56.117164 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:56.117225 (+    61us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:04:56.117266 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.117320 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.117485 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:56.055437  8029 consensus_queue.cc:1048] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:04:56.079826  8189 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:56.130975  8194 log.cc:927] Time spent T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:56.133433  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 674) took 1228 ms. Trace:
W20260709 13:04:56.135540  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.112890  8037 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.124027  8090 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:04:56.141192  8187 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.169s	user 0.001s	sys 0.001s
I20260709 13:04:56.137550  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:56.143038  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2777) took 1403 ms. Trace:
W20260709 13:04:56.137676  1566 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2749) took 2022 ms (client timeout 2043 ms). Trace:
W20260709 13:04:56.154973  1566 rpcz_store.cc:269] 0709 13:04:54.114878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.166010 (+ 51132us) service_pool.cc:224] Handling call
0709 13:04:56.137481 (+1971471us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b54004060d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:56.137590 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5417728}
I20260709 13:04:56.137270  1713 rpcz_store.cc:276] 0709 13:04:54.905273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.377874 (+472601us) service_pool.cc:224] Handling call
0709 13:04:55.380615 (+  2741us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:55.381124 (+   509us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:55.381158 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:55.403986 (+ 22828us) log.cc:844] Serialized 52 byte log entry
0709 13:04:55.858125 (+454139us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:55.858193 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.858297 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:56.133176 (+274879us) raft_consensus.cc:1685] finished
0709 13:04:56.133200 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.133363 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":155,"raft.queue_time_us":987,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":18971,"spinlock_wait_cycles":1024896,"thread_start_us":445254,"threads_started":1,"wal-append.queue_time_us":669345}
W20260709 13:04:56.147830  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 223508us
I20260709 13:04:56.120647  8176 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:56.149391  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 1082) took 1755 ms. Trace:
I20260709 13:04:56.158459  1565 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:56.135607  7948 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:04:56.147439  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 326847us
I20260709 13:04:56.162204  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:56.162631  1565 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:56.163465  1566 raft_consensus.cc:2410] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 17 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 6.
I20260709 13:04:56.163486  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 1086) took 1215 ms. Trace:
I20260709 13:04:56.166278  1565 rpcz_store.cc:276] 0709 13:04:54.947847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.159511 (+1211664us) service_pool.cc:224] Handling call
0709 13:04:56.162308 (+  2797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:56.163202 (+   894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.163416 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1920}
I20260709 13:04:56.167668  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:56.154563  1644 rpcz_store.cc:276] 0709 13:04:54.733613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.851214 (+117601us) service_pool.cc:224] Handling call
0709 13:04:56.136879 (+1285665us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b5400182c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:56.137075 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1507827584}
I20260709 13:04:56.128248  8195 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:04:56.140045  8189 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:56.145988  8017 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:56.162546  1687 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:56.134570  8145 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:56.158792  1572 rpcz_store.cc:276] 0709 13:04:54.393768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.385720 (+991952us) service_pool.cc:224] Handling call
0709 13:04:55.386078 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:55.386296 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:55.428558 (+ 42262us) log.cc:844] Serialized 14 byte log entry
0709 13:04:56.148329 (+719771us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:56.148415 (+    86us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:56.148516 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.149135 (+   619us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.149314 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":21017,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":1860,"raft.queue_time_us":178574,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":16808,"thread_start_us":717406,"threads_started":2,"wal-append.queue_time_us":698931}
I20260709 13:04:56.151305  7990 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:56.153041  8196 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:56.155208  8202 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:04:56.172499  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.144865  8037 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:56.157119  1538 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:56.174983  1687 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:56.160421  8185 raft_consensus.cc:697] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:56.147486  8090 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:56.147533  8193 consensus_queue.cc:1048] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:04:56.187122  1687 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:56.193946  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 1087) took 1196 ms. Trace:
I20260709 13:04:56.195403  1566 rpcz_store.cc:276] 0709 13:04:54.997230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.168224 (+1170994us) service_pool.cc:224] Handling call
0709 13:04:56.193315 (+ 25091us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:56.193521 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:56.193555 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:56.193600 (+    45us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:56.193667 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.193739 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.193880 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:56.184823  8137 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:56.197985  1538 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:56.206212  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 192678us
I20260709 13:04:56.210306  1538 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:56.189386  8017 raft_consensus.cc:697] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:56.205935  8137 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:56.215066  8090 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:56.215445  8190 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:04:56.216334  8037 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:56.189874  8191 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.025s
W20260709 13:04:56.244813  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 57793.3us
W20260709 13:04:56.217926  8197 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 13:04:56.219450  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.219376  7978 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:56.220417  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2721) took 2747 ms. Trace:
I20260709 13:04:56.221429  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2740) took 1051 ms. Trace:
I20260709 13:04:56.222326  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:56.169275  7948 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:56.224573  1538 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:56.190572  8202 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:56.216663  8198 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:04:56.253286  1715 rpcz_store.cc:276] 0709 13:04:55.169875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.512413 (+342538us) service_pool.cc:224] Handling call
0709 13:04:55.535048 (+ 22635us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:55.563161 (+ 28113us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:55.563206 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:55.563941 (+   735us) log.cc:844] Serialized 52 byte log entry
0709 13:04:56.054387 (+490446us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:56.054439 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.054523 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:56.221126 (+166603us) raft_consensus.cc:1685] finished
0709 13:04:56.221158 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.221358 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114127,"raft.queue_time_us":28811,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":159914,"thread_start_us":479177,"threads_started":1,"wal-append.queue_time_us":576310}
I20260709 13:04:56.254350  1719 rpcz_store.cc:276] 0709 13:04:53.473439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.427666 (+954227us) service_pool.cc:224] Handling call
0709 13:04:54.427941 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.823862 (+395921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.220142 (+1396280us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b5400246654. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:56.220349 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":908294400}
I20260709 13:04:56.246317  1687 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:56.252170  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:56.260419  1644 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:56.172219  7962 consensus_queue.cc:237] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:56.253880  1566 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
W20260709 13:04:56.269253  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 44774.1us
I20260709 13:04:56.248157  7926 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:56.248241  8090 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:56.240758  8203 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:56.292963  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:56.253306  8209 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:56.255385  8206 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:56.305364  1644 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:04:56.313517  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:56.313908  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:56.313091  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2813) took 1246 ms. Trace:
I20260709 13:04:56.300697  8037 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:56.300211  8195 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:56.317842  1572 rpcz_store.cc:276] 0709 13:04:55.066719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.180356 (+1113637us) service_pool.cc:224] Handling call
0709 13:04:56.180817 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:56.181096 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:56.182234 (+  1138us) log.cc:844] Serialized 14 byte log entry
0709 13:04:56.312661 (+130427us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:56.312728 (+    67us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:56.312786 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.312856 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.313027 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":753,"prepare.queue_time_us":730,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"thread_start_us":129193,"threads_started":1}
I20260709 13:04:56.320876  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:56.318605  1566 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:04:56.269259  8205 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:56.299954  7926 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.309023  8090 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.330969  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:56.312600  7861 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:56.311254  8203 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.339160  1719 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:56.341483  1719 raft_consensus.cc:606] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 22 LEADER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:56.338393  7926 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:56.342504  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:56.317299  8209 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.343811  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:56.346617  1572 raft_consensus.cc:2393] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:04:56.345012  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 489059us
I20260709 13:04:56.318765  8206 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:56.349256  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 53024.3us
I20260709 13:04:56.347463  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:56.329201  8037 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.350889  1566 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:56.259187  8185 consensus_queue.cc:237] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:56.353948  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3262) took 1829 ms. Trace:
I20260709 13:04:56.351272  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:56.353703  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:56.357203  1640 rpcz_store.cc:276] 0709 13:04:54.524014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.537533 (+ 13519us) service_pool.cc:224] Handling call
0709 13:04:54.538271 (+   738us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.737729 (+199458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.353676 (+1615947us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b540007ec54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:56.353879 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":363837696}
I20260709 13:04:56.358184  1566 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:56.354477  2314 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:56.351529  8206 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:56.340874  8090 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:56.344378  8203 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:56.314293  8215 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:04:56.315945  8017 consensus_queue.cc:237] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:56.344388  7861 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:56.406345  8176 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:56.376019  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.366570  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 1083) took 1871 ms. Trace:
I20260709 13:04:56.436622  1571 rpcz_store.cc:276] 0709 13:04:54.494839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.413043 (+918204us) service_pool.cc:224] Handling call
0709 13:04:55.413542 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:55.415820 (+  2278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:55.415865 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:55.498211 (+ 82346us) log.cc:844] Serialized 26 byte log entry
0709 13:04:55.935617 (+437406us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:55.935670 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.935759 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:56.366328 (+430569us) raft_consensus.cc:1685] finished
0709 13:04:56.366350 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.366499 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":400795,"threads_started":1,"wal-append.queue_time_us":605676}
I20260709 13:04:56.348985  8209 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:56.354187  8037 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:56.389456  8160 consensus_meta.cc:287] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.973s	user 0.015s	sys 0.000s
W20260709 13:04:56.410790  8181 log.cc:927] Time spent T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 1.026s	user 0.003s	sys 0.000s
I20260709 13:04:56.448911  1713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
W20260709 13:04:56.436318  8139 consensus_meta.cc:287] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.958s	user 0.011s	sys 0.004s
I20260709 13:04:56.460340  8139 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:56.412683  8218 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:04:56.364255  2314 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:56.457985  8202 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:56.458904  7978 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:56.366380  8211 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:56.496555  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:04:56.498834  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 134677us
I20260709 13:04:56.506259  2341 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:56.466085  8227 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.528571  8102 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:56.468474  8160 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:56.538064  2342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 848) took 1784 ms. Trace:
W20260709 13:04:56.505345  8174 consensus_meta.cc:287] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.944s	user 0.016s	sys 0.000s
I20260709 13:04:56.510423  8210 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:56.547904  8210 raft_consensus.cc:697] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:56.549093  8210 consensus_queue.cc:237] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:56.550493  8137 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:56.542599  8160 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:56.553675  7990 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:56.554515  7990 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:56.544559  8227 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:56.453788  8195 raft_consensus.cc:697] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:56.546820  8174 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:56.547444  8139 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:56.588085  8148 consensus_meta.cc:287] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.603s	user 0.013s	sys 0.001s
I20260709 13:04:56.591224  8196 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:56.591632  8209 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:56.594486  8195 consensus_queue.cc:237] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:56.594825  8203 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:56.596848  8145 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:56.539113  2342 rpcz_store.cc:276] 0709 13:04:54.753747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.753891 (+   144us) service_pool.cc:224] Handling call
0709 13:04:54.837998 (+ 84107us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:54.848636 (+ 10638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.848683 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:54.903610 (+ 54927us) log.cc:844] Serialized 26 byte log entry
0709 13:04:54.998443 (+ 94833us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:54.998503 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.998609 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:56.409965 (+1411356us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:56.537510 (+127545us) raft_consensus.cc:1685] finished
0709 13:04:56.537563 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.537954 (+   391us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":157,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":10973,"spinlock_wait_cycles":9769344,"thread_start_us":50748,"threads_started":1,"wal-append.queue_time_us":397585}
W20260709 13:04:56.659603  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 297088us
I20260709 13:04:56.731005  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:56.667222  1565 raft_consensus.cc:2393] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 19: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:56.450763  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2767) took 1574 ms. Trace:
I20260709 13:04:56.668812  1566 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 16. Current term is 17. Ops: []
W20260709 13:04:56.669430  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:56.675410  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 175723us
I20260709 13:04:56.678308  1713 raft_consensus.cc:606] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:56.687191  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:56.687822  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:56.597371  8148 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:56.738015  8203 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:56.739719  8148 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:56.603245  8182 log.cc:927] Time spent T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 1.137s	user 0.005s	sys 0.006s
I20260709 13:04:56.700325  2345 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:56.610087  8189 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:56.606267  8209 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:56.597409  8211 raft_consensus.cc:697] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:56.780699  8139 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.783584  8139 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:56.742940  7948 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:56.744937  8206 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:56.748592  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:56.749111  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:56.736008  8185 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:56.749977  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:56.750993  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:56.752386  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:56.753085  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:56.758490  1713 raft_consensus.cc:614] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:56.750124  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:56.774462  1538 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:56.775326  8097 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.246s	user 0.007s	sys 0.006s
I20260709 13:04:56.785709  8211 consensus_queue.cc:237] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:56.616041  8227 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:56.820245  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:56.820741  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:56.822901  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2722) took 3347 ms. Trace:
I20260709 13:04:56.800482  8185 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:56.824365  8185 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:56.825347  1569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2807) took 3240 ms (client timeout 1601 ms). Trace:
W20260709 13:04:56.825932  1569 rpcz_store.cc:269] 0709 13:04:53.585106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.586071 (+   965us) service_pool.cc:224] Handling call
0709 13:04:56.824817 (+3238746us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b54002b8754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:56.825071 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1995055488}
I20260709 13:04:56.828178  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:56.790771  8218 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.066s	user 0.000s	sys 0.003s
I20260709 13:04:56.829152  1569 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:56.831331  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2764) took 2205 ms. Trace:
I20260709 13:04:56.832907  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 1088) took 1751 ms. Trace:
I20260709 13:04:56.817319  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2723) took 3334 ms. Trace:
I20260709 13:04:56.834209  1717 rpcz_store.cc:276] 0709 13:04:53.482959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.593140 (+1110181us) service_pool.cc:224] Handling call
0709 13:04:54.593432 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:55.104394 (+510962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.816396 (+1712002us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b540002e9d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:56.817021 (+   625us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1131561856}
W20260709 13:04:56.803283  8097 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.333s	user 0.007s	sys 0.007s
I20260709 13:04:56.835806  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:56.836217  8097 raft_consensus.cc:697] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:56.836956  1717 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:04:56.837930  8097 consensus_queue.cc:237] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:56.735620  1567 rpcz_store.cc:276] 0709 13:04:54.876193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.066447 (+1190254us) service_pool.cc:224] Handling call
0709 13:04:56.066965 (+   518us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:56.076853 (+  9888us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:56.076903 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:56.079215 (+  2312us) log.cc:844] Serialized 26 byte log entry
0709 13:04:56.253934 (+174719us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:56.253988 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.254059 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:56.450440 (+196381us) raft_consensus.cc:1685] finished
0709 13:04:56.450472 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.450671 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11779,"raft.queue_time_us":95357,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":14806,"thread_start_us":129030,"threads_started":1,"wal-append.queue_time_us":239042}
I20260709 13:04:56.847164  8206 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:56.810672  1722 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:56.861227  8189 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:56.863978  2347 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:56.864836  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:56.811870  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:56.865808  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:56.866223  2344 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 12.
I20260709 13:04:56.866820  1567 raft_consensus.cc:1240] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
W20260709 13:04:56.848353  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.831840  1564 rpcz_store.cc:276] 0709 13:04:54.625362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.472670 (+847308us) service_pool.cc:224] Handling call
0709 13:04:55.473060 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:55.892867 (+419807us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:55.922443 (+ 29576us) log.cc:844] Serialized 14 byte log entry
0709 13:04:56.661798 (+739355us) log.cc:844] Serialized 14 byte log entry
0709 13:04:56.662886 (+  1088us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:56.664801 (+  1915us) log.cc:844] Serialized 26 byte log entry
0709 13:04:56.665043 (+   242us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:56.665132 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.665394 (+   262us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:56.830807 (+165413us) raft_consensus.cc:1685] finished
0709 13:04:56.830828 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.831050 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10771,"prepare.queue_time_us":299269,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":122,"raft.queue_time_us":645966,"raft.run_cpu_time_us":896,"raft.run_wall_time_us":49615,"thread_start_us":948501,"threads_started":3,"wal-append.queue_time_us":417970}
I20260709 13:04:56.853503  8193 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:56.794829  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2808) took 1265 ms. Trace:
I20260709 13:04:56.823571  1721 rpcz_store.cc:276] 0709 13:04:53.475223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.556864 (+1081641us) service_pool.cc:224] Handling call
0709 13:04:55.097767 (+540903us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:55.098851 (+  1084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.822417 (+1723566us) spinlock_profiling.cc:243] Waited 517 ms on lock 0x7b54002995d0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:56.822653 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1189175040}
I20260709 13:04:56.812848  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:56.867005  1713 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 16: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:56.850098  8139 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:56.852531  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 78493.4us
I20260709 13:04:56.833401  1570 rpcz_store.cc:276] 0709 13:04:55.081250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.831464 (+1750214us) service_pool.cc:224] Handling call
0709 13:04:56.832049 (+   585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:56.832402 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:56.832429 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:56.832490 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:56.832525 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.832576 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.832693 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:56.874061  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2826) took 1756 ms. Trace:
I20260709 13:04:56.874861  1567 rpcz_store.cc:276] 0709 13:04:55.110705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.842055 (+1731350us) service_pool.cc:224] Handling call
0709 13:04:56.842559 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:56.867331 (+ 24772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.867554 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:56.877727  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:56.880237  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:56.884033  8193 raft_consensus.cc:697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:04:56.885852  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 428683us
I20260709 13:04:56.891839  7948 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.894083  7738 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.913275  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:56.870622  1639 rpcz_store.cc:276] 0709 13:04:55.529289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.529529 (+   240us) service_pool.cc:224] Handling call
0709 13:04:55.557287 (+ 27758us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:55.557500 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:55.914060 (+356560us) log.cc:844] Serialized 14 byte log entry
0709 13:04:56.297795 (+383735us) log.cc:844] Serialized 14 byte log entry
0709 13:04:56.587716 (+289921us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b18003e5cc0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d662a6 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d662a6 00007f8245d66240
0709 13:04:56.588298 (+   582us) spinlock_profiling.cc:243] Waited 5.03 ms on lock 0x7b18003e5cc0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d662a6 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1
0709 13:04:56.588779 (+   481us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b18003e5cc0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:04:56.614295 (+ 25516us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b18003e5cc0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:04:56.696172 (+ 81877us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b18003e5cc0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287 00007f8248ea0208
0709 13:04:56.718426 (+ 22254us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:56.760533 (+ 42107us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:56.793806 (+ 33273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.794036 (+   230us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.794448 (+   412us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":588699,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":1652,"raft.queue_time_us":32812,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":16670,"spinlock_wait_cycles":100940416,"thread_start_us":713805,"threads_started":3,"wal-append.queue_time_us":397997}
W20260709 13:04:56.964344  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.954177  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 1089) took 1832 ms. Trace:
I20260709 13:04:56.968914  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:56.952709  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2821) took 1880 ms. Trace:
I20260709 13:04:56.968694  1570 rpcz_store.cc:276] 0709 13:04:55.121392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.903334 (+1781942us) service_pool.cc:224] Handling call
0709 13:04:56.953381 (+ 50047us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:56.953768 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:56.953801 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:56.953897 (+    96us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:04:56.953939 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.953991 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.954092 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:56.913363  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:56.886435  8193 consensus_queue.cc:237] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:56.980684  8088 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:56.982700  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:56.984053  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:56.983706  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.902817  7981 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:56.989102  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:56.990214  1713 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:56.991415  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 90096.8us
I20260709 13:04:56.992643  1538 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:56.994117  8195 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:56.994642  8195 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.995041  8195 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:56.986222  7948 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:56.964628  7929 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.975033  1571 rpcz_store.cc:276] 0709 13:04:55.072195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.449981 (+1377786us) service_pool.cc:224] Handling call
0709 13:04:56.450530 (+   549us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:56.451008 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:56.451050 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:56.451764 (+   714us) log.cc:844] Serialized 26 byte log entry
0709 13:04:56.878374 (+426610us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:56.878467 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.878598 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:56.952217 (+ 73619us) raft_consensus.cc:1685] finished
0709 13:04:56.952237 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.952494 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":90879,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":125,"thread_start_us":343358,"threads_started":1,"wal-append.queue_time_us":495766}
I20260709 13:04:57.010134  7948 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:57.010699  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.011138  1718 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 18: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:57.011889  8143 raft_consensus.cc:3055] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:57.012259  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:57.012580  8143 raft_consensus.cc:740] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:57.014005  8185 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:57.016613  8185 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:57.013311  1718 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:57.017889  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:57.014504  8090 consensus_meta.cc:287] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.624s	user 0.014s	sys 0.018s
W20260709 13:04:57.018002  7926 consensus_meta.cc:287] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.653s	user 0.008s	sys 0.027s
I20260709 13:04:57.025388  1569 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:04:57.023882  8090 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:57.028347  1639 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:04:57.030287  1717 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:57.032773  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:57.032953  7990 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:57.034117  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:57.036978  1640 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
W20260709 13:04:57.040503  8183 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.181s	user 0.001s	sys 0.001s
I20260709 13:04:57.048557  1569 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:57.043123  7926 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:57.045398  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2775) took 1166 ms. Trace:
I20260709 13:04:57.052374  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2774) took 1201 ms. Trace:
I20260709 13:04:57.054491  1571 rpcz_store.cc:276] 0709 13:04:55.828559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.027919 (+1199360us) service_pool.cc:224] Handling call
0709 13:04:57.028752 (+   833us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:57.029254 (+   502us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:57.029282 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:57.029348 (+    66us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:57.029388 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.029445 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:57.029582 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:57.054221  1565 rpcz_store.cc:276] 0709 13:04:55.879611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.044258 (+1164647us) service_pool.cc:224] Handling call
0709 13:04:57.044698 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:57.045014 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:57.045053 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:57.045123 (+    70us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:57.045185 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.045236 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:57.045339 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:57.050998  1645 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2778) took 2316 ms (client timeout 2026 ms). Trace:
W20260709 13:04:57.059356  1645 rpcz_store.cc:269] 0709 13:04:54.733900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.860156 (+126256us) service_pool.cc:224] Handling call
0709 13:04:57.050373 (+2190217us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b54000a5ad4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:57.050720 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1102920448}
I20260709 13:04:57.067291  1713 raft_consensus.cc:2410] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:04:57.069422  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:57.070802  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:57.074546  1570 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:57.062481  1538 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:57.069317  7738 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:57.077112  7929 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:57.082259  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.082448  7929 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:57.083637  8116 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:57.084553  8116 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:57.086222  7990 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:57.086861  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:57.087796  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2823) took 2010 ms. Trace:
I20260709 13:04:57.088411  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:57.091406  8227 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.092211  8227 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:57.059065  8088 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:57.094913  8077 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:57.093756  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:57.095691  1722 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:57.100008  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 1090) took 1944 ms. Trace:
I20260709 13:04:57.106825  1564 rpcz_store.cc:276] 0709 13:04:55.155436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.098468 (+1943032us) service_pool.cc:224] Handling call
0709 13:04:57.099114 (+   646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:57.099623 (+   509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:57.099654 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:57.099722 (+    68us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:57.099761 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.099821 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:57.099937 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:57.102814  8116 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:57.088258  1573 rpcz_store.cc:276] 0709 13:04:55.077380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.196709 (+119329us) service_pool.cc:224] Handling call
0709 13:04:57.087273 (+1890564us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b5400012f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:57.087540 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1487055616}
I20260709 13:04:57.119582  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2773) took 1547 ms. Trace:
I20260709 13:04:57.125392  1570 rpcz_store.cc:276] 0709 13:04:55.564010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.982220 (+1418210us) service_pool.cc:224] Handling call
0709 13:04:57.067007 (+ 84787us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:57.078095 (+ 11088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.111676 (+ 33581us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:57.120322  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:57.127224  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:57.128259  1713 raft_consensus.cc:2368] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15.
I20260709 13:04:57.094283  8143 consensus_queue.cc:260] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:57.142161  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:57.142504  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 109943us
I20260709 13:04:57.143049  1719 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:57.118348  8090 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:57.144311  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:57.144912  8088 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:57.103986  7970 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:57.184846  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3124) took 2996 ms. Trace:
W20260709 13:04:57.200730  8240 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.089s	user 0.003s	sys 0.000s
I20260709 13:04:57.208439  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:57.210136  1713 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:57.211512  1713 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:57.212033  8195 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:57.212911  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2741) took 2038 ms. Trace:
I20260709 13:04:57.214586  1718 raft_consensus.cc:2393] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:57.194310  1720 rpcz_store.cc:276] 0709 13:04:54.188741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.637713 (+448972us) service_pool.cc:224] Handling call
0709 13:04:55.537789 (+900076us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:55.551392 (+ 13603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.174900 (+1623508us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b5400248450. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:57.184580 (+  9680us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2068333952}
I20260709 13:04:57.217959  1715 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 19. Current term is 20. Ops: []
W20260709 13:04:57.217903  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 193048us
I20260709 13:04:57.194826  8143 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:57.230850  7970 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
W20260709 13:04:57.235846  8242 log.cc:927] Time spent T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Append to log took a long time: real 0.200s	user 0.001s	sys 0.003s
I20260709 13:04:57.236640  8097 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.237526  8097 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:57.240931  1567 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:57.237922  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2792) took 2143 ms. Trace:
I20260709 13:04:57.241560  8097 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:57.248100  8211 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:57.249262  8211 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:57.251991  1715 raft_consensus.cc:1275] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:04:57.247453  1647 rpcz_store.cc:276] 0709 13:04:55.086180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.086386 (+   206us) service_pool.cc:224] Handling call
0709 13:04:55.088722 (+  2336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:55.272363 (+183641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.139839 (+1867476us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54001f4354. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:57.229141 (+ 89302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":251849472}
I20260709 13:04:57.254662  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2819) took 2186 ms. Trace:
I20260709 13:04:57.255174  1538 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:57.255360  1566 rpcz_store.cc:276] 0709 13:04:55.068343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.361505 (+1293162us) service_pool.cc:224] Handling call
0709 13:04:56.362363 (+   858us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:56.789842 (+427479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.254318 (+464476us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b5400306454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:57.254575 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":701859072}
I20260709 13:04:57.258062  1713 raft_consensus.cc:3055] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:57.258709  1713 raft_consensus.cc:740] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:57.259230  7738 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:57.260170  1713 consensus_queue.cc:260] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:57.257530  7926 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:57.261953  8243 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:04:57.263104  8205 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:57.263718  8205 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:57.264307  8205 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:57.267249  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 1091) took 2026 ms. Trace:
I20260709 13:04:57.267915  1567 rpcz_store.cc:276] 0709 13:04:55.240479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.240115 (+1999636us) service_pool.cc:224] Handling call
0709 13:04:57.240659 (+   544us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:57.241572 (+   913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.267170 (+ 25598us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:57.261502  1713 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:57.213444  1716 rpcz_store.cc:276] 0709 13:04:55.174738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.519456 (+344718us) service_pool.cc:224] Handling call
0709 13:04:55.520055 (+   599us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:55.556939 (+ 36884us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:55.556978 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:55.564452 (+  7474us) log.cc:844] Serialized 26 byte log entry
0709 13:04:56.753022 (+1188570us) spinlock_profiling.cc:243] Waited 898 us on lock 0x7b1800241840. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a250d1 00007f8248e40a0f
0709 13:04:56.955106 (+202084us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:56.955205 (+    99us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.955316 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:57.212394 (+257078us) raft_consensus.cc:1685] finished
0709 13:04:57.212415 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:57.212665 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":198853,"raft.queue_time_us":36428,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":6550,"spinlock_wait_cycles":2065280,"thread_start_us":157035,"threads_started":1,"wal-append.queue_time_us":1408276}
I20260709 13:04:57.256875  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:57.254551  8227 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:57.297773  8195 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:57.305187  1611 connection.cc:256] RPC call timeout handler was delayed by -1.1219484460000331s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1585s for a response
I20260709 13:04:57.297880  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:35480 (request call id 28) took 1082 ms. Trace:
W20260709 13:04:57.297781  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.314108  8145 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.314827  8145 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:57.322446  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 65691.7us
I20260709 13:04:57.309520  1714 rpcz_store.cc:276] 0709 13:04:56.085803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.098203 (+ 12400us) service_pool.cc:224] Handling call
0709 13:04:57.085482 (+987279us) spinlock_profiling.cc:243] Waited 743 ms on lock 0x7b5400053f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e8b735 00007f8248e839fa 00007f824de4b4c9 00007f82473c7c54 00007f82473c7be2 00007f82473c7b11 00007f82473c7a8c 00007f82473c6d42 00007f82471c048c 00007f82471bf8c6
0709 13:04:57.167613 (+ 82131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1708621056}
I20260709 13:04:57.323002  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:57.324952  1722 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:04:57.325333  2347 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:04:57.329689  1611 connection.cc:256] RPC call timeout handler was delayed by -1.0886484460000361s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1622s for a response
W20260709 13:04:57.330559  1611 connection.cc:256] RPC call timeout handler was delayed by -1.0508484460000318s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1664s for a response
I20260709 13:04:57.330397  1713 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:04:57.324544  8145 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:57.351121  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.352703  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.363744  1538 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:57.338941  8195 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:57.377918  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:57.381726  1565 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:04:57.350065  8227 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:57.396569  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 58593.7us
W20260709 13:04:57.400910  8016 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.889s	user 0.005s	sys 0.021s
I20260709 13:04:57.402892  7929 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.407824  1646 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 22. Current term is 23. Ops: []
W20260709 13:04:57.413903  8016 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.934s	user 0.006s	sys 0.021s
I20260709 13:04:57.361543  7970 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.426496  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:57.373889  8195 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.434334  8139 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.436523  8029 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:57.437186  8029 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:57.437839  8029 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:57.418601  7929 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:57.461283  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:57.435372  8139 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:57.469419  7998 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.538s	user 0.003s	sys 0.027s
W20260709 13:04:57.469945  7998 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.602s	user 0.004s	sys 0.027s
I20260709 13:04:57.470221  8211 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:57.463219  2314 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:57.473568  8145 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:57.473892  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:57.476686  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:57.477142  1647 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:57.477514  1644 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:04:57.478210  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:57.474071  8145 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:57.479038  1647 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:57.479179  8145 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:57.476920  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:57.483669  8139 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:57.478574  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:57.486795  1719 raft_consensus.cc:3055] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:57.431457  8205 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:57.488503  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 147784us
W20260709 13:04:57.490641  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.490440  8205 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:57.492220  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:57.493528  1570 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:57.493940  8145 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:57.487906  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 63460.1us
I20260709 13:04:57.419656  7990 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:57.468076  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:57.443282  8195 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:57.487466  1719 raft_consensus.cc:740] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:57.498879  1565 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:57.512363  1716 raft_consensus.cc:2410] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
W20260709 13:04:57.509479  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 181280us
I20260709 13:04:57.487520  7929 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:57.517181  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:04:57.521030  8037 consensus_meta.cc:287] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 1.018s	user 0.005s	sys 0.023s
I20260709 13:04:57.520803  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:57.519423  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:57.523912  1538 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:57.527905  1567 raft_consensus.cc:1275] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:04:57.531833  8193 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:57.532532  8193 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.532991  8193 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:57.500079  7970 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:57.541312  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:57.510356  8227 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:57.522076  8037 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:57.556625  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:57.555621  1647 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:57.507978  1714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e89a11b4204e48568da409c196e33be4"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:04:57.544875  7929 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.559564  1648 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:57.540747  8139 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 13:04:57.561055  1714 raft_consensus.cc:606] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:04:57.541479  8236 consensus_queue.cc:1059] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.184s
I20260709 13:04:57.562827  1719 consensus_queue.cc:260] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:57.569150  7970 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:57.580258  1721 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:57.571146  8211 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 13:04:57.595870  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:57.597371  1719 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:57.575898  8077 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.575338  8247 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:57.589673  8211 consensus_queue.cc:1243] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:04:57.605317  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:57.606106  1570 raft_consensus.cc:2376] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17. Re-sending same reply.
I20260709 13:04:57.606400  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:57.608109   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:57.615906  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:57.615527  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:57.621477  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:57.622993  1564 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:57.620628  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:57.626353  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:57.640625  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:57.641799  8077 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:57.645062  8249 log.cc:927] Time spent T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:04:57.646504  8227 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:57.647186  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2766) took 2771 ms. Trace:
I20260709 13:04:57.647738  1568 rpcz_store.cc:276] 0709 13:04:54.875703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.878349 (+1002646us) service_pool.cc:224] Handling call
0709 13:04:55.904487 (+ 26138us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:57.210476 (+1305989us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b18003ef6e0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f8248e7d060 00007f8248e7dbfb 00007f8248e8864b
0709 13:04:57.210839 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:57.210874 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:57.211820 (+   946us) log.cc:844] Serialized 52 byte log entry
0709 13:04:57.371434 (+159614us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:57.371490 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.385021 (+ 13531us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:57.646587 (+261566us) raft_consensus.cc:1685] finished
0709 13:04:57.646621 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:57.646992 (+   371us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3282,"raft.queue_time_us":178128,"raft.run_cpu_time_us":450,"raft.run_wall_time_us":34060,"spinlock_wait_cycles":39024000,"thread_start_us":374794,"threads_started":2,"wal-append.queue_time_us":305879}
I20260709 13:04:57.640952  8247 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:57.653025  8247 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:57.653997  1642 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:04:57.654628  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:57.655723  1644 raft_consensus.cc:2393] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:57.656850  1573 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:57.642230  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:57.663745  1648 raft_consensus.cc:1275] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:57.665264  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:57.691164  1714 raft_consensus.cc:2393] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 19: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:57.609464  7929 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:57.727419  1644 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:04:57.727898  1642 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:04:57.727509  1719 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 13:04:57.736760  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 237139us
I20260709 13:04:57.657392   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:57.742019  2342 raft_consensus.cc:2410] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:04:57.644942  8077 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:57.791322  8037 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:04:57.792505  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.794232  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.794891  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:57.795750  1648 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19.
I20260709 13:04:57.796904  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:57.804044  1611 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:57.853835  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:57.855367  1564 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:04:57.854672  1571 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:57.855639  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:57.869086  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:57.859593  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:57.868813  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 324641us
I20260709 13:04:58.042814  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:58.044507  1573 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:58.046470  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:58.050283  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:58.053386  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 196981us
I20260709 13:04:58.057287  1648 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:57.873581  8077 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:58.071833  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:58.072530  1571 raft_consensus.cc:2376] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17. Re-sending same reply.
I20260709 13:04:58.062901  8146 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.079747  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:58.080572  2345 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:04:58.080488  8146 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:58.082034  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:58.082996  2344 raft_consensus.cc:2410] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:04:58.083295  8146 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:58.052677  8247 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.083899  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:58.084141  8247 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:58.087600  8247 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:58.089394  1645 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 18: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:58.090538  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:58.091408  1645 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:58.102090  1644 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:04:58.103454  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:58.105031  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:58.106712  7929 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:58.107764  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 303764us
I20260709 13:04:58.109951  8256 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:58.079746  8210 consensus_queue.cc:1059] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.270s
I20260709 13:04:58.119124  8210 raft_consensus.cc:3055] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:58.119575  8210 raft_consensus.cc:740] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:04:58.123198  8193 consensus_meta.cc:287] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.590s	user 0.008s	sys 0.004s
I20260709 13:04:58.123755  8193 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:58.127900  8037 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.128756  8037 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:58.131016  8037 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:58.132674  1573 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
W20260709 13:04:58.133409  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 84997.9us
I20260709 13:04:58.142290  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:04:58.145428  1647 consensus_meta.cc:287] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.566s	user 0.009s	sys 0.001s
I20260709 13:04:58.145922  1647 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
W20260709 13:04:58.151785  1646 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.001s	sys 0.071s
W20260709 13:04:58.152525  1646 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.002s	sys 0.071s
I20260709 13:04:58.156370  1644 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:58.063412  8227 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.078660  8077 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:58.167922  1643 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:04:58.159968  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:58.171988  1640 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.016s	sys 0.121s
W20260709 13:04:58.177399  1640 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.016s	sys 0.121s
I20260709 13:04:58.172341  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:58.172582  1645 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:58.175253  1647 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:04:58.167495  8195 consensus_meta.cc:287] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.630s	user 0.010s	sys 0.001s
W20260709 13:04:58.146951  1612 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.955s (SENT) [suppressed 34 similar messages]
W20260709 13:04:58.180729  1716 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.033s	sys 0.063s
W20260709 13:04:58.182940  1716 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.033s	sys 0.063s
I20260709 13:04:58.187284  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:58.171746  8248 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:58.198848  1645 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:04:58.204646  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 57708us
I20260709 13:04:58.177114  8077 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:58.197424  1568 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:04:58.199399  1643 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:04:58.212145  8260 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.212430  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:58.203025  8227 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:58.213011  8260 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:58.215514  8227 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:58.216123  8260 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:58.216398  8193 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:58.206578  8029 consensus_meta.cc:287] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.768s	user 0.010s	sys 0.005s
I20260709 13:04:58.161145  8210 consensus_queue.cc:260] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:58.191646  8195 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:58.225672  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 116000us
I20260709 13:04:58.222386  8029 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:58.232470  1569 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:58.233078  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:58.226964  8248 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:58.244284  1611 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:58.254105  1612 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 2.071s (SENT)
W20260709 13:04:58.256539  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.246347  8210 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:58.259195  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.248602  8227 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:04:58.261926  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 114603us
I20260709 13:04:58.268110  1719 raft_consensus.cc:3055] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:58.268582  1719 raft_consensus.cc:740] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:04:58.268671  8205 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.269959  7861 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:58.269549  1719 consensus_queue.cc:260] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:58.271065  1719 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:58.269383  8205 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:58.270192  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:58.274257  8205 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:58.269506  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:58.248234  8193 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:04:58.293841  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.296769  1719 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:04:58.306061  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:58.307206  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:04:58.311693  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.266772  8227 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:58.314390  8227 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:58.318630  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:58.319795  1639 raft_consensus.cc:2393] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:58.318292  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:58.322275  1645 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:04:58.325402  1720 raft_consensus.cc:2368] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15.
I20260709 13:04:58.327420  1644 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:04:58.303092  8260 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:04:58.335584  8248 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:58.336674  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 96378.4us
I20260709 13:04:58.338143  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:58.348628  1611 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:58.350098  1648 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:04:58.296109  8193 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:58.351034  2340 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:58.315676  8195 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:58.353837  8236 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.019s	sys 0.018s
I20260709 13:04:58.356226  7929 raft_consensus.cc:2764] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
W20260709 13:04:58.356444  8236 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.019s	sys 0.018s
I20260709 13:04:58.362221  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:58.350761  8029 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:58.368458  1611 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:58.370082  8236 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:58.371146  1611 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:58.371699  8077 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:58.372660  8236 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:58.375751  1611 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:58.377230  8236 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:58.377732  8236 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:58.378122  8236 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:58.379528  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:58.380342  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:58.353997  8205 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:58.381625  2346 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:58.381963  1639 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: [16.9-18.10]
I20260709 13:04:58.382122  8205 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:58.383327  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:58.384655  1611 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:58.394238  8267 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:58.398087  2343 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:58.409325  1686 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:58.410267  8077 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:58.411530  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:58.411959  8193 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:58.412019  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:58.412338  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:58.414222  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:58.415881  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:58.431370  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:58.441124  1641 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:04:58.442171  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 64229.5us
W20260709 13:04:58.443894  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.450295  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:58.451263  1647 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:58.452924  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:58.453938  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:58.454663  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:58.441375  8037 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:58.454566  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:58.457687  1641 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:58.458833  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:58.463583  1639 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:58.465932  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:58.467413  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:58.460671  1645 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:04:58.450227  8193 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:58.454460  8029 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.473371  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:58.475286  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:58.478511  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:58.479135  1639 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:58.480149  8212 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:58.481369  8268 log.cc:927] Time spent T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.094s	user 0.001s	sys 0.001s
I20260709 13:04:58.481122  8212 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:58.482563  1611 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:04:58.483505  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:58.484609  8212 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:58.485111  1645 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:58.486070  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:58.480460  1717 raft_consensus.cc:2368] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 16. Current term is 17.
I20260709 13:04:58.486794  1647 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:58.487409  1644 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
W20260709 13:04:58.489826  1572 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40670 (request call id 2841) took 2199 ms (client timeout 2784 ms). Trace:
W20260709 13:04:58.490340  1572 rpcz_store.cc:269] 0709 13:04:56.248559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.350991 (+102432us) service_pool.cc:224] Handling call
0709 13:04:58.441864 (+2090873us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b5400305a54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:58.447931 (+  6067us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1633166592}
I20260709 13:04:58.473054  1713 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:58.494011  8205 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:58.501564  1647 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:04:58.500355  8029 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:58.505537  1647 raft_consensus.cc:1275] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:58.505764  8247 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:58.518186  1538 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:58.518718  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:58.519829  1641 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:58.520152  2347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55890 (request call id 608) took 1206 ms. Trace:
I20260709 13:04:58.520668  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:58.522012  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:58.502884  8146 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:58.522694  2340 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
I20260709 13:04:58.478269  8193 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:58.523152  8146 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:58.524909  8193 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:58.525539  8146 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:58.526234  8029 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:58.520185  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:58.521181  1641 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:58.530622  2314 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:58.520641  2347 rpcz_store.cc:276] 0709 13:04:57.314245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.314881 (+   636us) service_pool.cc:224] Handling call
0709 13:04:57.315267 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:57.326195 (+ 10928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.519891 (+1193696us) spinlock_profiling.cc:243] Waited 3.03 ms on lock 0x7b54006d0454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:58.520069 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6970624}
I20260709 13:04:58.525533  8193 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:58.510154  8236 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:58.540627  1538 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:58.541173  8236 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:58.534833  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 52402.2us
I20260709 13:04:58.542184  1648 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 14: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:58.542900  2314 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:58.543889  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:58.546571  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:58.547938  1717 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:58.553959  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:58.555016  1716 raft_consensus.cc:2410] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:04:58.548264  8248 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:58.556695  8236 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.557264  8236 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:58.557322  8247 consensus_queue.cc:1059] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:58.558943  8247 raft_consensus.cc:3055] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:58.559319  8247 raft_consensus.cc:740] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:58.560284  8247 consensus_queue.cc:260] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:58.561204  8031 raft_consensus.cc:3055] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:58.561291  8247 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:58.561849  8031 raft_consensus.cc:740] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:58.562588  2345 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:04:58.562949  8031 consensus_queue.cc:260] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:58.577843  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:58.578513  8031 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:58.580744  8205 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:58.582365  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2815) took 1107 ms. Trace:
I20260709 13:04:58.582494  8029 consensus_queue.cc:1059] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:58.583541  8029 raft_consensus.cc:3055] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:58.584167  8029 raft_consensus.cc:740] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:58.582906  1640 rpcz_store.cc:276] 0709 13:04:57.474881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.475570 (+   689us) service_pool.cc:224] Handling call
0709 13:04:57.475966 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:57.573803 (+ 97837us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:57.573860 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:57.593898 (+ 20038us) log.cc:844] Serialized 26 byte log entry
0709 13:04:58.178299 (+584401us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:58.178341 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.581901 (+403560us) spinlock_profiling.cc:243] Waited 84.8 ms on lock 0x7b54002dc154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:58.581960 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:58.582013 (+    53us) raft_consensus.cc:1685] finished
0709 13:04:58.582031 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:58.582298 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5731,"raft.queue_time_us":8399,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":2925,"spinlock_wait_cycles":194927872,"thread_start_us":515228,"threads_started":1,"wal-append.queue_time_us":597937}
I20260709 13:04:58.564877  8236 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:58.565340  8259 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:58.566884  7929 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:58.590428  8259 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.591027  8259 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:58.567855  8260 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:58.568434  8248 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:58.567188  1612 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:58.573510  2342 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55890 (request call id 607) took 1551 ms (client timeout 1506 ms). Trace:
W20260709 13:04:58.595062  2342 rpcz_store.cc:269] 0709 13:04:57.021531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.021718 (+   187us) service_pool.cc:224] Handling call
0709 13:04:58.573065 (+1551347us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b54003f84d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:58.573346 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1594419968}
I20260709 13:04:58.577262  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:58.596539  1569 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
I20260709 13:04:58.589660  1647 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:58.587641  1538 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:58.593660  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:58.598354  1716 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:04:58.592024  8193 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:58.600278  1568 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:58.599341  1538 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:58.600700  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:58.602828  7990 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:58.585228  8029 consensus_queue.cc:260] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:58.622557  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:58.604704  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:04:58.609671  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 43861.1us
I20260709 13:04:58.625341  1713 raft_consensus.cc:2393] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:04:58.610638  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:58.603809  1538 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:58.617923  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:58.618551  1568 raft_consensus.cc:1240] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:58.610136  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:58.627970  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 57812.1us
I20260709 13:04:58.603950  7929 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:58.633824  8270 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:58.605792  7990 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.604048  8193 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:58.611958  8031 consensus_queue.cc:1059] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:04:58.611204  8205 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:58.614739  8227 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:58.638983  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:58.634908  1564 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17.
W20260709 13:04:58.638108  1722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 3134) took 1726 ms (client timeout 1581 ms). Trace:
W20260709 13:04:58.646767  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:58.643004  1722 rpcz_store.cc:269] 0709 13:04:56.911922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.004990 (+ 93068us) service_pool.cc:224] Handling call
0709 13:04:58.637869 (+1632879us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b5400425254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:58.638005 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53802880}
I20260709 13:04:58.614219  8029 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:58.666111  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:58.634857  7929 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:58.670462  7929 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:58.637058  8031 raft_consensus.cc:3055] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:58.635337  7990 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:58.646623  1639 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:04:58.675513  1537 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:58.635610  8193 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.635178  8270 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.635383  8211 consensus_queue.cc:1059] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:04:58.634523  8280 consensus_queue.cc:1059] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:04:58.680896  8193 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:58.637511  8247 consensus_queue.cc:1048] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:04:58.674434  8031 raft_consensus.cc:740] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:04:58.679770  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 49049us
I20260709 13:04:58.635675  8265 consensus_queue.cc:1059] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:04:58.637491  8205 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:58.689438  1538 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:04:58.694881  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.681371  8270 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:58.687507  8227 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:58.710446  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:58.722072  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:58.743319  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 53980.4us
W20260709 13:04:58.751905  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 54951.1us
I20260709 13:04:58.747365  8286 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:58.702755  8031 consensus_queue.cc:260] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:58.757155  8286 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:58.757936  8286 raft_consensus.cc:697] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:58.759549  8282 consensus_queue.cc:1048] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:58.759501  8286 consensus_queue.cc:237] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:58.761281  8265 raft_consensus.cc:3055] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:58.761965  8265 raft_consensus.cc:740] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:58.762933  8265 consensus_queue.cc:260] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:58.764438  8265 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:58.758096  8031 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:58.777273  1611 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:04:58.783923  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 107362us
I20260709 13:04:58.792258  8259 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:58.796164  8288 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:58.797029  8288 raft_consensus.cc:697] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:58.806290  7970 raft_consensus.cc:3055] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:58.808561  7970 raft_consensus.cc:740] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:58.822495  8205 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:58.830672  8288 consensus_queue.cc:237] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:58.848138  8259 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:58.863444  8270 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:58.865794  8270 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:59.094331  7990 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:59.096778  7990 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:59.125115  8288 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.125769  8288 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:59.127481  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.127765  8288 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:59.269973  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:04:59.277937  8029 consensus_meta.cc:287] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.612s	user 0.011s	sys 0.008s
I20260709 13:04:59.301994  1612 leader_election.cc:400] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15.
I20260709 13:04:59.302419  1612 leader_election.cc:403] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:59.303093  8267 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15.
I20260709 13:04:59.304730  1612 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:59.305516  8289 raft_consensus.cc:2764] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:04:59.310441  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:59.312201  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:59.313045  1640 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17.
I20260709 13:04:59.316465  8247 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:04:59.317196  8247 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:59.318115  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:59.314141  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:59.319815  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:59.320308  1640 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:59.323557  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:59.324484  1722 raft_consensus.cc:2410] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:04:59.326608  8247 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:59.327437  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:59.328456  1722 raft_consensus.cc:2410] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:04:59.329274  2345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55890 (request call id 610) took 1119 ms. Trace:
I20260709 13:04:59.329762  2345 rpcz_store.cc:276] 0709 13:04:58.210382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.210602 (+   220us) service_pool.cc:224] Handling call
0709 13:04:59.329020 (+1118418us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b5400600ed4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:59.329192 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":639715072}
W20260709 13:04:59.343837  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.344218  8247 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:59.344862  8247 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:59.345422  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.346299  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:59.347841  8247 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:59.349653  2314 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:59.350579  8211 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:59.352618  2314 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:59.353732  8211 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:59.354143  8211 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.354475  8211 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:59.363656  8146 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.364504  8146 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:59.370030  8146 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:59.363637  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:59.374801  8211 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:59.381233  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:59.379019  2343 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 13:04:59.379681  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:59.384148  2345 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:04:59.380003  8260 raft_consensus.cc:3055] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:59.386385  1722 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:04:59.388238  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:59.403352  7970 consensus_queue.cc:260] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:59.409955  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b82d3b49001741eda872554a7ecb2db3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:04:59.414108  7861 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:59.385260  8260 raft_consensus.cc:740] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:59.419618  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:59.420598  1647 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:04:59.420990  8260 consensus_queue.cc:260] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:59.422284  8260 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:59.423794  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:59.424475  1647 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:59.427106  8278 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.690s	user 0.006s	sys 0.003s
I20260709 13:04:59.430138  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3309) took 1014 ms. Trace:
W20260709 13:04:59.381392  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.430598  1645 rpcz_store.cc:276] 0709 13:04:58.415716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.490160 (+ 74444us) service_pool.cc:224] Handling call
0709 13:04:58.490534 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:58.500441 (+  9907us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:58.500474 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:58.534182 (+ 33708us) log.cc:844] Serialized 26 byte log entry
0709 13:04:58.649150 (+114968us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:58.649193 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.649261 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:59.426332 (+777071us) spinlock_profiling.cc:243] Waited 6.05 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:04:59.429748 (+  3416us) raft_consensus.cc:1685] finished
0709 13:04:59.429776 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.430063 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9198,"raft.queue_time_us":11840,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":3754,"spinlock_wait_cycles":13914880,"thread_start_us":102565,"threads_started":1,"wal-append.queue_time_us":150524}
I20260709 13:04:59.405441  7970 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:59.432886  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:59.435726  1645 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:04:59.438484  8146 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:04:59.439241  8146 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:59.367115  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2782) took 1946 ms. Trace:
I20260709 13:04:59.441089  1565 rpcz_store.cc:276] 0709 13:04:57.421155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.430508 (+  9353us) service_pool.cc:224] Handling call
0709 13:04:57.452645 (+ 22137us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:57.509674 (+ 57029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.366326 (+1856652us) spinlock_profiling.cc:243] Waited 43.7 ms on lock 0x7b5400012554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:59.367011 (+   685us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100414720}
W20260709 13:04:59.444561  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:59.448562  7929 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.777s	user 0.014s	sys 0.000s
W20260709 13:04:59.451231  1721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2744) took 2544 ms (client timeout 1955 ms). Trace:
W20260709 13:04:59.451709  1721 rpcz_store.cc:269] 0709 13:04:56.907374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.907540 (+   166us) service_pool.cc:224] Handling call
0709 13:04:59.450995 (+2543455us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b5400247a54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:59.451128 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1326975360}
I20260709 13:04:59.452033  2314 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:59.452868  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:59.454527  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:59.455741  8288 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:59.456203  8288 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.456539  8288 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:59.460376  1685 connection.cc:256] RPC call timeout handler was delayed by -2.1752808879999748s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25179999999999997s for a response
I20260709 13:04:59.469470  8259 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:59.470008  8259 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:04:59.470376  8259 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:59.471580  1537 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.014s	sys 0.003s
W20260709 13:04:59.472198  1537 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.014s	sys 0.003s
W20260709 13:04:59.473001  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 799411us
I20260709 13:04:59.476656  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:59.476933  1721 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:59.477499  1722 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17.
I20260709 13:04:59.479105  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:59.483016  8288 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:59.484429  8283 raft_consensus.cc:3055] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:59.485019  8283 raft_consensus.cc:740] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:59.480890  1537 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:59.488868  1685 connection.cc:256] RPC call timeout handler was delayed by -2.0186808880000058s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26920000000000005s for a response
I20260709 13:04:59.493088  1647 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
W20260709 13:04:59.494256  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:59.494679  1685 connection.cc:256] RPC call timeout handler was delayed by -1.8798424049999767s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
I20260709 13:04:59.391005  8211 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:59.493153  1640 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 20.
I20260709 13:04:59.486534  8139 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:59.507787  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:59.500276  1685 connection.cc:256] RPC call timeout handler was delayed by -1.8724584319999735s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2242s for a response
W20260709 13:04:59.514437  1685 connection.cc:256] RPC call timeout handler was delayed by -1.8584808879999741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28700000000000003s for a response
I20260709 13:04:59.449144  7929 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:59.514986  1685 connection.cc:256] RPC call timeout handler was delayed by -1.7657808879999948s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2973s for a response
I20260709 13:04:59.464735  8146 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:04:59.517791  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:59.485492  8288 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:59.533921  1687 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.907s (SENT)
I20260709 13:04:59.485889  8283 consensus_queue.cc:260] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:59.525998  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 52314.7us
I20260709 13:04:59.533764  8211 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:59.559319  1714 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:59.559191  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:04:59.560704  1641 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 20: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:04:59.565433  8283 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:59.566644  1565 raft_consensus.cc:1240] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:59.541358  7929 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:04:59.528736  8193 consensus_meta.cc:287] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.835s	user 0.014s	sys 0.001s
W20260709 13:04:59.530221  8031 consensus_meta.cc:287] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.764s	user 0.011s	sys 0.007s
W20260709 13:04:59.548202  1685 connection.cc:256] RPC call timeout handler was delayed by -1.5954536969999822s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
I20260709 13:04:59.542038  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:59.528827  8259 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:59.529171  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2822) took 2051 ms (client timeout 1652 ms). Trace:
I20260709 13:04:59.553154  1721 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16.
I20260709 13:04:59.574100  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:04:59.577024  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:59.581983  1566 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:59.577777  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 66420.5us
W20260709 13:04:59.580758  1642 rpcz_store.cc:269] 0709 13:04:57.497053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.500943 (+  3890us) service_pool.cc:224] Handling call
0709 13:04:59.528651 (+2027708us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54001ff9d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:59.528938 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":301888256}
I20260709 13:04:59.583235  2314 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:04:59.580264  1685 connection.cc:256] RPC call timeout handler was delayed by -1.576899917999981s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
I20260709 13:04:59.578843  1537 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:59.584579  1722 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:04:59.585947  1685 connection.cc:256] RPC call timeout handler was delayed by -1.5754707749999852s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17920000000000003s for a response
W20260709 13:04:59.586381  1685 connection.cc:256] RPC call timeout handler was delayed by -1.3648584320000055s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2806s for a response
W20260709 13:04:59.586768  1685 connection.cc:256] RPC call timeout handler was delayed by -1.2263557869999886s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1524s for a response
W20260709 13:04:59.587054  1685 connection.cc:256] RPC call timeout handler was delayed by -1.1111557869999729s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1652s for a response
W20260709 13:04:59.587389  1685 connection.cc:256] RPC call timeout handler was delayed by -1.1070022989999302s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1542s for a response
I20260709 13:04:59.584587  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:59.588701  2341 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:59.587806  1685 connection.cc:256] RPC call timeout handler was delayed by -1.0890022989999579s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1562s for a response
W20260709 13:04:59.591156  1685 connection.cc:256] RPC call timeout handler was delayed by -1.069030492999957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20840000000000003s for a response
W20260709 13:04:59.591624  1685 connection.cc:256] RPC call timeout handler was delayed by -1.0535557869999934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
I20260709 13:04:59.576964  8139 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
W20260709 13:04:59.592034  1685 connection.cc:256] RPC call timeout handler was delayed by -1.0228584320000209s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3186s for a response
I20260709 13:04:59.596089  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:59.606894  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 3319) took 1092 ms. Trace:
I20260709 13:04:59.607409  1648 rpcz_store.cc:276] 0709 13:04:58.514548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.519928 (+  5380us) service_pool.cc:224] Handling call
0709 13:04:59.606683 (+1086755us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b54002ae754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:04:59.606826 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30951296}
I20260709 13:04:59.597023  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:04:59.609117  2345 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 16: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:59.598726  8097 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:59.610443  8097 raft_consensus.cc:697] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:04:59.611470  8097 consensus_queue.cc:237] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:59.613600  2341 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 20.
I20260709 13:04:59.600750  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:59.614856  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 81045.4us
W20260709 13:04:59.618503  1687 leader_election.cc:336] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.026s (SENT)
I20260709 13:04:59.599040  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:59.150939   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:59.606977  1722 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:04:59.598626  8259 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:59.620107  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:59.620985  1721 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 16: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:59.622709  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:59.599469  8211 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:59.623778  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:59.620126  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:59.624418  1721 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:59.624785  1722 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:04:59.604398  1566 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
I20260709 13:04:59.576172  8193 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:59.626880  2312 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:59.629348  1714 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:04:59.630743  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:59.631371  1722 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:04:59.632297  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:59.633158  1714 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:04:59.633059  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:59.643494  1714 pending_rounds.cc:85] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 12
I20260709 13:04:59.627980  8195 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:59.643957  1714 pending_rounds.cc:107] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:04:59.648945  1714 raft_consensus.cc:2889] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:59.649166  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 61812.2us
W20260709 13:04:59.650190  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 67038.7us
I20260709 13:04:59.650128  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:59.651767  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.653241  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:04:59.654659  1643 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:59.652907  2314 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:59.632229  8145 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:59.643041  8236 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:59.664152  8139 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:59.666745  1537 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:59.650768  8300 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:59.679306  1721 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
I20260709 13:04:59.677788  1643 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16.
I20260709 13:04:59.678051  1722 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
W20260709 13:04:59.683024  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 64540.7us
I20260709 13:04:59.656342  8283 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.670660  1685 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:59.687132  1537 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:04:59.688541  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:04:59.693004  1721 raft_consensus.cc:2368] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 16. Current term is 17.
I20260709 13:04:59.694168  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:04:59.682008  8236 raft_consensus.cc:697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 20 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:04:59.687412  8211 raft_consensus.cc:697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 20 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:04:59.685894  8195 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:59.694917  8260 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:59.718904  8155 raft_consensus.cc:2764] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:04:59.712311  8300 raft_consensus.cc:697] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:04:59.721086  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 85632.1us
I20260709 13:04:59.716696  8236 consensus_queue.cc:237] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:04:59.712980  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 956707us
I20260709 13:04:59.715039  8029 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.727087  1537 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:04:59.731930  8297 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.068s	user 0.001s	sys 0.001s
I20260709 13:04:59.733117  8195 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:59.695237  8283 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:59.741072  1722 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:59.722297  8139 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:59.759874  8288 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:59.756721  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:59.768635  1717 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:04:59.770731  8300 consensus_queue.cc:237] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:59.771421  1714 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:59.772181  1722 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:04:59.741250  8029 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:59.776017  8029 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:59.782203  1685 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:59.733770  8195 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:04:59.745535  8139 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:04:59.787210  1611 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:59.750195  7990 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:59.808982  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 108718us
W20260709 13:04:59.810184  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:59.818151  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:59.829059  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:59.830204  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.791141  8139 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:59.873481  1538 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.129s	user 0.018s	sys 0.009s
W20260709 13:04:59.873979  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 117353us
I20260709 13:04:59.868891  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2813) took 2403 ms. Trace:
I20260709 13:04:59.786779  8193 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:59.791322  8029 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:59.882376  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 141217us
I20260709 13:04:59.841877  8288 raft_consensus.cc:2764] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:04:59.885270  1537 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:04:59.874848  1646 rpcz_store.cc:276] 0709 13:04:57.473878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.500417 (+ 26539us) service_pool.cc:224] Handling call
0709 13:04:57.522934 (+ 22517us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:57.548801 (+ 25867us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:57.548851 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:57.594037 (+ 45186us) log.cc:844] Serialized 52 byte log entry
0709 13:04:58.153117 (+559080us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:58.153180 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.229303 (+1076123us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b24002a3ea8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245c1e9c4 00007f8245c1cacb 00007f8245dd7dc9 00007f8245dd7a55 00007f8245dd6fb0 00007f8248a4d488 00007f8248a4dbe8 00007f8248a25398 00007f8248a2076a 00007f8248a1fbb6
0709 13:04:59.868312 (+639009us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b5400097c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:04:59.868390 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:59.868442 (+    52us) raft_consensus.cc:1685] finished
0709 13:04:59.868456 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.868814 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1599,"raft.queue_time_us":136,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":7218,"spinlock_wait_cycles":96516224,"thread_start_us":527942,"threads_started":1,"wal-append.queue_time_us":609271}
I20260709 13:04:59.906373  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:04:59.908581  1502 debug-util.cc:398] Leaking SignalData structure 0x7b08002f0cc0 after lost signal to thread 434
W20260709 13:04:59.963114  1502 debug-util.cc:398] Leaking SignalData structure 0x7b08002faf40 after lost signal to thread 1505
I20260709 13:04:59.908586  8004 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:59.907694  8029 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:59.910712  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:59.912698  1718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 3141) took 2599 ms (client timeout 2164 ms). Trace:
W20260709 13:04:59.910765  1573 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.236s	user 0.016s	sys 0.015s
I20260709 13:04:59.917387  1611 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:04:59.919397  2341 raft_consensus.cc:1240] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:04:59.921999  8031 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.976197  8031 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:04:59.976969  8221 consensus_queue.cc:1059] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:59.978374  8221 raft_consensus.cc:3055] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:59.978709  8031 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:04:59.924356  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:04:59.979913  2345 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 16: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:04:59.924902  8016 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.943540  8283 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:04:59.947121  8195 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:04:59.951761  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.953998  1464 catalog_manager.cc:5697] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 16 to 23. New cstate: current_term: 23 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:59.958184  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 72998us
I20260709 13:05:00.014510  1537 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:00.000752  8077 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:00.015571  8037 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:00.006556  8288 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:00.007542  8016 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:00.016940  8288 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:04:59.959488  1538 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.215s	user 0.019s	sys 0.009s
W20260709 13:05:00.017879  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 1.27324e+06us
I20260709 13:04:59.967885  8004 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:00.019032  8288 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:00.019465  8004 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:00.019888  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:00.020797  1568 raft_consensus.cc:2410] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 15 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
W20260709 13:04:59.972299  1718 rpcz_store.cc:269] 0709 13:04:57.313449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.912416 (+2598967us) service_pool.cc:211] Skipping call since client already timed out
0709 13:04:59.912595 (+   179us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:05:00.023336  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:00.016057  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 113651us
I20260709 13:05:00.024268  1537 leader_election.cc:400] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15.
I20260709 13:05:00.022207  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:04:59.988824  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:04:59.990427  1502 debug-util.cc:398] Leaking SignalData structure 0x7b080008a580 after lost signal to thread 1597
W20260709 13:05:00.026597  1573 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.352s	user 0.016s	sys 0.015s
I20260709 13:05:00.027462  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 1101) took 1434 ms. Trace:
W20260709 13:05:00.028080  1502 debug-util.cc:398] Leaking SignalData structure 0x7b0800038500 after lost signal to thread 1672
I20260709 13:05:00.027894  1573 rpcz_store.cc:276] 0709 13:04:58.592810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.592948 (+   138us) service_pool.cc:224] Handling call
0709 13:04:58.602922 (+  9974us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:58.616114 (+ 13192us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:04:58.616457 (+   343us) log.cc:844] Serialized 14 byte log entry
0709 13:05:00.027085 (+1410628us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:00.027155 (+    70us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:00.027212 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:00.027277 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:00.027400 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":219,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":72891,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":128,"spinlock_wait_cycles":2304,"thread_start_us":1409999,"threads_started":2,"wal-append.queue_time_us":81470,"wal-append.run_cpu_time_us":6394,"wal-append.run_wall_time_us":1061764}
W20260709 13:04:59.998786  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 120065us
I20260709 13:04:59.979022  8221 raft_consensus.cc:740] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:00.000990  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:00.026927  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2762) took 1606 ms. Trace:
I20260709 13:05:00.033457  1538 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:00.034924  1715 raft_consensus.cc:2410] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 19 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:05:00.022356  8029 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:00.015985  8077 raft_consensus.cc:697] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:00.016165  8037 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:05:00.037979  1502 debug-util.cc:398] Leaking SignalData structure 0x7b0800303040 after lost signal to thread 1746
I20260709 13:04:59.793776  8211 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:00.045073  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 56398.2us
I20260709 13:05:00.034186  1720 rpcz_store.cc:276] 0709 13:04:58.420830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.420986 (+   156us) service_pool.cc:224] Handling call
0709 13:04:58.421335 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:58.421667 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:58.422038 (+   371us) log.cc:844] Serialized 14 byte log entry
0709 13:04:58.460069 (+ 38031us) log.cc:844] Serialized 14 byte log entry
0709 13:04:58.616351 (+156282us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b1800361600. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d662a6 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1
0709 13:04:58.618063 (+  1712us) spinlock_profiling.cc:243] Waited 4.28 ms on lock 0x7b1800361600. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:04:59.776614 (+1158551us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b1800361600. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287 00007f8248ea0208
0709 13:05:00.025418 (+248804us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:00.025467 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:00.025514 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:00.026730 (+  1216us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:00.026866 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6253,"prepare.queue_time_us":7338,"prepare.run_cpu_time_us":236,"prepare.run_wall_time_us":1371,"raft.queue_time_us":18033,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":25541,"spinlock_wait_cycles":17615872,"thread_start_us":279419,"threads_started":2,"wal-append.queue_time_us":110165,"wal-append.run_cpu_time_us":1663,"wal-append.run_wall_time_us":1007823}
I20260709 13:05:00.038846  7990 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:00.039682  8288 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:00.047760  1502 debug-util.cc:398] Leaking SignalData structure 0x7b0800320640 after lost signal to thread 2373
W20260709 13:05:00.059008  1502 debug-util.cc:398] Leaking SignalData structure 0x7b0800336c20 after lost signal to thread 2374
I20260709 13:05:00.043427  8037 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:05:00.063218  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 45694.3us
W20260709 13:05:00.063915  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.057916  1611 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:00.041538  8270 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:05:00.086671  1612 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:00.092311  1612 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:05:00.094597  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 54156.7us
I20260709 13:05:00.093804  7929 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.065652  8221 consensus_queue.cc:260] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:00.095781  7929 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:00.096839  8221 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:00.098438  7929 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:00.094112  8288 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:00.089826  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 390753us
W20260709 13:05:00.110492  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 64284.7us
I20260709 13:05:00.080318  8236 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:00.104193  8016 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:00.122115  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:00.123535  1537 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:00.121037  8195 consensus_queue.cc:1059] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:05:00.123013  1464 catalog_manager.cc:5697] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:00.130506  2345 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:00.133978  8215 consensus_queue.cc:1059] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:00.134908  8215 raft_consensus.cc:3055] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:00.135294  8215 raft_consensus.cc:740] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:00.136277  8215 consensus_queue.cc:260] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:00.122004  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:00.141990  1719 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19.
I20260709 13:05:00.122830  2314 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:00.144835  2313 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:00.122658  7929 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:05:00.150089  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 57883.7us
I20260709 13:05:00.137583  8215 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:00.155184  1612 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:00.157881  1537 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:00.159436  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.148193  8236 raft_consensus.cc:697] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:00.149602  8301 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:00.173866  8282 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:00.174850  8195 raft_consensus.cc:3055] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:00.175307  8195 raft_consensus.cc:740] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:00.176204  8195 consensus_queue.cc:260] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:00.176609  8037 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:00.180390  8037 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:00.182148  1719 raft_consensus.cc:1240] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 23. Current term is 24. Ops: []
I20260709 13:05:00.149336  7990 raft_consensus.cc:697] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:00.184449  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:00.186494  8236 consensus_queue.cc:237] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:00.154520  7929 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.109935  8077 consensus_queue.cc:237] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:00.165505  8270 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:00.193912  8282 raft_consensus.cc:697] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:00.219444  8139 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:00.220736  8282 consensus_queue.cc:237] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:00.223306  8029 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:00.153053  8327 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:05:00.225090  1685 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:00.226768  8155 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:00.227154  1685 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:00.228317  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 70512.5us
I20260709 13:05:00.229358  1537 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:00.165890  8300 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:05:00.213857  1687 leader_election.cc:336] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.792s (SENT)
W20260709 13:05:00.218432  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 63368.2us
I20260709 13:05:00.211429  7929 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:00.177531  8195 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:00.224123  8004 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:00.231062  1537 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:00.182811  8301 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:00.232579  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 46170.5us
I20260709 13:05:00.239338  1612 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:00.244843  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.257957  1715 raft_consensus.cc:1240] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 22. Current term is 23. Ops: []
I20260709 13:05:00.235656  8037 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:00.238359  8031 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.251207  8301 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:00.244520  8328 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:00.260418  7929 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:00.253539  8300 raft_consensus.cc:697] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:00.254324  8139 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:00.274931  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 43949.1us
I20260709 13:05:00.231875  7990 consensus_queue.cc:237] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:00.273710  1565 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:05:00.282873  8337 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:00.262912  8029 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:00.267017  8031 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:00.291994  1537 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:00.303521  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 64279.4us
I20260709 13:05:00.306974  8300 consensus_queue.cc:237] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:00.309880  8004 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:00.312032  1537 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:00.313755  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.314431  8328 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:00.315331  8328 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:00.313373  1537 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:00.320961  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 66041.4us
I20260709 13:05:00.326072  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:00.335737  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 83064.1us
I20260709 13:05:00.321211  8324 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:05:00.337843  8029 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:00.338750  8155 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:00.339145  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:00.343161  8031 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:00.343782  8031 consensus_queue.cc:1059] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:05:00.344862  8031 raft_consensus.cc:3055] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:05:00.345225  8031 raft_consensus.cc:740] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:00.347095  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:00.347390  7929 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:00.295256  8327 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.348453  8327 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:00.350790  8327 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:00.347885  1565 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 20.
I20260709 13:05:00.350662  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:00.355808  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:00.358232  2312 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:00.359408  8077 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:00.359836  8259 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:00.360464  8259 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:00.360615  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.360862  8259 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:00.375089  1612 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:00.377029  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 51057.8us
I20260709 13:05:00.377915  1537 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:00.380362  1611 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:00.381760  1687 leader_election.cc:336] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.516s (SENT)
I20260709 13:05:00.383635  1612 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:00.385366  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:00.415256  1719 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 20.
I20260709 13:05:00.391589  8282 consensus_queue.cc:1059] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:00.394083  8259 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:00.353802  8327 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.373627  7929 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:00.401579  8077 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:00.388031  7990 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:00.398165  8029 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:05:00.431445  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 48979.3us
I20260709 13:05:00.437513  1612 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:00.428647  8288 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:00.435842  8029 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.426301  8305 consensus_queue.cc:1059] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.024s
W20260709 13:05:00.463106  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.438326  8338 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:00.450960  7990 raft_consensus.cc:697] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 20 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:00.434376  8327 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:00.453570  8077 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:00.448215  8332 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:00.476886  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 96654.5us
I20260709 13:05:00.458256  8283 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.464372  8029 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:00.479882  1611 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:00.485893  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 96399.6us
I20260709 13:05:00.495121  1721 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:00.499293  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:00.508466  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:00.510479  1721 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:05:00.510041  1611 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:00.512185  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:00.512248  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:00.513938  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:00.515241  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:00.516031  2345 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:00.477067  8077 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:00.512657  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:00.481343  8308 raft_consensus.cc:3055] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:05:00.521713  1713 raft_consensus.cc:2393] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 19: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:00.523061  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:00.524323  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:00.524461  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:00.437089  1464 catalog_manager.cc:5697] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 10 to 18. New cstate: current_term: 18 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:00.453248  8031 consensus_queue.cc:260] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:00.537827  2341 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 10.
W20260709 13:05:00.538362  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 100917us
I20260709 13:05:00.536540  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:00.541688  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.542870  1718 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:00.540279  2347 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 20 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:05:00.547745  1722 raft_consensus.cc:2410] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 17 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:05:00.549932  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:00.511931  8288 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:00.480002  8283 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:00.552884  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:00.554145  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:00.527118  8077 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:00.558029  1612 leader_election.cc:400] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 16. Current term is 17.
I20260709 13:05:00.562131  1612 leader_election.cc:403] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:00.562826  2344 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:00.558472  1537 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:00.559288  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:00.559347  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:00.557425  1570 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:05:00.526700  8308 raft_consensus.cc:740] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:00.556125  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:00.563958  1573 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:00.568231  1717 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:00.564713  2345 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:05:00.538619  8338 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:00.565476  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:00.571417  2341 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:05:00.579286  1572 raft_consensus.cc:2410] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:05:00.570960  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:00.570609  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:00.573472  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:00.544672  8270 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.581998  1568 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:00.583022  1570 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:05:00.585032  1571 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 13:05:00.590590  1716 raft_consensus.cc:1275] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:05:00.591485  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:00.592203  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:00.593266  1722 raft_consensus.cc:2410] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 18 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:05:00.594218  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:00.595224  8283 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:00.595419  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:00.596073  1722 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:00.596220  8308 consensus_queue.cc:260] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:00.585620  1571 pending_rounds.cc:85] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 10
I20260709 13:05:00.597836  1571 pending_rounds.cc:107] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:05:00.598119  1571 raft_consensus.cc:2889] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:00.592295  1718 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:00.602658  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 55665.7us
I20260709 13:05:00.603983  1611 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:00.607951  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:00.608220  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:00.608826  1567 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 19: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:00.599151  8288 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 16. Current term is 17.
I20260709 13:05:00.615821  1611 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:00.616729  8139 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:00.618336  8139 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:00.618981  8327 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:00.618907  8139 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:00.621222  8328 raft_consensus.cc:3055] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:00.621678  8328 raft_consensus.cc:740] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:00.622031  8139 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:00.624787  1569 raft_consensus.cc:2410] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:05:00.625905  8318 consensus_queue.cc:1059] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:00.598150  8270 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:00.633895  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:00.636983  8031 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:00.637565  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:00.642930  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:00.643242  8308 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:00.644193  1716 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:05:00.645685  1718 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:00.715725  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:00.646052  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:00.648802  8345 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:00.649351  8339 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:00.648828  8338 consensus_queue.cc:1048] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:00.652381  8146 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:00.648860  1685 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:00.654000  8004 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:00.656278  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:00.658234  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:00.737946  1566 raft_consensus.cc:2368] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 16. Current term is 17.
I20260709 13:05:00.658371  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:00.582558  7990 consensus_queue.cc:237] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:00.659047  8077 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:00.666026  1464 catalog_manager.cc:5697] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:00.604228  8259 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:00.600893  7929 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:05:00.674924  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:00.679616  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:00.763962  1568 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 16: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:00.680912  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:00.764611  1721 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
I20260709 13:05:00.681855  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:00.690935  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 134999us
I20260709 13:05:00.695670  1573 raft_consensus.cc:2410] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:05:00.699509  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.766010  1537 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:00.700409  8037 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:00.701040  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.703931  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:00.704592  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.766697  1573 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:00.767328  1647 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 18: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:00.709158  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.714236  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.716511  1719 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:00.716220  1717 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:05:00.727216  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:00.727406  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.728539  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.770856  1644 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:05:00.732087  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:00.744787  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:00.746120  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:00.755527  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:00.766189  1642 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:05:00.711131  1565 raft_consensus.cc:2410] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:05:00.768567  1640 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:00.776928  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:00.778110  1640 raft_consensus.cc:2376] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17. Re-sending same reply.
I20260709 13:05:00.779098  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:00.779732  1640 raft_consensus.cc:2376] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15. Re-sending same reply.
I20260709 13:05:00.780671  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.783269  1570 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:00.784349  1646 raft_consensus.cc:2410] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:05:00.794772  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:00.797144  1537 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:00.788612  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.798589  1647 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 16: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:00.799432  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:00.800457  1647 raft_consensus.cc:2393] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 16: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:00.803413  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:00.801484  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.808452  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.812577  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.812865  1642 raft_consensus.cc:2410] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 18 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:05:00.810556  1612 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:00.811465  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:00.822790  1645 raft_consensus.cc:2376] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19. Re-sending same reply.
I20260709 13:05:00.844321  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.847244  1641 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:05:00.849483  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.852150  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.676395  8328 consensus_queue.cc:260] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:00.781796  8145 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.853291  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:00.854075  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:00.830205  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.861946  1644 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:05:00.796751  8327 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.869024  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:05:00.870558  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 60089.9us
I20260709 13:05:00.891026  1645 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:05:00.759174  8301 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:00.895088  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 194522us
I20260709 13:05:00.895769  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.896274  1687 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:00.896545  1642 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:05:00.899415  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.874989  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:00.903508  1641 raft_consensus.cc:2376] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15. Re-sending same reply.
I20260709 13:05:00.878142  1719 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 21.
I20260709 13:05:00.907850  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.893023  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:00.871882  1640 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:00.815675  8195 consensus_queue.cc:1059] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 13:05:00.806980  8193 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:00.893325  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.913729  1644 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 19. Current term is 20. Ops: [18.13-19.14]
W20260709 13:05:00.918427  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 117534us
I20260709 13:05:00.845943  8282 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 13:05:00.920773  2314 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:00.887163  8333 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:00.921271  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:00.928574  1571 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
I20260709 13:05:00.929759  2342 raft_consensus.cc:2368] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 19. Current term is 20.
I20260709 13:05:00.930691  1719 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:05:00.922420  1537 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:00.875198  8349 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:00.939623  1642 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:05:00.923768  1538 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:00.922658  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:00.938726  1643 raft_consensus.cc:1240] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:00.941516  1646 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:00.870345  8145 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:00.950659  1643 raft_consensus.cc:1275] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:05:00.952677  1643 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:00.896044  8327 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:00.965895  1642 raft_consensus.cc:1240] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:00.967491  1643 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:05:00.931084  8195 consensus_queue.cc:1243] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer cf2f71592be8465fb18e36e15d0b2f89 log is divergent from this leader: its last log entry 20.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:05:00.968253  8195 raft_consensus.cc:3055] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:00.968817  8195 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:00.969151  1643 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:05:00.929875  8333 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.969784  8195 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:00.970530  8333 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:00.971199  8195 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:00.974614  1646 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:00.939226  8300 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:00.940415  8037 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:00.922060  8351 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:05:00.989183  8037 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:05:01.007887  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 87201.3us
I20260709 13:05:00.995980  8193 raft_consensus.cc:697] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:00.965617  8337 consensus_queue.cc:1059] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:05:01.020583  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:01.030836  1570 raft_consensus.cc:2393] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 16: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:05:01.034811  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 84067.9us
I20260709 13:05:01.016335  8037 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:01.021211  8363 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:01.029582  8145 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:01.034284  8270 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:01.039347  8355 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:05:01.098948  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 869748us
I20260709 13:05:01.019451  8211 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:05:01.077005  8363 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:01.030478  8350 consensus_queue.cc:1059] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.234s
I20260709 13:05:01.126781  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:01.127929  1570 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:05:01.082026  8309 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:05:01.130337  8327 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:01.133133  8029 consensus_meta.cc:287] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.631s	user 0.009s	sys 0.004s
I20260709 13:05:01.112651  8193 consensus_queue.cc:237] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:01.113044  8363 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:01.154748  8357 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:05:01.157155  8328 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:01.162375  8358 log.cc:927] Time spent T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
W20260709 13:05:01.163712  8352 log.cc:927] Time spent T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 13:05:01.117731  8301 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:01.170511  8301 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.121881  8355 raft_consensus.cc:697] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:01.184036  1570 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:05:01.236255  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:01.237201  1640 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:05:01.240424  1715 raft_consensus.cc:3055] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:01.240926  1715 raft_consensus.cc:740] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:05:01.245556  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 321784us
I20260709 13:05:01.248124  2314 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:01.246757  8288 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:01.249398  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 130871us
I20260709 13:05:01.254082  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:01.254915  1640 raft_consensus.cc:2410] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
W20260709 13:05:01.250279  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 283522us
I20260709 13:05:01.255379  1642 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:05:01.251133  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 330735us
I20260709 13:05:01.257387  1687 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:01.259063  1537 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:01.249579  1715 consensus_queue.cc:260] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:01.260432  1722 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:05:01.260850  1722 pending_rounds.cc:85] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 12
I20260709 13:05:01.261086  1722 pending_rounds.cc:107] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:05:01.261334  1722 raft_consensus.cc:2889] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:01.262544  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.262633  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:01.263522  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.263625  1643 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
W20260709 13:05:01.264079  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.264664  1570 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:01.265833  1538 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:01.171331  8301 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:01.143555  8029 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:01.269439  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:01.295846  1537 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:01.295411  1538 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:01.266980  8324 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:01.301565  1687 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:01.309588  1715 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:01.330211  8354 log.cc:927] Time spent T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.242s	user 0.002s	sys 0.000s
W20260709 13:05:01.332558  8364 log.cc:927] Time spent T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
W20260709 13:05:01.338181  8366 log.cc:927] Time spent T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.268s	user 0.000s	sys 0.001s
I20260709 13:05:01.285454  8327 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:05:01.435766  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.267119  8288 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:01.435981  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 168881us
I20260709 13:05:01.443441  1687 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:01.267436  8270 consensus_queue.cc:1048] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:05:01.314790  8324 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.310428  8282 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:01.283773  8221 consensus_queue.cc:1059] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.129s
I20260709 13:05:01.314358  8193 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:01.302983  8139 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.145s
I20260709 13:05:01.476410  1566 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:01.478487  1685 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:01.474946  1687 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:01.322839  8362 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:01.491369  8004 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:01.506747  8383 log.cc:927] Time spent T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:05:01.507201  8377 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:05:01.507016  8004 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:01.509059  8390 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:01.515797  1714 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:05:01.514846  1687 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:01.478554  8324 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:01.486130  8330 raft_consensus.cc:3055] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:05:01.498427  8031 consensus_meta.cc:287] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.782s	user 0.007s	sys 0.007s
I20260709 13:05:01.490751  8193 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.548986  8312 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:05:01.553879  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 258066us
I20260709 13:05:01.567636  2342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55890 (request call id 615) took 1060 ms. Trace:
I20260709 13:05:01.598034  1537 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:01.576282  1464 catalog_manager.cc:5697] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 12 to 16, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 16 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:01.502856  8215 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:01.597755  2342 rpcz_store.cc:276] 0709 13:05:00.507157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.507395 (+   238us) service_pool.cc:224] Handling call
0709 13:05:01.567312 (+1059917us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b54005b1154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:01.567548 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":910580480}
W20260709 13:05:01.586577  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 281938us
I20260709 13:05:01.449247  7990 consensus_queue.cc:1059] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.079s
W20260709 13:05:01.527915  8364 log.cc:927] Time spent T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:05:01.514444  8077 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:01.599462  1537 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:01.665271  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 65871.1us
I20260709 13:05:01.538555  8330 raft_consensus.cc:740] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:01.660414  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:01.598747  8376 log.cc:927] Time spent T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.290s	user 0.001s	sys 0.000s
I20260709 13:05:01.666409  1537 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:01.592712  8308 consensus_meta.cc:287] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.861s	user 0.014s	sys 0.000s
I20260709 13:05:01.483731  8387 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.156s
I20260709 13:05:01.593137  8193 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:01.700174  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:01.701481  1537 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:01.720149  1568 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:01.702905  1611 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:01.655959  8333 consensus_meta.cc:287] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.685s	user 0.014s	sys 0.000s
I20260709 13:05:01.627728  8351 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:01.596163  8391 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:01.615643  8004 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:01.624074  8221 raft_consensus.cc:3055] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:01.474642  8355 consensus_queue.cc:237] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:01.710477  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:05:01.714041  8365 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.348s	user 0.003s	sys 0.007s
I20260709 13:05:01.616539  8339 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.115s
I20260709 13:05:01.728796  1611 leader_election.cc:400] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 16. Current term is 17.
I20260709 13:05:01.717935  8139 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:01.714232  8215 raft_consensus.cc:697] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:01.719234  8155 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:05:01.752444  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 51052.7us
I20260709 13:05:01.729773  8380 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.732127  8351 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.720526  8388 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:05:01.754045  8372 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.304s	user 0.000s	sys 0.002s
I20260709 13:05:01.729758  8389 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:01.732127  8221 raft_consensus.cc:740] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:01.758021  1611 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:01.731974  8333 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:01.714367  8327 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:05:01.752987  8029 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:01.778589  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:01.774832  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 61570.3us
I20260709 13:05:01.779856  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:01.779675  1538 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:01.763598  8215 consensus_queue.cc:237] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:01.798085  8288 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:01.798388  1722 raft_consensus.cc:1275] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:05:01.802487  8221 consensus_queue.cc:260] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:01.807399  8382 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:01.765884  8351 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:01.809865  8392 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:05:01.764459  8381 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:01.771099  8388 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.815083  8388 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:01.765300  8155 raft_consensus.cc:697] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:01.786209  8139 raft_consensus.cc:697] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:01.820209  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3332) took 1073 ms. Trace:
W20260709 13:05:01.824801  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 45222.9us
W20260709 13:05:01.822311  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 64385.1us
I20260709 13:05:01.831359  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:01.824326  1644 rpcz_store.cc:276] 0709 13:05:00.746551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.928794 (+182243us) service_pool.cc:224] Handling call
0709 13:05:00.938193 (+  9399us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:00.938647 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:00.938679 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:00.950472 (+ 11793us) log.cc:844] Serialized 52 byte log entry
0709 13:05:01.111606 (+161134us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:01.111676 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:01.111808 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:01.819919 (+708111us) raft_consensus.cc:1685] finished
0709 13:05:01.819944 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:01.820138 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":659,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":34086,"thread_start_us":111776,"threads_started":1,"wal-append.queue_time_us":308951}
I20260709 13:05:01.855974  1716 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 20. Current term is 21. Ops: []
W20260709 13:05:01.856715  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 414825us
I20260709 13:05:01.854341  1718 raft_consensus.cc:3055] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:01.858946  1718 raft_consensus.cc:740] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:01.855161  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:01.860674  1721 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:05:01.856491  1715 raft_consensus.cc:1275] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:05:01.762168  8390 raft_consensus.cc:697] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:01.865072  1722 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
W20260709 13:05:01.847536  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 332648us
I20260709 13:05:01.859740  1718 consensus_queue.cc:260] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:01.869242  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:01.870599  1715 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:01.866102  1611 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:01.805476  8221 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:01.818524  8155 consensus_queue.cc:237] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:01.883160  1718 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:01.882943  1687 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:01.847018  8335 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:01.864168  8349 consensus_meta.cc:287] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.908s	user 0.014s	sys 0.000s
I20260709 13:05:01.903066  1611 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:01.823531  8381 raft_consensus.cc:697] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:05:01.899783  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 370849us
I20260709 13:05:01.912896  1611 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:01.879928  8395 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:05:01.919533  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.920941  8345 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:01.755111  8330 consensus_queue.cc:260] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:01.904906  8332 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:01.894902  8288 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.908862  1687 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:01.927858  1715 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 21.
I20260709 13:05:01.768600  8380 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:01.939749  1718 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:01.940481  8405 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:01.918973  8004 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:01.915470  8301 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:01.917711  8396 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:01.962733  1685 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:01.925711  8349 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:05:01.929280  8195 consensus_meta.cc:287] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.958s	user 0.014s	sys 0.002s
I20260709 13:05:01.923758  8395 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.942468  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2810) took 1016 ms. Trace:
I20260709 13:05:01.941427  8389 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.942225  8335 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.969672  8004 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:01.941401  8385 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:05:01.941596  8236 consensus_queue.cc:1059] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:05:01.944352  8327 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.958496  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3331) took 1218 ms. Trace:
I20260709 13:05:01.996505  1648 rpcz_store.cc:276] 0709 13:05:00.739915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.926334 (+186419us) service_pool.cc:224] Handling call
0709 13:05:00.926739 (+   405us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:00.942670 (+ 15931us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:00.996557 (+ 53887us) log.cc:844] Serialized 14 byte log entry
0709 13:05:01.150359 (+153802us) log.cc:844] Serialized 14 byte log entry
0709 13:05:01.174461 (+ 24102us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:01.247214 (+ 72753us) log.cc:844] Serialized 52 byte log entry
0709 13:05:01.248856 (+  1642us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:01.249762 (+   906us) log.cc:844] Serialized 14 byte log entry
0709 13:05:01.250007 (+   245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:01.250105 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:01.815864 (+565759us) spinlock_profiling.cc:243] Waited 9.97 ms on lock 0x7b18003825a0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:05:01.958179 (+142315us) raft_consensus.cc:1685] finished
0709 13:05:01.958210 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:01.958412 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55954,"prepare.queue_time_us":49311,"prepare.run_cpu_time_us":187,"prepare.run_wall_time_us":5586,"raft.queue_time_us":144773,"raft.run_cpu_time_us":524,"raft.run_wall_time_us":175593,"spinlock_wait_cycles":70452864,"thread_start_us":90815,"threads_started":1,"wal-append.queue_time_us":143994}
I20260709 13:05:01.930348  8381 consensus_queue.cc:237] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:01.960225  8037 consensus_meta.cc:287] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.858s	user 0.015s	sys 0.002s
I20260709 13:05:02.006873  8037 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:01.962996  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:05:02.010679  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.964968  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:02.012410  1714 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 21: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:01.913971  8077 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:01.972532  1571 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.010s	sys 0.011s
W20260709 13:05:02.016253  1571 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.011s	sys 0.012s
I20260709 13:05:01.973510  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:05:01.978514  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 69773.8us
I20260709 13:05:01.976982  1722 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:05:01.985446  1565 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 13:05:02.019584  1565 pending_rounds.cc:85] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 13
I20260709 13:05:01.974653  1567 rpcz_store.cc:276] 0709 13:05:00.926115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.926591 (+   476us) service_pool.cc:224] Handling call
0709 13:05:00.938156 (+ 11565us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:00.938367 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:01.546151 (+607784us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b180000fa60. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f824c87d3c7 00007f8248e8f9a7 00007f8248e9f0bf
0709 13:05:01.546396 (+   245us) log.cc:844] Serialized 14 byte log entry
0709 13:05:01.942057 (+395661us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:01.942114 (+    57us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:01.942167 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:01.942270 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:01.942417 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59145,"prepare.queue_time_us":10599,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":1604,"raft.queue_time_us":49944,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":5629,"spinlock_wait_cycles":25407232,"thread_start_us":365074,"threads_started":2,"wal-append.queue_time_us":386044}
I20260709 13:05:01.981726  8004 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:02.019611  1687 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:02.034188  1565 pending_rounds.cc:107] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 13:05:02.034763  1565 raft_consensus.cc:2889] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:02.034942  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 122110us
I20260709 13:05:02.035818  1611 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:02.037110  1611 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:01.896301  8390 consensus_queue.cc:237] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:01.976909  8395 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:01.981426  8335 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:01.983566  8396 raft_consensus.cc:697] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:01.952188  8288 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:02.054296  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:02.054996  1643 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16.
I20260709 13:05:02.031965  8077 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.046379  1611 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:02.036012  8349 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.047865  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:02.058295  1648 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:02.059549  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:01.936741  8139 consensus_queue.cc:237] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.060209  1718 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:02.035830  8195 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.074800  1568 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 20.
I20260709 13:05:02.034451  8402 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:02.042469  8405 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:02.044968  8333 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:02.040166  8345 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:02.027331  8327 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:02.086373  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 43987.8us
I20260709 13:05:01.991071  8389 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.082799  8355 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:05:02.144892  8399 log.cc:927] Time spent T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.208s	user 0.001s	sys 0.001s
W20260709 13:05:02.145702  8397 log.cc:927] Time spent T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.222s	user 0.002s	sys 0.000s
I20260709 13:05:02.087564  8077 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:02.147689  8403 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:02.148181  8404 consensus_queue.cc:1059] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.175s
I20260709 13:05:02.150786  8380 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:02.154086  8396 consensus_queue.cc:237] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.156713  8327 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:02.091488  8384 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:02.095582  8155 consensus_queue.cc:1059] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:05:02.060839  8324 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:02.088515  8402 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:02.149310  8288 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:02.217599  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:02.240764  1715 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:02.225464  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:02.225610  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2817) took 1073 ms. Trace:
I20260709 13:05:02.226909  1720 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:02.081966  8349 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.091817  8195 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.229060  1537 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:02.234609  1648 raft_consensus.cc:2410] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:05:02.211550  8345 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:02.241493  1571 rpcz_store.cc:276] 0709 13:05:01.152665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.177876 (+ 25211us) service_pool.cc:224] Handling call
0709 13:05:01.178377 (+   501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:01.178664 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:01.266132 (+ 87468us) log.cc:844] Serialized 14 byte log entry
0709 13:05:02.036152 (+770020us) log.cc:844] Serialized 14 byte log entry
0709 13:05:02.036425 (+   273us) log.cc:844] Serialized 14 byte log entry
0709 13:05:02.219741 (+183316us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:02.220907 (+  1166us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:02.220968 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.225331 (+  4363us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:02.225529 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18959,"prepare.queue_time_us":78876,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":42857,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":17189,"spinlock_wait_cycles":40099584,"thread_start_us":921642,"threads_started":2,"wal-append.queue_time_us":804167}
W20260709 13:05:02.236971  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 190488us
W20260709 13:05:02.237370  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 274551us
I20260709 13:05:02.243374  1718 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 20.
I20260709 13:05:02.250789  1611 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:02.223337  8037 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:02.221825  7990 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:02.228114  8355 raft_consensus.cc:697] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:02.255905  2312 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:02.228410  8387 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:02.261173  1611 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:02.266301  1720 raft_consensus.cc:3055] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:02.309176  1720 raft_consensus.cc:740] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:02.309988  1685 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:02.265658  1715 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 20.
W20260709 13:05:02.300274  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 45743us
I20260709 13:05:02.241688  8402 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:02.259168  8416 raft_consensus.cc:3055] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:02.259519  8380 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:02.321785  1537 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:02.266685  8327 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:02.316524  1720 consensus_queue.cc:260] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.347365  8221 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:02.347966  8221 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:02.348843  8221 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:02.349210  8413 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:05:02.347431  8419 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:05:02.322918  8416 raft_consensus.cc:740] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:02.354269  1720 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:02.239818  8384 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.311189  8211 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:05:02.315052  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 53897.4us
W20260709 13:05:02.406780  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:02.415761  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 105774us
I20260709 13:05:02.418026  1611 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:02.325426  8399 log.cc:927] Time spent T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:05:02.361433  8193 consensus_meta.cc:287] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.631s	user 0.011s	sys 0.003s
W20260709 13:05:02.331008  8328 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.173s	user 0.014s	sys 0.012s
W20260709 13:05:02.437130  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 377193us
I20260709 13:05:02.359498  8424 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:02.439316  1685 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:02.320772  8335 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:02.354950  8077 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:02.395311  8300 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.446866  8416 consensus_queue.cc:260] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.448342  8004 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.511281  8416 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:02.511830  8004 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:02.449417  8411 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:05:02.513890  1714 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:02.450588  8414 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:05:02.513408  8300 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:02.450203  8422 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:02.515444  8395 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.359534  8349 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:02.322012  8345 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:02.455895  8330 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:02.490875  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 47708.4us
I20260709 13:05:02.517747  8335 consensus_queue.cc:1059] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:05:02.494114  8373 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.855s	user 0.001s	sys 0.022s
W20260709 13:05:02.518925  8373 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.988s	user 0.001s	sys 0.022s
I20260709 13:05:02.451203  8418 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:02.519717  8418 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:02.496356  8351 consensus_meta.cc:287] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.686s	user 0.013s	sys 0.001s
I20260709 13:05:02.520056  8215 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:02.452486  8355 consensus_queue.cc:237] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.521085  8351 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.523016  8324 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:02.525447  8351 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:02.439158  8195 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:02.525507  8077 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:02.532441  8077 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.538120  8415 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.538776  8415 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:02.540711  8146 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
W20260709 13:05:02.540723  8363 consensus_meta.cc:287] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.298s	user 0.016s	sys 0.004s
I20260709 13:05:02.304776  8387 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.543987  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2819) took 1358 ms. Trace:
W20260709 13:05:02.544330  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.580399  1569 rpcz_store.cc:276] 0709 13:05:01.186061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.186325 (+   264us) service_pool.cc:224] Handling call
0709 13:05:01.186769 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:01.187248 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:01.187289 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:01.258906 (+ 71617us) log.cc:844] Serialized 26 byte log entry
0709 13:05:01.741264 (+482358us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:01.847004 (+105740us) log.cc:844] Serialized 14 byte log entry
0709 13:05:01.847272 (+   268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:01.847366 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:02.543773 (+696407us) raft_consensus.cc:1685] finished
0709 13:05:02.543798 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:02.543928 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61323,"prepare.queue_time_us":45091,"prepare.run_cpu_time_us":136,"prepare.run_wall_time_us":135,"raft.queue_time_us":11647,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":35104,"thread_start_us":307838,"threads_started":1,"wal-append.queue_time_us":587748}
I20260709 13:05:02.542842  8300 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:02.443627  8211 raft_consensus.cc:697] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:02.455816  8193 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.575332  8221 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.591632  8077 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:02.594061  8388 consensus_meta.cc:287] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.778s	user 0.015s	sys 0.012s
I20260709 13:05:02.595145  8335 raft_consensus.cc:3055] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:05:02.595813  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 77147.6us
I20260709 13:05:02.609305  1612 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:02.596745  8405 consensus_meta.cc:287] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.000s
I20260709 13:05:02.520146  8418 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:02.543445  8384 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:02.547010  1572 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:05:02.576815  8309 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:02.604957  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 283549us
W20260709 13:05:02.605580  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 167239us
I20260709 13:05:02.626559  1685 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:02.614225  1612 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:02.614835  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 174449us
I20260709 13:05:02.616950  1720 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 8. (term mismatch)
I20260709 13:05:02.642657  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:02.643705  2342 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:05:02.630383  1537 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:02.607439  8335 raft_consensus.cc:740] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:02.617810  8405 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:02.649546  2344 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:05:02.675573  8335 consensus_queue.cc:260] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.677011  8335 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:02.652674  8368 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.658205  8424 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:02.582086  8363 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.647430  8421 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:02.647071  8396 consensus_queue.cc:1059] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.013s
W20260709 13:05:02.663748  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.664234  8439 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:02.665050  8415 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:02.665656  8389 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:02.623885  8282 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.701498  8282 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:02.667647  8221 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:02.635178  8309 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:02.705148  8309 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:02.672504  8350 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:02.706382  8350 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:02.707152  8396 raft_consensus.cc:3055] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:02.707707  8396 raft_consensus.cc:740] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:02.708312  8029 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:02.678226  8368 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:02.708974  8029 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.710923  8368 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:02.615018  8077 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:02.650269  2345 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:02.693137  8421 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:02.680765  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:02.681728  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:02.636427  1720 pending_rounds.cc:85] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 7
W20260709 13:05:02.689076  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:02.717757  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.715390  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:02.718529  2344 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 24.
W20260709 13:05:02.692546  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 56570.8us
I20260709 13:05:02.714381  8029 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:02.720188  8363 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:02.693426  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 64874.5us
I20260709 13:05:02.726397  8193 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:02.727206  8435 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.727955  8435 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:02.730137  8333 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:02.730371  8345 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:02.731695  8288 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:02.731431  8345 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:02.731631  8371 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:02.733292  8371 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.735667  8371 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:02.732291  8288 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:02.735867  8438 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:02.615195  8388 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.730834  8333 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:02.736543  8438 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:02.737934  8288 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:02.738541  8388 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:02.653177  8384 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:02.743471  8384 raft_consensus.cc:697] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 20 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:02.744086  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:02.744896  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.744804  8384 consensus_queue.cc:237] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.679123  8211 consensus_queue.cc:237] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:02.742062  1717 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: [13.9-15.10]
I20260709 13:05:02.752995  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 1130) took 1268 ms. Trace:
I20260709 13:05:02.760511  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:02.761581  1715 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:05:02.763886  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:02.717195  1720 pending_rounds.cc:107] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 16.8
I20260709 13:05:02.767055  1720 raft_consensus.cc:2889] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:02.741351  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:02.770761  1715 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:02.760555  1565 rpcz_store.cc:276] 0709 13:05:01.484463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.484714 (+   251us) service_pool.cc:224] Handling call
0709 13:05:01.485187 (+   473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.035503 (+550316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.752641 (+717138us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b54001b5b54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:02.752908 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1149042432}
I20260709 13:05:02.775056  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:02.775703  1722 raft_consensus.cc:2410] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 19 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:05:02.775704  1569 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:02.776782  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:02.778505  1645 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:05:02.784226  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:02.784891  1648 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:02.786011  1569 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 20.
W20260709 13:05:02.788859  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 155676us
I20260709 13:05:02.790673  1685 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:02.793403  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:02.794596  1642 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 9.
I20260709 13:05:02.796232  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:02.797106  1719 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:02.729050  8387 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:02.801367  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:02.716346  8421 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:02.804585  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:02.805509  1568 raft_consensus.cc:2410] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:05:02.745839  8288 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.708875  8350 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:02.766033  8389 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.836511  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:02.822566  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:02.804559  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:02.747963  8097 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:02.738607  8333 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:02.825446  8288 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.826535  8360 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:02.715387  8439 raft_consensus.cc:697] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 21 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:05:02.846271  8398 log.cc:927] Time spent T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 1.129s	user 0.001s	sys 0.004s
W20260709 13:05:02.819537  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 49584.7us
I20260709 13:05:02.847431  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:02.739156  8438 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:02.828842  1568 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:05:02.829931  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:02.746968  8300 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:02.832131  8409 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.747598  8345 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:02.835175  1639 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:02.837232  1570 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 22.
I20260709 13:05:02.838389  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:02.795282  8029 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:02.708931  8396 consensus_queue.cc:260] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:02.827571  1685 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:02.849982  1569 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 18: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:02.855347  1573 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 23.
I20260709 13:05:02.899458  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:02.900776  1571 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:02.864602  1648 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16.
I20260709 13:05:02.865132  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:02.857343  2312 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:02.906976  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:02.887147  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:02.908138  1643 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:02.845263  8439 consensus_queue.cc:237] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.921913  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:02.923348  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2822) took 1649 ms. Trace:
I20260709 13:05:02.923712  1566 rpcz_store.cc:276] 0709 13:05:01.274479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.274626 (+   147us) service_pool.cc:224] Handling call
0709 13:05:01.297366 (+ 22740us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:01.498000 (+200634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.923089 (+1425089us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54002256d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:02.923290 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":411070848}
I20260709 13:05:02.925576  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:02.926249  1573 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:05:02.931640  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:02.828969  8312 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 13:05:02.922605  1567 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
W20260709 13:05:02.955267  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 98023.8us
W20260709 13:05:02.957280  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 43591.2us
W20260709 13:05:02.992189  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 277252us
I20260709 13:05:02.994738  1537 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:02.842497  8097 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:02.840477  8389 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:03.004405  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 176872us
I20260709 13:05:03.007632  1685 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:03.007985  1643 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:05:02.966122  8360 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:02.903736  8300 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:03.024885  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:02.986858  8309 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:02.904253  8029 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:03.025722  8328 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:03.024925  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:03.030524  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:02.984624  8438 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.029179  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:02.990335  8363 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:02.959530  8333 consensus_queue.cc:1059] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:05:02.886482  8409 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:03.035418  8309 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.031080  8435 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:03.107394  8288 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:03.108835  8097 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:03.111362  8418 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:03.113011  1644 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260709 13:05:03.114390  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 106821us
I20260709 13:05:03.115389  1685 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:03.038622  8419 consensus_queue.cc:1059] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 13:05:03.119688  1612 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:03.119920  8421 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:03.121373  1718 raft_consensus.cc:1240] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:03.123090  8435 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:03.062773  8438 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:03.126456  8438 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:03.044467  8328 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.133471  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3185) took 1279 ms. Trace:
I20260709 13:05:03.208678  1717 rpcz_store.cc:276] 0709 13:05:01.854336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.855544 (+  1208us) service_pool.cc:224] Handling call
0709 13:05:01.855893 (+   349us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:02.759802 (+903909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.133075 (+373273us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b5400249354. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:03.133383 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2036656896}
I20260709 13:05:03.138525  1647 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
W20260709 13:05:03.106298  8402 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.760s	user 0.012s	sys 0.001s
I20260709 13:05:03.115563  8409 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:03.130215  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:05:03.186079  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 66517.3us
I20260709 13:05:03.218784  1646 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:03.038026  8351 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.096s
I20260709 13:05:03.074365  8363 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:03.085696  8437 log.cc:927] Time spent T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:05:03.224498  8363 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:03.096511  8389 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:03.063537  8388 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:03.219401  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:03.096876  8300 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:03.226562  1612 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:03.229842  1612 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:03.221256  1685 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:03.129455  8029 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:03.162993  8309 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:03.240036  1685 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:03.283769  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 54026.2us
I20260709 13:05:03.207376  8390 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:03.212899  8328 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:03.298161  1612 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:05:03.233126  8416 consensus_meta.cc:287] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.721s	user 0.014s	sys 0.000s
W20260709 13:05:03.310552  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 70615.1us
I20260709 13:05:03.238250  7990 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.311748  1685 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:03.245741  8388 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:03.241591  8439 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:03.312942  8435 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:03.240339  8409 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:03.330318  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:03.415076  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 98417.1us
I20260709 13:05:03.299367  8333 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:03.331998  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 366528us
I20260709 13:05:03.427904  1687 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:03.238402  8371 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:03.304536  8390 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:03.335753  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:03.331570  2314 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:03.330382  8419 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.420224  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:03.477178  1715 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:05:03.421542  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:03.313038  8351 raft_consensus.cc:3055] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:03.321352  8029 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.536967  8029 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:03.537292  8288 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.321671  8388 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:03.327004  8439 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:03.332346  8435 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:03.313023  8360 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:03.478747  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 147210us
I20260709 13:05:03.591004  2314 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:03.218091  8402 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:03.413733  8416 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:03.599562  8416 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:03.600075  8416 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:03.411289  8345 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.518922  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:03.602070  8345 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:03.603329  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:03.604403  1645 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:05:03.605049  8345 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:03.519039  8418 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:03.326393  7990 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:03.535307  8404 raft_consensus.cc:3055] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:03.536175  8351 raft_consensus.cc:740] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:03.537971  8419 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:03.462173  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:03.543531  8221 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:03.580561  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:03.631973  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:03.539935  8029 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:03.463367  8371 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:03.474927  8390 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:03.587030  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:03.641794  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 56211.9us
I20260709 13:05:03.540032  8421 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:03.584156  8439 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:03.598913  8300 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:03.605530  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:03.551424  1646 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:05:03.582280  8388 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:03.610384  8405 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.622961  8404 raft_consensus.cc:740] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:03.589253  8435 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:03.644454  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:03.589936  8438 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:03.583606  8288 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:03.652092  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:03.635804  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:03.640558  8371 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:03.643594  8221 raft_consensus.cc:697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:03.709527  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:03.719412  1645 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
W20260709 13:05:03.671595  8195 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.015s	sys 0.043s
W20260709 13:05:03.673749  8350 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.026s	sys 0.032s
I20260709 13:05:03.649077  8029 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:03.706612  8300 raft_consensus.cc:697] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:03.788390  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:03.713485  8438 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:03.724713  8195 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.016s	sys 0.044s
W20260709 13:05:03.766251  8350 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.027s	sys 0.032s
I20260709 13:05:03.820292  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:03.706041  8405 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:03.828409  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 370043us
W20260709 13:05:03.810047  8335 consensus_meta.cc:287] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.133s	user 0.011s	sys 0.007s
I20260709 13:05:03.826059  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:03.773166  8236 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:03.766752  8384 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:03.706073  8351 consensus_queue.cc:260] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:03.825109  8029 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.848968  1645 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:03.849910  8335 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:03.850687  8335 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:03.831830  8211 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:03.857621  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:05:03.846208  8301 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.011s	sys 0.020s
W20260709 13:05:03.843029  8461 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:05:03.882537  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:03.884056  1646 raft_consensus.cc:2410] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 18 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:05:03.880599  8300 consensus_queue.cc:237] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:03.890455  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:03.894132  1720 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 24.
I20260709 13:05:03.891597  1645 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
W20260709 13:05:03.891674  1685 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.014s	sys 0.038s
W20260709 13:05:03.880286  8301 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.012s	sys 0.021s
W20260709 13:05:03.896615  1685 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.015s	sys 0.038s
I20260709 13:05:03.832082  8404 consensus_queue.cc:260] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:03.842660  8236 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:03.863695  8029 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:03.916127  1646 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:05:03.927126  1641 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:05:03.937301  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:03.846213  8384 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:03.910734  8309 consensus_meta.cc:287] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.589s	user 0.006s	sys 0.006s
I20260709 13:05:03.941497  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2869) took 1141 ms. Trace:
I20260709 13:05:03.948642  1646 rpcz_store.cc:276] 0709 13:05:02.800229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.893447 (+1093218us) service_pool.cc:224] Handling call
0709 13:05:03.912940 (+ 19493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:03.941159 (+ 28219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.941423 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:03.916939  8460 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.115s	user 0.000s	sys 0.002s
I20260709 13:05:03.941522  1720 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 23.
I20260709 13:05:03.897120  8221 consensus_queue.cc:237] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:04.052392  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 740617us
I20260709 13:05:04.053725  1685 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:04.078070  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:04.079559  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:04.080206  1720 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16.
I20260709 13:05:04.080336  8438 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:04.086272  8458 log.cc:927] Time spent T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
W20260709 13:05:04.087677  8457 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 13:05:04.089378  8465 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:05:04.090119  8029 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:04.092229  8405 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:03.951249  8309 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:04.093787  7990 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:04.094997  8309 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 24 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:03.947278  8402 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:04.104416  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 50769.2us
W20260709 13:05:04.140650  8455 log.cc:927] Time spent T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.431s	user 0.000s	sys 0.002s
W20260709 13:05:04.238566  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 133074us
I20260709 13:05:04.239740  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:04.142652  8402 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:04.241385  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:04.141832  8404 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:04.237084  8350 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.249215  8335 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:04.294740  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 55078.2us
I20260709 13:05:04.257503  8384 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:04.318575  2313 connection.cc:256] RPC call timeout handler was delayed by -1.8378810039999962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
I20260709 13:05:04.324957  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3348) took 1526 ms. Trace:
I20260709 13:05:04.325522  1642 rpcz_store.cc:276] 0709 13:05:02.798634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.842245 (+ 43611us) service_pool.cc:224] Handling call
0709 13:05:02.842729 (+   484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:02.899020 (+ 56291us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:02.899058 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:02.899904 (+   846us) log.cc:844] Serialized 26 byte log entry
0709 13:05:03.232280 (+332376us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:03.232318 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.232396 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:04.324704 (+1092308us) raft_consensus.cc:1685] finished
0709 13:05:04.324732 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:04.324892 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72632,"raft.queue_time_us":58656,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":34995,"thread_start_us":221893,"threads_started":1,"wal-append.queue_time_us":656963}
I20260709 13:05:04.330147  8236 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:04.287303  8405 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:04.329283  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:04.334702  1642 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:04.297314  8470 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.301015  8438 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.303411  8402 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:04.350296  8473 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:04.351279  8473 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:04.351925  2312 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:04.354550  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2835) took 1487 ms. Trace:
W20260709 13:05:04.355213  8463 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.217s	user 0.000s	sys 0.002s
I20260709 13:05:04.355504  8282 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:04.357270  1565 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2823) took 1583 ms (client timeout 1821 ms). Trace:
I20260709 13:05:04.289994  8350 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.359491  1646 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:04.370208  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3349) took 1530 ms. Trace:
I20260709 13:05:04.370764  1646 rpcz_store.cc:276] 0709 13:05:02.849984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.358936 (+1508952us) service_pool.cc:224] Handling call
0709 13:05:04.359299 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.369784 (+ 10485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.370117 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:04.364154  1565 rpcz_store.cc:269] 0709 13:05:02.774310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.776531 (+  2221us) service_pool.cc:224] Handling call
0709 13:05:04.357068 (+1580537us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b5400012554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:04.357211 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":616708480}
I20260709 13:05:04.354997  1564 rpcz_store.cc:276] 0709 13:05:02.867287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.867518 (+   231us) service_pool.cc:224] Handling call
0709 13:05:02.923981 (+ 56463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:03.036455 (+112474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:03.036496 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:03.036548 (+    52us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:03.036588 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.354332 (+1317744us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b5400406fd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:04.354374 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:04.354490 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257938176}
W20260709 13:05:04.362871  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.374711  1686 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:04.331656  8472 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:04.363926  8470 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.390224  8301 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.390904  8301 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:04.391683  8470 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:04.392663  8470 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:04.374293  8221 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:05:04.385251  8439 consensus_meta.cc:287] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.732s	user 0.011s	sys 0.001s
I20260709 13:05:04.393971  8221 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.394119  8439 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:04.396826  8439 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 24 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:04.393339  8300 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:05:04.393242  8470 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.404042  8470 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:04.386502  8388 consensus_meta.cc:287] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.733s	user 0.012s	sys 0.000s
I20260709 13:05:04.389329  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:04.384863  8309 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:05:04.407277  8450 log.cc:927] Time spent T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:05:04.363543  8438 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.389938  8472 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.411149  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2831) took 1644 ms. Trace:
I20260709 13:05:04.416214  1718 rpcz_store.cc:276] 0709 13:05:02.766375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.782813 (+ 16438us) service_pool.cc:224] Handling call
0709 13:05:02.783135 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:03.121968 (+338833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.410595 (+1288627us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b54003002d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:04.410774 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":776898944}
I20260709 13:05:04.417176  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2871) took 1576 ms. Trace:
I20260709 13:05:04.417894  1646 rpcz_store.cc:276] 0709 13:05:02.841241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.372119 (+1530878us) service_pool.cc:224] Handling call
0709 13:05:04.416587 (+ 44468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.416827 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:04.416852 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:04.416896 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:04.416936 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.416991 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:04.417103 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:04.385547  8402 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:04.338624  8236 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.397266  8221 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:04.411696  8416 consensus_meta.cc:287] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.811s	user 0.012s	sys 0.001s
I20260709 13:05:04.419142  8388 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.441860  1646 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:05:04.429602  8435 consensus_meta.cc:287] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.716s	user 0.009s	sys 0.004s
I20260709 13:05:04.416145  8309 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.431699  8472 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:04.432240  8350 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:04.433907  8438 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:04.434239  8328 consensus_meta.cc:287] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.021s	user 0.015s	sys 0.000s
I20260709 13:05:04.437096  8385 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.449929  8385 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.452628  8385 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:04.435719  8288 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:04.437855  8335 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:04.454036  8335 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.455070  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:04.455297  8301 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:04.345068  8405 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:04.456367  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:04.456992  8335 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:04.440531  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 3361) took 1431 ms. Trace:
I20260709 13:05:04.458232  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2864) took 1675 ms. Trace:
I20260709 13:05:04.460670  1645 rpcz_store.cc:276] 0709 13:05:03.009369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.819899 (+810530us) service_pool.cc:224] Handling call
0709 13:05:04.440255 (+620356us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b5400099554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:04.440440 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55472384}
I20260709 13:05:04.460891  1644 rpcz_store.cc:276] 0709 13:05:02.782625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.785758 (+  3133us) service_pool.cc:224] Handling call
0709 13:05:02.787754 (+  1996us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:03.113763 (+326009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.457939 (+1344176us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54002ab7d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:04.458164 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":747101440}
I20260709 13:05:04.462867  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:04.463300  1644 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:04.458832  1717 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:04.440244  8236 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.438436  8420 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:04.442615  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2872) took 1601 ms. Trace:
I20260709 13:05:04.475741  1646 rpcz_store.cc:276] 0709 13:05:02.841544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.421118 (+1579574us) service_pool.cc:224] Handling call
0709 13:05:04.441483 (+ 20365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.442344 (+   861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.442558 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:04.463989  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3350) took 1613 ms. Trace:
I20260709 13:05:04.476666  1644 rpcz_store.cc:276] 0709 13:05:02.850395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.462545 (+1612150us) service_pool.cc:224] Handling call
0709 13:05:04.462974 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.463754 (+   780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.463935 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4864}
I20260709 13:05:04.459926  8388 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:04.457515  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:04.444952  8390 consensus_meta.cc:287] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.748s	user 0.014s	sys 0.000s
I20260709 13:05:04.482324  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:04.484104  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 706) took 2432 ms. Trace:
I20260709 13:05:04.484862  1644 raft_consensus.cc:2410] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 18 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:05:04.443449  8309 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:04.484517  1640 rpcz_store.cc:276] 0709 13:05:02.051487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.051604 (+   117us) service_pool.cc:224] Handling call
0709 13:05:02.051902 (+   298us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:02.229932 (+178030us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:02.229970 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:02.237853 (+  7883us) log.cc:844] Serialized 52 byte log entry
0709 13:05:02.466355 (+228502us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:02.466430 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.481028 (+2014598us) spinlock_profiling.cc:243] Waited 40.4 ms on lock 0x7b54000cf3d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:04.481102 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:04.481156 (+    54us) raft_consensus.cc:1685] finished
0709 13:05:04.481171 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:04.484029 (+  2858us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1848,"raft.queue_time_us":140857,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":6718,"spinlock_wait_cycles":93008768,"thread_start_us":285615,"threads_started":2,"wal-append.queue_time_us":367610,"wal-append.run_cpu_time_us":2728,"wal-append.run_wall_time_us":1062634}
I20260709 13:05:04.481459  8420 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.457836  8328 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:04.495687  8420 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:04.495816  8328 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:04.457677  8405 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:04.496590  8338 raft_consensus.cc:3055] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:04.497079  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:04.497602  8400 raft_consensus.cc:3055] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:04.498093  8400 raft_consensus.cc:740] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:04.498278  1644 raft_consensus.cc:2410] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:05:04.499012  8400 consensus_queue.cc:260] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.500160  8400 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:04.503187  8419 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:04.510499  1639 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:04.511771  8473 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:04.512647  1572 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2804) took 3764 ms (client timeout 1511 ms). Trace:
W20260709 13:05:04.513274  1572 rpcz_store.cc:269] 0709 13:05:00.748037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.764551 (+ 16514us) service_pool.cc:224] Handling call
0709 13:05:04.512400 (+3747849us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b5400225bd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:04.512555 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3819361664}
W20260709 13:05:04.515398  1641 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2868) took 1715 ms (client timeout 1903 ms). Trace:
W20260709 13:05:04.515878  1641 rpcz_store.cc:269] 0709 13:05:02.799865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.020255 (+220390us) service_pool.cc:224] Handling call
0709 13:05:04.515182 (+1494927us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b54001f2f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:04.515315 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1847230336}
I20260709 13:05:04.520190  1646 raft_consensus.cc:1240] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:04.520932  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3352) took 1634 ms. Trace:
W20260709 13:05:04.521555  8449 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 1.629s	user 0.003s	sys 0.023s
I20260709 13:05:04.493232  8236 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:04.525806  8211 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:04.521318  1646 rpcz_store.cc:276] 0709 13:05:02.886497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.477882 (+1591385us) service_pool.cc:224] Handling call
0709 13:05:04.519957 (+ 42075us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.520661 (+   704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.520858 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:04.526386  8211 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:04.527040  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:04.527680  1646 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:05:04.528674  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:04.529384  1646 raft_consensus.cc:2393] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 18: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:04.529923  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2831) took 1669 ms. Trace:
I20260709 13:05:04.530392  1570 rpcz_store.cc:276] 0709 13:05:02.860650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.861376 (+   726us) service_pool.cc:224] Handling call
0709 13:05:02.861811 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:03.048169 (+186358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:03.048210 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:03.048250 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:03.048286 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.529673 (+1481387us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b5400307354. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:04.529727 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:04.529862 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":427983104}
I20260709 13:05:04.531883  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3191) took 2308 ms. Trace:
I20260709 13:05:04.497243  8338 raft_consensus.cc:740] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:04.532400  1714 rpcz_store.cc:276] 0709 13:05:02.223898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.224146 (+   248us) service_pool.cc:224] Handling call
0709 13:05:02.224979 (+   833us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.514443 (+289464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.531439 (+2016996us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b5400172854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:04.531656 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":663795584}
I20260709 13:05:04.533170  8338 consensus_queue.cc:260] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.534559  8338 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:04.534927  1647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 3351) took 1683 ms (client timeout 2054 ms). Trace:
W20260709 13:05:04.535336  1647 rpcz_store.cc:269] 0709 13:05:02.851327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.030206 (+178879us) service_pool.cc:224] Handling call
0709 13:05:04.534750 (+1504544us) spinlock_profiling.cc:243] Waited 97.1 ms on lock 0x7b54000a73d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:04.534870 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223328000}
I20260709 13:05:04.536551  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:04.443622  8416 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.538738  1640 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:04.539677  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3358) took 1556 ms. Trace:
I20260709 13:05:04.540004  1640 rpcz_store.cc:276] 0709 13:05:02.983120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.490565 (+1507445us) service_pool.cc:224] Handling call
0709 13:05:04.491066 (+   501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.539259 (+ 48193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.539593 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:04.541033  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:04.541975  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:04.542927  1713 raft_consensus.cc:1240] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:04.544987  1647 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:05:04.545997  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:04.547605  1647 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:05:04.549934  1639 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:05:04.543216  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:04.551019  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:04.552403  1647 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:04.549197  1538 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:04.554960  1538 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:04.555442  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:04.445474  8435 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.558017  1538 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:04.551198  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3354) took 1641 ms. Trace:
I20260709 13:05:04.562533  1639 rpcz_store.cc:276] 0709 13:05:02.910013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.479897 (+1569884us) service_pool.cc:224] Handling call
0709 13:05:04.510259 (+ 30362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.550932 (+ 40673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.551138 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:04.564815  1642 raft_consensus.cc:2410] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:05:04.543395  8333 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.595427  1647 raft_consensus.cc:2410] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
W20260709 13:05:04.574596  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 185975us
I20260709 13:05:04.596863  2313 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:04.577500  1639 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:05:04.601189  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2877) took 1015 ms. Trace:
I20260709 13:05:04.601795  1639 rpcz_store.cc:276] 0709 13:05:03.585397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.566840 (+981443us) service_pool.cc:224] Handling call
0709 13:05:04.567309 (+   469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.600760 (+ 33451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.601104 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:04.604156  1646 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:04.568111  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:04.595048  1538 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:04.591602  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.614926  1647 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:04.616418  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2878) took 1030 ms. Trace:
I20260709 13:05:04.617206  1647 rpcz_store.cc:276] 0709 13:05:03.585794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.614221 (+1028427us) service_pool.cc:224] Handling call
0709 13:05:04.614671 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.616035 (+  1364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.616335 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:04.619457  1644 raft_consensus.cc:1240] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:04.620014  1640 raft_consensus.cc:2410] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:05:04.621445  1687 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:04.568965  8423 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:04.558957  8236 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.559369  8363 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:04.538865  8390 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.594480  8475 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:04.633838  1647 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:05:04.594746  8421 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:04.594950  8402 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:04.595000  8318 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:04.636308  1538 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:04.637459  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:04.637423  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:04.617015  8301 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:04.639348  1718 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
W20260709 13:05:04.684934  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 63564.5us
W20260709 13:05:04.666797  1611 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.595s (SENT) [suppressed 29 similar messages]
I20260709 13:05:04.623440  8215 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.617167  8333 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.677547  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:04.622696  8380 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:04.676126  8416 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:04.676688  8465 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.642063  8423 raft_consensus.cc:697] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 20 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:04.692251  8390 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:04.635890  8363 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.642402  8475 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:04.682477  8371 consensus_meta.cc:287] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.904s	user 0.013s	sys 0.000s
I20260709 13:05:04.695488  1722 raft_consensus.cc:1240] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:04.697883  1464 catalog_manager.cc:5697] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 4 to 9, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 9 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:02.015300   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:04.702245  8470 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:04.632395  8355 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.111s
I20260709 13:05:04.703912  8389 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:04.707712  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:04.679082  8388 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:05:04.711133  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3368) took 1592 ms. Trace:
W20260709 13:05:04.764640  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 389915us
I20260709 13:05:04.771522  1646 rpcz_store.cc:276] 0709 13:05:03.118240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.557673 (+1439433us) service_pool.cc:224] Handling call
0709 13:05:04.603891 (+ 46218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.610485 (+  6594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.710808 (+100323us) spinlock_profiling.cc:243] Waited 45.6 ms on lock 0x7b54001fdbd0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:04.711026 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104939904}
I20260709 13:05:04.616796  8419 consensus_queue.cc:1059] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:05:04.714591  8405 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:05:04.715374  8454 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 1.493s	user 0.017s	sys 0.026s
I20260709 13:05:04.717475  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:04.623131  8328 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:04.718345  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:04.798272  1572 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
I20260709 13:05:04.754829  8363 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:04.718222  8402 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:04.722647  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:04.734025  8462 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:05:04.826889  1718 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 24: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:04.723788  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:04.691257  8472 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:04.731518  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:04.735773  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:04.829912  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:04.830628  1568 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:04.736678  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:04.739264  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:05:04.694880  8409 consensus_meta.cc:287] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.225s	user 0.005s	sys 0.011s
I20260709 13:05:04.716442  8435 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:04.743690  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b82d3b49001741eda872554a7ecb2db3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:04.743572  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:04.838287  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:04.745466  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:04.624058  8350 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.096s
I20260709 13:05:04.734478  8301 raft_consensus.cc:697] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 21 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:05:04.748208  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 57716.3us
I20260709 13:05:04.751015  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:04.751322  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:04.716416  7990 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:04.800374  8389 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:04.864009  7990 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.757599  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3359) took 1762 ms. Trace:
I20260709 13:05:04.872061  1644 rpcz_store.cc:276] 0709 13:05:02.995457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.498991 (+1503534us) service_pool.cc:224] Handling call
0709 13:05:04.499314 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.620046 (+120732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.757236 (+137190us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b5400183154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:04.757507 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":275649792}
I20260709 13:05:04.874549  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2837) took 1457 ms. Trace:
I20260709 13:05:04.874997  1722 rpcz_store.cc:276] 0709 13:05:03.417484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.418578 (+  1094us) service_pool.cc:224] Handling call
0709 13:05:03.418888 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.762648 (+1343760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.874311 (+111663us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540020a154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:04.874485 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2934618368}
I20260709 13:05:04.876240  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:04.879297  8423 consensus_queue.cc:237] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.711194  8097 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:04.881987  8097 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.882908  8333 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:04.703070  8391 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:04.646870  1642 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 9. (term mismatch)
I20260709 13:05:04.899670  1642 pending_rounds.cc:85] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 8
I20260709 13:05:04.900022  1642 pending_rounds.cc:107] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 18.9
I20260709 13:05:04.775511  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3367) took 1657 ms. Trace:
I20260709 13:05:04.901425  1641 rpcz_store.cc:276] 0709 13:05:03.117874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.516824 (+1398950us) service_pool.cc:224] Handling call
0709 13:05:04.517218 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.517467 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:04.569713 (+ 52246us) log.cc:844] Serialized 14 byte log entry
0709 13:05:04.775030 (+205317us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:04.775101 (+    71us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:04.775179 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.775265 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:04.775412 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32160,"prepare.queue_time_us":159414,"prepare.run_cpu_time_us":278,"prepare.run_wall_time_us":4780,"raft.queue_time_us":773,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":15738,"thread_start_us":203961,"threads_started":1}
I20260709 13:05:04.678663  8221 consensus_queue.cc:1059] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 13:05:04.639150  8236 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:04.787803  1646 raft_consensus.cc:1275] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 17 index: 7. (index mismatch)
I20260709 13:05:04.905397  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:04.688838  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 54989.1us
I20260709 13:05:04.720499  8318 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.718096  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2833) took 1396 ms. Trace:
I20260709 13:05:04.918965  1717 rpcz_store.cc:276] 0709 13:05:03.322207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.322351 (+   144us) service_pool.cc:224] Handling call
0709 13:05:03.322685 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.464793 (+1142108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.697621 (+232828us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540020ab54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:04.718011 (+ 20390us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2611854720}
I20260709 13:05:04.755164  8475 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:04.806475  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:04.726747  8215 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.738440  8380 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.830005  1573 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
W20260709 13:05:04.837596  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.747287  8465 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.949986  8301 consensus_queue.cc:237] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:04.952764  8465 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:04.838223  1456 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 94560.2us
W20260709 13:05:04.801210  8408 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.145s	user 0.014s	sys 0.025s
I20260709 13:05:04.755708  8416 consensus_queue.cc:1059] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:05:04.838959  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:04.839285  1564 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:05:04.848315  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:04.841975  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:04.843806  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:04.799486  8405 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:04.830011  8328 raft_consensus.cc:697] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:04.862566  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:04.856571  1567 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:05:04.891083  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:04.953953  1718 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:05:04.897763  1568 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:04.960407  8402 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:04.960271  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:04.838944  8409 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:04.961187  1571 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:05:04.961324  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:04.957237  1715 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:04.962172  1568 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:05:04.958245  1572 raft_consensus.cc:2376] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16. Re-sending same reply.
I20260709 13:05:04.963126  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:04.963456  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:04.963889  1568 raft_consensus.cc:2393] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:04.964700  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:04.964787  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:04.762683  8371 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:04.949244  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:04.958474  8405 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:04.966290  8221 raft_consensus.cc:3055] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:04.966713  8221 raft_consensus.cc:740] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:04.967409  1639 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:04.967660  8221 consensus_queue.cc:260] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.900825  1642 raft_consensus.cc:2889] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:04.829603  8472 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.969762  8472 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:04.965678  1647 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:04.827152  8419 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:04.973127  1722 raft_consensus.cc:2410] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:05:04.959225  8328 consensus_queue.cc:237] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:04.972683  8215 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:04.768641  8470 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:04.973166  8419 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:04.979740  8470 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:04.909483  1538 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:04.980996  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 71622.1us
W20260709 13:05:04.981727  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.938289  8309 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:04.918242  8333 consensus_queue.cc:1059] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
W20260709 13:05:04.954460  8408 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.302s	user 0.014s	sys 0.026s
I20260709 13:05:04.995247  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:04.718350  8421 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:04.931845  8395 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:04.927384  1719 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:04.980084  8419 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:04.981688  1538 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:04.999920  1720 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:04.969237  8221 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:05.007077  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:05.011394  1571 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
I20260709 13:05:05.001353  1564 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 24.
I20260709 13:05:05.009136  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2834) took 1593 ms. Trace:
I20260709 13:05:05.015121  1573 raft_consensus.cc:1240] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:05:05.012351  1713 rpcz_store.cc:276] 0709 13:05:03.416276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.417734 (+  1458us) service_pool.cc:224] Handling call
0709 13:05:03.418130 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.544525 (+1126395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.008853 (+464328us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540029a9d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:05.009061 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2576240000}
I20260709 13:05:05.021035  1641 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 13:05:05.021556  1641 pending_rounds.cc:85] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 10
I20260709 13:05:05.021857  1641 pending_rounds.cc:107] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:05:05.022262  1641 raft_consensus.cc:2889] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:05.025789  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 425742us
I20260709 13:05:04.964912  7990 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:04.997395  8470 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:04.931972  8389 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:05.037147  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:05.020283  8400 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:04.986901  8391 raft_consensus.cc:697] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:04.924903  8193 consensus_queue.cc:1059] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.099s
W20260709 13:05:05.046226  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 271662us
I20260709 13:05:05.047189  1641 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:05:05.051681  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 384816us
I20260709 13:05:05.053283  2313 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:05.051054  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3362) took 2028 ms. Trace:
I20260709 13:05:05.038465  8470 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:05.054422  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 294949us
I20260709 13:05:05.058175  1722 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:05.042730  8351 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:05.042750  8400 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.070722  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:05.112695  1718 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:05:05.072602  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.113103  1572 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:05:05.025488  8474 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
W20260709 13:05:05.119632  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 66417.5us
I20260709 13:05:05.123709  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.127899  2313 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:05.127908  1713 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:05:05.034574  8419 consensus_queue.cc:1059] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
W20260709 13:05:05.125324  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 76573.5us
I20260709 13:05:05.142614  2312 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:05.034315  8327 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:05:05.143668  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.118667  8400 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:05.150441  1713 raft_consensus.cc:2393] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 16: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:05.150240  8388 consensus_queue.cc:1048] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:05.150441  8097 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:05.152576  8395 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:05:05.151137  8492 log.cc:927] Time spent T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:05:05.060492  8470 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:05.171593  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.056229  1643 rpcz_store.cc:276] 0709 13:05:03.023150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.513670 (+1490520us) service_pool.cc:224] Handling call
0709 13:05:04.513990 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.514236 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:04.546499 (+ 32263us) log.cc:844] Serialized 14 byte log entry
0709 13:05:05.039668 (+493169us) log.cc:844] Serialized 14 byte log entry
0709 13:05:05.050538 (+ 10870us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:05.050620 (+    82us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:05.050700 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.050798 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.050965 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33240,"prepare.queue_time_us":55842,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":12390,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":4706,"spinlock_wait_cycles":14541952,"thread_start_us":491170,"threads_started":1}
I20260709 13:05:05.182474  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:05.153709  8468 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:05:05.193703  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 61151.4us
I20260709 13:05:05.194864  1537 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:05.197988  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.198741  1718 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:05:05.124661  8483 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.202731  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.199267  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:05.207520  1647 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:05.206066  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:05.209319  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.209321  1713 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 24: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:05.210310  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.210907  1718 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:05:05.211623  1716 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
I20260709 13:05:05.135870  8497 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:05.220201  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.214001  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.217267  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.223282  1718 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:05.221952  1716 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:05:05.224267  1721 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 19: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:05.060245  8484 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.147s
I20260709 13:05:05.221086  1713 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:05:05.228195  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.236583  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.239805  8318 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:05.241094  8211 consensus_meta.cc:287] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.714s	user 0.014s	sys 0.001s
I20260709 13:05:05.241567  8211 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:05.243755  8211 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:05.244395  8211 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:05.244762  8211 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.245121  8211 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:05.246564  8338 raft_consensus.cc:3055] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:05.246989  8338 raft_consensus.cc:740] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:05.247867  8338 consensus_queue.cc:260] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:05.249111  8338 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:05.250348  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:05.057071  8482 consensus_queue.cc:1059] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.154s
I20260709 13:05:05.239710  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:05.254863  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.237802  1713 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 14.
I20260709 13:05:05.256637  1718 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:05:05.257885  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:05.163964  8472 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:05.276054  2344 raft_consensus.cc:2368] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 24. Current term is 25.
I20260709 13:05:05.282166  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.288300  1716 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:05:05.294359  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.310070  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:05.310949  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.311672  1714 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:05.312666  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:05.314019  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.314244  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:05.314986  1721 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:05:05.316437  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.317132  1721 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:05.310989  2345 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:05.214434  8485 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:05:05.267230  8497 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:05.327191  1566 raft_consensus.cc:2393] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 18: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:05.332028  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.332376  1718 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:05:05.335183  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.336117  1713 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 13.
I20260709 13:05:05.341091  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.348977  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:05.218691  8483 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:05.352424  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.353511  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.354785  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:05.361589  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:05.355455  2345 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:05:05.363507  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:05.273018  8473 consensus_queue.cc:1059] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 13:05:05.359835  1714 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:05:05.327711  8435 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.322645  8309 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 25 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:05.373818  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:05:05.379428  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 236844us
I20260709 13:05:05.386967  1714 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:05.384874  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:05.382251  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 255177us
I20260709 13:05:05.389480  1537 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:05.236069  8391 consensus_queue.cc:237] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:05.388525  2312 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:05.392489  1537 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:05.353549  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3360) took 2355 ms. Trace:
I20260709 13:05:05.401337  1537 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:05.343688  8482 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:05:05.407006  1537 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:05.410595  1537 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:05.380117  8421 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:05.363806  8370 consensus_queue.cc:1059] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
W20260709 13:05:05.402436  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.427345  8370 raft_consensus.cc:3055] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:05.399338  1648 rpcz_store.cc:276] 0709 13:05:02.998055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.506601 (+1508546us) service_pool.cc:224] Handling call
0709 13:05:04.506927 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.507157 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:05.003421 (+496264us) log.cc:844] Serialized 14 byte log entry
0709 13:05:05.350056 (+346635us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:05.350139 (+    83us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:05.350198 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.353264 (+  3066us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.353459 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45875,"prepare.queue_time_us":30990,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":19590,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":7636,"thread_start_us":721575,"threads_started":2}
I20260709 13:05:05.427860  8370 raft_consensus.cc:740] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:05:05.428776  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.405232  8362 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:05.368891  8395 consensus_queue.cc:1059] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:05:05.412887  8419 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:05.418581  8193 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:05.408986  8478 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:05.393414  8474 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:05.425695  8400 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:05.414187  1537 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:05.436597  8474 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:05.438714  8494 log.cc:927] Time spent T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:05:05.391418  8423 consensus_queue.cc:1059] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:05:05.440955  1537 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:05.442422  8423 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:05.390867  8435 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:05.461401  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:05.445199  1537 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:05.459815  1714 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
W20260709 13:05:05.484016  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:05.492612  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 47506.7us
W20260709 13:05:05.454985  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 67265us
W20260709 13:05:05.503808  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 375886us
I20260709 13:05:05.510059  2343 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:05.428992  8370 consensus_queue.cc:260] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:05.521214  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:05.522040  1641 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:05.469375  8493 raft_consensus.cc:2804] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:05:05.504312  8328 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:05.535123  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:05.536300  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:05.498580  8499 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:05.538035  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:05.544533  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:05.545666  1567 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:05:05.542106  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:05.551154  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:05.522817  8421 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.524653  8324 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:05:05.562120  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:05.599699  2340 raft_consensus.cc:2368] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11.
I20260709 13:05:05.563181  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:05:05.564209  8475 consensus_meta.cc:287] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.565s	user 0.002s	sys 0.010s
I20260709 13:05:05.565822  8493 raft_consensus.cc:697] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 24 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:05.563201  8435 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:05.532923  8221 consensus_meta.cc:287] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.003s
I20260709 13:05:05.571660  8370 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:05.582901  8338 consensus_queue.cc:1059] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:05.585232  1573 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:05:05.614621  1573 pending_rounds.cc:85] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 11
I20260709 13:05:05.615242  1573 pending_rounds.cc:107] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:05:05.525250  8388 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.445286  8472 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:05.427881  8405 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:05.600685  1572 raft_consensus.cc:2368] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 19. Current term is 20.
I20260709 13:05:05.601295  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:05.608839  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:05.622900  1567 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:05:05.576215  8363 consensus_meta.cc:287] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.742s	user 0.010s	sys 0.001s
I20260709 13:05:05.615597  2344 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 21 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 20. Current term is 21. Ops: []
W20260709 13:05:05.616742  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.615661  1573 raft_consensus.cc:2889] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:05.599810  8215 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:05.619592  1722 raft_consensus.cc:2410] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:05:05.589385  8328 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:05.626773  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:05.631517  2343 raft_consensus.cc:2393] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:05.617301  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:05.643273  1641 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11.
I20260709 13:05:05.645121  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:05.651072  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:05.555274  8499 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:05.607681  8421 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:05.795306  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:05.796275  8400 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:05.806268  2342 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 13:05:05.806546  8508 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.284s	user 0.000s	sys 0.001s
I20260709 13:05:05.807689  8355 raft_consensus.cc:3055] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:05.845916  8355 raft_consensus.cc:740] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:05.847133  8493 consensus_queue.cc:237] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:05.852391  8405 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:05.810890  8435 raft_consensus.cc:3055] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:05.812021  8487 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:05.814282  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:05.644847  8439 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.617671  8374 consensus_queue.cc:1059] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.145s
I20260709 13:05:05.611639  8475 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:05.614214  8470 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:05.831506  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:05.845173  8517 log.cc:927] Time spent T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:05:05.853372  8396 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:05.624475  8388 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:05.866315  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2847) took 1128 ms. Trace:
I20260709 13:05:05.889024  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2843) took 1156 ms. Trace:
W20260709 13:05:05.911481  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.921234  1573 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 16: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:05.924780  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 3210) took 1155 ms. Trace:
I20260709 13:05:05.943367  1566 rpcz_store.cc:276] 0709 13:05:04.732728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.732904 (+   176us) service_pool.cc:224] Handling call
0709 13:05:05.888332 (+1155428us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b5400224cd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:05.888884 (+   552us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1082716416}
I20260709 13:05:05.653352  8363 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:05.928428  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2874) took 3075 ms. Trace:
W20260709 13:05:05.930997  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 440687us
W20260709 13:05:05.934722  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b82d3b49001741eda872554a7ecb2db3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:05.933912  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:05.935600  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:05.947806  1645 rpcz_store.cc:276] 0709 13:05:02.853360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.463994 (+1610634us) service_pool.cc:224] Handling call
0709 13:05:04.464305 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.464530 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:05.002884 (+538354us) log.cc:844] Serialized 14 byte log entry
0709 13:05:05.927839 (+924955us) log.cc:844] Serialized 14 byte log entry
0709 13:05:05.928044 (+   205us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:05.928096 (+    52us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:05.928152 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.928215 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.928352 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":637891,"prepare.queue_time_us":705648,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":73,"raft.queue_time_us":18915,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"spinlock_wait_cycles":15286784,"thread_start_us":755417,"threads_started":2,"wal-append.queue_time_us":406428}
I20260709 13:05:05.949167  1572 raft_consensus.cc:2368] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 23. Current term is 24.
W20260709 13:05:05.931803  1570 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.000s	sys 0.066s
W20260709 13:05:05.962714  1570 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.000s	sys 0.067s
I20260709 13:05:05.757725  8215 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:05.868701  8435 raft_consensus.cc:740] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:05.944195  1720 rpcz_store.cc:276] 0709 13:05:04.769234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.802952 (+ 33718us) service_pool.cc:224] Handling call
0709 13:05:05.924463 (+1121511us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54004079d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:05.924689 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":369533312}
I20260709 13:05:05.943145  1569 rpcz_store.cc:276] 0709 13:05:04.738367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.738488 (+   121us) service_pool.cc:224] Handling call
0709 13:05:04.755797 (+ 17309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.885033 (+129236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:04.885087 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:04.885145 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:04.885221 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.865952 (+980731us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54001b4754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:05.866021 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.866225 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296434048}
I20260709 13:05:05.946977  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:05.951056  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 1144) took 1230 ms. Trace:
I20260709 13:05:05.973793  1565 rpcz_store.cc:276] 0709 13:05:04.720274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.720542 (+   268us) service_pool.cc:224] Handling call
0709 13:05:04.720900 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.981212 (+260312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:04.981264 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:04.981329 (+    65us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:04.981389 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.950815 (+969426us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b54002251d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:05.950859 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.951001 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":597878528}
I20260709 13:05:05.975504  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:05.976187  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:05.954447  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:05.978806  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:05.980365  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:05:05.956664  8516 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:05:05.958493  8519 log.cc:927] Time spent T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:05:05.957719  1464 catalog_manager.cc:5697] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:05.959414  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:06.006683  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.961997  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:05:05.951692  8211 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.706s	user 0.008s	sys 0.008s
I20260709 13:05:06.007364  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:06.008199  2347 raft_consensus.cc:2393] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:06.007717  8211 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:05.974781  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:05:05.957587  8521 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:05:05.928174  8487 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:05.977463  1565 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 19: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:05.989429  8481 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:05.847148  8355 consensus_queue.cc:260] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:06.027592  8499 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:06.029055  8236 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:06.030812  8349 raft_consensus.cc:3055] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:06.031386  8349 raft_consensus.cc:740] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:06.032449  8349 consensus_queue.cc:260] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:06.034487  8355 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:06.035159  8470 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:06.037576  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:05.996078  8215 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:05.934273  8439 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:06.004315  8388 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:06.006904  1568 raft_consensus.cc:1275] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:05:06.056980  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2849) took 1316 ms. Trace:
I20260709 13:05:06.057687  1568 rpcz_store.cc:276] 0709 13:05:04.740410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.006257 (+1265847us) service_pool.cc:224] Handling call
0709 13:05:06.006586 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:06.056622 (+ 50036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.056902 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:05.988231  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 1141) took 1287 ms. Trace:
I20260709 13:05:06.016651  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:06.020825  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:06.061349  1569 raft_consensus.cc:2368] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16.
I20260709 13:05:06.024268  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:05.963387  8374 consensus_queue.cc:1059] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
W20260709 13:05:06.026321  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.052917  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:06.054440  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:06.069379  1565 raft_consensus.cc:2410] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:05:06.055286  2341 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 17: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:05:06.065538  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:05.969583  8338 consensus_queue.cc:1059] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:05:06.071782  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:06.062069  1570 rpcz_store.cc:276] 0709 13:05:04.700817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.713998 (+ 13181us) service_pool.cc:224] Handling call
0709 13:05:04.714506 (+   508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:04.715001 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:04.715042 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:04.715631 (+   589us) log.cc:844] Serialized 26 byte log entry
0709 13:05:05.175473 (+459842us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:05.963215 (+787742us) log.cc:844] Serialized 14 byte log entry
0709 13:05:05.987844 (+ 24629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.987937 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:05.987988 (+    51us) raft_consensus.cc:1685] finished
0709 13:05:05.988002 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.988161 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24292,"prepare.queue_time_us":244594,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":56556,"raft.run_cpu_time_us":397,"raft.run_wall_time_us":14207,"thread_start_us":1183074,"threads_started":2,"wal-append.queue_time_us":612009}
I20260709 13:05:06.072973  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:06.075006  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:06.078126  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:06.079038  1570 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:05:06.081285  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.089960  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:06.091053  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:06.095881  8487 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:06.099330  8524 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:05:06.100663  8435 consensus_queue.cc:260] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:06.101539  8349 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:06.102548  8374 raft_consensus.cc:3055] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:06.103541  8523 raft_consensus.cc:3055] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:06.104684  8421 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:05:06.038658  8371 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:06.109298  1572 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:05:06.109494  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:06.118191  1567 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 25.
I20260709 13:05:06.203088  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:06.121923  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:06.206358  1569 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:05:06.190349  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:06.207576  1565 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:06.116806  1573 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:05:06.195581  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:06.198297  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:06.050181  8525 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.177s
W20260709 13:05:06.211030  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.197921  8374 raft_consensus.cc:740] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:06.212599  8374 consensus_queue.cc:260] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:06.214368  8374 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:06.225517  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 272056us
I20260709 13:05:06.228557  1571 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:06.232961  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:05:06.233422  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.234537  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:06.235440  1716 raft_consensus.cc:2393] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 24: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:06.236348  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:06.237704  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:06.238457  1573 raft_consensus.cc:1240] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:05:06.240458  1573 raft_consensus.cc:1240] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:05:06.240630  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:06.241225  1717 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:06.199631  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:06.242233  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:06.242547  1570 raft_consensus.cc:2393] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 18: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:06.243891  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:06.245076  1573 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:05:06.204524  2312 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:06.247897  2342 raft_consensus.cc:1240] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:06.245914  1612 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:06.248714  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 44271.4us
I20260709 13:05:06.248996  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:06.249306  1612 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:06.251965  1686 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:06.251199  1612 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:06.255947  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:06.259990  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:06.198212  8523 raft_consensus.cc:740] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:06.262068  1717 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:05:06.225965  8387 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.297205  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:06.296454  1573 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:05:06.296341  1570 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:05:06.306519  1568 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
W20260709 13:05:06.261852  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 179680us
I20260709 13:05:06.248185  8409 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:06.247999  8282 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:06.234154  8475 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:06.295562  8473 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:05:06.296212  8435 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:06.322383  1568 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 20 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:06.261411  8408 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:06.264810  8415 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:06.331235  1570 raft_consensus.cc:1240] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:06.243991  8371 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 13:05:06.318333  8236 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:06.313040  8409 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:05:06.346021  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.348114  1568 raft_consensus.cc:3055] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:06.350111  1568 raft_consensus.cc:740] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:06.331661  8408 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.309927  8387 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:06.351990  8523 consensus_queue.cc:260] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:06.364889  8523 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:06.371644  8387 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:06.376063  8483 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:06.379402  2344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55890 (request call id 631) took 1068 ms. Trace:
I20260709 13:05:06.379905  2344 rpcz_store.cc:276] 0709 13:05:05.311298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.317962 (+  6664us) service_pool.cc:224] Handling call
0709 13:05:05.318399 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:05.657257 (+338858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.379085 (+721828us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b540065b3d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:06.379324 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":682523392}
I20260709 13:05:06.380770  8029 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.333796  8473 raft_consensus.cc:697] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 25 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:05:06.383180  1685 leader_election.cc:336] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.719s (SENT)
I20260709 13:05:06.350375  8409 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:06.356763  8529 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:05:06.383999  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:06.393628  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:06.393776  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.325529  8282 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:06.347743  8236 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.346009  1570 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:06.385152  1685 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:06.357450  8370 consensus_meta.cc:287] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.735s	user 0.013s	sys 0.000s
I20260709 13:05:06.381423  8029 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:06.355492  8408 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:06.407167  1567 raft_consensus.cc:2410] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:05:06.411653  1566 raft_consensus.cc:2410] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 13:05:06.416154  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 178057us
I20260709 13:05:06.419799  1564 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 14.
I20260709 13:05:06.419965  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:06.393136  8529 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.421860  1571 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:06.398869  8236 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:06.399250  8333 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:06.431452  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.437107  8380 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:06.438028  8211 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:06.438613  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 103050us
I20260709 13:05:06.438815  8473 consensus_queue.cc:237] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:06.441269  8497 raft_consensus.cc:3055] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:06.441763  8497 raft_consensus.cc:740] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:06.443056  8497 consensus_queue.cc:260] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:06.447026  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2866) took 1310 ms. Trace:
I20260709 13:05:06.447455  1572 rpcz_store.cc:276] 0709 13:05:05.136179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.013066 (+876887us) service_pool.cc:224] Handling call
0709 13:05:06.446800 (+433734us) spinlock_profiling.cc:243] Waited 70.9 ms on lock 0x7b54001275d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:06.446949 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":163042688}
I20260709 13:05:06.428610  8529 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:06.456419  8282 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:06.460067  1565 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:05:06.422168  8421 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:05:06.462240  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 211067us
I20260709 13:05:06.462625  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:06.467240  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.463773  1537 leader_election.cc:400] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 24 election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 24. Current term is 25.
I20260709 13:05:06.467967  1537 leader_election.cc:403] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 13:05:06.469964  1537 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:06.470131  1685 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:06.473484  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.472550  1612 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:06.479936  1612 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:06.464491  8389 raft_consensus.cc:3055] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:06.476398  8282 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for earlier term 24. Current term is 25.
I20260709 13:05:06.503615  8475 consensus_queue.cc:1059] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:06.504937  8475 raft_consensus.cc:3055] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:06.488583  1537 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:06.507217  1537 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:06.510164  8350 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:06.474542  8415 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:05:06.511375  1537 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:06.518026  1571 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17.
I20260709 13:05:06.525879  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:06.528028  1537 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:06.505369  8475 raft_consensus.cc:740] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:05:06.529687  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.462427  8333 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:06.533804  1537 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:06.485072  8405 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:05:06.536597  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 66613.4us
W20260709 13:05:06.537117  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.484844  8355 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:06.489076  8483 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:06.492550  8380 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:06.538883  1685 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:06.546068  8408 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:06.526754  8370 raft_consensus.cc:2764] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:05:06.549229  8349 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:06.550758  8409 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:06.493026  8389 raft_consensus.cc:740] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:06.534754  8327 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:06.581661  8355 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:06.542062  8282 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:06.556143  8380 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.542066  8405 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.569543  8421 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:06.540912  8481 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:05:06.627712  8029 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:06.653079  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.597599  8327 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:06.605369  8380 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:06.633806  8408 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:06.635339  8333 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:06.638681  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 99858.7us
I20260709 13:05:06.660629  1685 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:06.607549  8405 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:06.670497  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 136712us
I20260709 13:05:06.671772  1537 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:06.672403  1685 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:06.631938  8363 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:06.673363  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.676987  1537 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:06.604292  8355 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:06.657606  8327 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:06.686532  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:06.601089  8475 consensus_queue.cc:260] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:06.609871  8389 consensus_queue.cc:260] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:06.680981  8421 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:06.714886  1537 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:06.687747  8363 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:05:06.724436  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.680012  8468 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:05:06.733151  1537 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:06.685307  8483 raft_consensus.cc:697] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:05:06.740001  8536 log.cc:927] Time spent T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:05:06.719507  8542 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:06.724282  8349 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:06.742304  8349 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:06.742761  8349 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:06.744239  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:06.746243  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 73915.7us
I20260709 13:05:06.747530  1685 leader_election.cc:400] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16.
I20260709 13:05:06.695261  8481 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:05:06.750550  1685 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:06.752431  8537 log.cc:927] Time spent T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:05:06.722085  8363 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:06.754048  8363 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:06.754943  8421 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:06.762827  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:06.763568  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:06.764364  1717 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:05:06.765568  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:06.766410  1717 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:06.766067  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:06.766961  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:06.767855  1715 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 21.
I20260709 13:05:06.768358  1716 raft_consensus.cc:3055] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:06.769510  1716 raft_consensus.cc:740] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:06.770684  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:06.772570  1719 raft_consensus.cc:1275] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:05:06.731423  8389 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:06.773015  1719 pending_rounds.cc:85] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 8
I20260709 13:05:06.775826  1537 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:06.729601  8415 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:06.773499  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:06.773834  1714 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:06.729979  8475 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:06.779120  1719 pending_rounds.cc:107] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:05:06.798329  1717 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:06.798636  1719 raft_consensus.cc:2889] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:06.800324  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:06.800998  1713 raft_consensus.cc:3060] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:06.805975  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:06.807011  1719 raft_consensus.cc:2410] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:05:06.780097  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:06.792857  8529 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:06.770982  1716 consensus_queue.cc:260] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:06.762655  8541 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:06.784168  8355 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:06.783756  1537 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:06.785121  8370 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:06.786078  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:05:06.788362  8522 log.cc:927] Time spent T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.737s	user 0.007s	sys 0.002s
I20260709 13:05:06.754963  8518 raft_consensus.cc:3055] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:05:06.796692  2341 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55890 (request call id 627) took 1592 ms (client timeout 1507 ms). Trace:
I20260709 13:05:06.810390  1720 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:05:06.781546  8544 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:06.810941  1717 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:06.826385  1716 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:06.836525  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.841002  8529 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:06.801990  8543 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:06.846727  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 63083.7us
I20260709 13:05:06.852051  1714 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:05:06.851140  2341 rpcz_store.cc:269] 0709 13:05:05.204351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.204565 (+   214us) service_pool.cc:224] Handling call
0709 13:05:06.796404 (+1591839us) spinlock_profiling.cc:243] Waited 796 ms on lock 0x7b5400800054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:06.796592 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1830737152}
I20260709 13:05:06.863782  1537 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:06.859053  8470 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:06.859920  1713 raft_consensus.cc:2468] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 24.
I20260709 13:05:06.813396  8409 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:06.871567  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:06.880029  8481 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:06.859411  8518 raft_consensus.cc:740] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:06.880620  8481 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.881184  8481 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:06.881527  8518 consensus_queue.cc:260] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:06.883191  8518 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:06.850356  8370 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:06.859138  8541 raft_consensus.cc:697] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:05:06.897928  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 147417us
I20260709 13:05:06.899693  1685 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:06.892572  1716 raft_consensus.cc:1275] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:05:06.895159  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:06.901079  1720 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:06.894831  8370 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:06.873780  8468 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:06.902948  8545 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:06.903441  1685 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:06.912457  1685 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:06.915477  8415 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:06.915990  8415 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:06.915019  1685 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:06.923918  1722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2847) took 1855 ms (client timeout 1501 ms). Trace:
W20260709 13:05:06.924487  1722 rpcz_store.cc:269] 0709 13:05:05.069468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.070377 (+   909us) service_pool.cc:224] Handling call
0709 13:05:06.923676 (+1853299us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b5400209c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:06.923842 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1164001664}
I20260709 13:05:06.923924  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:06.916636  8416 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:06.926883  8416 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.927253  8416 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:06.928471  1716 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:06.940980  1722 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:05:06.944195  2313 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:06.945876  8484 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:06.945940  8545 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:06.947235  1721 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:05:06.870486  8543 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:06.950558  8545 consensus_queue.cc:1059] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:06.951390  8545 consensus_queue.cc:1243] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer 066ee0109aba4465ba17fa65e14ef504 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:05:06.951464  8543 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:06.952057  8421 raft_consensus.cc:3055] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:06.952428  8421 raft_consensus.cc:740] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:06.953500  8421 consensus_queue.cc:260] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:06.958346  8545 consensus_queue.cc:1059] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:06.958922  8327 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:06.959838  8545 raft_consensus.cc:3055] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:05:06.960351  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:06.960882  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.963871  1717 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:05:06.966912  8221 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:06.967317  8380 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:06.971303  8497 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:06.906165  8236 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:06.967468  8439 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:06.926909  8541 consensus_queue.cc:237] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:06.975750  8236 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:06.916419  8415 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:06.986107  1720 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 25.
I20260709 13:05:06.988895  8324 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:06.990800  8421 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:06.990451  8349 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:06.993680  8363 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:06.907308  8483 consensus_queue.cc:237] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:06.990244  8512 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.005s
I20260709 13:05:06.995528  8327 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:06.960412  8545 raft_consensus.cc:740] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:06.999421  8282 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:07.008864  8380 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:07.007223  8481 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:07.024327  8545 consensus_queue.cc:260] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.031687  8545 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:07.032117  8481 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:07.036211  8544 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:07.091787  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 215409us
I20260709 13:05:07.096419  1565 raft_consensus.cc:1275] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:05:07.137414  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:07.139753  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 1154) took 1206 ms. Trace:
W20260709 13:05:07.141278  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.141953  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.142361  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.142762  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.098832  1566 raft_consensus.cc:1240] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:07.145360  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 724) took 1692 ms. Trace:
I20260709 13:05:07.088162  1464 catalog_manager.cc:5697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:07.102972  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.114318  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 152413us
I20260709 13:05:07.145839  1648 rpcz_store.cc:276] 0709 13:05:05.453042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.474839 (+ 21797us) service_pool.cc:224] Handling call
0709 13:05:05.475794 (+   955us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:05.476185 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:06.298885 (+822700us) log.cc:844] Serialized 14 byte log entry
0709 13:05:06.520464 (+221579us) log.cc:844] Serialized 14 byte log entry
0709 13:05:06.940185 (+419721us) spinlock_profiling.cc:243] Waited 4.41 ms on lock 0x7b18004604e0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:05:07.139543 (+199358us) spinlock_profiling.cc:243] Waited 4.32 ms on lock 0x7b18004604e0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d12c0a 00007f824f452219 00007f8245d9aa40 00007f8245d954e4 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287 00007f8248ea0208
0709 13:05:07.144179 (+  4636us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:07.144310 (+   131us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:07.144483 (+   173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:07.144651 (+   168us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:07.145001 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":464882,"prepare.queue_time_us":482407,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":5983,"raft.queue_time_us":8592,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":6571,"spinlock_wait_cycles":20075648,"thread_start_us":571300,"threads_started":2,"wal-append.queue_time_us":305442}
I20260709 13:05:07.147513  1571 raft_consensus.cc:1240] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:07.140476  1573 rpcz_store.cc:276] 0709 13:05:05.933338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.305253 (+371915us) service_pool.cc:224] Handling call
0709 13:05:06.305763 (+   510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:06.374850 (+ 69087us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:06.374895 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:06.375450 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 13:05:06.569058 (+193608us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:06.569135 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:07.138818 (+569683us) spinlock_profiling.cc:243] Waited 68.6 ms on lock 0x7b5400013954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:07.138898 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:07.138993 (+    95us) raft_consensus.cc:1685] finished
0709 13:05:07.139010 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:07.139477 (+   467us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2074,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"spinlock_wait_cycles":157689088,"thread_start_us":193217,"threads_started":1,"wal-append.queue_time_us":287117}
I20260709 13:05:07.155128  1685 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:07.158161  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.128327  1568 consensus_queue.cc:260] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.161482  1568 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:07.173533  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.182885  8236 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:07.186046  8211 consensus_queue.cc:1048] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:05:07.186810  8324 consensus_queue.cc:1059] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:05:07.188140  8282 consensus_queue.cc:1048] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:07.189973  8333 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.190528  8327 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:07.191576  8327 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:07.192152  8327 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:07.191596  8333 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.184541  8236 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:07.215094  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.200831  1571 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 25 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:05:07.215888  8282 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:07.340590  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 202869us
I20260709 13:05:07.211611  8349 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:07.196870  8333 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:07.210621  8338 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.341179  8221 consensus_queue.cc:1059] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:05:07.348688  8484 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:07.193542  8324 consensus_queue.cc:1059] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.159s
I20260709 13:05:07.359846  8485 raft_consensus.cc:3055] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:07.360800  8485 raft_consensus.cc:740] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:07.366113  8543 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:07.379281  8338 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:07.380420  8236 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:07.381973  8282 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.382863  8371 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:07.384027  8484 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:07.385228  8282 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:07.404547  8544 consensus_queue.cc:1059] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:05:07.405966  8546 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:07.407666  8546 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.409898  8355 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:05:07.412060  8338 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:07.413107  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.414305  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.415043  8405 consensus_meta.cc:287] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.689s	user 0.006s	sys 0.014s
I20260709 13:05:07.415843  8405 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:07.421913  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.425904  1716 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17.
W20260709 13:05:07.426394  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 912784us
I20260709 13:05:07.468451  1537 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:07.428736  8543 raft_consensus.cc:697] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:07.433351  8324 consensus_queue.cc:1243] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer 066ee0109aba4465ba17fa65e14ef504 log is divergent from this leader: its last log entry 20.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:05:07.485450  8387 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.486645  8405 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:07.486486  8387 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.488871  8484 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:07.489127  8387 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:07.491969  8561 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.493121  8561 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.493435  8556 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.494841  8556 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.495757  8371 raft_consensus.cc:697] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:07.432065  8221 consensus_queue.cc:1243] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer 066ee0109aba4465ba17fa65e14ef504 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:05:07.504287  1640 raft_consensus.cc:1240] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:05:07.506009  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:07.506661  8556 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:07.512606  8493 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.049s
I20260709 13:05:07.514107  8338 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:07.517138  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 362209us
I20260709 13:05:07.518759  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:07.523131  8485 consensus_queue.cc:260] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:07.525123  8485 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:07.526180  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.527179  1648 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:07.527673  8211 raft_consensus.cc:3055] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:07.528184  8211 raft_consensus.cc:740] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:05:07.528488  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.529292  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:07.530766  8543 consensus_queue.cc:237] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.516860  8338 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.513900  8416 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:07.559109  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.568892  8416 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:07.570497  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.570583  8558 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.571672  8558 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.572808  1642 raft_consensus.cc:1240] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:07.574312  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:07.575305  1639 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:07.575909  1470 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38666 (request call id 217) took 29988 ms (client timeout 15000 ms). Trace:
I20260709 13:05:07.577219  8559 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:07.579789  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 205752us
I20260709 13:05:07.579896  8559 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.582218  8483 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:07.583271  8483 raft_consensus.cc:697] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:07.586172  8542 consensus_queue.cc:1048] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:07.587147  8542 consensus_queue.cc:1243] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 19.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:05:07.587208  8363 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:07.590284  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:07.578904  8139 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.591128  8559 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:07.578107  8350 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.570822  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:07.572724  1647 raft_consensus.cc:1275] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:05:07.576431  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.591564  8483 consensus_queue.cc:237] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:07.593758  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.612707  1639 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:07.613897  8139 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.614547  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:07.614866  1643 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:05:07.615414  1639 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:07.616686  8139 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:07.617725  1644 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:07.618134  1642 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 15: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:05:07.576833  1470 rpcz_store.cc:269] 0709 13:04:37.587276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.587565 (+   289us) service_pool.cc:224] Handling call
0709 13:05:07.575212 (+29987647us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:52.502709 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:52.503974 (+  1265us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:52.504160 (+   186us) write_op.cc:435] Acquired schema lock
0709 13:04:52.504190 (+    30us) tablet.cc:662] Decoding operations
0709 13:04:53.205390 (+701200us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:53.205533 (+   143us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:53.205564 (+    31us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:04:53.208345 (+  2781us) tablet.cc:701] Row locks acquired
0709 13:04:53.208403 (+    58us) write_op.cc:260] PREPARE: finished
0709 13:04:53.208635 (+   232us) write_op.cc:270] Start()
0709 13:04:53.208843 (+   208us) write_op.cc:276] Timestamp: P: 1783602293208548 usec, L: 0
0709 13:04:53.208871 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:04:53.261681 (+ 52810us) log.cc:844] Serialized 8713 byte log entry
0709 13:04:54.270354 (+1008673us) op_driver.cc:464] REPLICATION: finished
0709 13:04:54.409551 (+139197us) write_op.cc:301] APPLY: starting
0709 13:04:54.414482 (+  4931us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:55.004972 (+590490us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:55.005081 (+   109us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:56.862357 (+1857276us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:56.863290 (+   933us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=44,mrs_lookups=0
0709 13:04:56.863471 (+   181us) write_op.cc:312] APPLY: finished
0709 13:04:56.865287 (+  1816us) log.cc:844] Serialized 196 byte log entry
0709 13:04:57.002391 (+137104us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:57.176665 (+174274us) write_op.cc:454] Released schema lock
0709 13:04:57.198546 (+ 21881us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:57.256170 (+ 57624us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:57.256209 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":549849,"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":122895,"apply.run_cpu_time_us":87065,"apply.run_wall_time_us":2853942,"cfile_cache_hit":90,"cfile_cache_hit_bytes":96131,"num_ops":22,"prepare.queue_time_us":7383,"prepare.run_cpu_time_us":57786,"prepare.run_wall_time_us":1037453,"raft.queue_time_us":333033,"raft.run_cpu_time_us":7815,"raft.run_wall_time_us":111410,"replication_time_us":1039369,"spinlock_wait_cycles":7936,"thread_start_us":665968,"threads_started":5,"wal-append.queue_time_us":626309,"wal-append.run_cpu_time_us":110903,"wal-append.run_wall_time_us":2643261}]]}
I20260709 13:05:07.529479  8211 consensus_queue.cc:260] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:07.626747  8211 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:07.627267  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 1155) took 1525 ms. Trace:
I20260709 13:05:07.629422  8029 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:07.632371  8029 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:07.632959  8029 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:07.574834  8487 raft_consensus.cc:3055] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:07.639772  1639 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:05:07.651414  1645 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:05:07.642241  8487 raft_consensus.cc:740] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:07.657254  1642 raft_consensus.cc:1275] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:05:07.497463  8371 consensus_queue.cc:237] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:07.660586  1647 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:07.660058  1568 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:07.670951  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.673135  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2906) took 1566 ms. Trace:
I20260709 13:05:07.674245  1568 rpcz_store.cc:276] 0709 13:05:06.106737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.326817 (+220080us) service_pool.cc:224] Handling call
0709 13:05:06.344491 (+ 17674us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:06.462322 (+117831us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b4c00130640. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8247151a20 00007f8247162a29 00007f8247128644 00007f824714ff2d 00007f824714fb93 00007f8248e8018b 00007f8248e80cb5 00007f8248e7bf30 00007f8248e7d7de 00007f8248e87b49 00007f8248e88416
0709 13:05:07.116198 (+653876us) spinlock_profiling.cc:243] Waited 2.34 ms on lock 0x7b54001d4320. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e0306f 00007f8248e7bf60 00007f8248e7d7de 00007f8248e87b49 00007f8248e88416 00007f8248e84d8f 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62
0709 13:05:07.663573 (+547375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:07.673060 (+  9487us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64991872}
I20260709 13:05:07.661450  8561 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:07.665848  8416 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:07.692790  1641 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:07.694415  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 64918.5us
I20260709 13:05:07.627863  1569 rpcz_store.cc:276] 0709 13:05:06.101426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.316784 (+215358us) service_pool.cc:224] Handling call
0709 13:05:06.332778 (+ 15994us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:06.333070 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:06.347077 (+ 14007us) log.cc:844] Serialized 14 byte log entry
0709 13:05:06.531721 (+184644us) log.cc:844] Serialized 14 byte log entry
0709 13:05:07.144640 (+612919us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b18003ff2e0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287 00007f8248ea0208
0709 13:05:07.470586 (+325946us) log.cc:844] Serialized 14 byte log entry
0709 13:05:07.626260 (+155674us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:07.626374 (+   114us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:07.626555 (+   181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:07.626725 (+   170us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:07.627051 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":98291,"prepare.queue_time_us":169508,"prepare.run_cpu_time_us":140,"prepare.run_wall_time_us":147,"raft.queue_time_us":27270,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":8194,"spinlock_wait_cycles":6092672,"thread_start_us":415959,"threads_started":3,"wal-append.queue_time_us":253479}
I20260709 13:05:07.698808  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:05:07.699476  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 67599.1us
I20260709 13:05:07.699888  1639 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:07.707489  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.678951  8564 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:07.707564  1648 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 19. Current term is 20. Ops: [19.9-19.9]
I20260709 13:05:07.701753  1685 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:07.687065  8349 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.690234  8544 consensus_queue.cc:1243] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer 066ee0109aba4465ba17fa65e14ef504 log is divergent from this leader: its last log entry 15.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:05:07.703526  8546 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:07.727691  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:07.732455  1647 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:07.734005  8561 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:07.730554  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.738199  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:07.734669  7990 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 13:05:07.745785  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:07.746397  1685 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:07.750373  1643 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:05:07.751009  8564 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:07.752060  8546 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:07.754446  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b82d3b49001741eda872554a7ecb2db3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:07.760046  1717 raft_consensus.cc:1275] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:05:07.696730  8350 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.762120  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:07.743578  8561 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:07.778349  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.781010  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:07.781935  1717 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:07.785888  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:07.791778  8415 consensus_meta.cc:287] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.766s	user 0.011s	sys 0.013s
I20260709 13:05:07.792445  8415 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.795397  8338 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:07.732326  8349 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.797830  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:07.798301  1642 raft_consensus.cc:2410] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:05:07.776858  8405 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.799677  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.801023  1642 raft_consensus.cc:2393] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 15: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:07.800482  8405 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:07.696632  8487 consensus_queue.cc:260] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.805441  8487 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:07.805823  1721 raft_consensus.cc:606] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:07.806583  1721 raft_consensus.cc:614] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:07.808116  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:07.812276  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.812713  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.763789  8350 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:07.799665  8545 raft_consensus.cc:3055] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:07.766006  8564 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:07.799777  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:07.833667  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.775576  8558 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:07.748225  8416 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.827405  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.842573  1642 raft_consensus.cc:2393] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 15: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:07.790668  8561 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:07.846196  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.798522  8349 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:07.855141  1639 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
W20260709 13:05:07.856285  8327 consensus_meta.cc:287] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.663s	user 0.014s	sys 0.004s
I20260709 13:05:07.856805  8327 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:07.857365  8350 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:07.857565  1643 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 18: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:07.858022  8350 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.858851  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.859416  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:07.859678  1643 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:05:07.860837  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.861500  8282 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.862540  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.863600  1642 raft_consensus.cc:2393] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 18: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:07.864138  7990 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:07.864573  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.820122  8405 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:07.865221  8554 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:07.866299  1573 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:07.862373  8282 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.860313  8350 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:07.868353  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.871603  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.871989  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.872339  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.860841  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.871106  1644 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
W20260709 13:05:07.871346  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 56753.3us
I20260709 13:05:07.865437  1642 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:05:07.827173  8545 raft_consensus.cc:740] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:07.865953  1641 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:05:07.881449  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.867993  7990 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:07.859445  8327 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:07.919921  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:07.895320  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.898794  8421 consensus_meta.cc:287] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.880s	user 0.010s	sys 0.009s
I20260709 13:05:07.889134  8554 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.901428  8572 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:07.888546  8363 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:07.888797  8349 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.904285  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:07.906502  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.902359  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:05:07.907821  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 111452us
I20260709 13:05:07.911284  8545 consensus_queue.cc:260] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 17.7, Last appended by leader: 7, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:07.920099  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.922844  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:07.888388  1639 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:07.935834  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:07.939356  1646 raft_consensus.cc:2410] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 18 because replica has last-logged OpId of term: 17 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
W20260709 13:05:07.889849  8370 consensus_meta.cc:287] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.968s	user 0.022s	sys 0.006s
I20260709 13:05:07.940882  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:07.939442  1647 raft_consensus.cc:2376] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15. Re-sending same reply.
I20260709 13:05:07.943063  1645 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 18: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:05:07.953022  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.932531  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:07.937278  1643 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 23.
I20260709 13:05:07.959012  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:07.962154  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.963325  1644 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:07.963464  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.964114  1639 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 18: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:07.967763  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:07.968087  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.968657  1568 raft_consensus.cc:2410] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:05:07.969388  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.965143  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57572 (request call id 56) took 1001 ms. Trace:
I20260709 13:05:07.971858  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.962244  2313 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:07.975560  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:07.976639  1646 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 23.
I20260709 13:05:07.919974  8564 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:07.971791  1721 rpcz_store.cc:276] 0709 13:05:06.939617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.947073 (+  7456us) service_pool.cc:224] Handling call
0709 13:05:07.932469 (+985396us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b54004079d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e7f853 00007f824de4d60a 00007f82473d37d4 00007f82473d3762 00007f82473d3691 00007f82473d360c 00007f82473d28c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:07.941121 (+  8652us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1971684608}
I20260709 13:05:07.984566  1714 raft_consensus.cc:2410] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:05:07.931664  7990 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:07.992502  1639 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:05:07.988327  1643 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:05:07.989665  1641 raft_consensus.cc:2410] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:05:07.983820  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:07.998054  1644 raft_consensus.cc:1240] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: [17.7-17.7]
I20260709 13:05:07.991568  1640 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:05:08.001000  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:08.001863  8363 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:08.002533  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:08.003873  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:08.004571  8363 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:08.005210  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:05:08.005842  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.011417  1640 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:05:08.021889  1645 raft_consensus.cc:1275] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:05:08.023555  1644 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
W20260709 13:05:08.027805  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.031100  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:08.033056  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 109517us
I20260709 13:05:08.036164  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:07.962476  8349 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:08.038014  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.040946  1641 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:05:08.048454  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.962054  8370 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:08.067339  1713 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:08.059105  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:08.073604  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:07.979724  8572 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:08.084904  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:05:08.086012  8576 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:05:08.086433  1564 raft_consensus.cc:2368] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 16. Current term is 17.
I20260709 13:05:08.087551  8416 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:08.088310  8565 log.cc:927] Time spent T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:05:08.088392  8525 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:08.089111  8525 raft_consensus.cc:697] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:08.090112  1573 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:05:08.050704  8554 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:08.101042  1568 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:05:08.100901  8405 consensus_queue.cc:1059] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:05:08.126595  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:08.126928  8405 raft_consensus.cc:3055] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:08.079854  8493 consensus_queue.cc:1048] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.033s
I20260709 13:05:08.127574  8405 raft_consensus.cc:740] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:08.128814  8405 consensus_queue.cc:260] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:08.130311  8370 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:08.130648  8561 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:08.126506  8497 consensus_queue.cc:1059] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.119s
I20260709 13:05:08.130352  8405 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:08.127367  8569 log.cc:927] Time spent T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.215s	user 0.000s	sys 0.002s
W20260709 13:05:08.167584  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:08.166443  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 120752us
I20260709 13:05:08.118850  8282 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:08.119509  8416 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.091867  8564 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:08.193192  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:08.212064  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 74957.7us
I20260709 13:05:08.148392  8566 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.219326  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:08.181893  8415 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:08.224140  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 125538us
I20260709 13:05:08.219477  1717 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:08.230846  1721 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 18: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:08.239303  1567 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
W20260709 13:05:08.235394  8485 consensus_meta.cc:287] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.710s	user 0.018s	sys 0.003s
I20260709 13:05:08.247509  1715 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:08.249042  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:05:08.218690  1521 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 46758.6us
I20260709 13:05:08.259400  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:08.236279  8370 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.256984  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:08.223276  8416 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:08.264716  1538 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:08.264353  8415 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.195024  8525 consensus_queue.cc:237] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:08.285908  1714 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:05:08.256474  8473 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.024s	sys 0.069s
I20260709 13:05:08.289165  1721 raft_consensus.cc:2410] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:05:08.245329  8566 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:08.278708  8415 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:08.285079  8585 log.cc:927] Time spent T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
W20260709 13:05:08.287096  8584 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.207s	user 0.000s	sys 0.002s
W20260709 13:05:08.302493  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.302037  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:05:08.296770  1719 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.035s	sys 0.050s
I20260709 13:05:08.299726  8554 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:08.301365  8588 log.cc:927] Time spent T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
W20260709 13:05:08.290663  8473 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.025s	sys 0.070s
W20260709 13:05:08.307369  1719 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.035s	sys 0.051s
I20260709 13:05:08.307137  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:08.311388  2343 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 14.
I20260709 13:05:08.311357  1611 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:08.294715  8421 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:08.323544  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:08.326088  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.329078  8566 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:08.301890  8282 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.333794  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:08.335031  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:08.335337  8415 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:08.295059  8370 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:08.337432  8574 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:08.338176  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:05:08.338379  8211 consensus_meta.cc:287] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.711s	user 0.017s	sys 0.000s
I20260709 13:05:08.350841  1569 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:05:08.308873  8572 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:08.340265  8481 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.013s	sys 0.005s
I20260709 13:05:08.356132  2341 raft_consensus.cc:2410] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 13:05:08.323164  8349 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:08.361362  8487 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:05:08.393743  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.396087  1567 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:08.410769  1568 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 19: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:05:08.413261  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 285155us
I20260709 13:05:08.417040  1565 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:05:08.390444  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:08.433743  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 193576us
W20260709 13:05:08.446798  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:08.453864  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 272612us
I20260709 13:05:08.453122  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:08.397208  8566 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:08.416208  8564 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:08.465279  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:08.435976  8029 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.792s	user 0.014s	sys 0.003s
I20260709 13:05:08.391021  8487 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:08.470306  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.357314  8421 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:08.475960  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.476809  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:08.461985  8371 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:05:08.475036  8590 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.154s	user 0.000s	sys 0.002s
I20260709 13:05:08.434787  8282 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:08.483777  2345 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
W20260709 13:05:08.487398  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 365084us
W20260709 13:05:08.488868  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.459959  8566 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:05:08.478262  1612 connection.cc:256] RPC call timeout handler was delayed by -1.7913680870000235s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1553s for a response
I20260709 13:05:08.495862  8370 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:08.498068  8415 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:08.425513  8574 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:08.495271  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:08.505128  1569 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:08.508387  1612 connection.cc:256] RPC call timeout handler was delayed by -1.6615990599999577s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30600000000000005s for a response
I20260709 13:05:08.476967  8572 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:05:08.516160  1612 connection.cc:256] RPC call timeout handler was delayed by -1.6185680870000283s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17450000000000002s for a response
I20260709 13:05:08.511682  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:05:08.491425  8597 log.cc:927] Time spent T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Append to log took a long time: real 0.062s	user 0.001s	sys 0.002s
W20260709 13:05:08.518815  1612 connection.cc:256] RPC call timeout handler was delayed by -1.1802680870000017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22320000000000004s for a response
I20260709 13:05:08.511798  8416 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:08.508152  8566 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:08.525019  8572 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:08.547534  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 66010.6us
W20260709 13:05:08.549656  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.498068  1716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:05:08.629251  1716 raft_consensus.cc:606] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:08.630437  1716 raft_consensus.cc:614] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:08.545748  8415 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:08.524614  1538 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:08.592484  1718 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.014s	sys 0.018s
I20260709 13:05:08.527063  8371 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:08.632865  1718 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.016s	sys 0.024s
W20260709 13:05:08.633133  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 164528us
W20260709 13:05:08.626557  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 102088us
I20260709 13:05:08.477555  8029 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:08.640379  8405 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 13:05:08.648188  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:08.649008  8580 log.cc:927] Time spent T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.752s	user 0.004s	sys 0.000s
I20260709 13:05:08.652245  8371 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:08.652812  1612 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:08.655406  8415 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:08.656001  8574 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:08.657232  8282 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:08.654273  1687 leader_election.cc:336] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.520s (SENT)
W20260709 13:05:08.641443  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.664712  1567 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:05:08.674366  8396 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.674489  1687 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:08.675139  8396 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:08.677065  8421 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:08.679462  1629 tablet_service.cc:1558] Processing DeleteTablet for tablet 59d236b930214b1f91fb55ece6fecd68 with delete_type TABLET_DATA_TOMBSTONED (TS f5f4b2707a9647e5bb99e065068dea2b not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:47496
I20260709 13:05:08.655340  8572 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:08.698088  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:08.698483  1612 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:08.698899  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:08.682933  8561 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:08.674366  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 1156) took 2415 ms. Trace:
I20260709 13:05:08.700243  8396 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:08.700300  1570 rpcz_store.cc:276] 0709 13:05:06.253715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.398602 (+144887us) service_pool.cc:224] Handling call
0709 13:05:06.462025 (+ 63423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:06.468302 (+  6277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:06.468336 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:06.472668 (+  4332us) log.cc:844] Serialized 26 byte log entry
0709 13:05:07.145957 (+673289us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b180009c4c0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a250d1 00007f8248e40a0f
0709 13:05:07.462096 (+316139us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:07.462159 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:07.462344 (+   185us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:08.089772 (+627428us) raft_consensus.cc:1685] finished
0709 13:05:08.089819 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.668596 (+578777us) spinlock_profiling.cc:243] Waited 62.6 ms on lock 0x7b54001275d0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:08.668810 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7755,"raft.queue_time_us":6976,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":5376,"spinlock_wait_cycles":151157504,"thread_start_us":315694,"threads_started":1,"wal-append.queue_time_us":1347294}
I20260709 13:05:08.709892  8415 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:08.710453  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 56223.8us
I20260709 13:05:08.710909  8415 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:08.712519  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:08.714104  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:05:08.715912  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 201956us
I20260709 13:05:08.717368  8370 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:08.717430  8554 consensus_queue.cc:1059] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:08.718032  8370 raft_consensus.cc:697] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:08.719540  1685 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:08.724830  2349 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:05:08.710589  8574 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:05:08.731868  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 68212.4us
I20260709 13:05:08.740249  1567 raft_consensus.cc:1240] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:08.741209  1569 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:05:08.747960  1717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:05:08.749872  1717 raft_consensus.cc:606] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Received request to transfer leadership to TS f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:08.750319  1717 raft_consensus.cc:614] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:08.719833  8371 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:08.757782  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:08.753122  1686 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:08.741900  8481 raft_consensus.cc:3055] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:05:08.763437  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:08.764732  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.747382  8396 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.766539  8371 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:08.749159  8421 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:05:08.767387  8371 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:08.751780  8561 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.779417  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:08.780067  8600 log.cc:927] Time spent T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:05:08.780356  1713 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:05:08.766341  8396 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:08.719331  8370 consensus_queue.cc:237] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:08.791903  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.781495  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:08.784547  2349 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:05:08.794744  1713 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:08.783634  8589 log.cc:927] Time spent T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.485s	user 0.001s	sys 0.001s
I20260709 13:05:08.756557  8572 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:05:08.763648  8481 raft_consensus.cc:740] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:08.784847  8574 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:08.781296  8029 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:08.817812  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 98345.9us
W20260709 13:05:08.848511  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 74691.8us
I20260709 13:05:08.794416  8561 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:08.854560  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:08.854805  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.852962  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:08.854729  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 209243us
I20260709 13:05:08.859315  1685 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:08.810429  8574 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:08.863739  1717 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:05:08.800881  8604 consensus_queue.cc:1059] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 13:05:08.867367  8574 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:08.882845  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:05:08.887020  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 170202us
I20260709 13:05:08.893946  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2892) took 1164 ms. Trace:
I20260709 13:05:08.898517  1648 rpcz_store.cc:276] 0709 13:05:07.729179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.729349 (+   170us) service_pool.cc:224] Handling call
0709 13:05:07.788308 (+ 58959us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:07.821782 (+ 33474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:07.821815 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:07.877721 (+ 55906us) log.cc:844] Serialized 26 byte log entry
0709 13:05:08.045193 (+167472us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:08.045235 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.045311 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:08.782954 (+737643us) spinlock_profiling.cc:243] Waited 9.06 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:05:08.893264 (+110310us) raft_consensus.cc:1685] finished
0709 13:05:08.893389 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.893674 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1697,"raft.queue_time_us":26406,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":23296,"spinlock_wait_cycles":20833024,"thread_start_us":85994,"threads_started":1,"wal-append.queue_time_us":344513}
I20260709 13:05:08.908793  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:05:08.909300  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.909582  1717 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:08.917528  8607 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:08.918196  8396 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:08.919059  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 98262.9us
I20260709 13:05:08.919996  1685 leader_election.cc:400] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 16. Current term is 17.
I20260709 13:05:08.921526  1685 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:08.923141  1685 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:08.925081  8572 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:08.925590  8572 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:08.926136  8572 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:08.926146  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:08.917407  8610 tablet_replica.cc:333] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:05:08.930239  8607 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:08.930842  8607 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.931322  8607 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:08.935439  8566 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:08.940032  8421 consensus_queue.cc:1059] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:08.941578  8421 raft_consensus.cc:3055] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:08.942155  8421 raft_consensus.cc:740] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:08.942335  1611 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:08.943463  8481 consensus_queue.cc:260] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:08.944769  8481 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:08.946097  1685 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:08.946586  8561 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:08.950470  8614 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:08.914232  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 201816us
I20260709 13:05:08.951908  1611 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:08.955899  1713 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:08.958002  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2876) took 1003 ms. Trace:
I20260709 13:05:08.958851  1714 rpcz_store.cc:276] 0709 13:05:07.955086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.121302 (+166216us) service_pool.cc:224] Handling call
0709 13:05:08.957726 (+836424us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b5400248954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:08.957923 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355285376}
I20260709 13:05:08.959645  1537 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:08.962445  8282 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:08.963140  8282 raft_consensus.cc:697] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:05:08.963485  8603 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.066s	user 0.003s	sys 0.000s
I20260709 13:05:08.964574  8282 consensus_queue.cc:237] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:08.963219  8608 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:08.966423  8608 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:08.966830  8608 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:08.975798  1687 leader_election.cc:336] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.684s (SENT)
I20260709 13:05:08.976486  1687 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:08.976625  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:08.977862  1714 raft_consensus.cc:2393] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 18: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:08.979694  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:08.981621  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:08.988507  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3257) took 1130 ms. Trace:
I20260709 13:05:08.996932  1717 raft_consensus.cc:2410] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:05:08.993795  8566 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:08.989486  1721 rpcz_store.cc:276] 0709 13:05:07.857763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.417820 (+560057us) service_pool.cc:224] Handling call
0709 13:05:08.433337 (+ 15517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:08.462175 (+ 28838us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:08.462230 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:08.483261 (+ 21031us) log.cc:844] Serialized 26 byte log entry
0709 13:05:08.754015 (+270754us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:08.755432 (+  1417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.755535 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:08.981851 (+226316us) raft_consensus.cc:1685] finished
0709 13:05:08.981879 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.988411 (+  6532us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2557,"raft.queue_time_us":96,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":8919,"thread_start_us":232153,"threads_started":1,"wal-append.queue_time_us":352062}
I20260709 13:05:09.002789  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:09.003978  1647 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:09.006392  8543 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:09.007433  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:09.011026  1646 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:09.013746  8416 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.014511  8416 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:09.015184  1639 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:09.016582  1642 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:08.998392  8561 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:09.029300  8608 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:09.031611  8608 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:08.943487  8421 consensus_queue.cc:260] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:09.022197  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 46509us
I20260709 13:05:09.022634  1537 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:09.035449  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:09.022961  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:09.036837  8610 raft_consensus.cc:2243] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:09.037911  2343 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55890 (request call id 642) took 1304 ms. Trace:
I20260709 13:05:09.038085  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2869) took 1246 ms. Trace:
I20260709 13:05:09.038704  1715 rpcz_store.cc:276] 0709 13:05:07.791897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.817477 (+ 25580us) service_pool.cc:224] Handling call
0709 13:05:08.232079 (+414602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.250810 (+ 18731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.037840 (+787030us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b5400407c50. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:09.037992 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":952020736}
I20260709 13:05:09.040097  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3251) took 1318 ms. Trace:
I20260709 13:05:09.040172  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:09.041638  1715 raft_consensus.cc:2393] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 15: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:09.042698  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:09.043747  8605 raft_consensus.cc:3055] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:09.020035  8545 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:09.025506  8596 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.562s	user 0.005s	sys 0.000s
I20260709 13:05:09.046918  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 759) took 1314 ms. Trace:
I20260709 13:05:09.047051  1566 raft_consensus.cc:1240] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:09.047381  1719 rpcz_store.cc:276] 0709 13:05:07.742443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.744770 (+  2327us) service_pool.cc:224] Handling call
0709 13:05:07.745446 (+   676us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:07.746393 (+   947us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:07.746436 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:07.747319 (+   883us) log.cc:844] Serialized 52 byte log entry
0709 13:05:08.315697 (+568378us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:08.315754 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.315843 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:09.007474 (+691631us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:05:09.046612 (+ 39138us) raft_consensus.cc:1685] finished
0709 13:05:09.046633 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.046860 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2223,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":168,"spinlock_wait_cycles":32951168,"thread_start_us":567604,"threads_started":1,"wal-append.queue_time_us":611909}
I20260709 13:05:09.050153  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:09.050211  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:09.031828  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:09.051661  1719 raft_consensus.cc:2393] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 15: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:09.051839  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:09.052378  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:09.052672  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:09.040822  1718 rpcz_store.cc:276] 0709 13:05:07.721844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.727543 (+  5699us) service_pool.cc:224] Handling call
0709 13:05:07.748630 (+ 21087us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:07.867213 (+118583us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:07.932889 (+ 65676us) log.cc:844] Serialized 14 byte log entry
0709 13:05:08.633386 (+700497us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:08.633450 (+    64us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:08.633508 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.038849 (+405341us) spinlock_profiling.cc:243] Waited 8.36 ms on lock 0x7b18003204a0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d662a6 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419
0709 13:05:09.039769 (+   920us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b5400299fd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:09.039833 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.040012 (+   179us) inbound_call.cc:177] Queueing success response
0709 13:05:09.040219 (+   207us) spinlock_profiling.cc:243] Waited 40.5 ms on lock 0x7b18003204a0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
Metrics: {"mutex_wait_us":18269,"prepare.queue_time_us":190687,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":790,"raft.queue_time_us":118,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":27129,"spinlock_wait_cycles":384425856,"thread_start_us":714466,"threads_started":2,"wal-append.queue_time_us":723457}
I20260709 13:05:09.055055  8611 consensus_queue.cc:1048] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:05:09.017326  8416 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:09.044507  8605 raft_consensus.cc:740] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:09.058645  8605 consensus_queue.cc:260] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:09.037429  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:09.038559  2343 rpcz_store.cc:276] 0709 13:05:07.739241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.748298 (+  9057us) service_pool.cc:224] Handling call
0709 13:05:09.037690 (+1289392us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b5400467654. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:09.037799 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1054714240}
I20260709 13:05:09.024377  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:09.064172  1642 raft_consensus.cc:2376] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 11. Re-sending same reply.
I20260709 13:05:09.069567  8416 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:09.070650  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 118861us
I20260709 13:05:09.000291  1714 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
W20260709 13:05:09.071426  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 125407us
I20260709 13:05:09.044029  1715 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:09.074113  1713 raft_consensus.cc:1240] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:09.077612  8421 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:09.075721  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 67180.9us
I20260709 13:05:09.079008  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:09.079469  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 56892.4us
I20260709 13:05:09.081051  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:09.081256  1537 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:09.001610  8574 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:09.102129  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:09.094506  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 58164.4us
I20260709 13:05:09.084342  1720 raft_consensus.cc:1217] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Deduplicated request from leader. Original: 13.8->[16.9-20.11]   Dedup: 16.9->[18.10-20.11]
I20260709 13:05:09.098500  1717 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:09.100497  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:09.112183  1716 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:05:09.114670  1647 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17.
I20260709 13:05:09.116976  1714 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:05:09.120695  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:09.121003  1611 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:09.121958  2341 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:09.123003  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:09.123725  1611 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:09.122025  8566 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:09.070535  8487 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:09.124640  8566 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:09.125151  8566 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:09.126636  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:09.128997  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:09.129735  2341 raft_consensus.cc:2368] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11.
I20260709 13:05:09.129904  8608 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:09.131807  1611 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:09.131732  8607 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:09.133086  8211 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.154846  8211 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:09.155416  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:09.132936  8594 consensus_queue.cc:1059] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:09.134126  8572 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:09.135318  8485 consensus_queue.cc:1059] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:05:09.094069  8615 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:09.137923  1718 raft_consensus.cc:1240] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:09.138996  8415 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:09.142446  1715 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:09.145994  1713 raft_consensus.cc:2393] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:09.146436  8611 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:09.146485  1717 raft_consensus.cc:1240] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:09.114925  8605 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:09.109525  1720 pending_rounds.cc:85] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 9
I20260709 13:05:09.176030  1720 pending_rounds.cc:107] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 13:05:09.176365  1720 raft_consensus.cc:2889] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:09.177006  8211 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:09.177157  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45479us
I20260709 13:05:09.154038  8396 consensus_queue.cc:1048] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:05:09.094467  8610 raft_consensus.cc:2272] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:09.153973  8607 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:09.168326  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.121766  8595 raft_consensus.cc:3055] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:05:09.175222  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.115304  8543 raft_consensus.cc:697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:09.188261  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:09.188616  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:09.179492  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 79235.9us
I20260709 13:05:09.187486  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:09.187732  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:09.186228  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:09.191015  1643 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:09.070537  8416 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:09.189826  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:09.192546  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:09.192780  1646 raft_consensus.cc:2376] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16. Re-sending same reply.
I20260709 13:05:09.193066  1722 raft_consensus.cc:1275] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:05:09.193511  1641 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 15: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:09.170967  8608 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:09.172875  8615 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.197088  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:05:09.197919  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 86294.7us
I20260709 13:05:09.194397  1685 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:09.203635  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:09.204428  1716 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:09.171380  8029 consensus_queue.cc:1059] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:05:09.185124  8595 raft_consensus.cc:740] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:05:09.212746  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.198788  2314 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:09.214643  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:09.217793  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:09.218675  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:09.219244  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:09.225395  1647 raft_consensus.cc:2393] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 15: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:05:09.225445  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.226747  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:09.228890  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:09.201931  8615 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:09.231693  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:09.232812  1642 raft_consensus.cc:2393] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 18: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:09.233736  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:09.233934  8618 consensus_queue.cc:1059] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 13:05:09.237453  8595 consensus_queue.cc:260] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:09.243557  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 65421.9us
I20260709 13:05:09.242105  1646 raft_consensus.cc:2376] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16. Re-sending same reply.
I20260709 13:05:09.244385  1641 raft_consensus.cc:2393] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:09.214977  8608 raft_consensus.cc:697] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:09.232280  8282 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:09.188889  8543 consensus_queue.cc:237] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:09.245069  8594 raft_consensus.cc:3055] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:09.247578  8614 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:05:09.274156  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 46059.5us
I20260709 13:05:09.285382  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:09.263376  8595 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:09.260717  8282 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:09.263931  8594 raft_consensus.cc:740] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:05:09.290087  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:09.294552  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 139191us
I20260709 13:05:09.266916  8211 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:05:09.302410  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:09.302446  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:09.304008  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:09.305284  1687 leader_election.cc:336] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.835s (SENT)
W20260709 13:05:09.305187  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 106436us
I20260709 13:05:09.310622  1687 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:09.311997  8493 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:09.312721  8493 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:09.313731  8211 raft_consensus.cc:697] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:09.315325  8211 consensus_queue.cc:237] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:09.316136  8493 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:09.316969  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.319536  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:09.311403  8611 raft_consensus.cc:697] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:09.322721  8611 consensus_queue.cc:237] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:09.297791  8371 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:09.302065  8282 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:09.329226  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 44434.8us
I20260709 13:05:09.330082  8545 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:09.330255  1611 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:09.330797  8624 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:09.331444  8624 raft_consensus.cc:697] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:09.331749  1611 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:09.332553  8610 ts_tablet_manager.cc:1916] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:09.332516  8624 consensus_queue.cc:237] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:09.333809  8405 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:09.334301  8371 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:09.338032  8572 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:09.339809  8416 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:09.342808  8627 log.cc:927] Time spent T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:05:09.344998  8574 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:09.348124  1612 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:09.366569  1538 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:09.382122  8396 raft_consensus.cc:3055] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:09.339951  8594 consensus_queue.cc:260] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:09.382767  8396 raft_consensus.cc:740] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:09.383728  8396 consensus_queue.cc:260] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:09.385952  8396 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:09.388815  1538 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:09.391170  8602 raft_consensus.cc:3055] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:09.391939  8602 raft_consensus.cc:740] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:09.382022  8634 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:09.405073  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 104471us
W20260709 13:05:09.408053  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.346076  8608 consensus_queue.cc:237] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:09.406085  1687 leader_election.cc:336] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.841s (SENT)
W20260709 13:05:09.432544  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 43798.8us
I20260709 13:05:09.434180  1538 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:09.435539  8396 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:09.435590  8029 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:09.425232  8566 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:09.440244  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:09.440797  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 273219us
I20260709 13:05:09.441044  2349 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:05:09.444229  1537 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:09.402070  8211 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.383869  8594 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:09.466048  8633 consensus_queue.cc:1048] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:09.472255  1718 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:05:09.451596  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 103544us
I20260709 13:05:09.481833  1537 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:09.470400  8211 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:09.500316  1612 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:09.501610  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.480413  8639 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:09.429026  8602 consensus_queue.cc:260] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:09.514820  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 183185us
I20260709 13:05:09.507036  2313 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:09.522902  1685 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:09.527956  1718 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
I20260709 13:05:09.528898  1718 raft_consensus.cc:606] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:09.529310  1718 raft_consensus.cc:614] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:09.529768  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:09.538471  1715 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:09.502537  8611 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:05:09.535555  1612 raft_consensus.cc:2686] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:05:09.545473  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:09.512128  8639 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.547816  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:09.545213  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:09.550922  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:05:09.545185  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 44948.1us
I20260709 13:05:09.554636  1612 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:09.556232  1713 raft_consensus.cc:1275] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:05:09.517006  8629 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.565841  8643 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:09.572036  8566 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:09.572717  8635 log.cc:927] Time spent T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:05:09.590811  2341 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:09.573282  8211 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:09.537514  8029 consensus_queue.cc:1048] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:05:09.534917  8605 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:09.578657  1611 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:09.580025  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:05:09.582984  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 76099us
I20260709 13:05:09.544924  8611 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.544880  8421 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:09.591790  1537 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:09.598672  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 44159.3us
I20260709 13:05:09.598917  8649 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:09.603541  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 244295us
I20260709 13:05:09.609218  1612 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:09.553277  8639 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:09.611829  1566 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 22.
I20260709 13:05:09.621071  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:09.590291  8543 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:09.628212  8652 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:09.590903  8643 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:09.624720  2312 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:09.545399  8637 consensus_queue.cc:1059] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
W20260709 13:05:09.633028  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.611953  8611 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:09.616957  8566 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:09.643533  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:09.645042  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:09.635778  8643 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:09.646780  8405 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:09.647926  8633 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:09.647840  8405 consensus_queue.cc:1059] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:09.648839  8633 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:09.626164  8554 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:09.624773  8421 raft_consensus.cc:697] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:05:09.650741  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.631500  8647 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:05:09.664613  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:09.652293  8641 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:05:09.599630  8629 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:09.618259  8211 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:05:09.673468  8655 consensus_queue.cc:1059] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:09.649235  8405 consensus_queue.cc:1059] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
W20260709 13:05:09.691864  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 113322us
I20260709 13:05:09.640933  8566 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.702553  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:09.625177  8649 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:09.649819  8633 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:09.696050  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:09.635003  8543 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:09.703796  1612 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:09.706067  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:09.706346  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:09.682339  8647 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.677795  1717 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:09.713016  1646 raft_consensus.cc:2376] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17. Re-sending same reply.
W20260709 13:05:09.692962  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 48013.7us
I20260709 13:05:09.714646  1642 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
I20260709 13:05:09.717749  1537 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:09.729768  1611 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:09.731601  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.733810  1537 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:09.712460  8566 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:09.736532  1648 raft_consensus.cc:2410] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:05:09.737511  1715 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 21.
W20260709 13:05:09.736737  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 117599us
W20260709 13:05:09.743558  1538 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.659s (SENT)
I20260709 13:05:09.754065  1611 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:09.726733  8647 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:09.775698  8655 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:09.776887  8655 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:09.778615  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 44889.4us
I20260709 13:05:09.736508  8605 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:05:09.781821  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 157218us
I20260709 13:05:09.743073  8630 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:05:09.787585  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 44072.7us
W20260709 13:05:09.791173  1538 leader_election.cc:336] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.660s (SENT)
I20260709 13:05:09.791778  1538 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:09.771057  8615 consensus_meta.cc:287] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.513s	user 0.012s	sys 0.001s
I20260709 13:05:09.725886  8633 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:09.768640  8632 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.728863  8652 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 13:05:09.782531  8605 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.811440  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:09.812355  1565 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:05:09.806421  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:09.806632  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:05:09.806784  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 129870us
I20260709 13:05:09.807392  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:09.810760  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:09.790704  8630 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.806105  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:09.815392  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:09.813337  1572 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
W20260709 13:05:09.823431  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.822490  1571 raft_consensus.cc:2393] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 15: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:09.820451  1568 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:05:09.796648  8648 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:09.836180  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:09.817252  8605 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:05:09.838788  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 135097us
I20260709 13:05:09.841440  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:09.844086  1570 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:05:09.820861  8630 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:09.847265  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:09.844271  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:09.844877  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:09.847874  1572 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:05:09.819128  8634 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:09.845065  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:09.784435  8421 consensus_queue.cc:237] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:09.803097  8554 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:05:09.848881  1612 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:09.852422  1568 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 19: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:09.855417  1573 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:05:09.853405  1537 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:09.804435  8282 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:09.858901  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 67772.2us
W20260709 13:05:09.863070  1538 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.768s (SENT) [suppressed 41 similar messages]
I20260709 13:05:09.836848  8656 raft_consensus.cc:3055] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:09.854947  1685 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:05:09.852320  8371 consensus_meta.cc:287] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.517s	user 0.015s	sys 0.000s
I20260709 13:05:09.804200  8615 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:09.868482  1685 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:09.877754  1572 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:05:09.879371  1612 leader_election.cc:400] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11.
I20260709 13:05:09.879808  1612 leader_election.cc:403] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:09.877560  1713 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:05:09.884260  1719 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:05:09.846871  8648 raft_consensus.cc:697] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:09.833837  8632 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:09.875830  1685 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:09.892556  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:09.899922  1564 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:09.864738  1538 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:09.905566  1538 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:09.891157  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:09.908622  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:09.909798  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:09.910251  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:09.910797  1647 raft_consensus.cc:2410] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 19 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:05:09.910876  1644 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 11.
I20260709 13:05:09.862401  8349 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:09.868836  8624 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:09.803694  8655 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:09.913060  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:09.916254  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:09.908403  1685 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:09.920487  1645 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:09.914853  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:09.867352  8656 raft_consensus.cc:740] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:09.934826  8648 consensus_queue.cc:237] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:09.887327  8607 raft_consensus.cc:2764] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:05:09.913853  1538 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:09.897331  8371 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:09.899926  8652 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:09.944617  8656 consensus_queue.cc:260] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:09.968739  8656 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:09.924356  8661 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:09.973423  1641 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:09.937577  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:09.864720  8664 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:09.941675  8605 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:09.923445  8624 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.900281  8481 consensus_queue.cc:1048] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
W20260709 13:05:09.917737  8603 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:05:09.956148  1569 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:05:09.926178  1567 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:05:09.915364  8634 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:09.926656  1640 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:09.928318  1648 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:09.914460  8564 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:10.015965  8564 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:10.017206  8564 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:09.965222  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 51474.4us
I20260709 13:05:09.978178  1566 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:10.024500  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:09.942598  8631 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.521s	user 0.001s	sys 0.003s
I20260709 13:05:09.943935  8655 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:09.999084  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 138097us
I20260709 13:05:09.977216  8607 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:09.942409  8615 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:10.004323  8624 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:10.019466  8605 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:10.020699  1648 raft_consensus.cc:3055] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:10.034258  1648 raft_consensus.cc:740] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:10.021364  8371 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:10.022472  8282 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:10.025200  1538 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:09.995707  2314 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:10.025315  8634 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:10.004719  8649 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:10.029248  8651 log.cc:927] Time spent T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 13:05:10.010648  1645 raft_consensus.cc:1275] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
I20260709 13:05:10.027242  8554 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:10.030149  8594 consensus_meta.cc:287] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.004s
W20260709 13:05:10.052963  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 57376.7us
I20260709 13:05:10.023375  8543 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:10.008906  8652 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 10. Current term is 11.
W20260709 13:05:10.055272  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 146922us
W20260709 13:05:10.033845  8608 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.013s	sys 0.004s
I20260709 13:05:10.050294  1648 consensus_queue.cc:260] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:10.058460  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.028944  8629 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:10.058197  8633 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:10.078311  8632 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:10.088594  8564 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:10.093369  1566 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:05:10.055922  8634 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:10.095695  1685 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:10.096676  1648 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:10.122063  1685 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:05:10.122179  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 368182us
W20260709 13:05:10.124102  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b82d3b49001741eda872554a7ecb2db3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:05:10.126259  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 505301us
W20260709 13:05:10.128329  1687 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.834s (SENT)
W20260709 13:05:10.129441  1687 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.015s (SENT)
I20260709 13:05:10.131744  1687 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:10.120497  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 92466.8us
I20260709 13:05:10.057529  8571 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:10.096845  8608 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.013s	sys 0.004s
I20260709 13:05:10.116921  8607 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:10.167547  8607 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:10.110332  8564 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:10.114317  8658 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.057365  8371 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:10.093540  8566 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:10.151574  8282 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:10.154268  8543 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:10.188450  1611 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:10.199077  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 173901us
I20260709 13:05:10.169873  8675 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:10.217734  1643 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:05:10.215972  1538 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:10.248922  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:10.268388  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.239214  1646 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:05:10.265797  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 116178us
I20260709 13:05:10.275707  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:10.279439  1645 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:05:10.240682  1538 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:10.210242  8349 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:10.219367  8564 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:10.213886  8675 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:10.276212  8614 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.009s	sys 0.009s
I20260709 13:05:10.241850  8633 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:10.291069  8639 consensus_meta.cc:287] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.651s	user 0.009s	sys 0.007s
I20260709 13:05:10.239449  8282 raft_consensus.cc:697] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:10.208952  8543 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:10.294024  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 132581us
I20260709 13:05:10.287492  1648 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 13:05:10.294400  8614 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.868s	user 0.009s	sys 0.010s
I20260709 13:05:10.272159  8674 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:05:10.305903  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.322299  8543 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:10.306432  8671 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.237803  8658 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:10.294377  8675 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:10.335575  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 50709.8us
I20260709 13:05:10.321339  8629 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:10.323117  8674 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:10.343406  1685 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:05:10.347407  8677 log.cc:927] Time spent T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
W20260709 13:05:10.365703  8611 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.716s	user 0.013s	sys 0.000s
W20260709 13:05:10.367913  8643 consensus_meta.cc:287] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.721s	user 0.008s	sys 0.004s
I20260709 13:05:10.342792  8671 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:10.318343  8639 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:10.328751  8564 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:10.375713  1611 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:10.397413  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:10.428843  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:10.350929  8282 consensus_queue.cc:237] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:10.416137  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.321106  8648 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
W20260709 13:05:10.432399  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 89058.2us
I20260709 13:05:10.433418  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:10.434008  1573 raft_consensus.cc:2393] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 22: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:10.435307  8639 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:10.379726  8663 raft_consensus.cc:3055] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:10.456236  8663 raft_consensus.cc:740] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:10.457603  8658 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:10.431576  8421 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:05:10.459733  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 84160.1us
I20260709 13:05:10.460000  8421 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:10.427649  8647 consensus_meta.cc:287] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.629s	user 0.012s	sys 0.001s
I20260709 13:05:10.426836  8643 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:10.462083  8647 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:10.464573  8647 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:10.442526  8628 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 1.002s	user 0.002s	sys 0.004s
W20260709 13:05:10.442341  8630 consensus_meta.cc:287] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.576s	user 0.011s	sys 0.002s
W20260709 13:05:10.443730  8670 log.cc:927] Time spent T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.143s	user 0.001s	sys 0.002s
I20260709 13:05:10.382128  8566 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:10.373595  8633 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 13:05:10.448671  8634 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:10.449235  8681 log.cc:927] Time spent T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:05:10.381762  8602 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:10.369621  8607 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:10.396067  8674 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:10.453925  8564 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:10.455179  8657 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:10.457306  8663 consensus_queue.cc:260] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:10.381021  8624 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:10.468518  1642 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:10.442001  8680 log.cc:927] Time spent T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.053s	user 0.000s	sys 0.002s
W20260709 13:05:10.443434  8683 log.cc:927] Time spent T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:05:10.477741  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3263) took 1733 ms. Trace:
I20260709 13:05:10.492074  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:05:10.520684  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 280076us
I20260709 13:05:10.493114  1611 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:10.442013  8682 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:10.524961  1611 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:10.499841  8421 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:10.525833  8624 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:10.525885  8682 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:10.500550  8654 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 13:05:10.501274  8637 raft_consensus.cc:3055] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:10.527125  8637 raft_consensus.cc:740] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:10.528549  8564 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:10.528705  8637 consensus_queue.cc:260] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:10.464897  8643 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:10.529991  8671 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:10.485193  8630 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:10.530484  8643 consensus_queue.cc:1048] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:10.531303  1611 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:10.426337  8611 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:10.532192  8671 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:10.532419  8630 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:10.532776  8671 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:10.533277  8671 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:10.537595  8692 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:10.538655  8692 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:10.521831  8595 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.542315  8595 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:10.544544  8595 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:10.520583  8607 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:10.545341  8607 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:10.545429  8634 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:10.545825  8607 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:10.546147  8656 consensus_meta.cc:287] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.577s	user 0.006s	sys 0.012s
I20260709 13:05:10.546797  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:10.547117  8663 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:10.548029  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:10.548362  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:10.549597  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:10.550384  1568 raft_consensus.cc:2393] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 16: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:10.553328  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:10.519125  8610 ts_tablet_manager.cc:1929] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 10.8
I20260709 13:05:10.554410  8610 log.cc:1199] T 59d236b930214b1f91fb55ece6fecd68 P f5f4b2707a9647e5bb99e065068dea2b: Deleting WAL directory at /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-1-root/wals/59d236b930214b1f91fb55ece6fecd68
I20260709 13:05:10.556008  8610 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47496 (request call id 0) took 1877 ms. Trace:
I20260709 13:05:10.556273  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:10.559329  2314 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:10.520160  1720 rpcz_store.cc:276] 0709 13:05:08.744320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.053410 (+309090us) service_pool.cc:224] Handling call
0709 13:05:09.083611 (+ 30201us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:09.177185 (+ 93574us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:05:09.177220 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:09.177902 (+   682us) log.cc:844] Serialized 52 byte log entry
0709 13:05:09.300211 (+122309us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:09.300278 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.300396 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:10.265336 (+964940us) spinlock_profiling.cc:243] Waited 9.15 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:05:10.467175 (+201839us) raft_consensus.cc:1685] finished
0709 13:05:10.467397 (+   222us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:10.477572 (+ 10175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4427,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":7495,"spinlock_wait_cycles":21043968,"thread_start_us":121744,"threads_started":1,"wal-append.queue_time_us":186648}
I20260709 13:05:10.529268  8566 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:10.502259  8371 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:10.554143  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:05:10.563748  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:10.569201  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:10.569873  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.573987  1641 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:05:10.534340  8611 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:10.541482  8692 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:10.513108  1566 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:05:10.594655  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:10.600078  1538 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:10.603624  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:10.605135  1566 raft_consensus.cc:2393] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:10.576946  8657 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:10.611939  1538 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:10.605893  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 130105us
I20260709 13:05:10.641664  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:10.615468  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:10.642403  1715 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:10.642746  1720 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:10.616626  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:10.643570  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:10.646139  1718 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:10.649060  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:10.649950  1566 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 18: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:10.618501  1685 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:10.661896  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:10.662695  1564 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
W20260709 13:05:10.663657  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.666087  1537 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:05:10.666448  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 536925us
I20260709 13:05:10.667117  8396 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:10.668030  8396 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:10.668309  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:10.668423  8396 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:10.580298  8371 consensus_queue.cc:1059] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:05:10.670113  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:10.670827  8371 raft_consensus.cc:3055] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:10.671559  8371 raft_consensus.cc:740] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:05:10.673666  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 71418.5us
I20260709 13:05:10.556420  8610 rpcz_store.cc:276] 0709 13:05:08.678782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.679106 (+   324us) service_pool.cc:224] Handling call
0709 13:05:08.916681 (+237575us) ts_tablet_manager.cc:1151] Deleting tablet 59d236b930214b1f91fb55ece6fecd68
0709 13:05:08.916818 (+   137us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:05:10.156299 (+1239481us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:10.514595 (+358296us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:10.555910 (+ 41315us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111262,"tablet-delete.queue_time_us":222567,"thread_start_us":179186,"threads_started":1}
I20260709 13:05:10.679193  8543 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:10.682152  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:10.685789  8637 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:10.686206  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:10.687150  1567 raft_consensus.cc:2410] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:05:10.666159  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:10.670853  1721 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:10.667184  1687 leader_election.cc:336] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.102s (SENT)
I20260709 13:05:10.692852  1687 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:10.618217  8657 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:10.581928  8595 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:05:10.688576  1715 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:10.612813  8689 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:10.698401  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:10.599567  8633 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:05:10.716722  8633 consensus_queue.cc:1243] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:05:10.698833  1537 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:10.694808  8657 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:10.657011  8611 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:05:10.714756  8691 log.cc:927] Time spent T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:05:10.715091  8543 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:10.715489  8673 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:10.737474  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:10.715734  8607 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:10.706516  8370 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:05:10.730165  1538 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:10.739826  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:10.736692  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:10.694634  8481 consensus_queue.cc:1048] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:05:10.718353  8689 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:10.734463  8029 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:10.743832  8689 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:10.734215  8630 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:10.744248  8029 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:10.747537  8564 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:05:10.748941  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 81810.8us
I20260709 13:05:10.750661  8667 raft_consensus.cc:3060] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:10.754277  8673 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:10.755107  1641 raft_consensus.cc:1275] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:05:10.756064  8675 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:10.756397  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:10.756840  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:10.723066  8611 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:10.696213  8692 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:05:10.758059  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:10.758955  1645 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 16: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:10.761690  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:10.762760  1564 raft_consensus.cc:2393] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 19: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:10.767000  1648 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:10.757004  8673 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:10.757205  1643 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 18: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:10.747714  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:10.781261  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:10.783560  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:10.789589  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:10.796989  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:10.729152  8396 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:10.799436  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:10.816708  1715 raft_consensus.cc:2393] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:10.802582  1564 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:10.806916  8675 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:10.807700  8029 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:10.811228  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:10.819716  1716 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 20: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:10.814287  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:10.758102  8611 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:10.785866  8564 raft_consensus.cc:697] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:10.828138  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:10.835212  1715 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:10.831923  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:10.832473  1719 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:10.832571  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:10.836805  1648 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:10.840032  1716 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 19: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:10.795775  8649 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.839918  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:10.833283  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:10.810177  8543 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:10.809191  8630 raft_consensus.cc:697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:10.846033  1567 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 18: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:10.805086  8671 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:10.844496  1572 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
I20260709 13:05:10.847342  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2934) took 1052 ms. Trace:
I20260709 13:05:10.858292  1569 rpcz_store.cc:276] 0709 13:05:09.794586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.802565 (+  7979us) service_pool.cc:224] Handling call
0709 13:05:09.802867 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:10.009581 (+206714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.847075 (+837494us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b5400406fd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:10.847279 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":290400128}
I20260709 13:05:10.835239  1643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:55704
I20260709 13:05:10.848189  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:10.862869  1643 raft_consensus.cc:606] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:10.863443  1643 raft_consensus.cc:614] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:10.847973  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:10.842854  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:10.865181  1645 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:05:10.860618  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:10.870198  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:10.871516  1640 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 20: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:10.872257  1538 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:10.791684  8571 consensus_queue.cc:1059] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
W20260709 13:05:10.880225  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.889284  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:10.889730  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:10.891202  1645 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:10.891510  8564 consensus_queue.cc:237] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:10.892725  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:10.893792  8370 consensus_queue.cc:1048] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:10.856570  8649 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:10.896168  8370 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:10.894337  1719 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:10.899147  1645 raft_consensus.cc:2393] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:05:10.909737  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.920408  1715 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:10.933069  1644 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:10.877776  8554 consensus_meta.cc:287] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.805s	user 0.013s	sys 0.002s
W20260709 13:05:10.934409  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 315927us
I20260709 13:05:10.938750  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:10.938750  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:10.968245  2343 raft_consensus.cc:2368] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 22. Current term is 23.
I20260709 13:05:10.939877  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:10.943351  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:10.943825  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:10.936625  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:10.971735  2349 raft_consensus.cc:2393] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 14: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:10.972648  1645 raft_consensus.cc:2393] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 18: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:10.915946  8663 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.945719  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:10.943742  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:10.974182  1719 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:10.915299  8493 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:10.947928  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:10.951813  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.937316  8595 raft_consensus.cc:3055] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:10.948273  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:10.958039  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:10.972015  2345 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 21.
I20260709 13:05:10.967445  1685 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:10.978369  1612 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:10.977149  2346 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:05:10.996215  1686 leader_election.cc:400] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 22. Current term is 23.
I20260709 13:05:10.999399  1686 leader_election.cc:403] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:11.005203  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:11.009398  1648 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:11.006911  1612 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:11.017582  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 267914us
I20260709 13:05:11.017976  1686 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:11.020081  8566 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:11.022034  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:10.971622  8554 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:11.027331  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:11.021998  1685 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:11.024279  1686 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:05:11.042546  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.076635  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:10.981551  8595 raft_consensus.cc:740] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:11.077672  1643 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 18: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:11.020361  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:11.082379  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:11.082875  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:11.003970  8649 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:11.083729  1643 raft_consensus.cc:2393] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:11.084383  1566 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:05:11.083660  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:11.091159  1640 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:05:10.980103  8652 consensus_queue.cc:1059] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.096s
W20260709 13:05:11.096360  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 72182.6us
I20260709 13:05:11.096727  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:11.026974  8624 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:05:11.027679  8566 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:11.102068  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.976363  8674 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:11.028237  8543 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:11.113274  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:11.029443  8675 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:11.026974  8656 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:11.126183  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3431) took 1235 ms. Trace:
I20260709 13:05:11.122676  1716 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:05:10.981354  8663 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:11.140003  1721 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:05:11.003232  8493 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:11.100246  8700 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.133199  1642 rpcz_store.cc:276] 0709 13:05:09.890385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.890535 (+   150us) service_pool.cc:224] Handling call
0709 13:05:09.912380 (+ 21845us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:10.518373 (+605993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.125825 (+607452us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b54002db254. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:11.126101 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1277573376}
W20260709 13:05:11.141894  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 269674us
I20260709 13:05:11.150660  1538 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:11.134542  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:11.146504  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:11.104244  8566 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:11.159008  1538 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:11.133278  8652 consensus_queue.cc:1243] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:05:11.127295  8607 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:11.118785  8675 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:11.163929  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:11.199580  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:11.163916  1643 raft_consensus.cc:2393] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 18: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:11.167186  1538 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:10.984302  8630 consensus_queue.cc:237] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:11.151737  8396 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:11.202739  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:11.211869  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.201100  1642 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 18: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:11.220106  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:11.221081  1642 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:05:11.171468  8481 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:05:11.197948  8698 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.351s	user 0.000s	sys 0.002s
W20260709 13:05:11.229167  1720 consensus_meta.cc:287] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.575s	user 0.015s	sys 0.000s
I20260709 13:05:11.229933  1720 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:11.227182  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:11.201309  8615 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:11.211493  8678 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:11.210237  8675 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:11.155261  8700 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:11.233016  1565 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:11.234891  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:11.238022  8396 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:05:11.239153  8701 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:05:11.240077  8702 log.cc:927] Time spent T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 13:05:11.246901  8667 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:11.207705  8647 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:11.232110  1644 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:11.154436  8595 consensus_queue.cc:260] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:11.222932  8481 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:11.251379  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:11.287977  1642 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:11.233441  8712 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:11.308802  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:11.312326  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:11.286051  8656 raft_consensus.cc:697] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:11.316082  1565 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:11.315215  1644 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:05:11.319389  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:11.319998  1644 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:05:11.321451  8674 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:11.292914  8678 raft_consensus.cc:697] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:11.326373  1569 raft_consensus.cc:2468] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
I20260709 13:05:11.320786  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:11.306021  8481 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:11.300459  8396 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:11.304141  8647 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:11.382416  8493 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:11.383461  8707 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:05:11.386561  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:11.389137  8678 consensus_queue.cc:237] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:11.400328  8637 consensus_meta.cc:287] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.714s	user 0.013s	sys 0.000s
I20260709 13:05:11.369460  8663 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:11.408588  1644 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 21.
I20260709 13:05:11.325093  8711 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.440052  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:11.440325  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:11.444933  8595 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:11.424839  8029 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:11.421656  1573 raft_consensus.cc:1240] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 11. Current term is 12. Ops: []
W20260709 13:05:11.423863  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.441574  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:11.431982  8714 log.cc:927] Time spent T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:05:11.438032  8349 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.439882  8711 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:11.467490  8349 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:11.474885  8700 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:11.477890  8602 consensus_meta.cc:287] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.970s	user 0.012s	sys 0.004s
I20260709 13:05:11.478684  8618 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.480504  8656 consensus_queue.cc:237] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:11.485149  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.486538  8648 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.487334  8648 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:11.490721  8657 consensus_meta.cc:287] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.753s	user 0.009s	sys 0.005s
I20260709 13:05:11.420586  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2945) took 1618 ms. Trace:
I20260709 13:05:11.497916  8481 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:11.499707  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:11.500387  1568 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:11.449463  1644 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 18: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:11.446499  8029 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:11.462785  8615 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:11.464057  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:11.450017  8370 consensus_meta.cc:287] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.553s	user 0.015s	sys 0.000s
I20260709 13:05:11.465530  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:11.502933  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:11.511829  1565 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 21.
I20260709 13:05:11.512403  1644 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 20: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:11.503489  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:11.507054  1646 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:05:11.492854  1570 rpcz_store.cc:276] 0709 13:05:09.802377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.846289 (+ 43912us) service_pool.cc:224] Handling call
0709 13:05:09.846591 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.846810 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:09.870564 (+ 23754us) log.cc:844] Serialized 14 byte log entry
0709 13:05:10.434439 (+563875us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b1800299fe0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a2523a 00007f8248e8fa2a
0709 13:05:11.225683 (+791244us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.331742 (+106059us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.418645 (+ 86903us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:11.418733 (+    88us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:11.420215 (+  1482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.420293 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.420509 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":185155,"prepare.queue_time_us":522474,"prepare.run_cpu_time_us":170,"prepare.run_wall_time_us":8801,"raft.queue_time_us":149673,"raft.run_cpu_time_us":621,"raft.run_wall_time_us":19285,"spinlock_wait_cycles":30045568,"thread_start_us":786219,"threads_started":2,"wal-append.queue_time_us":968768}
I20260709 13:05:11.514629  1640 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:11.514544  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:11.514946  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:11.517056  1644 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:05:11.518164  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:11.518632  1644 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:11.511188  8647 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:11.524983  8029 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:11.479489  8618 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:11.543566  1641 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:05:11.548887  1572 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:11.544622  1648 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:05:11.528843  8624 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 22. Current term is 23.
I20260709 13:05:11.506597  8602 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:11.520720  2312 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:11.517230  8370 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:11.525568  8689 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.781s	user 0.008s	sys 0.010s
I20260709 13:05:11.520558  8721 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.574357  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:11.491490  8657 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:11.522778  8611 consensus_meta.cc:287] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.682s	user 0.015s	sys 0.000s
I20260709 13:05:11.558043  2341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55890 (request call id 651) took 1162 ms. Trace:
I20260709 13:05:11.595289  1646 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:05:11.560367  1642 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:05:11.586102  1568 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 23.
I20260709 13:05:11.591178  8721 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:11.594081  1644 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:11.595584  2341 rpcz_store.cc:276] 0709 13:05:10.390142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.404406 (+ 14264us) service_pool.cc:224] Handling call
0709 13:05:10.404995 (+   589us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:10.766421 (+361426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:10.766480 (+    59us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:05:10.801588 (+ 35108us) log.cc:844] Serialized 78 byte log entry
0709 13:05:11.233628 (+432040us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b18000720a0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a250d1 00007f8248e40a0f
0709 13:05:11.284609 (+ 50981us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:11.539001 (+254392us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b18000720a0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d95455 00007f824c87d3c7 00007f8248e8f9a7 00007f8248e9f0bf
0709 13:05:11.539230 (+   229us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.539517 (+   287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.541332 (+  1815us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b54005a4454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:11.541381 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:11.541427 (+    46us) raft_consensus.cc:1685] finished
0709 13:05:11.541440 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.551331 (+  9891us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30815,"prepare.queue_time_us":185920,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":66746,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":10217,"spinlock_wait_cycles":802742144,"thread_start_us":89387,"threads_started":2,"wal-append.queue_time_us":520568}
I20260709 13:05:11.582213  8689 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:11.517004  8646 consensus_queue.cc:1059] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:05:11.618270  8646 raft_consensus.cc:3055] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:11.618714  8646 raft_consensus.cc:740] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:11.619518  1571 raft_consensus.cc:2368] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 17. Current term is 18.
I20260709 13:05:11.619640  8646 consensus_queue.cc:260] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:11.621075  8646 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:11.595919  8647 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.622061  8647 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:11.622682  1565 raft_consensus.cc:1240] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:11.596902  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 125527us
I20260709 13:05:11.563845  8658 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:11.625867  8658 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:11.626286  8658 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:11.630301  8211 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.630947  8211 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:11.633016  8371 consensus_queue.cc:260] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:11.634313  8371 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:11.634382  8675 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:11.637058  8675 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:11.606057  8611 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:11.558862  8602 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:11.639340  8611 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:11.639575  8602 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:11.640633  8657 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:11.642469  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:11.642876  8566 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:11.645134  8721 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:11.647903  1537 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:11.631481  8639 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:11.645073  8566 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:11.655126  1639 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:05:11.678655  1646 pending_rounds.cc:85] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 10
I20260709 13:05:11.698709  1646 pending_rounds.cc:107] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:05:11.699167  1646 raft_consensus.cc:2889] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:11.666980  8689 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:11.705160  8632 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.655551  8721 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:11.713253  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:11.717106  1572 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19.
I20260709 13:05:11.661029  8618 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:11.717166  8632 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:11.720660  8632 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:11.654459  8647 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:11.689491  8634 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:11.722080  8634 raft_consensus.cc:697] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 20 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:11.723246  8634 consensus_queue.cc:237] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:11.726501  8632 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.727149  8632 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:11.713932  8211 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:11.716955  8721 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:11.737778  8721 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:11.743022  8671 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:11.749119  8632 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:11.753082  8703 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:11.753803  8703 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:11.754065  8692 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:11.754284  8703 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:11.756031  8719 log.cc:927] Time spent T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.310s	user 0.001s	sys 0.000s
I20260709 13:05:11.776523  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:11.783393  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.787986  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3441) took 1865 ms. Trace:
I20260709 13:05:11.777376  2341 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:11.821054  1538 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:11.822300  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.788647  1647 rpcz_store.cc:276] 0709 13:05:09.922931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.923275 (+   344us) service_pool.cc:224] Handling call
0709 13:05:09.945516 (+ 22241us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:10.062410 (+116894us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:10.062441 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:10.073627 (+ 11186us) log.cc:844] Serialized 26 byte log entry
0709 13:05:10.255014 (+181387us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:10.287010 (+ 31996us) log.cc:844] Serialized 14 byte log entry
0709 13:05:10.901341 (+614331us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b1800178840. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d662a6 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e00733 00007f8248e85dbf 00007f8248e8483e
0709 13:05:11.787414 (+886073us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b5400472aa0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e00733 00007f8248e85dbf 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c
0709 13:05:11.787507 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.787641 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:11.787703 (+    62us) raft_consensus.cc:1685] finished
0709 13:05:11.787721 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.787907 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1205,"prepare.queue_time_us":29480,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":3871,"raft.queue_time_us":68949,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":18884,"spinlock_wait_cycles":61346816,"thread_start_us":297008,"threads_started":2,"wal-append.queue_time_us":194096,"wal-append.run_cpu_time_us":4930,"wal-append.run_wall_time_us":1243632}
I20260709 13:05:11.827992  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:11.694139  8639 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:11.838812  1538 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:11.839131  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:11.841250  8658 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:11.842367  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:11.843102  2349 raft_consensus.cc:2368] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 21. Current term is 22.
I20260709 13:05:11.771420  8396 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:11.827047  8543 raft_consensus.cc:697] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:11.846869  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:11.847697  8649 raft_consensus.cc:3055] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:11.848384  8649 raft_consensus.cc:740] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:11.848863  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:11.850162  8689 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:11.835806  8721 consensus_queue.cc:1059] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:05:11.852468  8721 raft_consensus.cc:3055] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:11.852870  8721 raft_consensus.cc:740] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:11.856343  1640 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 20: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:11.856276  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:11.859997  1569 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:11.841252  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:11.858322  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:11.790373  8671 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.789156  8637 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:11.815670  8632 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:05:11.867916  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.868345  1538 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:11.881134  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:11.884227  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:05:11.887462  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 870116us
I20260709 13:05:11.867616  8685 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:11.910600  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3305) took 1178 ms. Trace:
W20260709 13:05:11.893982  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:11.912791  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:11.913578  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.896559  1647 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:11.914968  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:11.915818  1647 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 21: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:11.921147  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:11.919911  8421 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.923892  8421 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:11.929235  1720 raft_consensus.cc:1275] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 17 index: 7. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
I20260709 13:05:11.845254  8543 consensus_queue.cc:237] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:11.936726  1716 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:11.904268  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:11.943840  1640 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:05:11.894543  1685 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:11.908741  1569 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 22.
I20260709 13:05:11.879144  8688 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:11.909946  8689 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:11.871873  8671 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:11.959672  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:11.960497  1720 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 21: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:11.950596  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:05:11.952945  1687 connection.cc:256] RPC call timeout handler was delayed by -1.753385819000016s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
W20260709 13:05:11.967809  1687 connection.cc:256] RPC call timeout handler was delayed by -1.7516199650000317s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1937s for a response
I20260709 13:05:11.874873  8632 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:11.876376  8637 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:11.947947  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:11.954394  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:11.974130  1715 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
I20260709 13:05:11.883657  8564 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.849483  8649 consensus_queue.cc:260] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:11.921249  8675 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:11.993309  8675 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:11.876192  8721 consensus_queue.cc:260] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:11.996905  8721 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:12.002034  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:12.007716  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:12.009826  8561 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:11.955252  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:12.011940  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:12.012703  1647 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
I20260709 13:05:12.014914  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:12.020280  8671 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:12.009302  8564 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:11.980054  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:12.024807  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:11.984930  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 90513.2us
I20260709 13:05:11.985519  1686 leader_election.cc:400] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 21. Current term is 22.
I20260709 13:05:12.026346  1686 leader_election.cc:403] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:05:12.027872  2343 raft_consensus.cc:1240] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:11.992479  8649 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:12.030090  8648 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:11.952312  8639 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:11.911069  1714 rpcz_store.cc:276] 0709 13:05:10.732548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.732734 (+   186us) service_pool.cc:224] Handling call
0709 13:05:10.733175 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:10.733461 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:11.018956 (+285495us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.028744 (+  9788us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.072408 (+ 43664us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.295759 (+223351us) spinlock_profiling.cc:243] Waited 5.58 ms on lock 0x7b4800190158. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ddaeaa1 00007f824dda5b37 00007f824dda59ca 00007f824de9d26f 00007f824de9cf37 00007f824deb155c 00007f824deb1508 00007f824deb1488 00007f824deb1443 00007f824deb06ec 00007f824cddaa96
0709 13:05:11.542407 (+246648us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b180027fa00. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d98522 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:05:11.751662 (+209255us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b180027fa00. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d12c0a 00007f824f452219 00007f8245d9aa40 00007f8245d954e4 00007f8245d95455 00007f8248e7d060 00007f8248e8f716 00007f8248ea0287 00007f8248ea0208
0709 13:05:11.910139 (+158477us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:11.910210 (+    71us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:11.910275 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.910355 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.910529 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1279735,"prepare.queue_time_us":695404,"prepare.run_cpu_time_us":136,"prepare.run_wall_time_us":141,"raft.queue_time_us":689983,"raft.run_cpu_time_us":3024,"raft.run_wall_time_us":270344,"spinlock_wait_cycles":57820800,"thread_start_us":413973,"threads_started":2}
I20260709 13:05:12.034565  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:12.035965  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:11.961933  8688 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:12.038089  8561 raft_consensus.cc:697] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 20 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:12.039935  8561 consensus_queue.cc:237] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:12.041448  1611 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:12.042533  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:12.043015  1611 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:12.043541  1647 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
I20260709 13:05:12.044579  1611 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:12.047839  8736 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:05:12.050923  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 65539.5us
I20260709 13:05:12.055279  8481 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:12.055873  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:12.079903  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.050961  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2923) took 1237 ms. Trace:
I20260709 13:05:12.081439  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:11.990530  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:12.065598  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:12.027467  8703 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:12.062849  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:12.050395  1713 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 23.
W20260709 13:05:12.088100  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 101120us
I20260709 13:05:11.987737  1612 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:12.094466  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.081914  1641 rpcz_store.cc:276] 0709 13:05:10.813585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.813888 (+   303us) service_pool.cc:224] Handling call
0709 13:05:12.050729 (+1236841us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b5400098b54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:12.050861 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1449294720}
I20260709 13:05:12.085777  2346 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:12.053560  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:12.078025  2347 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:05:12.113883  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:12.116441  8718 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:12.115844  8656 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:05:12.113782  8730 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.082s	user 0.004s	sys 0.000s
W20260709 13:05:12.114979  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 70561.2us
I20260709 13:05:12.137820  1611 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:12.117581  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:12.139030  1642 raft_consensus.cc:2410] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:05:12.114004  1647 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
W20260709 13:05:12.118400  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 149790us
I20260709 13:05:12.051131  8624 raft_consensus.cc:3060] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:12.128170  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:12.055608  8711 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:12.147855  1642 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:12.131563  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:12.150156  1714 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:12.040001  8632 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:12.115186  8732 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.097s	user 0.001s	sys 0.008s
I20260709 13:05:12.079835  8647 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:05:12.114290  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 127672us
I20260709 13:05:12.126549  1716 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17.
I20260709 13:05:12.201264  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:12.082803  8671 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:12.113586  8733 log.cc:927] Time spent T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.092s	user 0.000s	sys 0.004s
W20260709 13:05:12.143826  1687 connection.cc:256] RPC call timeout handler was delayed by -1.6076199650000262s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2097s for a response
W20260709 13:05:12.203336  1687 connection.cc:256] RPC call timeout handler was delayed by -1.600385818999996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17930000000000001s for a response
I20260709 13:05:12.180897  8688 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:12.140259  1538 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:12.207340  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 67142.3us
I20260709 13:05:12.153285  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:12.159996  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:12.211366  1715 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17.
I20260709 13:05:12.161799  8624 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 21. Current term is 22.
I20260709 13:05:12.163048  8564 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:12.174105  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:12.136354  8656 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.153379  8718 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:12.152094  8421 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:12.187825  8703 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:12.189931  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:12.226994  8685 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:12.149499  8349 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:12.200918  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:12.201822  1567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2881) took 1482 ms (client timeout 1596 ms). Trace:
W20260709 13:05:12.203716  1687 connection.cc:256] RPC call timeout handler was delayed by -1.5806199649999826s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
W20260709 13:05:12.230106  1687 connection.cc:256] RPC call timeout handler was delayed by -1.4426679599999943s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16770000000000002s for a response
W20260709 13:05:12.230952  1687 connection.cc:256] RPC call timeout handler was delayed by -1.2949469020000492s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2217s for a response
W20260709 13:05:12.231464  1687 connection.cc:256] RPC call timeout handler was delayed by -1.2448858189999896s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21880000000000002s for a response
W20260709 13:05:12.232143  1687 connection.cc:256] RPC call timeout handler was delayed by -1.1537679599999819s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1998s for a response
I20260709 13:05:12.083096  8689 raft_consensus.cc:697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:12.206326  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:12.238061  1716 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:05:12.217304  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:12.218986  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:05:12.221319  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:12.230033  1567 rpcz_store.cc:269] 0709 13:05:10.715694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.730338 (+ 14644us) service_pool.cc:224] Handling call
0709 13:05:12.193334 (+1462996us) spinlock_profiling.cc:243] Waited 60.6 ms on lock 0x7b54004060d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:12.198323 (+  4989us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":139304704}
I20260709 13:05:12.230940  1612 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:12.240499  1715 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 18: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:12.216413  1720 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 22.
I20260709 13:05:12.242074  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:12.242347  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:12.243176  1644 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 18: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:12.242960  1642 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:12.244305  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:12.245144  1642 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:12.215780  8656 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:12.215477  8667 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:12.241657  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:12.263784  1612 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:12.265558  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:12.262244  1643 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:12.258827  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:12.261083  1538 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:12.278954  8689 consensus_queue.cc:237] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:12.260623  8692 consensus_meta.cc:287] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.004s
I20260709 13:05:12.284881  1642 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:12.240103  8671 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:12.260349  8349 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.237437  8703 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:12.260537  8421 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.261513  8647 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:12.281934  8685 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:12.314189  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:12.267920  8667 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:12.268522  8624 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:05:12.336524  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 71087.6us
I20260709 13:05:12.337850  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3465) took 1419 ms. Trace:
I20260709 13:05:12.333300  8718 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:12.342044  1646 rpcz_store.cc:276] 0709 13:05:10.917937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.596180 (+678243us) service_pool.cc:224] Handling call
0709 13:05:11.678420 (+ 82240us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:11.724269 (+ 45849us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:11.724308 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:11.725076 (+   768us) log.cc:844] Serialized 52 byte log entry
0709 13:05:11.859149 (+134073us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:11.859203 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.859275 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:12.337569 (+478294us) raft_consensus.cc:1685] finished
0709 13:05:12.337589 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.337778 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9669,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"thread_start_us":132957,"threads_started":1,"wal-append.queue_time_us":257766}
I20260709 13:05:12.344077  1713 raft_consensus.cc:2410] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:05:12.338431  1538 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:12.293170  8692 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:12.293200  8595 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:12.270390  8371 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:05:12.294298  8671 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.293965  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:12.364061  1716 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
W20260709 13:05:12.338675  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 200898us
I20260709 13:05:12.351230  1538 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:12.351759  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 56300.5us
I20260709 13:05:12.354780  8736 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:05:12.359069  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.366760  1611 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:12.340073  8685 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.345265  8667 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:12.371068  1717 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:05:12.294128  8632 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:05:12.371320  1538 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:12.345976  8647 raft_consensus.cc:697] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:12.381505  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:12.340713  8421 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:12.382225  8639 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:12.382648  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:12.382586  1611 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:12.356974  8718 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:12.357321  8721 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:12.295774  8349 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:12.359012  8595 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.340291  8703 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:12.389024  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.400097  1716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e89a11b4204e48568da409c196e33be4"
dest_uuid: "130bc5ff39b847b1b3204626f284f838"
mode: GRACEFUL
new_leader_uuid: "066ee0109aba4465ba17fa65e14ef504"
 from {username='slave'} at 127.0.0.1:57572
W20260709 13:05:12.405699  8737 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:05:12.410037  1716 raft_consensus.cc:606] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Received request to transfer leadership to TS 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:12.410444  1716 raft_consensus.cc:614] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:12.366930  8671 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:12.374436  8685 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:12.349010  8624 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:12.406548  1713 raft_consensus.cc:2393] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 16: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:12.377590  8633 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:12.406802  1687 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:12.420197  1611 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:12.423614  2346 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:05:12.427660  1687 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:12.394181  8611 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.434113  8647 consensus_queue.cc:237] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:12.437992  8692 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:12.438719  8692 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:12.439184  8692 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:12.442297  8744 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:05:12.411202  8673 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:12.444038  8673 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.444435  8673 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:12.446993  8624 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:12.447383  1640 raft_consensus.cc:2410] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:05:12.450336  8703 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:12.428916  8633 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:12.451769  1716 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:05:12.452207  8633 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:12.403463  8721 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.455464  8721 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:12.455766  1611 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:12.459019  8632 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.459703  8632 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:12.429366  8718 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:12.462088  8371 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:05:12.450343  8742 log.cc:927] Time spent T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:05:12.457204  1611 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:12.467396  1687 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:12.483515  1714 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
I20260709 13:05:12.413846  8595 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:12.368866  8646 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:12.484844  1612 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:12.498373  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 64498.8us
W20260709 13:05:12.465353  8740 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.080s	user 0.002s	sys 0.000s
I20260709 13:05:12.507040  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:12.507843  1646 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
W20260709 13:05:12.512162  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 54877.5us
W20260709 13:05:12.514735  1687 leader_election.cc:336] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.846s (SENT)
I20260709 13:05:12.514986  1611 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:12.515492  1687 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:12.462821  8718 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:12.433157  8607 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:12.530207  8632 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:12.534649  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 749) took 1657 ms. Trace:
I20260709 13:05:12.535145  1648 rpcz_store.cc:276] 0709 13:05:10.877155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.570096 (+692941us) service_pool.cc:224] Handling call
0709 13:05:11.605301 (+ 35205us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:11.677525 (+ 72224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:11.677560 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:11.698260 (+ 20700us) log.cc:844] Serialized 26 byte log entry
0709 13:05:11.908215 (+209955us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:11.908278 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.534179 (+625901us) spinlock_profiling.cc:243] Waited 44.2 ms on lock 0x7b54001a9354. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:12.534251 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:12.534301 (+    50us) raft_consensus.cc:1685] finished
0709 13:05:12.534316 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.534579 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5026,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":88,"spinlock_wait_cycles":101547264,"thread_start_us":167968,"threads_started":1,"wal-append.queue_time_us":425504}
W20260709 13:05:12.539379  8722 log.cc:927] Time spent T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 1.100s	user 0.005s	sys 0.000s
I20260709 13:05:12.542243  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3302) took 1884 ms. Trace:
I20260709 13:05:12.542735  1718 rpcz_store.cc:276] 0709 13:05:10.658016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.676986 (+ 18970us) service_pool.cc:224] Handling call
0709 13:05:10.677388 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:10.735223 (+ 57835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:10.735270 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:10.780422 (+ 45152us) log.cc:844] Serialized 26 byte log entry
0709 13:05:11.283137 (+502715us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:11.283601 (+   464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.283725 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:12.538563 (+1254838us) spinlock_profiling.cc:243] Waited 8.95 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:05:12.541842 (+  3279us) raft_consensus.cc:1685] finished
0709 13:05:12.541870 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.542165 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56417,"raft.queue_time_us":111573,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":16778,"spinlock_wait_cycles":20583808,"thread_start_us":474362,"threads_started":1,"wal-append.queue_time_us":506761}
I20260709 13:05:12.545936  1719 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:12.547631   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:05:12.464967  8611 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:12.549669  8421 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:12.556564  1718 raft_consensus.cc:1275] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:05:12.505596  8736 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:12.522012  8749 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:05:12.578050  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 63129us
I20260709 13:05:12.588398  1611 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:12.594079  1611 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:12.588275  8727 log.cc:927] Time spent T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:05:12.516271  8646 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:12.554451  8750 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:12.591235  8639 raft_consensus.cc:697] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:05:12.627700  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 112967us
I20260709 13:05:12.592691  8649 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:12.629423  1687 leader_election.cc:336] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.043s (SENT)
I20260709 13:05:12.595858  8718 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:12.591828  8746 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:12.633200  1687 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:12.630106  1611 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:12.595894  8421 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:12.631951  8673 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:12.619117  8749 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:12.668565  1611 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:12.634428  8211 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:12.633718  8750 raft_consensus.cc:697] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 19 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:12.680589  8750 consensus_queue.cc:237] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:12.678977  8211 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:12.686094  8721 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:12.694470  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 65071.6us
I20260709 13:05:12.694911  8721 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:12.695324  1687 leader_election.cc:336] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.076s (SENT)
W20260709 13:05:12.696650  1687 leader_election.cc:336] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.130s (SENT)
I20260709 13:05:12.698849  8646 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:12.701948  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:12.703022  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:12.703300  8633 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:12.704012  1569 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
I20260709 13:05:12.704633  8349 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:12.629040  8371 consensus_queue.cc:1048] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:05:12.688509  8673 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:12.697556  1687 leader_election.cc:336] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.133s (SENT)
I20260709 13:05:12.708926  1687 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:12.708940  1611 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:12.669950  8607 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:12.670742  8649 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:12.721484  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:12.723204  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:12.723603  1714 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:12.717401  8718 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:05:12.737596  1687 leader_election.cc:336] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.169s (SENT)
I20260709 13:05:12.740114  1611 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:12.743382  1714 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:05:12.744336  1611 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:12.747241  8611 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:12.749892  8649 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:12.728130  8736 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:12.727661  8633 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:12.732039  8675 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:12.758754  1611 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:12.758314  8718 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:12.738919  8764 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:05:12.761747  8762 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:05:12.761482  8764 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.763619  8764 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:12.686179  8639 consensus_queue.cc:237] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:12.760550  8673 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:12.771762  8685 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:12.789261  8685 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:12.726635  8692 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:12.775909  8656 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:12.779676  1611 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:12.781451  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2897) took 1584 ms. Trace:
I20260709 13:05:12.822041  1564 rpcz_store.cc:276] 0709 13:05:11.197583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.197843 (+   260us) service_pool.cc:224] Handling call
0709 13:05:11.223735 (+ 25892us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:11.231315 (+  7580us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:11.255196 (+ 23881us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.507239 (+252043us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.508742 (+  1503us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:11.536180 (+ 27438us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b1800127b40. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:05:11.557436 (+ 21256us) log.cc:844] Serialized 52 byte log entry
0709 13:05:11.557667 (+   231us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:11.557966 (+   299us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.559971 (+  2005us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.570921 (+ 10950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.571042 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:11.657469 (+ 86427us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b1800127b40. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d98522 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:05:11.710719 (+ 53250us) spinlock_profiling.cc:243] Waited 6.68 ms on lock 0x7b480036ffd8. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ddaeaa1 00007f824dda5b37 00007f824dda59ca 00007f824de9d26f 00007f824de9cf37 00007f824deb155c 00007f824deb1508 00007f824deb1488 00007f824deb1443 00007f824deb06ec 00007f824cddaa96
0709 13:05:12.461239 (+750520us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b1800127b40. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d983f3 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:05:12.530786 (+ 69547us) spinlock_profiling.cc:243] Waited 6.88 ms on lock 0x7b1800127b40. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d98522 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
0709 13:05:12.781116 (+250330us) raft_consensus.cc:1685] finished
0709 13:05:12.781141 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.781378 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38943,"prepare.queue_time_us":993494,"prepare.run_cpu_time_us":195,"prepare.run_wall_time_us":4742,"raft.queue_time_us":50742,"raft.run_cpu_time_us":1899,"raft.run_wall_time_us":165649,"spinlock_wait_cycles":104083712,"thread_start_us":123920,"threads_started":1,"wal-append.queue_time_us":267521}
I20260709 13:05:12.823705  8607 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:12.824482  8720 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:12.783669  8751 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.825510  8751 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:12.825973  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3449) took 2071 ms. Trace:
I20260709 13:05:12.826318  1639 rpcz_store.cc:276] 0709 13:05:10.754665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.755930 (+  1265us) service_pool.cc:224] Handling call
0709 13:05:10.756303 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.675075 (+918772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.825728 (+1150653us) spinlock_profiling.cc:243] Waited 898 ms on lock 0x7b54001fe5d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:12.825917 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2065882496}
I20260709 13:05:12.771827  8667 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:12.827296  8692 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:12.828891  8667 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:12.784871  8595 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:12.829869  1716 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:12.831319  8595 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:12.838330  1713 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:12.773608  8211 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:12.790915  8671 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:12.795629  8658 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 22 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:12.798508  8747 log.cc:927] Time spent T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.244s	user 0.002s	sys 0.000s
W20260709 13:05:12.800139  8715 log.cc:927] Time spent T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 1.359s	user 0.000s	sys 0.004s
I20260709 13:05:12.801244  8649 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:12.808208  1686 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:05:12.886426  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 78288.6us
I20260709 13:05:12.815714  2313 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:12.888427  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 72789us
I20260709 13:05:12.889165  2313 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:12.875962  8671 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:12.891160  8692 raft_consensus.cc:2764] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:05:12.891372  8211 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:12.865106  8667 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.927809  1573 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:12.907316  2313 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:12.914813  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:12.930430  1569 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 20: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:12.924427  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:12.931174  1567 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 18: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:05:12.937362  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 124320us
I20260709 13:05:12.938058  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:12.939044  1639 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:05:12.943197  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:12.945771  1639 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:12.947772  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:12.950076  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:12.953814  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:12.954944  1641 raft_consensus.cc:2376] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 21. Re-sending same reply.
I20260709 13:05:12.957108  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:12.958468  1641 raft_consensus.cc:2410] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:05:12.961719  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:12.962891  8667 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:12.936059  8658 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:12.965670  8658 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.966641  8658 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:12.969126  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3296) took 2332 ms. Trace:
I20260709 13:05:12.970521  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:12.972467  8752 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:12.973052  8752 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:12.973834  8607 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:12.974776  8671 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:12.975137  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:12.975414  8671 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:12.975826  1717 raft_consensus.cc:2376] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 23. Re-sending same reply.
I20260709 13:05:12.976603  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:12.977988  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:12.937095  8764 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:12.972523  8649 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.980034  8649 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:12.981006  8764 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:12.979385  1646 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:05:12.982698  1648 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:12.983215  1645 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 20: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:12.976866  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:12.984377  8649 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:12.973425  8752 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:12.984550  1572 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:12.982604  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 44380.6us
I20260709 13:05:12.987102  2312 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:12.988687  2312 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:12.969034  8751 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:12.994277  8618 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:12.995067  8618 raft_consensus.cc:697] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 23 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:12.995251  8595 consensus_queue.cc:1059] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:12.996073  8618 consensus_queue.cc:237] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:12.996544  8595 raft_consensus.cc:3055] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:12.997231  8595 raft_consensus.cc:740] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:12.998667  8595 consensus_queue.cc:260] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:12.987049  1643 raft_consensus.cc:2368] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16.
I20260709 13:05:12.984992  1564 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:05:13.006424  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.006860  8667 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:12.974490  1571 raft_consensus.cc:1275] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:05:12.970028  1722 rpcz_store.cc:276] 0709 13:05:10.596107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.596232 (+   125us) service_pool.cc:224] Handling call
0709 13:05:10.617535 (+ 21303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:10.937662 (+320127us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:10.937706 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:11.009781 (+ 72075us) log.cc:844] Serialized 26 byte log entry
0709 13:05:11.307723 (+297942us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:11.307768 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.307870 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:12.799531 (+1491661us) spinlock_profiling.cc:243] Waited 6.73 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:05:12.904785 (+105254us) raft_consensus.cc:1685] finished
0709 13:05:12.904825 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.928125 (+ 23300us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":79103,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":128,"spinlock_wait_cycles":15481088,"thread_start_us":492015,"threads_started":2,"wal-append.queue_time_us":362576}
I20260709 13:05:13.008716  8671 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:13.009374  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2909) took 1903 ms. Trace:
I20260709 13:05:13.009817  1719 rpcz_store.cc:276] 0709 13:05:11.106187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.118736 (+ 12549us) service_pool.cc:224] Handling call
0709 13:05:12.545745 (+1427009us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.546513 (+   768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.009159 (+462646us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54002995d0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:13.009306 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3225145728}
I20260709 13:05:13.011536  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39914 (request call id 3479) took 1157 ms. Trace:
I20260709 13:05:13.011612  8595 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:13.011982  1640 rpcz_store.cc:276] 0709 13:05:11.854417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.024456 (+170039us) service_pool.cc:224] Handling call
0709 13:05:13.011315 (+986859us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b54002ab7d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:13.011468 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":969292544}
I20260709 13:05:13.018011  8771 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:13.020048  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:13.024906  1722 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:13.027357  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:13.027544  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:13.029834  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:13.030077  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:13.030550  1714 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:05:13.031225  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:13.031929  1715 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:13.033620  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:13.034224  1721 raft_consensus.cc:1275] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:05:13.035459  1718 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:05:13.036610  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:13.037240  1714 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:10.677536  1457 catalog_manager.cc:5028] TS f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): tablet 59d236b930214b1f91fb55ece6fecd68 (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
I20260709 13:05:13.037705  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:13.038134  1457 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 2.36015e+06us
I20260709 13:05:13.038749  1716 raft_consensus.cc:2393] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 23: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:05:13.039799  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:13.040544  1715 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:13.041842  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:13.040611   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:13.043517  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:13.044612  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 137347us
I20260709 13:05:13.046288  8656 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:13.047660  2313 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:13.048352  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 779) took 1125 ms. Trace:
I20260709 13:05:13.048465  8646 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:13.049026  8646 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.049681  8646 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:13.053023  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3312) took 1119 ms. Trace:
I20260709 13:05:13.053453  1713 rpcz_store.cc:276] 0709 13:05:11.933341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.436754 (+503413us) service_pool.cc:224] Handling call
0709 13:05:12.837811 (+401057us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.838993 (+  1182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.052787 (+213794us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b54003a2fd0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:13.052949 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":921475840}
I20260709 13:05:13.055485  8615 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.056056  8615 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:13.056981  8717 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 1.615s	user 0.005s	sys 0.000s
W20260709 13:05:13.058127  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.058809  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:13.059820  8615 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:13.061806  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:13.062624  1571 raft_consensus.cc:2368] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 21. Current term is 22.
I20260709 13:05:13.063856  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:13.064594  1571 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:13.067814  8671 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:13.031984  2345 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:13.070530  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:13.070775  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:13.071717  2349 raft_consensus.cc:2393] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 22: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:05:13.072464  1564 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:13.073568  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:13.074265  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:13.074385  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:13.075747  1641 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:13.048873  1720 rpcz_store.cc:276] 0709 13:05:11.923530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.081101 (+157571us) service_pool.cc:224] Handling call
0709 13:05:13.048168 (+967067us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b5400212d54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:13.048287 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":308595072}
I20260709 13:05:13.077303  1573 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 19. Current term is 20. Ops: []
W20260709 13:05:13.078284  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 89548.1us
I20260709 13:05:13.078963  1566 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:13.079682  2312 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:13.080256  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:13.084225  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:13.085265  1566 raft_consensus.cc:2368] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 21. Current term is 22.
I20260709 13:05:13.085875  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:13.090240  1643 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:05:13.095000  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:13.096199  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:13.076041  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:13.097908  1566 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:05:13.098146  1567 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:13.097847  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:13.102497  1641 raft_consensus.cc:1275] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:05:13.104250  1641 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:13.080156  1714 raft_consensus.cc:2410] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:05:13.110127  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:13.110723  1571 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:05:13.111454  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:13.112186  2346 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:05:13.114166  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:13.115964  1643 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:05:13.119083  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:13.119084  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:13.120680  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:13.120808  1570 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:05:13.121435  1714 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:13.122128  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:13.122751  2341 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:13.127727  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:13.128295  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:13.128643  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:13.129101  1722 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:05:13.129523  1565 raft_consensus.cc:3060] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:13.127704  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:13.130312  1639 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:05:13.130821  1715 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 20: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:13.127701  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:13.132078  1648 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 20: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:13.132336  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:13.135612  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:13.136487  1641 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:05:13.127835  1538 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:13.138267  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:13.140257  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:13.141839  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:13.099187  8689 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.015s	sys 0.055s
I20260709 13:05:13.141041  1640 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:05:13.151298  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:13.152657  2346 raft_consensus.cc:2393] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 19: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:05:13.146030  1643 raft_consensus.cc:1240] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:13.158397  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:13.159318  1573 raft_consensus.cc:2393] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:13.137753  8766 raft_consensus.cc:2804] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:13.164077  8766 raft_consensus.cc:697] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:13.164378  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:13.166004  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:13.167373  1571 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:13.168493  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:13.169390  8760 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:13.170085  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:13.171747  1566 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:05:13.174054  8633 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:13.174678  1567 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:05:13.179677  8211 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:13.129276  8282 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:13.182600  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:13.191075  1573 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:05:13.186568  1571 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:13.072187  8671 raft_consensus.cc:697] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:13.185500  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:13.119799  8637 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:13.201704  1645 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:13.170459  8760 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:13.190832  8717 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:05:13.194801  1572 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
I20260709 13:05:13.195364  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:13.196224  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:13.214969  2349 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 12.
I20260709 13:05:13.192322  1568 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:13.206974  8769 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.143s	user 0.001s	sys 0.002s
W20260709 13:05:13.148551  8689 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.015s	sys 0.056s
I20260709 13:05:13.197551  1714 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
I20260709 13:05:13.240202  1565 raft_consensus.cc:2468] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:05:13.245358  1569 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:05:13.247262  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:13.248592  1645 raft_consensus.cc:2393] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:13.208760  8760 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:13.261107  1722 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
I20260709 13:05:13.263779  1645 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:13.199376  8282 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:13.272133  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:13.259352  1537 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:13.266948  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:13.274144  1643 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
I20260709 13:05:13.275358  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:13.268211  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:13.276473  1647 raft_consensus.cc:2393] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:13.268637  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:13.277002  1639 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:13.278003  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:13.279140  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:13.279196  1537 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:13.273375  1645 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:13.279786  1639 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:05:13.280673  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:13.281176  1639 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:13.244189  8685 consensus_queue.cc:1048] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:05:13.286800  1537 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:13.297160  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:13.299225  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:13.300187  1643 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
W20260709 13:05:13.285851  1464 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47518 (request call id 100) took 35578 ms (client timeout 15000 ms). Trace:
I20260709 13:05:13.302665  1537 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:13.305315  1537 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:13.310053  1646 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:05:13.316192  2341 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:05:13.318778  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:13.327508  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:13.328312  1569 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
I20260709 13:05:13.329417  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:05:13.330827  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 251187us
I20260709 13:05:13.332697  1687 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:13.332847  1645 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:13.287006  8667 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:13.301276  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:13.166471  8766 consensus_queue.cc:237] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:13.275038  8671 consensus_queue.cc:237] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:13.292273  8629 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:13.302578  8554 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:13.318825  8712 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:05:13.523377  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.337603  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:13.334728  8692 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.201s
I20260709 13:05:13.384500  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:13.565459  1565 raft_consensus.cc:3060] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:13.536091  1643 raft_consensus.cc:2376] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 21. Re-sending same reply.
I20260709 13:05:13.567411  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:05:13.432663  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 64217us
I20260709 13:05:13.534739  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:13.307257  8685 consensus_queue.cc:1243] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 17.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:05:13.335301  1687 leader_election.cc:400] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 pre-election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16.
I20260709 13:05:13.569509  1645 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
W20260709 13:05:13.570124  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 234894us
I20260709 13:05:13.571074  1687 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:13.321722  8633 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:13.527340  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 195676us
W20260709 13:05:13.557379  8774 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.158s	user 0.004s	sys 0.002s
W20260709 13:05:13.540165  8658 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.572s	user 0.018s	sys 0.000s
I20260709 13:05:13.529117  1639 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:13.357900  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:13.370048  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:13.357685  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:13.355100  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:13.568274  1643 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 20: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:13.579491  2312 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:13.358425  8370 consensus_queue.cc:1059] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.180s
I20260709 13:05:13.588428  1572 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 20: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:13.333681  8561 raft_consensus.cc:2764] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:05:13.582091  1611 leader_election.cc:400] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 17. Current term is 18.
I20260709 13:05:13.602377  1611 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:13.537910  8736 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:13.585949  1647 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:13.591135  1568 raft_consensus.cc:2393] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 12: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:13.597070  2314 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:13.598292  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:05:13.578265  8775 log.cc:927] Time spent T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.224s	user 0.000s	sys 0.006s
I20260709 13:05:13.582733  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:13.610380  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:13.617825  1643 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:13.579845  8607 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:13.605985  1611 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:13.608057  1646 raft_consensus.cc:2376] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15. Re-sending same reply.
I20260709 13:05:13.600709  1565 raft_consensus.cc:2468] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
W20260709 13:05:13.620575  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 300240us
I20260709 13:05:13.620931  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:13.621964  1643 raft_consensus.cc:2393] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 20: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:13.418732  8646 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:13.629575  1643 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:05:13.598690  8771 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:13.637300  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:13.583359  8658 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:13.638422  1643 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:13.610737  8750 raft_consensus.cc:2804] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:13.609404  8282 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:13.709554  8595 consensus_meta.cc:287] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.697s	user 0.017s	sys 0.000s
I20260709 13:05:13.689972  1537 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:05:13.711872  8779 log.cc:927] Time spent T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:05:13.636021  8371 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:13.617460  8736 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:13.654719  8561 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:13.633399  8607 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:13.734709  8667 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:13.740367  1646 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:13.749128  1722 raft_consensus.cc:1275] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:13.755079  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:13.674230  8554 consensus_queue.cc:1059] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 13:05:13.762283  1569 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:13.764427  1537 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:13.714010  8771 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:13.654371  8370 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.085s
W20260709 13:05:13.803263  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 232292us
I20260709 13:05:13.713793  8282 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.078s
I20260709 13:05:13.790125  1612 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:13.817708  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 53371.8us
I20260709 13:05:13.790820  8667 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.718102  8639 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.122s
I20260709 13:05:13.731385  8751 consensus_queue.cc:1048] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.110s
W20260709 13:05:13.855563  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 65528.2us
I20260709 13:05:13.878569  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:13.858220  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 51954.7us
I20260709 13:05:13.880074  1538 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:13.860234  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:13.853258  1647 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:05:13.883267  1645 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:13.807106  8771 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:13.843976  8554 consensus_queue.cc:1243] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:05:13.846619  8396 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:13.871539  8752 consensus_meta.cc:287] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.886s	user 0.015s	sys 0.000s
W20260709 13:05:13.934806  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49549us
W20260709 13:05:13.928241  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 49791.8us
I20260709 13:05:13.786566  8756 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 13:05:13.846246  1687 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:13.873013  8667 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:13.958092  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 111927us
I20260709 13:05:13.944366  1646 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:13.934347  8771 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:13.932871  8750 raft_consensus.cc:697] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 19 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:13.965996  8750 consensus_queue.cc:237] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:13.962064  1687 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:13.867754  8656 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:05:13.973994  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 345814us
W20260709 13:05:13.931734  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 334689us
I20260709 13:05:13.891299  8561 consensus_queue.cc:1059] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 13:05:13.955646  8554 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:05:13.991467  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 306589us
I20260709 13:05:13.994776  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:13.992717  1611 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:13.960361  8692 raft_consensus.cc:2764] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:05:13.969776  8658 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:14.016074  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 136102us
I20260709 13:05:13.953040  8752 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:14.021503  8752 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:13.971853  8370 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:05:14.023801  1538 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:13.979241  8786 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:14.028652  8795 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:14.035647  8554 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.035519  8370 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:05:14.029338  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 169203us
I20260709 13:05:14.050633  1537 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:14.017004  8646 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:13.996622  8656 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:14.043831  8370 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:14.055740  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 78698us
W20260709 13:05:14.051435  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 104539us
I20260709 13:05:14.037264  8798 raft_consensus.cc:2804] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:14.062811  1537 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:14.066144  8802 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:14.067914  8802 consensus_queue.cc:1059] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:14.069252  8711 raft_consensus.cc:3055] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:14.069696  8711 raft_consensus.cc:740] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:14.070605  8711 consensus_queue.cc:260] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:14.072274  8711 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:14.082571  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:14.083900  1537 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:14.063920  1538 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:14.064560  8615 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:14.085394  8615 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:14.065030  8639 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:14.083616  8282 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:14.087875  8282 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:14.063485  1687 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:14.090161  1537 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:14.091323  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:14.095366   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:14.045223  8692 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
W20260709 13:05:14.105258  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 112873us
I20260709 13:05:14.048624  8554 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:14.109853  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:14.059659  8370 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:14.088722  8282 raft_consensus.cc:697] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:14.102963  8639 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:14.100525  8615 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:14.115602  1685 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:14.112840  8711 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:14.120867  2348 raft_consensus.cc:1240] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:05:14.114254  8736 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:14.119257  1611 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:14.137180  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:14.111074  8561 consensus_queue.cc:1059] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:05:14.124138  8615 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.141253  8561 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:14.180239  8561 raft_consensus.cc:697] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:14.181689  8561 consensus_queue.cc:237] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:14.182770  8790 log.cc:927] Time spent T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
W20260709 13:05:14.183951  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.184523  8804 consensus_queue.cc:1048] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:14.186347  8810 consensus_queue.cc:1059] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:14.141722  8736 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:14.142158  8808 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:05:14.146741  8813 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:14.147815  8758 raft_consensus.cc:3055] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:14.148680  8807 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:14.282037  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:14.123940  8806 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:14.286191  8806 raft_consensus.cc:697] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 19 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:14.286644  8646 consensus_queue.cc:1059] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:05:14.287703  8396 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:14.287942  8646 raft_consensus.cc:3055] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:14.150125  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:14.151849  1611 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:14.149523  8796 log.cc:927] Time spent T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:05:14.289131  1572 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 18: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:05:14.289467  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 137665us
I20260709 13:05:14.171908  8656 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:14.177923  8615 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:14.291726  8656 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:14.292397  8656 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:14.191223  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:14.294003  1564 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:14.286379  1571 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:05:14.212647  1570 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 13:05:13.302445  1464 rpcz_store.cc:269] 0709 13:04:37.698757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.699171 (+   414us) service_pool.cc:224] Handling call
0709 13:05:13.274802 (+35575631us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:07.607717 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:07.701880 (+ 94163us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:07.701946 (+    66us) write_op.cc:435] Acquired schema lock
0709 13:05:07.701969 (+    23us) tablet.cc:662] Decoding operations
0709 13:05:07.737600 (+ 35631us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:07.769084 (+ 31484us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:07.774779 (+  5695us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:05:07.818512 (+ 43733us) tablet.cc:701] Row locks acquired
0709 13:05:07.818583 (+    71us) write_op.cc:260] PREPARE: finished
0709 13:05:07.818874 (+   291us) write_op.cc:270] Start()
0709 13:05:07.819096 (+   222us) write_op.cc:276] Timestamp: P: 1783602307818769 usec, L: 0
0709 13:05:07.834391 (+ 15295us) op_driver.cc:348] REPLICATION: starting
0709 13:05:08.032174 (+197783us) log.cc:844] Serialized 3218 byte log entry
0709 13:05:08.901820 (+869646us) op_driver.cc:464] REPLICATION: finished
0709 13:05:09.102611 (+200791us) write_op.cc:301] APPLY: starting
0709 13:05:09.102730 (+   119us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:09.385086 (+282356us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:09.385274 (+   188us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:09.925408 (+540134us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:10.045152 (+119744us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0709 13:05:10.054087 (+  8935us) write_op.cc:312] APPLY: finished
0709 13:05:10.166012 (+111925us) log.cc:844] Serialized 81 byte log entry
0709 13:05:10.287226 (+121214us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:10.293400 (+  6174us) write_op.cc:454] Released schema lock
0709 13:05:10.317763 (+ 24363us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:10.436412 (+118649us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:10.440378 (+  3966us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6548010,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":190616,"apply.run_cpu_time_us":35979,"apply.run_wall_time_us":1242524,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37583,"num_ops":8,"prepare.queue_time_us":283951,"prepare.run_cpu_time_us":34730,"prepare.run_wall_time_us":775691,"raft.queue_time_us":217361,"raft.run_cpu_time_us":11781,"raft.run_wall_time_us":123346,"replication_time_us":1054480,"thread_start_us":665109,"threads_started":5,"wal-append.queue_time_us":362830,"wal-append.run_cpu_time_us":40807,"wal-append.run_wall_time_us":2788959}]]}
I20260709 13:05:14.296010  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:14.276629  8805 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:14.277940  8633 raft_consensus.cc:3055] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:14.287420  8806 consensus_queue.cc:237] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:14.288515  8646 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:14.187518  8639 consensus_queue.cc:1059] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:05:14.169989  8282 consensus_queue.cc:237] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:14.179076  8812 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:14.297765  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:14.304627  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3501) took 1032 ms. Trace:
I20260709 13:05:14.358424  1566 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:05:14.304905  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:14.301710  8811 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:14.271304  8758 raft_consensus.cc:740] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:14.356101  1572 raft_consensus.cc:1275] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:14.358342  1640 rpcz_store.cc:276] 0709 13:05:13.272182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.282814 (+ 10632us) service_pool.cc:224] Handling call
0709 13:05:13.283127 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.283351 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:05:13.621773 (+338422us) log.cc:844] Serialized 14 byte log entry
0709 13:05:14.049372 (+427599us) log.cc:844] Serialized 14 byte log entry
0709 13:05:14.304184 (+254812us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:14.304254 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:14.304324 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.304415 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:14.304555 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101122,"prepare.queue_time_us":451610,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":119,"raft.queue_time_us":322115,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":37909,"thread_start_us":977859,"threads_started":4,"wal-append.queue_time_us":564149}
I20260709 13:05:14.295907  8692 raft_consensus.cc:3055] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:05:14.362706  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 304574us
I20260709 13:05:14.407902  1612 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:14.367736  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
W20260709 13:05:14.273470  8778 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.734s	user 0.012s	sys 0.003s
W20260709 13:05:14.296165  8760 consensus_meta.cc:287] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 1.024s	user 0.015s	sys 0.000s
W20260709 13:05:14.366132  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 302659us
W20260709 13:05:14.435539  1642 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.024s	sys 0.069s
I20260709 13:05:14.436075  1611 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:14.271909  8736 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:14.436215  1642 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.025s	sys 0.069s
I20260709 13:05:14.435345  1687 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:14.354331  8689 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.486418  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:14.481300  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:14.489105  1648 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:05:14.492290  2349 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:14.367143  8750 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.357764  8812 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.305658  8633 raft_consensus.cc:740] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:14.306934  8607 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:14.479681  8811 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:14.500422  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 368426us
I20260709 13:05:14.282069  8807 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:14.506975  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:14.434834  8805 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.412830  8797 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:05:14.484248  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 48997.4us
I20260709 13:05:14.371076  8658 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.127s
I20260709 13:05:14.410238  8692 raft_consensus.cc:740] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:14.438216  8760 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:14.437593  8798 raft_consensus.cc:697] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 19 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:14.439424  8809 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:14.370694  8795 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.129s
I20260709 13:05:14.526938  1687 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:14.527654  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:14.486090  8801 raft_consensus.cc:3055] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:14.559042  1722 raft_consensus.cc:3055] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:05:14.570117  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 43510.3us
I20260709 13:05:14.566740  1685 leader_election.cc:400] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Vote denied by peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999) with higher term. Message: Invalid argument: T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 21. Current term is 22.
I20260709 13:05:14.579778  1648 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 20.
I20260709 13:05:14.512907  8812 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:14.598402  1687 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:14.595654  1722 raft_consensus.cc:740] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:14.523867  8607 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:14.507351  8750 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:14.609957  1722 consensus_queue.cc:260] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:14.623261  1722 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:14.606174  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45367.4us
I20260709 13:05:14.629446  1721 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:14.582909  8795 consensus_queue.cc:1243] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer 066ee0109aba4465ba17fa65e14ef504 log is divergent from this leader: its last log entry 20.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:05:14.583696  8801 raft_consensus.cc:740] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:14.594586  8819 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.525817  8805 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:14.524369  8811 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:14.712275  1715 raft_consensus.cc:3055] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:14.712718  1715 raft_consensus.cc:740] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:14.515138  8758 consensus_queue.cc:260] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:14.514971  8646 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:14.508324  8689 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:14.711911  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 83695.6us
I20260709 13:05:14.661036  1722 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 13:05:14.714956  1722 pending_rounds.cc:85] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 11
I20260709 13:05:14.715251  1722 pending_rounds.cc:107] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:05:14.631587  8815 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:14.715514  1722 raft_consensus.cc:2889] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:14.646623  8656 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:14.625291  8607 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:14.714047  1715 consensus_queue.cc:260] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:14.727589  1715 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:14.731206  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:14.729409  8814 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.707623  8808 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:14.705155  8818 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.667431  8824 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:14.569226  8797 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:14.724490  8815 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.722625  8658 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.758085  1719 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:14.761065  1573 raft_consensus.cc:2393] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 18: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:14.622150  8633 consensus_queue.cc:260] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:14.734048  8758 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:14.748663  8792 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.375s	user 0.000s	sys 0.001s
I20260709 13:05:14.734833  8656 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.752341  8824 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:14.802541  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 87762us
I20260709 13:05:14.667865  8819 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:14.761425  8817 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:05:14.810042  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.804013  8808 consensus_queue.cc:1059] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 13:05:14.736661  8798 consensus_queue.cc:237] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:14.809820  8825 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:05:14.765048  8815 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:14.769809  8809 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.819216  1685 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:14.851146  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 252820us
I20260709 13:05:14.712513  8692 consensus_queue.cc:260] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:14.787576  8756 consensus_queue.cc:1059] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.080s
I20260709 13:05:14.777235  8658 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:14.631201  8791 consensus_queue.cc:1048] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.101s
I20260709 13:05:14.805733  8824 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:14.884565  8792 log.cc:927] Time spent T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:05:14.759086  8818 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:14.818142  8817 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.747222  8814 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:14.910293  8807 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:14.895589  8831 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:05:14.911408  8820 log.cc:927] Time spent T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
W20260709 13:05:14.914424  8777 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.361s	user 0.009s	sys 0.010s
I20260709 13:05:14.915720  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3506) took 1591 ms. Trace:
W20260709 13:05:14.916409  8667 consensus_meta.cc:287] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.945s	user 0.012s	sys 0.000s
I20260709 13:05:14.851862  8760 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:14.856171  1564 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.562s	user 0.011s	sys 0.003s
I20260709 13:05:14.801419  8656 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:14.967886  8656 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:14.968664  8656 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:14.947232  8837 consensus_queue.cc:1059] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:05:14.950443  8807 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:14.974128  8807 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:14.976783  8807 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:14.913363  8797 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:14.944497  8808 consensus_queue.cc:1059] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:05:14.758906  8801 consensus_queue.cc:260] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:14.978515  8836 consensus_queue.cc:1059] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:14.979409  8801 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:14.979561  1568 pending_rounds.cc:85] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Aborting all ops after (but not including) 13
I20260709 13:05:14.980177  1568 pending_rounds.cc:107] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 13:05:14.980741  1568 raft_consensus.cc:2889] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:14.981276  8819 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:14.863509  8810 consensus_queue.cc:1059] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:05:14.982142  8810 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.982630  8810 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:14.983170  8795 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.983958  8795 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:14.984683  8810 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:14.859803  8809 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:14.898038  8736 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:14.921027  8370 consensus_meta.cc:287] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.786s	user 0.009s	sys 0.008s
I20260709 13:05:14.953773  1645 rpcz_store.cc:276] 0709 13:05:13.324464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.642469 (+318005us) service_pool.cc:224] Handling call
0709 13:05:13.883020 (+240551us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.883779 (+   759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.915446 (+1031667us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54000a69d0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:14.915642 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551803648}
I20260709 13:05:14.884202  8721 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.080s
W20260709 13:05:15.013424  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.015004  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:14.961581  1564 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 20.
I20260709 13:05:15.015811  2349 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:05:15.016491  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.017258  1564 raft_consensus.cc:2393] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 18: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
W20260709 13:05:14.950649  8821 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.329s	user 0.001s	sys 0.000s
I20260709 13:05:14.940793  8817 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:14.997596  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.029544  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:14.977449  8807 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:14.985496  8810 raft_consensus.cc:3055] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:15.030944  1570 raft_consensus.cc:2393] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 21: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:15.037035  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.031395  1564 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 18: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:14.969318  8656 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:15.011950  8771 consensus_meta.cc:287] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 1.048s	user 0.008s	sys 0.008s
I20260709 13:05:15.011893  8825 raft_consensus.cc:3055] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:15.042819  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:14.961872  8667 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.045276  1570 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:15.020377  8689 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:15.047849  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:15.020438  8808 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:15.021010  8750 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:15.053339  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2971) took 1956 ms. Trace:
W20260709 13:05:15.060861  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 200044us
I20260709 13:05:15.028824  8736 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:15.038903  8810 raft_consensus.cc:740] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:15.012593  8370 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.042623  8832 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.086637  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:15.064962  8786 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:05:15.065938  1642 rpcz_store.cc:276] 0709 13:05:13.096493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.120260 (+ 23767us) service_pool.cc:224] Handling call
0709 13:05:13.137995 (+ 17735us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:13.138355 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.138394 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:13.174799 (+ 36405us) log.cc:844] Serialized 26 byte log entry
0709 13:05:13.306649 (+131850us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:13.631168 (+324519us) log.cc:844] Serialized 14 byte log entry
0709 13:05:14.483396 (+852228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.483478 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:14.913669 (+430191us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:05:15.053006 (+139337us) raft_consensus.cc:1685] finished
0709 13:05:15.053132 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:15.053275 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55622,"prepare.queue_time_us":544889,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":59,"raft.queue_time_us":107305,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":41292,"spinlock_wait_cycles":4980096,"thread_start_us":1245559,"threads_started":3,"wal-append.queue_time_us":241607}
I20260709 13:05:15.096467  1687 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:05:15.141954  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 45592.5us
I20260709 13:05:15.126292  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.130189  1565 raft_consensus.cc:1240] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:15.133137  8813 raft_consensus.cc:3055] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:15.144559  8813 raft_consensus.cc:740] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:05:15.097198  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 241925us
I20260709 13:05:15.047173  8825 raft_consensus.cc:740] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:15.040664  8807 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.083796  8607 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:15.091143  8750 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.091638  8791 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:15.111008  8841 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:15.111706  8788 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.118315  8832 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:15.120082  8658 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:15.138667  8827 log.cc:927] Time spent T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:05:15.084306  8826 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:15.124912  8633 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:15.139989  8834 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:05:15.145354  8813 consensus_queue.cc:260] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.150352  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:15.143484  1687 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:15.239884  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 96484.3us
W20260709 13:05:15.170882  8833 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:05:15.146670  8736 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:15.124943  8615 consensus_meta.cc:287] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.834s	user 0.012s	sys 0.000s
I20260709 13:05:15.126904  8805 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:15.170332  8844 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:15.252779  8844 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:15.157617  8851 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:05:15.254961  2315 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.255174  8851 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:15.158931  8667 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:15.130957  8810 consensus_queue.cc:260] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.245890  8837 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:15.258227  8810 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:15.258791  8837 raft_consensus.cc:697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 21 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:15.259704  8837 consensus_queue.cc:237] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:15.253374  8844 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:15.166955  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.266106  1564 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
I20260709 13:05:15.162482  8809 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:15.266446  8843 log.cc:927] Time spent T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:05:15.157166  8839 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:15.200978  8671 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.586s	user 0.005s	sys 0.020s
W20260709 13:05:15.268142  8671 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.653s	user 0.005s	sys 0.020s
I20260709 13:05:15.268142  8839 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.155476  8371 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.053859  8771 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:15.275161  2341 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:15.275619  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:15.276155  8371 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.276818  8736 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:15.279489  8839 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:15.284768  8812 consensus_meta.cc:287] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.662s	user 0.016s	sys 0.004s
I20260709 13:05:15.285367  8371 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:15.162956  8795 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:15.164391  8818 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:15.247462  8814 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:15.270983  8656 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:15.287766  8835 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
I20260709 13:05:15.291842  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.292711  1571 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 12.
I20260709 13:05:15.288867  8844 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:15.298607  8370 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:15.299620  8832 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:15.301108  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:15.302326  2349 raft_consensus.cc:2368] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 19. Current term is 20.
I20260709 13:05:15.304596  1686 leader_election.cc:400] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 19. Current term is 20.
I20260709 13:05:15.305033  1686 leader_election.cc:403] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 13:05:15.305303  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.249784  8615 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:15.251892  8607 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:05:15.235853  8791 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:15.229317  8841 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.281698  8825 consensus_queue.cc:260] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.282405  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:15.071758  8808 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.341586  8811 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:15.363709  8799 log.cc:927] Time spent T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.302s	user 0.010s	sys 0.011s
I20260709 13:05:15.337250  8817 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.364778  8808 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:15.232347  8788 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:15.283663  8771 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:15.281571  8851 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:15.349277  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2964) took 2254 ms (client timeout 1903 ms). Trace:
I20260709 13:05:15.286396  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:15.350688  1567 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:15.370301  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 83981.6us
I20260709 13:05:15.370821  8756 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:15.306142  8786 raft_consensus.cc:3060] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:15.345321  8637 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:15.375954  8637 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.307557  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40670 (request call id 2984) took 1181 ms. Trace:
I20260709 13:05:15.312850  8844 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:15.377882  1565 rpcz_store.cc:276] 0709 13:05:14.126811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.126945 (+   134us) service_pool.cc:224] Handling call
0709 13:05:14.131985 (+  5040us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:15.143675 (+1011690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:15.307288 (+163613us) spinlock_profiling.cc:243] Waited 998 ms on lock 0x7b5400190fd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:15.307496 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2294929280}
I20260709 13:05:15.378364  8637 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:15.317950  8807 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:15.368041  8788 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:15.245155  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 94863.1us
I20260709 13:05:15.380901  1685 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:15.318353  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 1.23007e+06us
I20260709 13:05:15.382448  1538 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:15.382565  1685 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:15.226907  8750 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.328356  1612 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:15.332726  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2915) took 1037 ms. Trace:
I20260709 13:05:15.336189  8815 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.338256  8751 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.285246  8812 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.242628  1687 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:15.348376  8816 raft_consensus.cc:3055] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:15.371639  8756 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:05:15.369232  1644 rpcz_store.cc:269] 0709 13:05:13.094562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.094701 (+   139us) service_pool.cc:224] Handling call
0709 13:05:15.348878 (+2254177us) spinlock_profiling.cc:243] Waited 2.18 s on lock 0x7b5400097754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:15.349167 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5007179008}
W20260709 13:05:15.327061  8850 log.cc:927] Time spent T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:05:15.238790  8826 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.352666  8615 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:15.368108  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.357543  8791 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:15.354094  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:15.535017  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.404284  1573 rpcz_store.cc:276] 0709 13:05:14.295048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.295747 (+   699us) service_pool.cc:224] Handling call
0709 13:05:15.332291 (+1036544us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b54004060d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:15.332637 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1061202688}
W20260709 13:05:15.407261  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:15.431082  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 188398us
I20260709 13:05:15.406821  1538 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:15.542435  1687 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:15.542794  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.543589  1568 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:05:15.546038  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.547034  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.548307  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.549595  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.551440  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.552186  1570 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
W20260709 13:05:15.408995  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 80833.1us
I20260709 13:05:15.379276  8807 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:15.362605  8825 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:15.414255  1611 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:15.368145  8817 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:15.526957  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 95389.5us
I20260709 13:05:15.355407  2341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 890) took 1420 ms. Trace:
I20260709 13:05:15.361889  8813 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:15.343042  8595 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:15.530717  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:15.575265  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 161099us
I20260709 13:05:15.362596  1564 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:15.365588  8646 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:15.359661  8841 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:15.410585  8607 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:15.634840  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 228104us
I20260709 13:05:15.544960  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.568256  1687 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:15.638007  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 70071.8us
I20260709 13:05:15.405505  8844 consensus_queue.cc:1059] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:05:15.406112  8808 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:15.654318  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:15.566599  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:15.542498  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.556565  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.430370  8819 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:15.560801  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:15.575575  2341 rpcz_store.cc:276] 0709 13:05:13.935362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.935551 (+   189us) service_pool.cc:224] Handling call
0709 13:05:13.935838 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:15.275696 (+1339858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:15.355117 (+ 79421us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400549054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:15.355335 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3079245568}
I20260709 13:05:15.410728  8804 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:15.582984  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:15.583743  8371 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:15.585132  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:15.596395  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40656 (request call id 2917) took 1296 ms. Trace:
I20260709 13:05:15.597874  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3515) took 2049 ms. Trace:
I20260709 13:05:15.615963  8595 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:15.701020  2346 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:15.574420  8561 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:15.475555  8751 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.628681  8750 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:15.630327  8815 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:15.574791  8814 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 13:05:15.431963  8797 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.646962  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.647254  1644 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:05:15.430094  8816 raft_consensus.cc:740] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:15.431785  8851 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:15.650367  1713 raft_consensus.cc:1275] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:05:15.531708  8756 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:15.408215  8637 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:15.656661  1573 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
W20260709 13:05:15.658975  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.674665  1648 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:05:15.676926  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.574468  8771 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.678187  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:15.680212  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:15.580430  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.673367  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:15.342592  8824 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:15.691073  1714 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.940s	user 0.044s	sys 0.086s
I20260709 13:05:15.699689  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:15.735908  1567 rpcz_store.cc:276] 0709 13:05:14.300290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.300506 (+   216us) service_pool.cc:224] Handling call
0709 13:05:14.300978 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:15.370285 (+1069307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:15.596085 (+225800us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54001275d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:15.596313 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2413065600}
I20260709 13:05:15.430073  8370 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:05:15.737802  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 394100us
I20260709 13:05:15.707094  1564 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19.
I20260709 13:05:15.739059  2345 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:15.851236  2312 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:15.740186  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.742821  1640 raft_consensus.cc:1240] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:05:15.744074  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:15.853132  2341 raft_consensus.cc:2393] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 19: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:05:15.745615  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:15.854682  1720 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
W20260709 13:05:15.855077  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.747925  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:15.748037  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:15.533772  8836 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.162s
I20260709 13:05:15.525050  8831 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.171s
I20260709 13:05:15.868638  2312 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:15.652747  8817 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:15.652945  8793 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:15.759886  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:15.752283  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:15.762298  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:15.783766  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.746690  8371 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.647903  8807 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.685936  8819 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:15.854820  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:15.942283  1639 raft_consensus.cc:2368] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 19. Current term is 20.
I20260709 13:05:15.735520  1641 rpcz_store.cc:276] 0709 13:05:13.548456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.875046 (+326590us) service_pool.cc:224] Handling call
0709 13:05:13.875448 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:14.096181 (+220733us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:14.096232 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:14.096953 (+   721us) log.cc:844] Serialized 26 byte log entry
0709 13:05:14.366799 (+269846us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:14.411689 (+ 44890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:15.597384 (+1185695us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b54001fdbd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:15.597443 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:15.597500 (+    57us) raft_consensus.cc:1685] finished
0709 13:05:15.597515 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:15.597806 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":506789632,"thread_start_us":269314,"threads_started":1,"wal-append.queue_time_us":426032}
I20260709 13:05:15.763881  2347 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 21.
I20260709 13:05:15.856057  1647 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:05:15.948329  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:15.905189  8788 raft_consensus.cc:2804] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:05:15.856478  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45193.8us
I20260709 13:05:15.870071  1612 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:15.856024  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
W20260709 13:05:15.877569  1714 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.127s	user 0.044s	sys 0.093s
I20260709 13:05:15.879554  1645 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:05:15.811539  2344 raft_consensus.cc:1240] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:05:15.883205  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:15.877048  1570 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:05:15.653270  8758 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:15.886317  1568 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:05:15.887454  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:15.892158  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:15.949810  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:15.957113  1644 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17.
I20260709 13:05:15.953480  1566 raft_consensus.cc:3060] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:15.954025  1569 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:15.895404  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:15.909657  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:15.962296  2343 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:15.915246  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:15.917291  1720 raft_consensus.cc:1240] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: [14.11-16.12]
W20260709 13:05:15.892993  8827 log.cc:927] Time spent T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:05:15.955307  1611 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:15.807094  8554 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.007392  2343 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:15.750288  8804 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.939103  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:15.803139  8751 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:15.763165  8795 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:15.850682  8370 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.759651  8561 raft_consensus.cc:697] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:15.943760  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:15.795943  2313 leader_election.cc:336] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.949s (SENT)
W20260709 13:05:16.042234  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 246327us
I20260709 13:05:15.953302  8863 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.894585  8692 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:15.945333  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:15.985212  8822 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:15.905301  8371 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:15.761667  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55890 (request call id 667) took 1572 ms. Trace:
I20260709 13:05:15.892320  8853 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:15.910513  8815 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.954835  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:15.784188  8797 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.955825  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504"
I20260709 13:05:15.965694  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:16.060213  1645 raft_consensus.cc:2393] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 16: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:16.060437  1719 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
W20260709 13:05:15.966977  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 97028.6us
I20260709 13:05:15.868266  8851 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.969185  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3347) took 1458 ms. Trace:
I20260709 13:05:16.062808  1714 rpcz_store.cc:276] 0709 13:05:14.510923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.511412 (+   489us) service_pool.cc:224] Handling call
0709 13:05:14.511818 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:14.512337 (+   519us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:14.750384 (+238047us) log.cc:844] Serialized 14 byte log entry
0709 13:05:15.968469 (+1218085us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:15.968547 (+    78us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:15.968626 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:15.968721 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:15.968996 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24367,"prepare.queue_time_us":382754,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":52489,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":24156,"spinlock_wait_cycles":1576832,"thread_start_us":1392860,"threads_started":2,"wal-append.queue_time_us":1212126}
I20260709 13:05:15.956785  8810 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:15.970286  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:15.973727  8758 raft_consensus.cc:697] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:15.905675  8857 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 13:05:16.085557  8857 consensus_queue.cc:1243] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:05:15.909060  8793 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:15.891768  8771 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:15.939085  8817 raft_consensus.cc:697] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:15.968595  1611 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:15.977494  8824 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:15.986510  2349 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 21.
I20260709 13:05:15.904973  8637 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:15.985821  8814 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:15.986158  8791 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:15.968474  8658 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:15.977829  8595 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.002533  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:16.089063  8813 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.947134  8750 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:15.951629  8816 consensus_queue.cc:260] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.010630  1717 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:16.017498  1537 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:15.890583  8671 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.949851  8819 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:16.001410  8756 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:16.142204  8658 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.133216  8791 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.089828  8815 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:16.045514  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 366695us
I20260709 13:05:15.948894  1647 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:16.022289  8029 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.009s	sys 0.007s
W20260709 13:05:16.050060  2313 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:34553 timed out after 1.859s (SENT) [suppressed 26 similar messages]
I20260709 13:05:16.050298  1646 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:05:16.053234  1713 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:05:15.982627  8812 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.053927  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:15.983994  8788 raft_consensus.cc:697] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:15.988544  8826 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.039713  8370 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:16.064661  2314 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:16.040867  8554 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.067737  1720 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
W20260709 13:05:16.067905  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:16.216354  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:16.217094  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.068579  1645 raft_consensus.cc:1275] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:05:16.091718  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:16.176542  8814 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:16.223147  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:16.224872  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:16.096565  1612 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:16.054306  8808 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:16.073400  8786 consensus_meta.cc:287] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.687s	user 0.010s	sys 0.003s
I20260709 13:05:16.125483  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:16.237326  1719 raft_consensus.cc:2410] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:05:16.043391  8795 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:16.057885  8867 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:16.134654  1648 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:05:16.193444  8871 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:16.150280  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:16.151798  1642 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 21.
I20260709 13:05:16.115105  8811 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.157096  1566 raft_consensus.cc:2468] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:05:16.161489  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:16.258582  1715 raft_consensus.cc:2393] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 18: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:16.054950  8863 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:16.258713  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 290123us
I20260709 13:05:16.175271  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:16.123452  8859 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:15.956018  8818 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.096769  8810 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.089994  8853 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:16.183488  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 166072us
W20260709 13:05:16.182781  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 291565us
I20260709 13:05:16.189406  1716 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:16.177866  8736 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:16.187254  8029 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.010s	sys 0.007s
I20260709 13:05:16.221474  1647 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19.
I20260709 13:05:16.249051  8826 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 6
I20260709 13:05:16.182265  8658 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:16.234673  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 138265us
I20260709 13:05:16.296789  1612 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:16.158707  8813 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:16.227928  8816 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:16.243764  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 179149us
I20260709 13:05:16.304543  2314 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:16.247125  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:16.192138  8667 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:16.003304  8807 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.175300  8750 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:16.177968  8671 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:16.254482  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 204414us
I20260709 13:05:16.228780  8814 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.215543  8647 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.319653  8647 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:16.320577  8873 log.cc:927] Time spent T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:05:16.322175  8647 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.156699  8758 consensus_queue.cc:237] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:16.322672  8868 log.cc:927] Time spent T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:05:16.324034   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:05:16.260670  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:05:16.325662  8864 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 13:05:16.326560  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:16.327704  8869 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:05:16.259727  1717 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:05:16.329180  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:16.269244  8810 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:16.329778  1717 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 20.
I20260709 13:05:16.330094  8637 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.330703  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:16.331341  1717 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 17.
I20260709 13:05:16.331017  8797 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.267362  8859 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.332304  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:16.249292  8786 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 19. Current term is 20.
I20260709 13:05:16.185261  8791 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:16.313942  8667 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.238279  8831 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:16.274777  2346 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:05:16.279340  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:16.268982  8756 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:16.264665  8804 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.313747  8877 consensus_queue.cc:1059] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:05:16.271462  8811 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.335717  8877 raft_consensus.cc:3055] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:16.336083  8877 raft_consensus.cc:740] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:16.272023  8871 raft_consensus.cc:697] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:16.337419  8871 consensus_queue.cc:237] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:16.312896  8788 consensus_queue.cc:237] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:16.341261  8851 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.256243  8867 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:16.270582  8798 consensus_queue.cc:1059] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 13:05:16.227090  8633 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.046s
W20260709 13:05:16.263690  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 54415.8us
W20260709 13:05:16.279826  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.344440  1537 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:16.282732  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "066ee0109aba4465ba17fa65e14ef504" is_pre_election: true
I20260709 13:05:16.307457  1720 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:16.345988  1566 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 14.
I20260709 13:05:16.263003  1611 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:16.238560  8824 consensus_queue.cc:1059] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
W20260709 13:05:16.312119  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.175370  8561 consensus_queue.cc:237] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.281308  8736 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.347255  8824 raft_consensus.cc:3055] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:16.314214  8844 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:05:16.347728  8824 raft_consensus.cc:740] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:16.317591  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:16.348381  1715 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:16.245162  8808 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.309890  8828 consensus_queue.cc:1059] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:05:16.356724  1717 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:05:16.332435  1639 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 23.
I20260709 13:05:16.274057  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:16.365329  1716 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:05:16.334139  8831 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.267493  8818 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.282347  8853 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:16.372253  8750 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:16.373030  8771 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.374276  8671 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.374980  8867 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.375636  8867 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:16.377262  8811 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.379104  8815 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.346290  8801 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:16.259013  8879 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:16.348628  8824 consensus_queue.cc:260] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:16.232565  8554 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:16.327836  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:16.394774  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 131833us
I20260709 13:05:16.395100  1719 raft_consensus.cc:2468] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 22.
I20260709 13:05:16.396418  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:16.357928  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:16.359484  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 62797us
I20260709 13:05:16.397683  1719 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:16.398821  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:16.399200  1612 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:16.399916  1647 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 21.
I20260709 13:05:16.251386  8817 consensus_queue.cc:237] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.366967  8807 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:16.367290  8793 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:16.367682  8736 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:16.396354  1611 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:16.377980  8808 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:16.336971  8877 consensus_queue.cc:260] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.408501  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:16.413166  1720 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:16.411056  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:16.359019  8814 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:16.358999  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 51219.4us
I20260709 13:05:16.370155  1715 raft_consensus.cc:2468] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
I20260709 13:05:16.422802  1611 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:16.392139  8801 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.427331  1717 raft_consensus.cc:2393] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 16: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:16.432155  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:16.394371  8879 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.433331  1715 raft_consensus.cc:2410] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:05:16.436834  1611 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:16.432315  1714 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 20.
I20260709 13:05:16.442804  1611 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:16.410308  8804 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:16.432698  8801 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:16.429124  1687 leader_election.cc:400] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 19. Current term is 20.
I20260709 13:05:16.447149  8815 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:16.456210  1612 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:16.454293  2312 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:16.421733  8629 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:16.435254  8282 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.425284  8870 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.431447  8750 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:16.449911  8851 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:16.425592  8756 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.474404  1612 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:16.451370  8877 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:16.460083  1687 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:16.447772  8804 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:16.490284  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:05:16.500422  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.491786  1720 raft_consensus.cc:2410] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:05:16.451063  8807 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:16.438925  8879 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:16.449522  8863 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:16.477531  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 173049us
I20260709 13:05:16.488830  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:16.509759  1644 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:05:16.461829  8824 raft_consensus.cc:3060] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:16.459939  8828 raft_consensus.cc:2804] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:16.518607  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:16.500684  1643 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:16.491554  1645 raft_consensus.cc:1240] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:16.496776  8851 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:16.503525  1647 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:05:16.502880  8808 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:16.464177  8815 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.504271  8756 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.537855  8815 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:16.539690  8756 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.509264  8818 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.540525  8808 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:16.540985  8808 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:16.524915  8851 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:16.542249  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:16.542827  8888 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:16.485905  8633 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.543344  8888 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.499032  8831 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:16.543655  8633 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.492676  8797 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:16.512953  8736 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.548955  8736 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.515683  1612 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:16.493443  8637 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.563645  8637 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.566896  8637 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.493863  8647 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:16.497869  1687 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:16.503257  8804 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:16.567867  8647 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.570495  8647 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.462111  8813 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:16.471253  8793 consensus_queue.cc:1048] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:05:16.503523  8807 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.527190  8886 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:16.586135  8807 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:16.586648  8371 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.588217  1641 raft_consensus.cc:1240] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 19. Ops: []
I20260709 13:05:16.573984  8791 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:16.573364  8825 consensus_meta.cc:287] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.954s	user 0.012s	sys 0.009s
I20260709 13:05:16.528331  8554 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:16.492077  8629 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.596925  8629 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.483515  8750 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.598137  8791 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.497315  8870 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:16.599107  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:16.600574  8692 consensus_meta.cc:287] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.547s	user 0.012s	sys 0.002s
I20260709 13:05:16.600834  8870 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:16.550444  8633 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.525544  8828 raft_consensus.cc:697] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:16.516738  8841 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:16.557102  1719 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:16.510324  8859 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:16.618351  8825 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:16.619234  8825 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:16.620038  8825 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:16.620294  8814 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.603940  8750 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:16.626125  8886 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:16.628482  8371 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.632982  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:16.633951  1720 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:05:16.636473  1720 pending_rounds.cc:85] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 6
I20260709 13:05:16.636803  1720 pending_rounds.cc:107] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 17.7
I20260709 13:05:16.637050  1720 raft_consensus.cc:2889] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:16.571691  8888 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:16.562063  8671 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:16.582533  8370 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.494457  8282 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.666616  8814 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.667598  8282 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.670998  8828 consensus_queue.cc:237] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.585959  8637 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:16.550213  8689 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.602869  1721 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
W20260709 13:05:16.602973  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 105214us
I20260709 13:05:16.600723  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:16.592958  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:16.606262  1643 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:05:16.612493  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:16.672477  8808 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.637377  8819 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:16.057440  2348 rpcz_store.cc:276] 0709 13:05:14.188779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.188936 (+   157us) service_pool.cc:224] Handling call
0709 13:05:14.189227 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:14.189441 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:14.514892 (+325451us) log.cc:844] Serialized 14 byte log entry
0709 13:05:15.325298 (+810406us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b18004bd960. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8248a2d4bc 00007f8245d7b48b 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a2523a
0709 13:05:15.747314 (+422016us) log.cc:844] Serialized 14 byte log entry
0709 13:05:15.760969 (+ 13655us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:15.761078 (+   109us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:15.761160 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:15.761274 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:15.761459 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2230,"prepare.queue_time_us":1269564,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":84,"raft.queue_time_us":15327,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":260,"spinlock_wait_cycles":43142016,"thread_start_us":970607,"threads_started":3,"wal-append.queue_time_us":433946}
I20260709 13:05:16.690371  1722 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:16.692080  1719 pending_rounds.cc:85] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 10
W20260709 13:05:16.577195  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 61593.1us
I20260709 13:05:16.697906  1719 pending_rounds.cc:107] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:05:16.698405  1719 raft_consensus.cc:2889] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:16.699512  1612 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:16.716754  1612 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:16.723356  1570 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 20: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:16.688846  8637 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:16.724097  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:16.727927  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.692785  8736 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:16.732749  1640 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 12.
I20260709 13:05:16.690815  8851 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:16.723605  8814 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:16.740163  8814 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:16.740598  8814 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:16.742384  8851 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:16.746178  8886 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:16.749168  8879 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:16.723315  8859 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:16.689549  8671 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.696130  8689 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.773981  8751 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:16.777341  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 60715.2us
I20260709 13:05:16.783006  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 2994) took 1113 ms. Trace:
I20260709 13:05:16.783568  1642 rpcz_store.cc:276] 0709 13:05:15.670505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.670664 (+   159us) service_pool.cc:224] Handling call
0709 13:05:16.782755 (+1112091us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b5400182c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:16.782920 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":632470400}
I20260709 13:05:16.695735  8813 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:05:16.779901  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 3001) took 1032 ms. Trace:
I20260709 13:05:16.790108  1641 rpcz_store.cc:276] 0709 13:05:15.747534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.048611 (+301077us) service_pool.cc:224] Handling call
0709 13:05:16.049025 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:16.588805 (+539780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.779631 (+190826us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b54004114d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:16.779835 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1239036672}
I20260709 13:05:16.733614  8637 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:16.726394  8831 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:16.793799  1612 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:05:16.798419  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 309017us
I20260709 13:05:16.806716  1687 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:16.753394  8859 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.781093  8825 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:16.819656  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 365924us
I20260709 13:05:16.815541  8751 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:16.812543  8822 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:05:16.833896  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 1.2583e+06us
I20260709 13:05:16.834992  1685 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:16.837267  1685 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:16.820912  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:16.848588  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:16.849010  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:16.849566  1643 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:05:16.852015  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:16.852619  1642 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:05:16.853154  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:16.857266  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:16.858111  2346 raft_consensus.cc:2393] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 18: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:05:16.858209  1687 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:16.859620  1612 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:16.860922  1612 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:16.821327  8879 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:16.863317  1686 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:16.863437  1612 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:16.805843  8831 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:16.867264  2344 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:05:16.868316  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:16.872987  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:16.873870  1643 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:05:16.868959  2314 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:16.871335  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:16.861086  8561 raft_consensus.cc:2804] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:16.877101  2340 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:05:16.878468  8859 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:16.879069  8859 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.879562  8859 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:16.868054  8793 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:16.881367  8793 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:16.841024  8751 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:16.862412  8816 consensus_queue.cc:1059] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:05:16.872804  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:16.883212  2345 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:05:16.840176  8817 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:16.875193  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:16.841326  8822 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:16.953749  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:16.911545  2314 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:16.927932  2346 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:16.932529  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:16.870720  8853 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:16.917013  1537 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:05:16.883383  8658 consensus_meta.cc:287] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.566s	user 0.007s	sys 0.011s
I20260709 13:05:17.039575  8658 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:16.862087  8863 consensus_queue.cc:1059] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 13:05:17.041905  8633 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:17.042459  8633 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.042810  8633 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:16.953563  8029 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:16.881983  8751 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:17.025822  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:16.863354  8841 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:16.868592  8882 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:16.881805  8793 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:17.031774  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 120325us
W20260709 13:05:17.034571  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 117707us
I20260709 13:05:17.035501  1648 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:16.953330  8879 raft_consensus.cc:2804] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:17.037737  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:17.047587  8029 raft_consensus.cc:697] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:16.889847  8813 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:16.871918  8808 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:17.054881  8893 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 13:05:17.057914  8370 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:17.062579  8895 log.cc:927] Time spent T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.222s	user 0.000s	sys 0.000s
I20260709 13:05:17.048686  8689 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:17.182039  8029 consensus_queue.cc:237] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:17.070525  8853 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:17.073544  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 210255us
I20260709 13:05:16.894856  8371 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:16.864001  8786 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.106s
I20260709 13:05:17.071782  8890 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:17.166682  8828 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.894342  8797 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:17.172096  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:16.974819  8819 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:17.193990  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3361) took 1304 ms. Trace:
I20260709 13:05:17.195809  1715 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19.
I20260709 13:05:17.047400  8816 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.048607  8817 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:16.911468  8822 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:17.155494  8857 raft_consensus.cc:3055] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:17.157255  8841 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:17.018147  8815 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:17.203769  8888 consensus_meta.cc:287] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.539s	user 0.004s	sys 0.014s
I20260709 13:05:17.160660  1647 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:17.219201  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.175187  8561 raft_consensus.cc:697] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 20 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:05:17.203003  8897 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.339s	user 0.005s	sys 0.000s
W20260709 13:05:17.199076  8801 consensus_meta.cc:287] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.726s	user 0.020s	sys 0.015s
I20260709 13:05:17.245043  8671 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:17.222544  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 364319us
W20260709 13:05:17.219875  2312 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:17.264041  1648 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:05:17.264303  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 51656.8us
W20260709 13:05:17.253274  8750 consensus_meta.cc:287] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.631s	user 0.013s	sys 0.000s
I20260709 13:05:17.194293  8871 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.174983  8882 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:17.232110  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3363) took 1290 ms. Trace:
I20260709 13:05:17.209508  8863 raft_consensus.cc:3055] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:17.205622  8879 raft_consensus.cc:697] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:17.209434  8813 raft_consensus.cc:697] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:17.245589  1644 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:17.215081  8689 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:17.215405  8371 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:17.224505  8890 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.228346  8554 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:17.240134  8857 raft_consensus.cc:740] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:17.244760  8841 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:17.266733  1687 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:05:17.270308  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:17.267457  1716 rpcz_store.cc:276] 0709 13:05:15.941490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.669024 (+727534us) service_pool.cc:224] Handling call
0709 13:05:16.669709 (+   685us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:16.670155 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:16.670184 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:16.670917 (+   733us) log.cc:844] Serialized 26 byte log entry
0709 13:05:16.816129 (+145212us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:16.816359 (+   230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.816433 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:17.231816 (+415383us) raft_consensus.cc:1685] finished
0709 13:05:17.231842 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:17.232025 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4502,"raft.run_cpu_time_us":605,"raft.run_wall_time_us":8017,"thread_start_us":144672,"threads_started":1,"wal-append.queue_time_us":200300}
I20260709 13:05:17.292660  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:17.245496  8816 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.267704  8859 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:17.342206  2349 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 618) took 1777 ms (client timeout 2173 ms). Trace:
W20260709 13:05:17.342751  2349 rpcz_store.cc:269] 0709 13:05:15.565688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.566199 (+   511us) service_pool.cc:224] Handling call
0709 13:05:17.341969 (+1775770us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b54006d0454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:17.342137 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":760375680}
I20260709 13:05:17.345029  8857 consensus_queue.cc:260] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.346422  8857 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:17.348589  8751 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.352782  8831 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:17.353350  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 86698us
I20260709 13:05:17.353544  8831 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:17.354115  8831 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:17.354535  8831 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:17.354954  1687 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:17.355918  1687 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:17.359494  1687 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:17.297256  8892 log.cc:927] Time spent T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.342s	user 0.000s	sys 0.001s
W20260709 13:05:17.299373  8901 log.cc:927] Time spent T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
I20260709 13:05:17.290230  8689 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.362563  8633 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.274886  8871 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.364753  8689 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:17.280413  8814 consensus_meta.cc:287] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.539s	user 0.008s	sys 0.004s
I20260709 13:05:17.296748  8554 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:17.368091  8554 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:17.368980  8814 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.313925  8561 consensus_queue.cc:237] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.272783  8750 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.290498  8890 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:17.373808  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 804) took 1093 ms. Trace:
I20260709 13:05:17.374358  1715 rpcz_store.cc:276] 0709 13:05:16.280353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.600247 (+319894us) service_pool.cc:224] Handling call
0709 13:05:17.373077 (+772830us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b5400212854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:17.373729 (+   652us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":643558144}
I20260709 13:05:17.376420  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39930 (request call id 3002) took 1626 ms. Trace:
I20260709 13:05:17.376856  1639 rpcz_store.cc:276] 0709 13:05:15.749722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.949554 (+199832us) service_pool.cc:224] Handling call
0709 13:05:17.376216 (+1426662us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b54001f3954. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:17.376354 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":862256384}
I20260709 13:05:17.378182  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:17.379076  1639 raft_consensus.cc:2393] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 18: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:17.379397  8750 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:17.380215  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:17.383762  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:17.386234  8814 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:17.387559  8859 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:17.388523  8859 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:17.389243  8859 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:17.391072  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 2998) took 1711 ms. Trace:
I20260709 13:05:17.274574  1647 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:05:17.291189  8371 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:17.318780  8813 consensus_queue.cc:237] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.273475  8882 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:17.400427  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:17.322202  8825 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:17.405046  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:05:17.322827  8804 consensus_meta.cc:287] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.753s	user 0.027s	sys 0.008s
I20260709 13:05:17.406142  2341 raft_consensus.cc:2368] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 18. Current term is 19.
I20260709 13:05:17.324368  8879 consensus_queue.cc:237] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:17.271658  8863 raft_consensus.cc:740] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:17.324973  8841 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.415483  8798 raft_consensus.cc:3055] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:17.416162  8798 raft_consensus.cc:740] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:17.416289  8841 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:17.417356  8798 consensus_queue.cc:260] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:17.418144  8795 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:17.384646  8793 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.262485  8801 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.418684  8798 raft_consensus.cc:3060] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:17.256021  8888 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:17.338900  8816 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:17.355710  1462 catalog_manager.cc:5697] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:17.356755  8824 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:17.360141  8877 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:17.242264  8828 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:17.437873  8824 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.438658  8824 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:17.395109  1647 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:17.397037  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.320981  8815 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:17.293503  1716 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:05:17.444604  1612 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:17.392726  1646 rpcz_store.cc:276] 0709 13:05:15.679402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.679540 (+   138us) service_pool.cc:224] Handling call
0709 13:05:15.679890 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:16.185206 (+505316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.390697 (+1205491us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b54002add54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:17.390974 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":851125120}
W20260709 13:05:17.431325  8876 log.cc:927] Time spent T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 1.100s	user 0.005s	sys 0.007s
I20260709 13:05:17.403250  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:17.284981  8658 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:17.437381  1462 catalog_manager.cc:5697] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 11 to 14, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 14 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:17.404301  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:17.297260  8904 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260709 13:05:17.403142  2312 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:17.471864  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 68765us
I20260709 13:05:17.409121  1715 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:05:17.388954  8633 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:17.419862  2347 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 620) took 1838 ms (client timeout 2117 ms). Trace:
I20260709 13:05:17.441068  8801 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:17.327008  8853 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:17.423569  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:17.480131  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:17.480680  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.434386  2343 raft_consensus.cc:2368] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15.
I20260709 13:05:17.435984  8888 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:17.384724  1717 raft_consensus.cc:2376] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18. Re-sending same reply.
I20260709 13:05:17.443667  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:17.484156  1716 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:17.402338  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:17.401163  8871 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:17.453971  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:17.454954  8750 consensus_queue.cc:1059] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:17.455453  8656 raft_consensus.cc:3055] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:17.455828  8816 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:17.456876  8814 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:17.441457  8877 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:17.457772  8841 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:17.500566  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:17.502224  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:17.460765  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:17.503760  1644 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:17.505474  1644 raft_consensus.cc:1240] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:17.463215  8751 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:17.464645  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 3357) took 1578 ms. Trace:
I20260709 13:05:17.469501  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:17.470799  1612 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:17.473328  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:17.474866  2347 rpcz_store.cc:269] 0709 13:05:15.581384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.584580 (+  3196us) service_pool.cc:224] Handling call
0709 13:05:17.419665 (+1835085us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b5400164754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:17.419795 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55061376}
I20260709 13:05:17.405992  8804 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:17.472790  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:17.481966  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:17.527906  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.529075  1686 leader_election.cc:400] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 18. Current term is 19.
I20260709 13:05:17.529505  1686 leader_election.cc:403] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:05:17.485131  2349 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 15.
I20260709 13:05:17.490332  2313 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:17.501435  1646 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:05:17.521620  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3364) took 1575 ms. Trace:
I20260709 13:05:17.531845  1719 rpcz_store.cc:276] 0709 13:05:15.946694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.691314 (+744620us) service_pool.cc:224] Handling call
0709 13:05:16.691812 (+   498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:16.699145 (+  7333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:16.699195 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:16.699786 (+   591us) log.cc:844] Serialized 26 byte log entry
0709 13:05:16.830407 (+130621us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:16.830462 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.830558 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:17.521333 (+690775us) raft_consensus.cc:1685] finished
0709 13:05:17.521360 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:17.521543 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":130207,"threads_started":1,"wal-append.queue_time_us":350867}
I20260709 13:05:17.533754  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:17.534600  1719 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 23.
I20260709 13:05:17.535801  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:17.523173  1639 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:05:17.538443  1639 raft_consensus.cc:1275] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:05:17.523253  1647 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:05:17.522719  1713 rpcz_store.cc:276] 0709 13:05:15.886433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.886572 (+   139us) service_pool.cc:224] Handling call
0709 13:05:17.464414 (+1577842us) spinlock_profiling.cc:243] Waited 96.3 ms on lock 0x7b5400210f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:17.464543 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222830208}
I20260709 13:05:17.545154  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:17.546429  1713 raft_consensus.cc:2410] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
W20260709 13:05:17.524721  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 53972.8us
I20260709 13:05:17.526885  1572 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 18. Current term is 19. Ops: []
W20260709 13:05:17.527331  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 54622us
I20260709 13:05:17.536633  1719 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 15.
I20260709 13:05:17.551151  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:17.548821  2312 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:17.508215  8637 consensus_meta.cc:287] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.699s	user 0.011s	sys 0.003s
I20260709 13:05:17.560788  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:17.555675  2312 leader_election.cc:336] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:17.560907  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:17.485956  8817 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:17.486214  8656 raft_consensus.cc:740] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:17.500171  8877 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:17.565274  1686 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:05:17.552654  8758 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.227s	user 0.001s	sys 0.010s
I20260709 13:05:17.499961  8814 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:17.574270  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:17.577113  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:17.576066  1719 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 25.
I20260709 13:05:17.580629  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:17.581764  8793 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:17.584652  8814 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:17.573935  8758 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.249s	user 0.002s	sys 0.010s
W20260709 13:05:17.597812  1722 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.006s	sys 0.015s
W20260709 13:05:17.598572  1722 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.007s	sys 0.015s
I20260709 13:05:17.560487  8888 raft_consensus.cc:2804] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:17.562189  8826 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:17.576462  8736 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:17.508497  8816 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:17.622682  1716 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19.
I20260709 13:05:17.629449  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:17.755825  1462 catalog_manager.cc:5697] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 8 to 14, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 14 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:17.779556  1640 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44424 (request call id 772) took 1910 ms (client timeout 1594 ms). Trace:
I20260709 13:05:17.786584  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:05:17.790817  1685 leader_election.cc:336] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:17.799269  1715 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 16.
I20260709 13:05:17.655153  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:17.805503  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:17.815289  1718 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:17.817406  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:17.649870  8804 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:17.810529  1643 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:17.830976  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:17.651597  8814 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:17.821687  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46982 (request call id 1234) took 1539 ms. Trace:
I20260709 13:05:17.825367  8282 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.826174  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:05:17.826421  2312 leader_election.cc:336] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:17.702900  8371 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.573675  8637 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.676810  8667 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:17.670687  8595 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.308s	user 0.008s	sys 0.016s
I20260709 13:05:17.656613  8656 consensus_queue.cc:260] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:17.872939  1642 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:17.854178  8481 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:17.810045  1640 rpcz_store.cc:269] 0709 13:05:15.868899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.869211 (+   312us) service_pool.cc:224] Handling call
0709 13:05:17.778946 (+1909735us) spinlock_profiling.cc:243] Waited 769 ms on lock 0x7b54001a9854. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:17.779204 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1769047296}
W20260709 13:05:17.641108  8807 consensus_meta.cc:287] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 1.053s	user 0.017s	sys 0.003s
I20260709 13:05:17.884135  8656 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:17.883823  8807 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.695773  8816 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:17.883174  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:17.883332  8857 consensus_meta.cc:287] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.536s	user 0.012s	sys 0.004s
I20260709 13:05:17.888592  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:17.704185  8822 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.716027  8824 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.679780  8826 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:17.685061  8828 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:17.788939  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:05:17.813043  1685 leader_election.cc:336] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:17.680027  8831 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.630627  8853 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 18. Current term is 19.
W20260709 13:05:17.909781  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.501251  8863 consensus_queue.cc:260] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.910480  8831 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:17.915194  1713 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 22: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:17.696512  8859 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.922389  8877 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.711584  8882 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.928529  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:17.623023  8888 raft_consensus.cc:697] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:17.930073  8828 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:17.654196  8890 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:17.811369  1462 catalog_manager.cc:5697] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 9 to 12, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 12 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:17.846067  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:17.846737  2347 raft_consensus.cc:2368] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13.
W20260709 13:05:17.852325  1714 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 797) took 1649 ms (client timeout 1859 ms). Trace:
I20260709 13:05:17.931700  8667 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.852581  8282 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.841662  1572 rpcz_store.cc:276] 0709 13:05:16.282207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.282342 (+   135us) service_pool.cc:224] Handling call
0709 13:05:17.412466 (+1130124us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b5800011198. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:05:17.412809 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:17.547146 (+134337us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.820843 (+273697us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54002251d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:17.821311 (+   468us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270442624}
I20260709 13:05:17.863999  8756 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:17.865058  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3367) took 1900 ms. Trace:
I20260709 13:05:17.865103  2346 raft_consensus.cc:2368] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 18. Current term is 19.
I20260709 13:05:17.633270  8736 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.962926  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 809) took 1138 ms. Trace:
I20260709 13:05:17.963158  8899 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:17.963522  1715 rpcz_store.cc:276] 0709 13:05:16.825945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.503395 (+677450us) service_pool.cc:224] Handling call
0709 13:05:17.962604 (+459209us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b5400211454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:17.962839 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":535414016}
I20260709 13:05:17.964008  8899 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:17.964744  8282 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:17.965022  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:17.965929  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:17.966771  8756 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:17.967592  8814 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:17.969740  2346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55874 (request call id 630) took 1027 ms. Trace:
I20260709 13:05:17.970230  2346 rpcz_store.cc:276] 0709 13:05:16.928161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.830179 (+902018us) service_pool.cc:224] Handling call
0709 13:05:17.955701 (+125522us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:17.971549  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3520) took 1039 ms. Trace:
I20260709 13:05:17.972196  1642 rpcz_store.cc:276] 0709 13:05:16.932431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.932562 (+   131us) service_pool.cc:224] Handling call
0709 13:05:16.932883 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:17.873817 (+940934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.971073 (+ 97256us) spinlock_profiling.cc:243] Waited 934 ms on lock 0x7b54001c0554. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:17.971470 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2147238272}
I20260709 13:05:17.899580  1685 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:17.873004  8595 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.522s	user 0.009s	sys 0.016s
I20260709 13:05:18.010804  2348 raft_consensus.cc:1240] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:17.578832  8817 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:18.012971  8890 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 22 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:17.952077  1722 rpcz_store.cc:276] 0709 13:05:15.963982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.723328 (+759346us) service_pool.cc:224] Handling call
0709 13:05:16.756599 (+ 33271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:16.828765 (+ 72166us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:05:17.027311 (+198546us) log.cc:844] Serialized 14 byte log entry
0709 13:05:17.599103 (+571792us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:17.599170 (+    67us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:17.599228 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.864326 (+265098us) spinlock_profiling.cc:243] Waited 71.8 ms on lock 0x7b5400248454. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:17.864418 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:17.864828 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":416699,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":1456,"raft.queue_time_us":111,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":2229,"spinlock_wait_cycles":165230976,"thread_start_us":739844,"threads_started":2,"wal-append.queue_time_us":647876}
I20260709 13:05:17.883893  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:17.889400  1646 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:17.899056  1645 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:17.899288  1648 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:17.899362  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:17.884313  1570 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40656 (request call id 2944) took 1926 ms (client timeout 1627 ms). Trace:
I20260709 13:05:17.909046  8481 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:18.018043  1570 rpcz_store.cc:269] 0709 13:05:15.955364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.955580 (+   216us) service_pool.cc:224] Handling call
0709 13:05:17.861322 (+1905742us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b5400223dd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:17.881623 (+ 20301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1486400384}
I20260709 13:05:18.022851  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:18.023873  1718 raft_consensus.cc:2468] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:18.025206  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:17.928808  8882 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:17.641004  8912 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.028191  2313 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:05:17.682384  8915 log.cc:927] Time spent T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:05:18.049868  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39930 (request call id 3013) took 1202 ms. Trace:
I20260709 13:05:17.925382  8859 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:17.931667  8371 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:18.015960  8817 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:18.050413  1641 rpcz_store.cc:276] 0709 13:05:16.847606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.847794 (+   188us) service_pool.cc:224] Handling call
0709 13:05:16.848279 (+   485us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:16.861996 (+ 13717us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:16.862027 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:16.870457 (+  8430us) log.cc:844] Serialized 52 byte log entry
0709 13:05:17.269435 (+398978us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:17.782436 (+513001us) log.cc:844] Serialized 14 byte log entry
0709 13:05:17.783251 (+   815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.783481 (+   230us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:18.049285 (+265804us) raft_consensus.cc:1685] finished
0709 13:05:18.049311 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.049573 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13296,"prepare.queue_time_us":456355,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":60373,"raft.run_cpu_time_us":528,"raft.run_wall_time_us":1749,"thread_start_us":628938,"threads_started":2,"wal-append.queue_time_us":572660}
I20260709 13:05:17.898252  8826 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:17.946856  1714 rpcz_store.cc:269] 0709 13:05:16.202704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.224592 (+ 21888us) service_pool.cc:224] Handling call
0709 13:05:17.851811 (+1627219us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b540038fed4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:17.852076 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":247332608}
I20260709 13:05:17.962069  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2937) took 2270 ms. Trace:
I20260709 13:05:17.976434  8756 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:17.880419  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:17.228732  1720 rpcz_store.cc:276] 0709 13:05:15.889416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.635900 (+746484us) service_pool.cc:224] Handling call
0709 13:05:16.636226 (+   326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:16.641791 (+  5565us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:16.641838 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:16.642707 (+   869us) log.cc:844] Serialized 52 byte log entry
0709 13:05:16.839478 (+196771us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:17.155590 (+316112us) log.cc:844] Serialized 14 byte log entry
0709 13:05:17.193404 (+ 37814us) log.cc:844] Serialized 14 byte log entry
0709 13:05:17.193593 (+   189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.193691 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:17.193742 (+    51us) raft_consensus.cc:1685] finished
0709 13:05:17.193752 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:17.193928 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29799,"raft.queue_time_us":86913,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":20858,"thread_start_us":394569,"threads_started":2,"wal-append.queue_time_us":372982}
I20260709 13:05:17.980652  1643 raft_consensus.cc:2468] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13.
I20260709 13:05:17.986981  8667 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:17.977267  8370 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.938849  8828 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:17.997061  1716 raft_consensus.cc:2393] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 18: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:05:17.998373  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.066738  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:18.068171  1720 raft_consensus.cc:2468] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 23.
I20260709 13:05:18.070250  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:18.071058  8807 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:18.071346  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 3381) took 1229 ms. Trace:
I20260709 13:05:18.071910  1714 rpcz_store.cc:276] 0709 13:05:16.841500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.069796 (+1228296us) service_pool.cc:224] Handling call
0709 13:05:18.071278 (+  1482us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.069418  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:17.963030  8736 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:18.028594  8912 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:18.077874  1717 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 20.
I20260709 13:05:18.006551  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:18.016479  8637 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:18.016947  1713 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19.
I20260709 13:05:18.017738  1715 raft_consensus.cc:2368] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 18. Current term is 19.
I20260709 13:05:18.016034  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:17.925379  8877 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:18.082777  8828 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:18.007500  8888 consensus_queue.cc:237] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:17.897588  8824 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:18.029189  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:18.032051  8595 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:18.084297  8595 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:18.035880  8863 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:18.038393  8899 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:18.039417  8481 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:18.046484  8825 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:05:18.046793  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 233950us
I20260709 13:05:18.059883  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:18.061049  1648 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19.
I20260709 13:05:18.116101  1648 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:18.056282  1721 rpcz_store.cc:276] 0709 13:05:15.679401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.679828 (+   427us) service_pool.cc:224] Handling call
0709 13:05:17.933790 (+2253962us) spinlock_profiling.cc:243] Waited 728 ms on lock 0x7b5400213754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:17.949907 (+ 16117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1673896320}
I20260709 13:05:18.073261  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:18.119910  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:17.990868  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:18.075274  1720 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:18.073840  8370 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:18.081856  8853 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.034181  8831 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.082828  1722 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:05:18.088681  8595 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:18.090348  1538 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:18.060429  8822 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:18.102653  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3522) took 1169 ms. Trace:
W20260709 13:05:18.124917  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 91705.8us
I20260709 13:05:18.077721  1462 catalog_manager.cc:5697] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 6 to 17, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 17 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:18.133001  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 808) took 1308 ms. Trace:
I20260709 13:05:18.134812  1645 rpcz_store.cc:276] 0709 13:05:16.932986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.933239 (+   253us) service_pool.cc:224] Handling call
0709 13:05:16.933684 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.017124 (+1083440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.089470 (+ 72346us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b5400098154. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:18.102541 (+ 13071us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2218364672}
I20260709 13:05:18.144778  8853 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:18.145885  1640 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:05:18.147716  8853 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:18.158600  1639 raft_consensus.cc:1240] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: []
W20260709 13:05:18.164018  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:18.164698  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:18.165120  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.167166  1714 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:18.143754  1713 rpcz_store.cc:276] 0709 13:05:16.824471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.917614 (+1093143us) service_pool.cc:224] Handling call
0709 13:05:18.132746 (+215132us) spinlock_profiling.cc:243] Waited 49.8 ms on lock 0x7b540002dad4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:18.132914 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114634496}
W20260709 13:05:18.174161  1685 leader_election.cc:336] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:18.180352  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:18.184551  8667 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:18.185971  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:18.191172  1685 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:18.201009  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 110954us
I20260709 13:05:18.201329  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:18.202989  1716 raft_consensus.cc:2393] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:18.203537  1642 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:18.195734  8831 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:18.215485  8370 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:18.216200  8921 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.217029  8921 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:18.218025  8831 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:18.218304  1538 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:18.208403  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:18.222096  8921 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:18.226018  1715 raft_consensus.cc:2393] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 20: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:05:18.230855  8933 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:05:18.237010  8912 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:18.193600  8481 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:05:18.238790  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 108402us
I20260709 13:05:18.240339  2313 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:18.247205  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:18.248203  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:18.245851  8637 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.196933  8853 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:18.257313  8890 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:18.253376  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:18.252933  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:18.252846  8637 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:18.263847  1716 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 21: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:18.264191  1713 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:18.219054  1462 catalog_manager.cc:5697] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 6 to 11, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 11 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:18.265125  1721 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:18.248607  8370 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:18.263279  8890 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:18.285914  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:18.290530  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:18.261257  8853 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:18.285215  8890 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:18.297942  8658 consensus_queue.cc:1059] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:18.301930  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:18.304450  8667 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:18.308102  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 89959.7us
I20260709 13:05:18.305305  8667 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:18.310668  1715 raft_consensus.cc:2468] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:18.294683  8831 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:18.322237  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:18.326159  1716 raft_consensus.cc:2393] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:18.335032  1538 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:18.337858  8667 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:18.339213  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:18.344514  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:18.346474  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.322373  8658 consensus_queue.cc:1243] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer 066ee0109aba4465ba17fa65e14ef504 log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:05:18.361187  1721 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:18.363963  1717 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:18.364594  8370 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:05:18.376884  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:18.378079  2312 leader_election.cc:336] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:18.355446  8831 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:18.385262  1715 raft_consensus.cc:1240] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 19. Current term is 20. Ops: []
W20260709 13:05:18.389760  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 69539.5us
I20260709 13:05:18.394129  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3368) took 2373 ms. Trace:
I20260709 13:05:18.395129  1715 rpcz_store.cc:276] 0709 13:05:16.018493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.353509 (+2335016us) service_pool.cc:224] Handling call
0709 13:05:18.376813 (+ 23304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.388981 (+ 12168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.389197 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.392380  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:18.376681  8370 raft_consensus.cc:697] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:18.410377  1643 raft_consensus.cc:1240] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:05:18.412855  1647 raft_consensus.cc:2393] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:05:18.412967  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 80979.7us
I20260709 13:05:18.414069  1612 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:18.426384  8853 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:18.398882  1462 catalog_manager.cc:5697] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:18.435847  1462 catalog_manager.cc:5697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 10 to 12, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 12 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:18.441031  8889 raft_consensus.cc:3055] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:18.443214  2348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55888 (request call id 895) took 1608 ms. Trace:
I20260709 13:05:18.443665  2348 rpcz_store.cc:276] 0709 13:05:16.835211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.835422 (+   211us) service_pool.cc:224] Handling call
0709 13:05:17.421223 (+585801us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.011481 (+590258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.143034 (+131553us) spinlock_profiling.cc:243] Waited 566 ms on lock 0x7b5400164754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:18.442911 (+299877us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b5400164750. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:18.443133 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2649116160}
I20260709 13:05:18.445715  8889 raft_consensus.cc:740] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:18.446807  8889 consensus_queue.cc:260] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:18.448338  8889 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:18.452165  1720 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:18.453860  1714 raft_consensus.cc:2468] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
W20260709 13:05:18.458218  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 44209us
I20260709 13:05:18.459209  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:18.451638  8828 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:18.468375  8826 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:18.471473  1643 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 19. Current term is 20. Ops: [17.11-19.12]
W20260709 13:05:18.481981  8936 log.cc:927] Time spent T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.002s
I20260709 13:05:18.482591  1642 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:18.486065  1537 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:18.454510  8637 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:18.487852  1715 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:18.486905  1640 raft_consensus.cc:2468] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15.
I20260709 13:05:18.492317  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3370) took 2445 ms. Trace:
I20260709 13:05:18.492847  1715 rpcz_store.cc:276] 0709 13:05:16.046954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.453402 (+2406448us) service_pool.cc:224] Handling call
0709 13:05:18.481172 (+ 27770us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.492015 (+ 10843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.492223 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:18.499715  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 131875us
I20260709 13:05:18.506181  1719 raft_consensus.cc:2410] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 21 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
I20260709 13:05:18.498308  8888 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.477504  8736 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:18.514627  8934 log.cc:927] Time spent T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.293s	user 0.000s	sys 0.008s
I20260709 13:05:18.513427  1537 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:18.506245  8370 consensus_queue.cc:237] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:18.507999  8667 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:18.518185  8667 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.522498  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2965) took 1311 ms. Trace:
I20260709 13:05:18.523065  1719 rpcz_store.cc:276] 0709 13:05:17.211298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.982484 (+771186us) service_pool.cc:224] Handling call
0709 13:05:18.522205 (+539721us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b540002e4d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:18.522395 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":973134592}
I20260709 13:05:18.525246  1719 raft_consensus.cc:1275] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 13:05:18.526553  1719 pending_rounds.cc:85] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 9
I20260709 13:05:18.509994  8826 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:18.498396  8899 consensus_queue.cc:1059] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:05:18.526911  1719 pending_rounds.cc:107] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:05:18.535408  1718 raft_consensus.cc:2368] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 15. Current term is 16.
I20260709 13:05:18.537424  1722 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:18.536047  1719 raft_consensus.cc:2889] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:18.544023  8282 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:18.544697  8282 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:18.544868  8756 raft_consensus.cc:3055] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:18.545590  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:18.546492  1643 raft_consensus.cc:2368] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 18. Current term is 19.
I20260709 13:05:18.545212  8282 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:18.545854  8756 raft_consensus.cc:740] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:18.549028  8756 consensus_queue.cc:260] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:18.545071  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:18.551571  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:18.526408  8667 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:18.554858  8371 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:18.555370  8371 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:18.556074  8371 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:18.555264  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3373) took 2503 ms. Trace:
I20260709 13:05:18.558391  1719 rpcz_store.cc:276] 0709 13:05:16.051513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.524196 (+2472683us) service_pool.cc:224] Handling call
0709 13:05:18.524726 (+   530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.554867 (+ 30141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.555159 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.560600  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 2978) took 1282 ms. Trace:
I20260709 13:05:18.561051  1714 rpcz_store.cc:276] 0709 13:05:17.278727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.119422 (+840695us) service_pool.cc:224] Handling call
0709 13:05:18.560398 (+440976us) spinlock_profiling.cc:243] Waited 46.4 ms on lock 0x7b5400210f54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:18.560531 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":106682752}
I20260709 13:05:18.565237  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34078 (request call id 811) took 1120 ms. Trace:
I20260709 13:05:18.565851  1718 rpcz_store.cc:276] 0709 13:05:17.444992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.024773 (+579781us) service_pool.cc:224] Handling call
0709 13:05:18.564973 (+540200us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b54003a2fd4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:18.565165 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1164952960}
I20260709 13:05:18.566520  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 794) took 2389 ms. Trace:
I20260709 13:05:18.568357  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3530) took 1278 ms. Trace:
I20260709 13:05:18.569269  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 796) took 2366 ms. Trace:
I20260709 13:05:18.568267  1719 rpcz_store.cc:276] 0709 13:05:16.178953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.565107 (+2386154us) service_pool.cc:224] Handling call
0709 13:05:18.565576 (+   469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.566002 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:18.566028 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:18.566104 (+    76us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:05:18.566162 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.566224 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.566339 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.569968  1718 rpcz_store.cc:276] 0709 13:05:16.202457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.568148 (+2365691us) service_pool.cc:224] Handling call
0709 13:05:18.568618 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.568926 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:18.568950 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:18.568990 (+    40us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:05:18.569019 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.569080 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.569193 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.571255  1719 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:18.572199  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 799) took 2345 ms. Trace:
I20260709 13:05:18.572549  1718 raft_consensus.cc:1275] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 13:05:18.569541  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 71384.2us
I20260709 13:05:18.567055  1642 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:05:18.567021  8756 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:18.531235  8888 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:18.577389  1612 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:18.582754  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.606961  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:18.607600  1647 raft_consensus.cc:1240] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:18.636468  1643 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:05:18.569039  1644 rpcz_store.cc:276] 0709 13:05:17.260887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.512595 (+251708us) service_pool.cc:224] Handling call
0709 13:05:17.513387 (+   792us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:17.515719 (+  2332us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:17.950191 (+434472us) log.cc:844] Serialized 14 byte log entry
0709 13:05:18.174474 (+224283us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:18.249705 (+ 75231us) log.cc:844] Serialized 26 byte log entry
0709 13:05:18.250076 (+   371us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:18.287494 (+ 37418us) log.cc:844] Serialized 14 byte log entry
0709 13:05:18.287668 (+   174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.287722 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:18.509108 (+221386us) spinlock_profiling.cc:243] Waited 4 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:05:18.530827 (+ 21719us) raft_consensus.cc:1685] finished
0709 13:05:18.530859 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.539238 (+  8379us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4144,"prepare.queue_time_us":236032,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":2821,"raft.queue_time_us":55608,"raft.run_cpu_time_us":490,"raft.run_wall_time_us":45587,"spinlock_wait_cycles":11091456,"thread_start_us":438118,"threads_started":2,"wal-append.queue_time_us":227987}
I20260709 13:05:18.637770  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:18.608554  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:18.638741  1643 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:18.639575  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:18.590497  2313 leader_election.cc:400] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 15. Current term is 16.
I20260709 13:05:18.588577  8736 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:18.609609  1646 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:18.611757  1641 raft_consensus.cc:1275] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:05:18.642259  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:18.642468  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:18.643085  1646 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:18.643214  1641 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:05:18.644011  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:18.644234  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:18.613322  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:18.613063  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:18.646765  1639 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:18.610752  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:18.619577  8595 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:18.650213  1646 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 21: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:18.651234  2340 raft_consensus.cc:1240] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:18.640326  1648 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:18.653355  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3371) took 2602 ms. Trace:
I20260709 13:05:18.638025  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:18.639902  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:18.572769  1719 rpcz_store.cc:276] 0709 13:05:16.226705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.570383 (+2343678us) service_pool.cc:224] Handling call
0709 13:05:18.570916 (+   533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.571846 (+   930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.572120 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.586733  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 795) took 2387 ms. Trace:
I20260709 13:05:18.658710  1641 raft_consensus.cc:1275] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:05:18.658455  1714 rpcz_store.cc:276] 0709 13:05:16.199759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.566900 (+2367141us) service_pool.cc:224] Handling call
0709 13:05:18.585999 (+ 19099us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.586313 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:18.586340 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:18.586472 (+   132us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:05:18.586513 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.586561 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.586669 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.640360  2313 leader_election.cc:403] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:05:18.653800  1720 rpcz_store.cc:276] 0709 13:05:16.050899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.466385 (+2415486us) service_pool.cc:224] Handling call
0709 13:05:18.510294 (+ 43909us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.563317 (+ 53023us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:18.563353 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:18.652468 (+ 89115us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:18.652514 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.652586 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.653269 (+   683us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.573616  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 801) took 2325 ms. Trace:
I20260709 13:05:18.638046  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:18.607235  1716 raft_consensus.cc:1240] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:18.664853  1718 rpcz_store.cc:276] 0709 13:05:16.248306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.571455 (+2323149us) service_pool.cc:224] Handling call
0709 13:05:18.571996 (+   541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.573258 (+  1262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.573497 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.591296  8798 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:18.663528  1639 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:05:18.677783  8798 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:18.678395  8595 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:18.679261  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 88977.7us
I20260709 13:05:18.678468  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:18.605955  2312 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:18.680847  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 74928.6us
I20260709 13:05:18.627454  8888 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:18.704619  1537 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:18.655376  8863 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:18.690804  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2953) took 2677 ms. Trace:
I20260709 13:05:18.708199  1716 rpcz_store.cc:276] 0709 13:05:15.999741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.606531 (+2606790us) service_pool.cc:224] Handling call
0709 13:05:18.607019 (+   488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.676112 (+ 69093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.676622 (+   510us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.708046  8863 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:18.679334  8595 raft_consensus.cc:3060] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:18.714046  8831 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:18.715358  1719 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 20.
I20260709 13:05:18.710413  1716 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:18.716858  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3375) took 2443 ms. Trace:
I20260709 13:05:18.717319  1716 rpcz_store.cc:276] 0709 13:05:16.273862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.709631 (+2435769us) service_pool.cc:224] Handling call
0709 13:05:18.710238 (+   607us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.716159 (+  5921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.716755 (+   596us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.721123  1716 raft_consensus.cc:1275] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:05:18.722219  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2962) took 1511 ms. Trace:
I20260709 13:05:18.722687  1716 rpcz_store.cc:276] 0709 13:05:17.210519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.718844 (+1508325us) service_pool.cc:224] Handling call
0709 13:05:18.719493 (+   649us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.721886 (+  2393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.722128 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.726146  1644 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:05:18.734338  8863 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:18.736477  8595 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for earlier term 15. Current term is 16.
I20260709 13:05:18.607888  1462 catalog_manager.cc:5697] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 10 to 13, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 13 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:18.757853  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:35480 (request call id 34) took 1576 ms. Trace:
I20260709 13:05:18.758414  1719 rpcz_store.cc:276] 0709 13:05:17.181930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.716784 (+1534854us) service_pool.cc:224] Handling call
0709 13:05:18.757717 (+ 40933us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.624423  8282 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:18.762565  8371 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:18.763513  8282 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:18.764348  8282 consensus_queue.cc:1059] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:05:18.742046  2312 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:18.747192  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2963) took 1536 ms. Trace:
W20260709 13:05:18.747759  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49023.7us
W20260709 13:05:18.766618  2312 consensus_peers.cc:597] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:18.767552  2312 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:18.767926  8371 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:18.748452  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 45020.4us
W20260709 13:05:18.769558  2312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:18.755573  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.772924  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:18.770473  1462 catalog_manager.cc:5697] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 11 to 13, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 13 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:18.775369  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:18.775902  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.776566  8831 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.777210  8831 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:18.769250  8798 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:18.782104  1716 rpcz_store.cc:276] 0709 13:05:17.210806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.745945 (+1535139us) service_pool.cc:224] Handling call
0709 13:05:18.746377 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.746740 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:18.746765 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:18.746856 (+    91us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:18.746915 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.746987 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.747130 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.785413  1648 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
W20260709 13:05:18.786237  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 208921us
I20260709 13:05:18.791643  1717 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:18.793386  1721 raft_consensus.cc:1240] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:18.798873  1713 consensus_meta.cc:287] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.514s	user 0.011s	sys 0.004s
I20260709 13:05:18.799399  1713 raft_consensus.cc:2468] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
W20260709 13:05:18.802249  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 226123us
I20260709 13:05:18.806095  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3369) took 2761 ms. Trace:
I20260709 13:05:18.806653  1721 rpcz_store.cc:276] 0709 13:05:16.038465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.367043 (+2328578us) service_pool.cc:224] Handling call
0709 13:05:18.793112 (+426069us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.794076 (+   964us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.799670 (+  5594us) spinlock_profiling.cc:243] Waited 418 ms on lock 0x7b5400247050. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e848ad 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:18.799828 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":961202944}
I20260709 13:05:18.797497  8863 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:18.819905  2312 leader_election.cc:336] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:18.802323  8912 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:18.834196  8736 consensus_queue.cc:1059] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:05:18.837893  1538 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:18.819878  8828 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:18.843209  1721 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:18.862494  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3386) took 1067 ms. Trace:
I20260709 13:05:18.862784  1721 rpcz_store.cc:276] 0709 13:05:17.794815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.825862 (+1031047us) service_pool.cc:224] Handling call
0709 13:05:18.842970 (+ 17108us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.862253 (+ 19283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.862433 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.832305  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:18.865252  1720 raft_consensus.cc:1275] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:05:18.866148  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 802) took 2614 ms. Trace:
I20260709 13:05:18.866568  1720 rpcz_store.cc:276] 0709 13:05:16.252313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.789195 (+2536882us) service_pool.cc:224] Handling call
0709 13:05:18.830148 (+ 40953us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.865774 (+ 35626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.866072 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:18.873219  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 63412.3us
W20260709 13:05:18.861125  8890 consensus_meta.cc:287] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.555s	user 0.009s	sys 0.004s
I20260709 13:05:18.837399  8798 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.856069  8736 raft_consensus.cc:3055] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:18.859798  8370 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:18.860023  8828 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:18.900611  1718 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:18.904608  1538 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:18.834841  8863 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:18.905573  8831 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:18.907012  8751 raft_consensus.cc:493] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:18.907521  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.912014  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 812) took 1462 ms. Trace:
I20260709 13:05:18.907660  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3385) took 1114 ms. Trace:
I20260709 13:05:18.923233  1718 rpcz_store.cc:276] 0709 13:05:17.449815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.838853 (+1389038us) service_pool.cc:224] Handling call
0709 13:05:18.888915 (+ 50062us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.905239 (+ 16324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.911917 (+  6678us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.870309  8888 consensus_queue.cc:1048] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:05:18.903293  8828 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:18.897971  8370 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.910204  8899 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:05:18.918910  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b82d3b49001741eda872554a7ecb2db3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:18.920423  8941 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:18.894047  8736 raft_consensus.cc:740] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:18.923853  1714 rpcz_store.cc:276] 0709 13:05:17.794182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.820976 (+1026794us) service_pool.cc:224] Handling call
0709 13:05:18.838698 (+ 17722us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.904022 (+ 65324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:18.904057 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:18.907319 (+  3262us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:18.907371 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.907438 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.907590 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.948515  1612 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:18.894414  8798 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:18.953203  1720 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:18.960057  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34078 (request call id 813) took 1062 ms. Trace:
I20260709 13:05:18.960636  1720 rpcz_store.cc:276] 0709 13:05:17.898142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.889238 (+991096us) service_pool.cc:224] Handling call
0709 13:05:18.922983 (+ 33745us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.959734 (+ 36751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.959993 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.891518  8890 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:18.966117  2340 raft_consensus.cc:2468] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:05:18.946202  8370 raft_consensus.cc:3060] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:18.924183  8942 raft_consensus.cc:3055] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:18.977857  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3374) took 2926 ms. Trace:
W20260709 13:05:18.972697  1456 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 54049.3us
I20260709 13:05:18.946318  8899 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.979262  8942 raft_consensus.cc:740] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:18.983302  8853 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.982864  8890 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:18.947431  8941 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:18.988392  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 283889us
I20260709 13:05:18.992141  1714 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:18.976162  8943 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.068s	user 0.001s	sys 0.001s
W20260709 13:05:19.007552  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 142285us
I20260709 13:05:18.992841  8637 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:18.991485  8899 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:18.935168  8831 consensus_queue.cc:1059] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 13:05:19.050382  1538 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:19.053735  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 180700us
I20260709 13:05:18.993251  8751 raft_consensus.cc:515] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:19.078446  1714 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:19.120785  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3372) took 3061 ms. Trace:
I20260709 13:05:19.129268  1686 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:18.989364  1722 rpcz_store.cc:276] 0709 13:05:16.051740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.553999 (+2502259us) service_pool.cc:224] Handling call
0709 13:05:18.589000 (+ 35001us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.589364 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:18.861626 (+272262us) log.cc:844] Serialized 14 byte log entry
0709 13:05:18.977269 (+115643us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:18.977354 (+    85us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:18.977437 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.977543 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.977770 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10679,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":104,"thread_start_us":332148,"threads_started":2}
I20260709 13:05:19.047873  8941 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:19.062070  1611 consensus_peers.cc:597] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:19.094564  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:19.186944  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 57734.9us
I20260709 13:05:19.128054  8912 raft_consensus.cc:3055] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:05:19.189972  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 61235.2us
W20260709 13:05:19.193359  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 135222us
I20260709 13:05:19.180990  1715 rpcz_store.cc:276] 0709 13:05:16.051291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.505322 (+2454031us) service_pool.cc:224] Handling call
0709 13:05:18.551797 (+ 46475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.619886 (+ 68089us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:18.869091 (+249205us) log.cc:844] Serialized 14 byte log entry
0709 13:05:19.109838 (+240747us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:19.110103 (+   265us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:19.110191 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.111230 (+  1039us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.112686 (+  1456us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9197,"prepare.queue_time_us":489168,"raft.queue_time_us":135394,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":21560,"spinlock_wait_cycles":7503616,"thread_start_us":413461,"threads_started":3}
W20260709 13:05:19.073303  2314 leader_election.cc:336] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.594s (SENT)
W20260709 13:05:19.122869  8951 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
W20260709 13:05:19.079156  8889 consensus_meta.cc:287] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: Time spent flushing consensus metadata: real 0.630s	user 0.008s	sys 0.007s
I20260709 13:05:19.056561  8637 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:19.209496  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 221569us
I20260709 13:05:19.122942  8863 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:19.155076  8958 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.125799  8946 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:19.196460  8912 raft_consensus.cc:740] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:19.236656  8946 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.237164  8946 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:19.215273  1611 leader_election.cc:336] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:19.238752  1611 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:19.241060  1611 consensus_peers.cc:597] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:19.230098  1612 leader_election.cc:400] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 18. Current term is 19.
I20260709 13:05:19.242326  1612 leader_election.cc:403] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:05:19.241879  8958 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:19.244330  1612 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:19.235435  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2977) took 1964 ms. Trace:
I20260709 13:05:19.247022  1717 rpcz_store.cc:276] 0709 13:05:17.271298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.794848 (+1523550us) service_pool.cc:224] Handling call
0709 13:05:18.828198 (+ 33350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:18.858670 (+ 30472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:18.858706 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:18.886522 (+ 27816us) log.cc:844] Serialized 26 byte log entry
0709 13:05:18.992452 (+105930us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:18.992756 (+   304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.992904 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:19.235105 (+242201us) raft_consensus.cc:1685] finished
0709 13:05:19.235136 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.235345 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":82109,"threads_started":1,"wal-append.queue_time_us":335000}
I20260709 13:05:19.247892  8958 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:19.248745  8371 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) with higher term. Message: Invalid argument: T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 18. Current term is 19.
I20260709 13:05:19.249436  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:19.237385  8912 consensus_queue.cc:260] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:19.250905  8912 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:19.249511  8371 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:19.252033  8371 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.252383  8371 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:19.074246  8942 consensus_queue.cc:260] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:19.255728  8756 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:05:19.256186  8756 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 25 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:19.256578  8756 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:05:19.257226  8942 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:19.260746  8371 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:18.997454  8481 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.205s
I20260709 13:05:19.261909  1612 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:19.262223  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 167848us
W20260709 13:05:19.264070  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 141899us
W20260709 13:05:19.262405  1611 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:19.047516  8853 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:19.268577  1611 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:19.183298  8831 consensus_queue.cc:1059] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:05:19.211755  8637 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:19.276685  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.077893  8736 consensus_queue.cc:260] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:19.290303  1687 leader_election.cc:400] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 18. Current term is 19.
W20260709 13:05:19.351253  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 71366.5us
I20260709 13:05:19.352398  1687 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:19.272948  8958 raft_consensus.cc:2804] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:19.299623  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2974) took 2048 ms. Trace:
I20260709 13:05:19.302796  8751 leader_election.cc:290] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:19.265017  8946 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:19.356902  8751 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:19.354200  8958 raft_consensus.cc:697] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:19.358664  8946 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:19.358880  8798 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:19.313508  8637 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:19.315668  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:19.359867  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.359134  8958 consensus_queue.cc:237] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:19.322170  8831 consensus_queue.cc:1243] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:05:19.330387  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 419006us
I20260709 13:05:19.361094  8637 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:19.332959  8853 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:19.336644  8667 consensus_meta.cc:287] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.711s	user 0.009s	sys 0.003s
W20260709 13:05:19.362593  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.362423  8667 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:19.345867  8826 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:19.364112  8826 raft_consensus.cc:697] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:05:19.348681  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 58340.5us
I20260709 13:05:19.354044  8863 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:05:19.370280  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 79959.5us
I20260709 13:05:19.355501  1719 rpcz_store.cc:276] 0709 13:05:17.251666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.759621 (+1507955us) service_pool.cc:224] Handling call
0709 13:05:18.760098 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:18.800819 (+ 40721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:18.800859 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:18.818498 (+ 17639us) log.cc:844] Serialized 26 byte log entry
0709 13:05:18.948091 (+129593us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:18.948142 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.948247 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:19.299237 (+350990us) raft_consensus.cc:1685] finished
0709 13:05:19.299272 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.299532 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":87027,"threads_started":1,"wal-append.queue_time_us":359624}
W20260709 13:05:19.366547  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 75808.5us
I20260709 13:05:19.310925  8899 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:19.377516  1537 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:05:19.380885  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.278266  8950 consensus_queue.cc:1059] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.111s
W20260709 13:05:19.384812  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.365268  8826 consensus_queue.cc:237] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:19.370040  8863 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:19.392364  1612 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:19.392979  8863 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:19.406355  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:19.400960  8888 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:05:19.407689  8370 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:19.391934  8853 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:19.410283  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 2979) took 2123 ms. Trace:
I20260709 13:05:19.394204  8889 consensus_queue.cc:1048] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 13:05:19.412961  8942 raft_consensus.cc:2804] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:19.397763  8658 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:05:19.418393  1713 rpcz_store.cc:276] 0709 13:05:17.287161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.800287 (+1513126us) service_pool.cc:224] Handling call
0709 13:05:18.819115 (+ 18828us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:18.895930 (+ 76815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:18.895966 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:18.925594 (+ 29628us) log.cc:844] Serialized 26 byte log entry
0709 13:05:19.390048 (+464454us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:19.390118 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.390199 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:19.409844 (+ 19645us) raft_consensus.cc:1685] finished
0709 13:05:19.409869 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.410078 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1124,"raft.queue_time_us":27308,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":31758,"thread_start_us":403410,"threads_started":1,"wal-append.queue_time_us":455954}
W20260709 13:05:19.422760  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 181114us
I20260709 13:05:19.400089  8736 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:19.426142  1611 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:19.402374  8637 raft_consensus.cc:2804] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:19.445142  8899 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:19.446400  8371 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:19.447707  8756 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:19.429554  8853 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:19.450124  1612 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:19.450493  8637 raft_consensus.cc:697] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:19.433399  8828 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:19.450986  8961 raft_consensus.cc:3055] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:19.451469  8282 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.473863  8961 raft_consensus.cc:740] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:19.474615  8282 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:19.475943  8798 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.475729  8961 consensus_queue.cc:260] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:19.476727  8798 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:19.455873  8751 consensus_queue.cc:1048] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:05:19.452543  8667 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:19.395314  8963 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:05:19.454368  8370 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:19.433885  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:19.478883  1612 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:19.456113  1611 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:19.427935  8942 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:19.469946  8828 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:19.471477  8637 consensus_queue.cc:237] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:19.500618  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 83470.5us
I20260709 13:05:19.502833  1611 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:19.501029  8667 raft_consensus.cc:2804] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:19.526268  8667 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:19.495498  8946 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.527323  8667 raft_consensus.cc:3060] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:19.527719  8946 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:19.457949  8912 raft_consensus.cc:3055] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:19.505481  8942 raft_consensus.cc:3060] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:19.538875  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 118578us
I20260709 13:05:19.477586  8961 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:19.507355  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.482684  8853 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:19.491920  8756 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 26 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:19.492503  8798 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:19.479492  8371 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:05:19.572288  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 64043.9us
I20260709 13:05:19.495923  8899 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.584580  1537 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:19.588662  8863 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:19.543104  8282 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:19.563400  8828 raft_consensus.cc:2804] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:19.542359  8912 raft_consensus.cc:740] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:05:19.598161  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 96266.9us
I20260709 13:05:19.487650  8941 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:19.614347  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 158310us
W20260709 13:05:19.615685  1611 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:19.616344  1611 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:19.602759  8798 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:19.605509  8371 raft_consensus.cc:697] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:19.625856  8756 consensus_queue.cc:1059] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:19.626926  8756 raft_consensus.cc:3055] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:19.627256  8756 raft_consensus.cc:740] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:19.627460  8863 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:19.628057  8756 consensus_queue.cc:260] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:19.630299  8946 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:19.612226  8828 raft_consensus.cc:697] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 19 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:19.614265  8481 raft_consensus.cc:2804] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:19.631569  8481 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:19.632195  8481 raft_consensus.cc:3060] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:19.632087  8828 consensus_queue.cc:237] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:19.604998  8899 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:19.636011  8899 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:19.637297  8371 consensus_queue.cc:237] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:19.630124  8912 consensus_queue.cc:260] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:19.629490  8756 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:19.647276  1537 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:19.643766  8826 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:19.658977  1611 leader_election.cc:336] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:19.659857  1611 leader_election.cc:336] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:19.661727  1611 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:19.678435  8941 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:19.658094  8912 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:19.677629  8942 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:19.690445  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:19.691859  1640 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 20: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:05:19.693136  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.660466  8899 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:19.669117  1537 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:05:19.690030  1611 consensus_peers.cc:597] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:19.705837  1611 leader_election.cc:336] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:19.709345  1611 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:19.721724  8978 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:19.723908  8975 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:19.724663  8975 raft_consensus.cc:697] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:19.725744  8975 consensus_queue.cc:237] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:19.732810  8667 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:19.734458  1648 raft_consensus.cc:1240] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:19.756735  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:19.758116  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:19.758649  1648 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:19.759591  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:19.760950  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:19.761567  1648 raft_consensus.cc:2468] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 15.
I20260709 13:05:19.736809  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:19.763840  1640 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:19.739724  8826 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:19.741147  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:19.741899  8964 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:19.765503  1647 raft_consensus.cc:2410] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 21 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:05:19.742555  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:19.766441  1639 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:19.743860  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:19.745208  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:19.746469  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:19.747570  1646 raft_consensus.cc:1240] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:19.748600  1642 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 13:05:19.749253  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:19.749696  8899 raft_consensus.cc:697] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:05:19.750204  8973 log.cc:927] Time spent T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:05:19.750882  8946 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:05:19.751606  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45822.7us
I20260709 13:05:19.774178  1641 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:19.767809  1642 pending_rounds.cc:85] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 12
I20260709 13:05:19.752481  8282 consensus_queue.cc:1059] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:19.753710  8957 consensus_queue.cc:1059] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:05:19.754810  8974 log.cc:927] Time spent T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.051s	user 0.002s	sys 0.000s
I20260709 13:05:19.752393  8370 consensus_queue.cc:1059] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:19.775444  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:19.778654  1639 raft_consensus.cc:1240] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:19.781670  1642 pending_rounds.cc:107] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 17.13
W20260709 13:05:19.782052  1611 leader_election.cc:336] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:19.782135  1642 raft_consensus.cc:2889] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:19.756750  8667 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 24 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:19.783068  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.783893  1648 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 19: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:05:19.787463  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:19.795827  1640 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
W20260709 13:05:19.799882  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 47196.4us
I20260709 13:05:19.804677  8942 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:19.806342  8899 consensus_queue.cc:237] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:19.811647  1611 leader_election.cc:336] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:19.812180  1611 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:19.813282  8826 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:19.813488  1611 leader_election.cc:336] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:19.825464  1611 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:19.838121  8978 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:19.823194  8941 raft_consensus.cc:3055] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:19.830271  8667 raft_consensus.cc:3055] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:19.857782  8667 raft_consensus.cc:740] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:19.842633  8637 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:19.842686  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:19.858976  1642 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:19.853477  8941 raft_consensus.cc:740] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:19.860492  8941 consensus_queue.cc:260] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:19.849916  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.861575  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:19.861938  8941 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:19.863484  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:19.863996  1645 raft_consensus.cc:1240] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
W20260709 13:05:19.885002  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 71527.6us
I20260709 13:05:19.886121  1641 raft_consensus.cc:2393] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 13: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:05:19.862999  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:19.895404  8912 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.896248  8912 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:19.906103  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:19.907207  2340 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:05:19.909541  8912 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:19.886022  1611 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:19.922830  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:19.923739  1648 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 21: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:05:19.926194  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:19.929454  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 335317us
I20260709 13:05:19.910859  8978 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:05:19.938087  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 52116.9us
W20260709 13:05:19.939162  1611 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:19.939599  1611 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:19.944671  2340 raft_consensus.cc:1275] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:19.934316  8978 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:19.949669  8978 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:19.952863  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:19.958374  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 95509.3us
I20260709 13:05:19.959646  1612 leader_election.cc:400] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15.
I20260709 13:05:19.959954  1612 leader_election.cc:403] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:19.960640  8826 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 14. Current term is 15.
I20260709 13:05:19.962898  8978 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:19.965487  8978 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:19.971475  1612 leader_election.cc:400] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13.
W20260709 13:05:19.972030  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.972004  1612 leader_election.cc:403] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 13:05:19.972757  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:19.983793  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 44653.3us
W20260709 13:05:19.984916  1611 leader_election.cc:336] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:19.988524  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.974859  8371 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:19.974164  8957 consensus_queue.cc:1243] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:05:19.930716  8667 consensus_queue.cc:260] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:19.994303  8831 consensus_queue.cc:1048] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.006s
W20260709 13:05:20.016402  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.004760  1642 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 22.
I20260709 13:05:19.987362  8828 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 12. Current term is 13.
I20260709 13:05:20.018908  1612 leader_election.cc:400] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 18. Current term is 19.
W20260709 13:05:20.024158  1611 leader_election.cc:336] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:20.024724  1611 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:20.040745  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:20.050534  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:20.052757  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:20.053515  1714 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:20.053794  8595 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.054077  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:20.054464  8595 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:20.057173  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:20.036085  8989 log.cc:927] Time spent T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.065s	user 0.000s	sys 0.002s
I20260709 13:05:20.062242  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:20.063834  8756 consensus_queue.cc:1059] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:20.064842  8756 raft_consensus.cc:3055] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:20.065237  8756 raft_consensus.cc:740] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:20.066282  8756 consensus_queue.cc:260] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:20.067589  8756 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:20.064075  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:20.037199  8798 consensus_queue.cc:1059] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:05:20.070225  2340 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:20.063359  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:20.072291  1713 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 19: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:05:20.076572  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:20.077489  1719 raft_consensus.cc:2393] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 18: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:20.077075  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:20.078163  1713 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:20.079443  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:20.082594  8667 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:20.085544  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:20.086241  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:20.086583  2349 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 19: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:05:20.088236  2343 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:05:20.094700  2340 raft_consensus.cc:2468] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:05:20.096350  2346 raft_consensus.cc:1240] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:20.077181  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:20.035821  8282 raft_consensus.cc:3055] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:20.099228  8282 raft_consensus.cc:740] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:20.100315  8282 consensus_queue.cc:260] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:20.104184  8371 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:20.104786  8481 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:20.105756  8282 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:20.108131  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.063467  1714 raft_consensus.cc:2410] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
W20260709 13:05:20.124993  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:20.182461  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:20.185572  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:20.187307  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 163148us
W20260709 13:05:20.188146  1611 leader_election.cc:336] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:20.188612  1611 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:20.192437  8751 raft_consensus.cc:3055] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:20.192939  8751 raft_consensus.cc:740] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:20.194265  1719 raft_consensus.cc:1240] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:20.194728  8637 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:05:20.196095  1611 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:20.198027  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:20.198710  1715 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:20.198887  1714 raft_consensus.cc:1240] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: []
W20260709 13:05:20.199718  1611 leader_election.cc:336] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:20.199968  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:20.201272  1722 raft_consensus.cc:2410] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:05:20.202065  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:05:20.203272  1611 leader_election.cc:336] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:20.203568  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:20.204504  1718 raft_consensus.cc:2393] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:20.204244  1611 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:20.205128  8826 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:20.205665  8964 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:20.206036  8751 consensus_queue.cc:260] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:20.209414  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:20.210536  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:20.211494  1714 raft_consensus.cc:2393] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 18: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:20.206908  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:20.207650  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:20.216687  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:20.217378  2340 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:20.224164  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:20.228190  1716 raft_consensus.cc:1275] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:05:20.229220  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:20.230118  1720 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 19: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:20.224095  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:20.233035  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:20.233928  2349 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:05:20.209393  8751 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:20.239058  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:20.240185  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:05:20.245283  1612 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.247009  1642 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:20.251626  2340 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 22.
I20260709 13:05:20.270234  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:20.273629  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:20.274200  1721 raft_consensus.cc:1240] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:20.275131  8481 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:20.275650  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:20.275695  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:20.277491  1715 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 21.
I20260709 13:05:20.275529  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:20.278563  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.279991  2343 raft_consensus.cc:2368] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 21. Current term is 22.
I20260709 13:05:20.281471  1644 raft_consensus.cc:2393] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 22: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:20.276080  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:20.282737  1716 raft_consensus.cc:2410] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:05:20.283403  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:20.276868  1639 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 22: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:20.235922  8831 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.284678  1716 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 13:05:20.286303  1716 pending_rounds.cc:85] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 7
I20260709 13:05:20.281420  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:20.287292  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:20.287591  1715 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:20.286216  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:20.288908  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:20.289336  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.289987  1645 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 18.
W20260709 13:05:20.274374  8992 log.cc:927] Time spent T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:05:20.291917  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:20.295223  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.286720  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0d191c5dc46468a777fba0a666b32" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:20.287890  2346 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:20.286728  1716 pending_rounds.cc:107] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 13:05:20.297438  1716 raft_consensus.cc:2889] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:20.293757  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:20.298066  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:20.276857  8961 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:20.297293  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:20.298847  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:20.301156  1644 raft_consensus.cc:1275] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:20.310508  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:05:20.310526  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 100951us
I20260709 13:05:20.298642  1641 raft_consensus.cc:2376] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19. Re-sending same reply.
I20260709 13:05:20.310165  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:20.311220  1645 raft_consensus.cc:2376] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 13. Re-sending same reply.
I20260709 13:05:20.315291  2349 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 22: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:05:20.316308  2348 raft_consensus.cc:2393] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 15: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:05:20.316351  2347 raft_consensus.cc:2393] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 13: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:05:20.316895  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:05:20.317595  2314 leader_election.cc:336] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.873s (SENT)
I20260709 13:05:20.291965  8975 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.277990  8963 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:05:20.298847  8946 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.301014  8941 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.327591  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.331760  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:20.332484  1719 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19.
I20260709 13:05:20.333766  2313 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:20.329521  8946 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:20.316027  8961 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:20.338752  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:20.338973  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:20.339418  8941 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:20.327884  8993 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.291539  8831 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:20.315833  8370 consensus_queue.cc:1059] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:05:20.326368  8957 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:20.340479  8921 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:05:20.361996  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 47064.9us
I20260709 13:05:20.355953  1644 raft_consensus.cc:2376] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 15. Re-sending same reply.
W20260709 13:05:20.363421  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.364151  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.365003  1639 raft_consensus.cc:2393] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 16: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:20.339337  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:20.369046  8957 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:20.371634  8957 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:20.373998  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:20.374203  1685 leader_election.cc:336] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.602s (SENT)
I20260709 13:05:20.336223  8595 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:20.391186  1642 raft_consensus.cc:2410] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:05:20.397349  8942 raft_consensus.cc:3055] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:20.398041  8942 raft_consensus.cc:740] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:20.398954  8942 consensus_queue.cc:260] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:20.400570  8942 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:20.386094  1645 raft_consensus.cc:2393] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:20.361967  8993 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:20.403177  8993 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:20.380483  8961 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:20.404968  8961 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:20.405516  8961 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:20.408587  8961 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:20.410392  1685 leader_election.cc:336] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.640s (SENT)
W20260709 13:05:20.413076  8995 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 13:05:20.325716  8975 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:20.415766  8975 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:20.417837  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 51981.7us
W20260709 13:05:20.418742  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.337235  8899 consensus_queue.cc:1059] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 13:05:20.427893  1645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0"
dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:47496
I20260709 13:05:20.428980  1645 raft_consensus.cc:606] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Received request to transfer leadership
I20260709 13:05:20.429387  1645 raft_consensus.cc:614] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:20.430192  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:20.430500  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.431080  1641 raft_consensus.cc:2393] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 20: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:20.430991  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.432035  1639 raft_consensus.cc:2393] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 15: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:05:20.432212  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.433105  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:20.433574  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.433892  1644 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:20.434024  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:20.429240  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:20.434898  1643 raft_consensus.cc:2393] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:20.433076  1648 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:05:20.438721  2314 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.836s (SENT)
I20260709 13:05:20.439436  2314 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:20.365785  8921 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.441876  8921 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:20.447345  2345 raft_consensus.cc:1240] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:20.340482  8946 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:20.458513  1685 leader_election.cc:336] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.688s (SENT)
I20260709 13:05:20.459043  1685 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:20.453227  1644 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:20.464427  1612 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:20.454690  2314 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:20.438874  8941 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:20.473006  2314 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:20.475183  1612 leader_election.cc:400] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 21. Current term is 22.
I20260709 13:05:20.475519  1612 leader_election.cc:403] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:05:20.466367  8595 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:20.470885  8961 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:20.483032  2314 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:20.484257  8751 raft_consensus.cc:2804] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:20.485054  8751 raft_consensus.cc:697] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:20.478091  9005 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:20.490335  8941 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.491135  9005 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.491652  8963 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:20.492148  8595 consensus_queue.cc:1048] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 13:05:20.493678  9005 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:20.494812  1687 leader_election.cc:336] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 2.228s (SENT)
I20260709 13:05:20.495049  8595 raft_consensus.cc:2804] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:20.495343  1687 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:20.495572  8595 raft_consensus.cc:697] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:20.496601  8595 consensus_queue.cc:237] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:20.497246  8961 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:05:20.505798  1611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.511598  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.511871  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.513192  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.513737  1641 raft_consensus.cc:1240] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 19. Current term is 20. Ops: [17.11-19.12]
I20260709 13:05:20.513599  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.514434  1639 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:05:20.515018  1647 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:05:20.510113  1612 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:20.518741  1648 raft_consensus.cc:2393] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 18: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:20.519942  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.520797  1647 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:05:20.525166  1644 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:20.526391  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.542167  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.530471  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.544416  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.542918  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.546484  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.546897  1648 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:05:20.469676  8996 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:05:20.546768  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.547312  1640 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 21.
I20260709 13:05:20.530236  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:20.548220  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:20.543358  1646 raft_consensus.cc:2468] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:05:20.549088  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:20.515796  8941 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:20.551296  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:20.552043  1643 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:20.498446  8961 consensus_queue.cc:1059] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
W20260709 13:05:20.567768  1685 leader_election.cc:336] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.797s (SENT)
I20260709 13:05:20.562326  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:20.566021  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:20.568632  1685 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:20.553889  1648 raft_consensus.cc:2376] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18. Re-sending same reply.
I20260709 13:05:20.569110  1644 raft_consensus.cc:2376] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17. Re-sending same reply.
I20260709 13:05:20.570034  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.570967  1648 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:20.571794  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:20.572522  1648 raft_consensus.cc:2393] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 22: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:20.476269  8282 consensus_queue.cc:1059] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 18.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:05:20.542032  8826 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:20.575984  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.577193  1640 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 21: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:20.573473  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.525544  8371 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:20.574982  1639 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 21: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:05:20.579005  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 69993.1us
W20260709 13:05:20.583011  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 100079us
I20260709 13:05:20.510417  8736 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:05:20.587339  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.586028  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:20.525018  8964 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b for earlier term 21. Current term is 22.
I20260709 13:05:20.595498  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.599073  1644 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:05:20.542037  8370 consensus_queue.cc:1059] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:05:20.584739  8826 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.610026  8826 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:20.610404  8370 raft_consensus.cc:3055] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:20.610787  1714 raft_consensus.cc:1275] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:05:20.550457  8977 consensus_queue.cc:1059] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.064s
I20260709 13:05:20.611785  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.610069  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:20.612468  2314 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:05:20.615857  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.557511  8941 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:20.626392  1643 raft_consensus.cc:2393] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 22: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:20.611245  8370 raft_consensus.cc:740] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:20.595459  8751 consensus_queue.cc:237] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:20.633888  8370 consensus_queue.cc:260] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:20.635083  8828 raft_consensus.cc:3055] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:20.635464  8370 raft_consensus.cc:3060] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:20.635565  8828 raft_consensus.cc:740] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:20.629060  1538 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:20.630129  2314 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:20.630656  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:20.639627  1611 leader_election.cc:336] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:20.639907  1647 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:05:20.640631  1611 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:20.641222  1612 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:20.644403  1720 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:05:20.644860  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:20.646059  1640 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 16: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:20.616736  9003 raft_consensus.cc:3055] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:20.647842  1538 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:20.600731  9010 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:20.651774  9013 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:20.655997  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:20.666846  1612 leader_election.cc:304] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:20.590471  8371 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:20.676291  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 108569us
I20260709 13:05:20.673830  1687 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:20.678474  1687 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:20.674193  9018 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.611552  8912 consensus_queue.cc:1048] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:05:20.680672  1718 raft_consensus.cc:1275] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:05:20.595253  9002 consensus_queue.cc:1048] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:05:20.650880  8941 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:05:20.684176  1685 leader_election.cc:336] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.799s (SENT)
I20260709 13:05:20.674474  8978 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:20.674728  8798 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:05:20.692580  1685 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:20.692607  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 53021us
W20260709 13:05:20.692138  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 62082.4us
I20260709 13:05:20.653475  9003 raft_consensus.cc:740] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:05:20.695739  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 48018us
I20260709 13:05:20.676712  8977 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:20.698449  2314 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:20.662109  8957 consensus_queue.cc:1059] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:05:20.688191  9010 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:20.680431  8637 consensus_queue.cc:1048] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:05:20.692281  8941 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.691231  8736 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.694445  8961 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:20.733690  8961 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.734338  8961 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:20.738749  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 54585.6us
W20260709 13:05:20.740334  1685 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.813s (SENT)
W20260709 13:05:20.741489  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:20.742022  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:20.742321  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.742383  8997 raft_consensus.cc:2804] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:05:20.742734  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.742626  9011 raft_consensus.cc:3055] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:20.743451  9011 raft_consensus.cc:740] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:05:20.743661  9009 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:05:20.743556  9005 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:20.744514  9011 consensus_queue.cc:260] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:20.745837  9011 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:20.754987  9011 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:20.757750  8941 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:20.758148  8736 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:20.687279  8756 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:20.762598  1685 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.841s (SENT)
W20260709 13:05:20.764371  1685 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.880s (SENT)
I20260709 13:05:20.766242  9005 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:20.694808  8831 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:20.771256  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.762763  8658 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:20.776648  1716 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 22.
I20260709 13:05:20.761286  8736 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
W20260709 13:05:20.798763  1685 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:20.799394  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 132626us
I20260709 13:05:20.695307  9013 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:05:20.772591  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 442785us
I20260709 13:05:20.809973  1612 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:20.652820  8828 consensus_queue.cc:260] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:20.819175  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 140757us
I20260709 13:05:20.826282  1687 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:20.805871  8978 raft_consensus.cc:2749] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:20.828715  2312 leader_election.cc:336] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:20.819618  8637 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:20.830010  1538 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:20.810305  8997 raft_consensus.cc:697] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 15 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:20.817859  9013 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:20.839241  1687 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:20.696465  9018 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:20.842783  1538 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:20.810102  8946 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:05:20.838866  8637 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.862053  8637 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:20.839874  8912 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:05:20.863159  1611 consensus_peers.cc:597] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:20.863755  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 144257us
W20260709 13:05:20.864420  1611 leader_election.cc:336] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:20.864830  1611 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:20.852125  8921 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:20.853189  9021 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:20.873143  1714 raft_consensus.cc:1275] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:05:20.845587  1687 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:20.827162  8957 consensus_queue.cc:1059] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:05:20.840545  8756 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:20.840996  8798 consensus_queue.cc:1048] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:05:20.881424  1641 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 22.
I20260709 13:05:20.842330  9013 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:20.857815  8899 raft_consensus.cc:3055] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:05:20.901191  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 58422.3us
W20260709 13:05:20.950543  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 105064us
I20260709 13:05:20.951810  1687 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:20.952119  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 87752.6us
I20260709 13:05:20.863221  8912 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:20.955283  8912 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:20.834748  9003 consensus_queue.cc:260] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:20.958802  9003 raft_consensus.cc:3060] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:20.953091  1687 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:20.875099  8751 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:20.977967  1687 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:20.970198  9021 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.992990  1687 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:21.002271  8831 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.006264  9021 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.007296  8831 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.006855  9014 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:20.994607  8736 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:20.987320  9020 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:21.008929  9021 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:20.993391  9011 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:05:20.984896  1611 leader_election.cc:336] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:20.998601  1685 leader_election.cc:336] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.979s (SENT)
I20260709 13:05:20.946995  8282 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:05:21.006302  2312 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:21.041965  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 68281.8us
W20260709 13:05:21.043970  2312 leader_election.cc:336] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:20.950770  8899 raft_consensus.cc:740] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:21.010082  9020 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.997975  8997 consensus_queue.cc:237] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:21.009469  8736 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.069263  1538 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:20.999500  8942 consensus_meta.cc:287] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.598s	user 0.012s	sys 0.002s
I20260709 13:05:21.044992  1687 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:21.073712  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:21.075929  1611 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:21.071122  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:21.087991  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:20.994750  9002 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:21.043826  8957 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.992296  8986 raft_consensus.cc:3055] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:21.012236  8371 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:21.010490  8828 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:21.092216  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:21.113044  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:21.118615  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 133746us
I20260709 13:05:21.071864  8736 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:21.108958  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:21.123692  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 125126us
W20260709 13:05:21.124528  1685 leader_election.cc:336] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.982s (SENT)
I20260709 13:05:21.070457  9018 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:21.124984  1685 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:21.052951  9011 raft_consensus.cc:697] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:20.991679  8912 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.106613  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:21.131352  1715 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:21.132513  1718 raft_consensus.cc:2410] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:05:20.994782  8826 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:21.106438  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 294081us
W20260709 13:05:21.119752  1611 leader_election.cc:336] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.134375  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:21.128091  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 59008.7us
I20260709 13:05:21.135061  1717 raft_consensus.cc:2468] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:21.071942  9020 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:21.123210  1721 raft_consensus.cc:1240] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:21.086187  9021 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:21.059151  8941 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.135522  1538 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:21.116227  8986 raft_consensus.cc:740] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:21.127753  8282 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:21.145501  1612 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:21.155323  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:21.151476  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.152215  1720 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:21.152426  1716 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:05:21.156705  9011 consensus_queue.cc:237] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.135809  8751 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:21.150193  8370 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.162490  8751 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:21.162220  8370 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.163097  8751 raft_consensus.cc:697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 22 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:05:21.164031  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 44309us
I20260709 13:05:21.164690  8370 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:21.143921  8798 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:05:21.165470  8798 consensus_queue.cc:1243] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:05:21.164942  1611 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:21.166672  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.166679  1611 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:21.167150  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.136689  9018 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.169512  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:21.125545  8957 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:21.173199  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 48711.2us
I20260709 13:05:21.173470  8831 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:21.174336  8831 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:21.146131  8942 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:21.176304  8941 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:21.176667  1685 leader_election.cc:336] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.923s (SENT)
W20260709 13:05:21.177726  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.179029  8826 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:21.180155  9028 consensus_queue.cc:1048] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:21.142464  8899 consensus_queue.cc:260] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:21.171319  2312 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 170 similar messages]
I20260709 13:05:21.182654  8899 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:21.131863  9013 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.187089  8957 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:21.187552  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.188125  9013 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:21.144376  9021 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.188971  9021 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:21.164047  8751 consensus_queue.cc:237] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:21.195199  8912 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:21.196455  8921 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:21.197453  1611 leader_election.cc:336] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.198814  8798 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:21.198544  8961 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.200492  8637 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.201225  8961 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:21.198582  1611 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.202387  1611 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:21.202730  8637 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:21.205837   431 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 1598)
W20260709 13:05:21.206142  1685 leader_election.cc:336] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.021s (SENT)
I20260709 13:05:21.206616  1685 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:21.207690  1611 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:21.207819  1685 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.048s (SENT)
W20260709 13:05:21.208678  1611 leader_election.cc:336] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:21.208913  1685 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.077s (SENT)
I20260709 13:05:21.150933  9019 consensus_queue.cc:1048] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:05:21.209239  1611 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:21.210685  2312 leader_election.cc:336] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.130486  8756 raft_consensus.cc:697] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 22 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:21.206650  8371 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:21.221688  1537 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:21.227291  1685 leader_election.cc:336] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.159s (SENT)
I20260709 13:05:21.228530  1685 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:21.209954  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 118638us
W20260709 13:05:21.242276  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.243464  8282 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:21.247980  8986 consensus_queue.cc:260] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.253564  9026 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:21.258564  9003 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:21.260160  9003 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:05:21.260363  1537 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:21.261579  9003 raft_consensus.cc:3060] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:21.223110  9014 raft_consensus.cc:697] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:21.241504  8798 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:21.227314  8942 raft_consensus.cc:697] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:21.227226  8961 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:05:21.265084  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:21.261612  1537 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:21.279896  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 56241.7us
I20260709 13:05:21.262202  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:21.242717  8912 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:21.284577  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 75914.4us
W20260709 13:05:21.286741  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 131744us
I20260709 13:05:21.218389  9018 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:21.289517  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:21.315696  1685 leader_election.cc:336] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.172s (SENT)
W20260709 13:05:21.268615  9029 log.cc:927] Time spent T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.096s	user 0.002s	sys 0.001s
W20260709 13:05:21.317818  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.319741  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:21.318596  8481 raft_consensus.cc:3060] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:21.321373  9027 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.324092  2314 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:21.315727  1611 consensus_peers.cc:597] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:21.326033  1611 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.326460  1611 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:21.241376  8736 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:21.324708  2312 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:21.270895  8975 raft_consensus.cc:3055] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:21.330034  8370 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:21.334600  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:05:21.341399  1611 leader_election.cc:336] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.343003  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:21.344751  1611 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:21.345986  1611 leader_election.cc:336] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.332412  1714 pending_rounds.cc:85] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 10
I20260709 13:05:21.332358  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:21.338145  1721 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:21.334096  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 43778.8us
I20260709 13:05:21.331015  1537 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:21.334956  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:21.336123  2349 raft_consensus.cc:1275] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:21.349128  1611 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:21.349323  1714 pending_rounds.cc:107] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:05:21.352936  1714 raft_consensus.cc:2889] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:21.353073  1537 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:21.350893  1685 leader_election.cc:336] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.202s (SENT)
I20260709 13:05:21.351473  1720 raft_consensus.cc:2393] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 21: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:05:21.351689  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.354070  1685 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:21.351617  1717 raft_consensus.cc:2393] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 19: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:21.350569  2347 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 19: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
W20260709 13:05:21.361290  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.416580  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:05:21.417088  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.417470  1643 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:05:21.415619  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 69646.3us
I20260709 13:05:21.321920  9027 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:21.418877  1611 leader_election.cc:336] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.420753  1643 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:05:21.422725  1611 leader_election.cc:336] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.423346  1611 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:21.416359  8370 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:21.366259  9011 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:21.424981  8637 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:21.279572  8942 consensus_queue.cc:237] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.342578  9019 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:21.428836  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:21.429767  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 76752.4us
W20260709 13:05:21.433300  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 82427.1us
W20260709 13:05:21.433918  1685 leader_election.cc:336] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.219s (SENT)
I20260709 13:05:21.434055  1537 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:21.337949  8736 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:21.432318  1611 leader_election.cc:336] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.347184  8826 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:21.344113  8986 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:21.347453  9005 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:21.420011  9029 log.cc:927] Time spent T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:05:21.437036  8282 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:21.341917  8751 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:21.438544  1537 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:21.440719  1611 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:21.457581  1721 raft_consensus.cc:2468] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 20.
I20260709 13:05:21.431349  8370 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.464186  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:21.349092  8975 raft_consensus.cc:740] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:21.283421  9020 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.451578  9036 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:21.470184  9027 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:21.471599  9020 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:21.445358  9026 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:21.447356  8978 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:05:21.451754  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.460587  1537 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:21.442277  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:21.465036  1611 leader_election.cc:336] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:21.476529  1687 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Timed out: RequestConsensusVote RPC to 127.0.107.194:44293 timed out after 1.696s (SENT)
I20260709 13:05:21.270771  9014 consensus_queue.cc:237] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.478842  9021 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:21.466636  8942 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:21.466755  1641 raft_consensus.cc:2393] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 20: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:21.330003  8756 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:21.483482  8481 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:21.467329  8370 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.440042  9019 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.468595  9018 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:21.340587  9003 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:05:21.477440  1611 consensus_peers.cc:597] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:21.486518  1537 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:21.484697  8975 consensus_queue.cc:260] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.491017  1641 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:05:21.501088  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:21.493801  8282 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:21.498567  2313 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:05:21.499295  1611 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.465976  8826 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:21.503980  1537 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:21.501263  1685 leader_election.cc:336] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.517616  1685 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:21.516135  1687 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:21.520778  8370 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:21.520807  9018 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:21.521606  8966 raft_consensus.cc:2804] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:21.525185  8966 raft_consensus.cc:697] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:21.526312  8966 consensus_queue.cc:237] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:21.536266  1641 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:05:21.490903  8481 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:21.538905  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:21.537952  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:21.544796  9003 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:21.545501  9003 raft_consensus.cc:697] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 21 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:21.541611  1641 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:05:21.550899  9019 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:21.503153  9026 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:21.516296  8942 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:05:21.554440  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:21.555473  9038 log.cc:927] Time spent T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.087s	user 0.002s	sys 0.000s
I20260709 13:05:21.555610  1647 raft_consensus.cc:1240] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:21.556052  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:21.556777  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:21.556850  1642 raft_consensus.cc:2468] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:05:21.558245  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:21.559343  1642 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 22.
I20260709 13:05:21.560559  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:21.561421  1645 raft_consensus.cc:2393] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:21.546471  9003 consensus_queue.cc:237] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.563484  1640 raft_consensus.cc:2468] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:21.564235  1648 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:05:21.562691  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:21.554471  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 64119.3us
I20260709 13:05:21.569572  1648 raft_consensus.cc:1240] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:21.569797  1537 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:21.572330  1643 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:05:21.573876  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:05:21.574229  2312 consensus_peers.cc:597] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:21.574678  1643 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:21.579146  1537 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:21.577680  2312 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:21.583649  8282 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:21.584465  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:21.539951  9018 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:21.585448  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:21.586867  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:21.540498  8975 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:21.587527  1645 raft_consensus.cc:2410] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 22 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 9.
I20260709 13:05:21.587587  1640 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:05:21.585978  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:21.581279  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:21.586627  8282 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:05:21.589789  1648 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 16.
I20260709 13:05:21.591387  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:21.592309  1648 raft_consensus.cc:3060] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:21.589048  1647 raft_consensus.cc:2468] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
W20260709 13:05:21.594144  1611 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.595057  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:21.592890  1687 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:21.595968  1640 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:05:21.597301  1687 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:21.598426  1643 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 21.
I20260709 13:05:21.595038  1611 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:21.599907  1642 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 20. Current term is 21. Ops: []
W20260709 13:05:21.601048  1611 leader_election.cc:336] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:21.602615  1611 leader_election.cc:336] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.602788  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:21.603116  1611 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:21.603425  1647 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:21.599328  1687 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:21.605396  8282 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:21.608145  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:05:21.608628  1611 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.609164  1640 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:21.608692  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:21.609748  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 108504us
I20260709 13:05:21.610350  1645 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:05:21.609306  1611 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:21.596097  8481 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:21.622462  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:21.605583  8826 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:21.623489  1645 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:21.604235  8736 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:05:21.626075  1611 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.626629  1611 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:21.605315  9018 raft_consensus.cc:697] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 22 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:21.628153  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:21.612171  8961 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:21.610941  9019 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:21.632117  1646 raft_consensus.cc:2393] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:21.601527  8912 consensus_queue.cc:1059] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:05:21.633466  9026 consensus_queue.cc:1059] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:05:21.636605  1685 leader_election.cc:336] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.664443  1685 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:21.631312  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:21.640250  1538 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:21.645573  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:21.641782  9014 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:21.647775  1648 raft_consensus.cc:2468] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 21.
I20260709 13:05:21.650144  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:21.651371  9021 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:21.628635  8736 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.682629  8736 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:21.636022  8961 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.668723  1640 raft_consensus.cc:2468] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 22.
I20260709 13:05:21.628193  9035 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:21.673768  9028 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:21.664692  9018 consensus_queue.cc:237] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.674878  1538 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:21.675585  9036 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:21.669741  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
W20260709 13:05:21.667953  1611 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.697858  2349 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:21.685056  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 53849.1us
I20260709 13:05:21.688062  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:21.699421  1646 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:05:21.687525  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.699805  1537 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:05:21.700712  1611 consensus_peers.cc:597] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:21.679289  9014 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.709662  1538 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:21.701611  1537 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:21.704151  1611 consensus_peers.cc:597] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:21.713177  1611 consensus_peers.cc:597] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:21.705628  1647 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:05:21.637578  8942 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:05:21.715433  1644 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:21.718058  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:21.756210  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 60247.1us
W20260709 13:05:21.758515  1611 consensus_peers.cc:597] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:21.721393  2345 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:05:21.685056  9002 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:21.689406  8961 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:21.719276  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:21.786481  1721 raft_consensus.cc:2393] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 19: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:21.725347  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:21.725842  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:21.762796  1687 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:21.788211  1687 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:21.789081  1643 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 23. Current term is 24. Ops: []
I20260709 13:05:21.769985  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:21.686221  8370 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:21.782369  2347 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:05:21.791745  1722 raft_consensus.cc:2393] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 22: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:21.782671  9033 consensus_queue.cc:1048] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 13:05:21.790653  1687 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:21.799041  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 169167us
W20260709 13:05:21.800160  1685 leader_election.cc:336] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.800642  1685 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:21.719082  9014 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:21.761737  9027 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:21.805193  1646 raft_consensus.cc:1275] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:05:21.812357  1687 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:21.810633  8282 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:21.718698  8899 raft_consensus.cc:2804] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:21.821054  2314 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:21.786248  9036 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.827710  1645 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:05:21.797559  8912 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:21.803315  9011 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:21.756363  8946 consensus_queue.cc:1059] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:05:21.718943  9026 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.100s
I20260709 13:05:21.719422  8941 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:21.809239  9020 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:21.817158  9021 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:21.835058  9019 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.837000  8912 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:21.877501  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 77370.4us
I20260709 13:05:21.849038  9011 raft_consensus.cc:697] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 21 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:21.825519  8975 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:21.823092  8899 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.826434  9005 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:21.887246  9020 raft_consensus.cc:2804] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:21.889791  9020 raft_consensus.cc:697] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 21 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:21.868604  8912 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:21.904331  8756 raft_consensus.cc:3055] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:21.904917  8756 raft_consensus.cc:740] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:21.891321  9020 consensus_queue.cc:237] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.885874  8282 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:21.886158  1685 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.842514  8658 consensus_queue.cc:1059] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.028s
I20260709 13:05:21.834815  9036 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.861136  8798 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.895841  9027 raft_consensus.cc:3055] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:21.871001  9019 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:21.842154  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 132601us
I20260709 13:05:21.858959  9021 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.906678  8756 consensus_queue.cc:260] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.908569  2349 raft_consensus.cc:2468] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:05:21.876839  9030 raft_consensus.cc:3055] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:21.913136  1685 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:21.913304  8282 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.915800  8961 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:21.921875  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 109611us
I20260709 13:05:21.922371  9026 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.971825  1687 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:21.973060  1687 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:21.921916  8986 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.923403  8481 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:21.929895  1717 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:05:21.933923  9004 consensus_queue.cc:1059] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:21.936358  9011 consensus_queue.cc:237] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.946672  9035 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:21.947288  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.892313  8899 raft_consensus.cc:3060] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:21.987396  1714 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:21.967044  9021 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:21.953667  1538 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:21.942305  9045 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
W20260709 13:05:21.966722  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 87095.2us
I20260709 13:05:21.968958  9030 raft_consensus.cc:740] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:05:21.976257  1611 leader_election.cc:336] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:21.983664  1642 raft_consensus.cc:1275] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:05:22.003046  1611 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:21.982393  8921 consensus_queue.cc:1048] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:05:21.986356  2347 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:05:21.952090  9027 raft_consensus.cc:740] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:05:21.951535  2312 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:22.000272  1686 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:22.002029  1685 consensus_peers.cc:597] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:21.970208  8946 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:05:21.949723  8941 consensus_queue.cc:1059] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
W20260709 13:05:22.020282  1685 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.021006  1685 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:22.005841  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:22.007396  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 53833.5us
I20260709 13:05:22.009444  1718 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 13:05:22.009768  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 58348.9us
I20260709 13:05:21.992694  9004 consensus_queue.cc:1243] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer cf2f71592be8465fb18e36e15d0b2f89 log is divergent from this leader: its last log entry 20.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:05:21.986940  9065 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:22.022398  1716 raft_consensus.cc:2410] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:05:21.955330  8798 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:22.026576  9036 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:21.992904  9033 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:22.035254  1611 leader_election.cc:336] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.001834  8756 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:22.043344  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:22.044109  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 43940.2us
I20260709 13:05:22.035766  1611 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:22.047662  1538 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:22.044306  2345 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:05:22.027305  9019 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:22.064157  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:22.065503  1716 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:05:21.979182  9026 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:21.983577  8986 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:22.052502  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:22.075196  1538 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:22.075301  1720 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:22.077889  9019 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:22.004099  9035 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:22.072016  2312 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.084163  1538 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:22.072150  1611 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.085057  1611 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:22.030961  8942 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:22.037107  8975 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:22.077273  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:22.044845  8637 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:22.093712  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 73444.2us
W20260709 13:05:22.093945  1611 consensus_peers.cc:597] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.096975  1611 leader_election.cc:336] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.097680  1611 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:22.099570  1685 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.100148  1685 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:22.073737  9036 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.084179  8370 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:05:22.103523  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 129145us
I20260709 13:05:22.104674  1641 raft_consensus.cc:1275] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:05:22.047171  9033 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:22.050982  9052 consensus_queue.cc:1059] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.062s
I20260709 13:05:22.084050  9068 raft_consensus.cc:2804] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:22.125336  9060 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:22.128114  9059 log.cc:927] Time spent T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.107s	user 0.002s	sys 0.000s
I20260709 13:05:22.082726  8751 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:05:22.112588  9019 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:22.143301  9072 raft_consensus.cc:2804] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:22.143767  9019 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.111492  8370 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.114462  8975 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:22.114178  8978 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:22.110366  9058 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:22.159327  9074 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:05:22.145682  1685 leader_election.cc:336] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.159485  9026 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:22.144191  9019 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:22.090754  9027 consensus_queue.cc:260] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:22.108780  9036 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:22.098140  9030 consensus_queue.cc:260] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:22.151885  8942 raft_consensus.cc:2804] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:22.153110  8370 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:05:22.206321   431 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 1598)
W20260709 13:05:22.182358  1685 leader_election.cc:336] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.160379  8481 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:05:22.233619  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:22.250209  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 67866us
W20260709 13:05:22.254068  1685 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.183990  8961 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:05:22.186934  9035 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:22.183372  9072 raft_consensus.cc:697] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 20 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:05:22.258852  1685 leader_election.cc:336] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.273778  1685 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:22.258116  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 128714us
W20260709 13:05:22.278630  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 140295us
I20260709 13:05:22.246373  9058 raft_consensus.cc:697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:22.185010  9066 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.082s
I20260709 13:05:22.287524  1538 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:22.249606  8942 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.283638  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:22.292109  1641 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:22.241731  8975 consensus_queue.cc:1059] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:05:22.183977  8997 consensus_queue.cc:1048] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 13:05:22.293681  1538 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:22.258522  9027 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:22.297369  1537 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:22.295130  8942 raft_consensus.cc:3060] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:22.303630  9033 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:22.298287  8946 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:22.307101  2347 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:05:22.312677  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:22.299748  1685 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.313912  1685 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:22.308241  8736 consensus_meta.cc:287] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.625s	user 0.005s	sys 0.009s
I20260709 13:05:22.322084  1648 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 23. Current term is 24. Ops: []
I20260709 13:05:22.324651  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:22.325754  9076 consensus_queue.cc:1048] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
W20260709 13:05:22.329844  9071 log.cc:927] Time spent T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:05:22.320477  8941 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:22.332767  8941 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.333238  8941 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:22.338395  8798 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:22.337966  9081 consensus_queue.cc:1059] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 13:05:22.306238  8986 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:22.340793  1721 raft_consensus.cc:1275] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:05:22.342255  1538 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:22.359405  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 59666.3us
W20260709 13:05:22.360883  1685 leader_election.cc:336] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.361483  1685 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:22.326118  9075 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:22.389170  2312 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.392074  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 49979us
I20260709 13:05:22.341318  8736 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:22.428488  1645 raft_consensus.cc:1275] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:05:22.415550  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:22.431358  1639 raft_consensus.cc:2393] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:05:22.411599  9014 consensus_meta.cc:287] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.580s	user 0.010s	sys 0.002s
I20260709 13:05:22.394626  8751 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:22.398475  9081 raft_consensus.cc:3060] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:22.443221  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 375810us
W20260709 13:05:22.454752  9083 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.074s	user 0.002s	sys 0.000s
W20260709 13:05:22.455102  9062 log.cc:927] Time spent T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:05:22.454183  9085 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:22.465133  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:22.468744  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.475927  1722 raft_consensus.cc:1240] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:05:22.475044  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:22.455646  9077 raft_consensus.cc:2804] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:22.483913  1645 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:22.484897  1639 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:05:22.412999  8946 consensus_queue.cc:1059] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:05:22.486425  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:22.501149  1717 raft_consensus.cc:1240] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 21 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:22.506618  1641 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:05:22.507951  1646 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 19. Current term is 20. Ops: [19.9-19.9]
W20260709 13:05:22.515208  8912 consensus_meta.cc:287] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.612s	user 0.012s	sys 0.000s
I20260709 13:05:22.517007  8736 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:22.517920  8946 raft_consensus.cc:3055] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:22.486804  9077 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.525144  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:22.406507  9058 consensus_queue.cc:237] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:22.475008  8798 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.511241  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:22.533475  1645 raft_consensus.cc:2468] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:22.489019  8986 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.470243  9086 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:22.502465  9036 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:22.456823  9014 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:22.602795  8826 consensus_meta.cc:287] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.971s	user 0.013s	sys 0.002s
I20260709 13:05:22.603304  8826 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:22.603852  9014 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:22.604424  1719 raft_consensus.cc:1240] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:05:22.395840  9072 consensus_queue.cc:237] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:22.608881  1716 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:05:22.566437  9087 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.568004  9082 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:22.570138  9093 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:22.570771  9088 log.cc:927] Time spent T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:05:22.572347  8370 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:22.540331  9077 raft_consensus.cc:3060] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:22.541069  8912 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:22.528594  8946 raft_consensus.cc:740] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:22.585093  8282 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:22.589469  9054 log.cc:927] Time spent T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.697s	user 0.003s	sys 0.000s
I20260709 13:05:22.598265  9089 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.600153  8997 consensus_queue.cc:1059] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:05:22.509797  9068 raft_consensus.cc:697] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 22 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:22.470717  8751 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:22.616668  2345 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
W20260709 13:05:22.608769  1685 leader_election.cc:336] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:22.644980  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 87013.2us
W20260709 13:05:22.646044  1685 leader_election.cc:336] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:22.647799  1685 leader_election.cc:336] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.648343  1685 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:22.619944  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:22.610841  1687 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:22.573469  8756 consensus_meta.cc:287] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.503s	user 0.009s	sys 0.000s
I20260709 13:05:22.625535  1646 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:05:22.631242  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:05:22.653352  1685 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.653895  1685 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:22.633894  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:22.632784  8899 consensus_meta.cc:287] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.627s	user 0.014s	sys 0.000s
W20260709 13:05:22.660802  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 285045us
I20260709 13:05:22.556636  8798 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:22.671293  1611 leader_election.cc:336] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.672457  1611 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:22.617817  8986 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:22.624980  9087 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:22.629691  9093 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:22.656416  9082 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:22.684790  1611 leader_election.cc:336] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.677888  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 3026) took 1037 ms. Trace:
I20260709 13:05:22.712103  1713 rpcz_store.cc:276] 0709 13:05:21.640050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.640728 (+   678us) service_pool.cc:224] Handling call
0709 13:05:21.641047 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:21.669509 (+ 28462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:21.669542 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:21.682325 (+ 12783us) log.cc:844] Serialized 26 byte log entry
0709 13:05:21.860472 (+178147us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:21.860542 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.860639 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:22.588428 (+727789us) spinlock_profiling.cc:243] Waited 1.32 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:05:22.677518 (+ 89090us) raft_consensus.cc:1685] finished
0709 13:05:22.677544 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.677820 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6660,"raft.queue_time_us":25726,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":3685,"spinlock_wait_cycles":3033984,"thread_start_us":104674,"threads_started":1,"wal-append.queue_time_us":198026}
W20260709 13:05:22.649528  1519 auto_rebalancer.cc:893] Removing replace marker failed: Replica f5f4b2707a9647e5bb99e065068dea2b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:22.620707  9074 consensus_queue.cc:1048] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
W20260709 13:05:22.688858  1685 consensus_peers.cc:597] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.675722  8282 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:22.715046  1519 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:05:22.683598  8481 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.717765  8282 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:22.713495  9021 consensus_meta.cc:287] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.694s	user 0.015s	sys 0.000s
I20260709 13:05:22.674546  9002 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:05:22.717760  1519 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:05:22.683691  8798 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:22.722491  1685 consensus_peers.cc:597] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:22.693853  9057 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:22.683481  8921 consensus_queue.cc:1048] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 13:05:22.749043  2345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:22.672963  8899 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:22.743593  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 78631.3us
I20260709 13:05:22.673458  8975 consensus_queue.cc:1059] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.059s
I20260709 13:05:22.736986  8282 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:22.741505  1611 leader_election.cc:336] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.757756  2345 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 22.
I20260709 13:05:22.758455  1611 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:22.742246  1685 leader_election.cc:336] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.759274  1685 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:22.760068  9011 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:22.752938  9003 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.018s	sys 0.001s
W20260709 13:05:22.760733  9003 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.018s	sys 0.002s
I20260709 13:05:22.650672  9089 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:22.732429  9033 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.717342  9091 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:05:22.760538  1685 leader_election.cc:336] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.732945  8481 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:22.746732  9066 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.794140  1646 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:05:22.734568  8736 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.035s
W20260709 13:05:22.836928  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 83936.4us
W20260709 13:05:22.778111  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:22.789831  1685 leader_election.cc:336] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.790078  8798 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:22.676990  1464 catalog_manager.cc:5697] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 13 to 14, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 14 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:22.756920  9002 raft_consensus.cc:697] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 20 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:22.775094  8975 consensus_queue.cc:1243] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer cf2f71592be8465fb18e36e15d0b2f89 log is divergent from this leader: its last log entry 20.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:05:22.734745  9021 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:22.819635  9066 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:22.717245  8946 consensus_queue.cc:260] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:22.741544  9068 consensus_queue.cc:237] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:22.822182  8370 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:22.823298  8912 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:22.823518  9087 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:22.825512  9093 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:22.826121  8942 consensus_meta.cc:287] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.000s
I20260709 13:05:22.821424  9086 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:22.827505  8595 raft_consensus.cc:3055] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:05:22.829348  8481 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:22.829972  9004 raft_consensus.cc:3055] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:22.754587  9113 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.840078  1538 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:22.846041  1611 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:22.924767  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 78777us
I20260709 13:05:22.839766  1685 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:22.848150  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 457956us
I20260709 13:05:22.849258  9052 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:22.927712  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 87735us
W20260709 13:05:22.930392  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 140579us
W20260709 13:05:22.931342  1685 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:22.932634  1685 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.929448  1538 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:22.935873  1538 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:22.884193  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 45201.1us
W20260709 13:05:22.865760  8941 consensus_meta.cc:287] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.532s	user 0.009s	sys 0.005s
I20260709 13:05:22.866228  9081 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:22.796564  9033 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:22.955854  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:22.911116  8595 raft_consensus.cc:740] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
W20260709 13:05:22.937534  1685 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:22.941277  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:22.944419  8282 consensus_queue.cc:1059] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:22.959959  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:22.925016  8946 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:23.007560  1645 raft_consensus.cc:3060] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:23.007898  8282 raft_consensus.cc:3055] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:23.008284  8282 raft_consensus.cc:740] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:23.009011  8282 consensus_queue.cc:260] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.007015  1685 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:23.029811  9111 raft_consensus.cc:2804] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:23.030551  9111 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.031183  9111 raft_consensus.cc:3060] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:23.034082  9020 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:23.035988  8595 consensus_queue.cc:260] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:22.941700  9113 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.039999  9113 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.042013  1645 raft_consensus.cc:2468] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
W20260709 13:05:23.030802  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 93304.8us
I20260709 13:05:22.911942  8942 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:22.942811  8912 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:22.942409  8481 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:22.925593  9004 raft_consensus.cc:740] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:05:22.987676  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.946732  8370 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:23.010224  8282 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:22.957048  1646 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:05:22.938252  1612 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:05:23.048347  1685 consensus_peers.cc:597] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.098219  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49947.7us
W20260709 13:05:23.099145  1685 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:23.100518  1685 leader_election.cc:336] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:23.100989  1685 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:22.951992  8975 consensus_queue.cc:1059] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.114s
W20260709 13:05:23.097321  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 159171us
I20260709 13:05:22.952613  8941 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:23.068826  9027 consensus_meta.cc:287] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.760s	user 0.009s	sys 0.009s
I20260709 13:05:23.047128  8986 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:23.124464  1685 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:23.066529  9081 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:22.947871  9052 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.230533  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:23.174227  1464 catalog_manager.cc:5697] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:23.185122  1687 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:23.200805  8912 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:23.094329  8756 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.202512  8899 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.095268  8370 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:23.189327  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 68561.6us
W20260709 13:05:23.198604  9077 consensus_meta.cc:287] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.566s	user 0.012s	sys 0.000s
I20260709 13:05:23.122915  9102 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.248344  9102 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.002061  9002 consensus_queue.cc:237] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.124033  9035 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:23.250743  9035 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.251174  9035 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:23.166199  2345 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:05:23.133077  9089 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.251883  9089 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.252388  9089 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.243805  8826 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.144032  9082 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:23.229964  1685 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:23.255590  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 131140us
W20260709 13:05:23.256556  1685 leader_election.cc:336] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:23.232810  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 47782.4us
I20260709 13:05:23.239666  8370 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:23.253736   431 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 1598)
I20260709 13:05:23.230957  8282 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.263633  8282 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.257570  1717 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
I20260709 13:05:23.229547  9033 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:23.241493  8756 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.278950  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:23.279810  1713 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19.
I20260709 13:05:23.286214  9021 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:23.288033  9053 raft_consensus.cc:3055] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:23.288512  9053 raft_consensus.cc:740] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:23.289332  9053 consensus_queue.cc:260] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.290694  9053 raft_consensus.cc:3060] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:23.237571  9020 raft_consensus.cc:2804] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:05:23.245590  9019 consensus_meta.cc:287] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.995s	user 0.015s	sys 0.000s
I20260709 13:05:23.269440  8899 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.293682  9082 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:23.270161  8798 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:23.328111  9116 log.cc:927] Time spent T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:05:23.329473  9117 log.cc:927] Time spent T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:05:23.236423  9081 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.270285  8826 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.263460  9077 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:23.353960  1685 leader_election.cc:336] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:23.337145  8978 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.334110  9020 raft_consensus.cc:697] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:23.365929  9020 consensus_queue.cc:237] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:23.372004  9090 log.cc:927] Time spent T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.860s	user 0.003s	sys 0.012s
I20260709 13:05:23.374177  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 809) took 1176 ms. Trace:
I20260709 13:05:23.307514  9019 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.374667  1647 rpcz_store.cc:276] 0709 13:05:22.198215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.201844 (+  3629us) service_pool.cc:224] Handling call
0709 13:05:22.237828 (+ 35984us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:22.241837 (+  4009us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:22.241872 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:22.260476 (+ 18604us) log.cc:844] Serialized 26 byte log entry
0709 13:05:22.391848 (+131372us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:22.391921 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.392025 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:23.318525 (+926500us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b4400022b4c. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245cbec85 00007f8245cbf6a2 00007f8248a311db 00007f8248a2caaf 00007f8248a2086e 00007f8248a1fbb6 00007f8248a1f7d8 00007f8248a29221 00007f8248a291d9 00007f8248a29169
0709 13:05:23.373554 (+ 55029us) raft_consensus.cc:1685] finished
0709 13:05:23.373589 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.374058 (+   469us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20324,"raft.queue_time_us":5978,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":3708,"spinlock_wait_cycles":52492032,"thread_start_us":89563,"threads_started":1,"wal-append.queue_time_us":179842}
W20260709 13:05:23.376219  1685 leader_election.cc:336] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:23.337476  8899 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.401329  8826 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.431274  9134 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.346585  8798 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.378940  8978 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.422214  8481 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:23.444916  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.421077  9124 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:23.452584  9126 log.cc:927] Time spent T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:05:23.472211  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3437) took 1747 ms. Trace:
I20260709 13:05:23.472913  1719 rpcz_store.cc:276] 0709 13:05:21.724716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.726012 (+  1296us) service_pool.cc:224] Handling call
0709 13:05:21.726364 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:22.604915 (+878551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.471837 (+866922us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b5400213c54. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:23.472156 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2018447744}
I20260709 13:05:23.480011  9124 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.506942  9019 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:23.508232  8370 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.509052  9035 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:23.508675  9011 consensus_meta.cc:287] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.748s	user 0.009s	sys 0.005s
I20260709 13:05:23.441511  9134 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.516923  9011 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:23.527622  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:23.530151  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:23.525854  2313 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:23.526396  1687 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:23.529803  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:23.536649  1719 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 21.
I20260709 13:05:23.537603  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.538763  1643 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 21.
I20260709 13:05:23.537937  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:23.541826  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:23.541916  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.543116  1643 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:23.543205  1641 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:23.544085  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:23.544713  1645 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:23.540496  1719 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:23.545060  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.545717  1641 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:23.546702  1722 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:05:23.546985  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:23.552059  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:23.552991  1722 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:23.558022  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.558866  1640 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:23.559082  1644 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 22. Current term is 23. Ops: []
I20260709 13:05:23.562911  1718 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:23.564828  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:23.568056  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:23.568360  1722 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:23.551313  8921 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:23.550963  9076 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:23.544028  9068 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:23.545619  8912 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:23.585044  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:23.592911  1645 raft_consensus.cc:2468] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 21.
I20260709 13:05:23.558056  9011 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:23.565627  9123 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.602852  1643 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 23.
I20260709 13:05:23.604080  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:23.607246  2341 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:23.607383  1644 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 22. Current term is 23. Ops: []
I20260709 13:05:23.575388  9130 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.579615  8921 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.589507  9076 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.615185  8921 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:23.617190  9133 log.cc:927] Time spent T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.195s	user 0.002s	sys 0.000s
I20260709 13:05:23.635588  1645 raft_consensus.cc:1275] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
W20260709 13:05:23.636521  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.621621  9076 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:23.620239  9102 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.604046  8912 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.645116  8978 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.605123  9068 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.609993  9011 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.611444  9123 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.656243  9068 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:23.657433  9004 consensus_queue.cc:260] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.650270  1687 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:23.659615  8912 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:23.660233  9141 log.cc:927] Time spent T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.090s	user 0.002s	sys 0.000s
I20260709 13:05:23.661008  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42278 (request call id 3042) took 1041 ms. Trace:
I20260709 13:05:23.663507  8826 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.618041  9130 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.676129  1720 rpcz_store.cc:276] 0709 13:05:22.619185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.659665 (+1040480us) service_pool.cc:224] Handling call
0709 13:05:23.660200 (+   535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.660488 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:23.660529 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:23.660632 (+   103us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:23.660688 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.660765 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.660925 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:23.679360  2341 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:23.680897  1612 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:23.689608  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.695802  9089 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:23.698318  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.685328  9020 raft_consensus.cc:2804] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:23.678054  8826 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.697955  9033 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:23.706187  9066 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:23.721166  9110 log.cc:927] Time spent T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.680s	user 0.004s	sys 0.008s
I20260709 13:05:23.707361  9004 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:23.721874  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39914 (request call id 3609) took 1254 ms. Trace:
I20260709 13:05:23.723209  1639 rpcz_store.cc:276] 0709 13:05:22.467611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.516292 (+ 48681us) service_pool.cc:224] Handling call
0709 13:05:22.516740 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:22.538351 (+ 21611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:22.538386 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:22.548858 (+ 10472us) log.cc:844] Serialized 26 byte log entry
0709 13:05:23.432844 (+883986us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b1800390fa0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d7ae07 00007f824ebd5ab9 00007f8245d9769b 00007f8245d9593f 00007f8245d97c02 00007f8248a1f20b 00007f8248a24d23 00007f8248a250d1 00007f8248e40a0f
0709 13:05:23.497382 (+ 64538us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:23.497433 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.497522 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:23.717824 (+220302us) raft_consensus.cc:1685] finished
0709 13:05:23.717846 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.721784 (+  3938us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21085,"raft.queue_time_us":31724,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":2443,"spinlock_wait_cycles":60990336,"thread_start_us":64283,"threads_started":1,"wal-append.queue_time_us":978730}
I20260709 13:05:23.726514  9123 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.717876  9134 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.728070  9082 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:23.735666  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42278 (request call id 3045) took 1114 ms. Trace:
I20260709 13:05:23.736425  9019 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.736232  1717 rpcz_store.cc:276] 0709 13:05:22.621606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.621923 (+   317us) service_pool.cc:224] Handling call
0709 13:05:23.735458 (+1113535us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b5400213754. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:23.735597 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1384475392}
I20260709 13:05:23.736826  9011 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:23.737126  9019 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.738619  9081 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:23.739486  9019 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.740267  8756 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.740809  8756 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.716182  9020 raft_consensus.cc:697] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 21 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:23.743124  9020 consensus_queue.cc:237] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.743980  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42276 (request call id 3445) took 1431 ms. Trace:
I20260709 13:05:23.744354  1718 rpcz_store.cc:276] 0709 13:05:22.312702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.312964 (+   262us) service_pool.cc:224] Handling call
0709 13:05:22.313366 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.567146 (+1253780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.743701 (+176555us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400212354. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e85e01 00007f8248e8483e 00007f824de49419 00007f82473c0bd4 00007f82473c0b62 00007f82473c0a91 00007f82473c0a0c 00007f82473bfcc2 00007f82471c048c 00007f82471bf8c6
0709 13:05:23.743914 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2872792448}
I20260709 13:05:23.712356  8899 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:23.741295  8756 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.723932  9033 raft_consensus.cc:697] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:23.728173  9130 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.734885  8826 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:23.746116  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42276 (request call id 3444) took 1433 ms (client timeout 1793 ms). Trace:
W20260709 13:05:23.732815  9111 consensus_meta.cc:287] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.701s	user 0.006s	sys 0.006s
W20260709 13:05:23.751299  1716 rpcz_store.cc:269] 0709 13:05:22.312294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.312471 (+   177us) service_pool.cc:224] Handling call
0709 13:05:23.745954 (+1433483us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b54002995d4. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e892fa 00007f824de4ab17 00007f82473c56d4 00007f82473c5662 00007f82473c5591 00007f82473c550c 00007f82473c47c2 00007f82471c048c 00007f82471bf8c6 00007f82471bf26f
0709 13:05:23.746059 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":679737216}
I20260709 13:05:23.751173  9011 consensus_queue.cc:1048] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:05:23.753911  9111 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.747025  9033 consensus_queue.cc:237] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.756389  9111 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.759503  8370 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.765865  8826 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.772946  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.761027  9123 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:23.764065  8370 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:05:23.787086  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.785512  9123 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.792325  9123 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:23.793742  1640 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:23.793536  9076 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.797361  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.797256  9011 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:23.795969  8370 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.798379  9011 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.799443  9130 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.800661  8370 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:23.800024  9130 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.801045  9011 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:23.801930  8798 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.803431  9130 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:23.803844  9076 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:23.798836  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:23.805012  1640 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:23.795125  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.773041  8826 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.806137  1648 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:05:23.808707  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:23.809342  1648 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:23.806635  2314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.814921  8899 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:23.815595  8899 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.817241  9010 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.817840  9010 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.819202  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:23.819847  1647 raft_consensus.cc:3060] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:23.820297  9010 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:23.820914  8826 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.824072  1538 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:23.824909  1538 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:23.829511  1647 raft_consensus.cc:2468] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
W20260709 13:05:23.830595  1536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.810897  8370 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.832927  1687 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:23.837900  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.838716  1647 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:23.839684  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:23.840755  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.841269  1647 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 20.
I20260709 13:05:23.835346  8370 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.844031  8370 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.845748  9134 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:23.846459  9134 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.846912  9134 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:23.847127  1640 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 21.
I20260709 13:05:23.848253  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:23.836843  1538 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:23.849913  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.850878  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.851387  1647 raft_consensus.cc:2468] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 21.
I20260709 13:05:23.852106  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:23.853029  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.826193  9081 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.854298  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.854054  9081 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.826404  8756 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:23.830037  9053 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:23.855204  9053 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.855544  9053 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:23.856261  9081 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:23.862003  8899 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:23.865393  1647 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Rejecting Update request from peer cf2f71592be8465fb18e36e15d0b2f89 for earlier term 23. Current term is 24. Ops: []
I20260709 13:05:23.870890  1648 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:23.871335  8921 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.873251  2347 raft_consensus.cc:2393] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 21: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:05:23.878142  1538 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:23.880527  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:23.881237  1648 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:23.893778  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.895270  1642 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19.
I20260709 13:05:23.902798  1648 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 21.
I20260709 13:05:23.902815  1538 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:23.874377  8921 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.874748  9134 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.909499  9130 consensus_queue.cc:1059] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:23.911325  8899 raft_consensus.cc:3055] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:05:23.883908  8756 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:23.911942  8899 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:23.902235  9086 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:23.913463  9086 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.913795  8899 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.915462  8899 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:23.912235  9134 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.918280  9065 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:23.918918  9065 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.921186  9065 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:23.902978  8370 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:23.921936  8370 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.922444  8370 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:23.922890  9052 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:23.923946  9086 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:23.926055  9087 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.926625  9087 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.932382  8370 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.923027  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.933048  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.935791  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.884027  9146 raft_consensus.cc:2804] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:23.880255  9033 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:23.929423  9087 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:23.940871  1648 raft_consensus.cc:1275] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:05:23.941397  8370 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.942206  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.942540  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:23.932863  9111 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:23.943403  1648 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:23.936769  8756 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.944597  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:23.947604  1640 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:23.943796  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.946269  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.950742  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:23.951748  1642 raft_consensus.cc:2393] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 18: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:23.952365  8751 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:23.948904  8756 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:23.955629  8595 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:23.943279  9146 raft_consensus.cc:697] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 20 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:23.958416  9146 consensus_queue.cc:237] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.943802  9033 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.961706  1648 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:05:23.963505  9033 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.964015  9033 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:23.964479  9033 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.968461  8912 consensus_queue.cc:1048] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:23.969051  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:23.950155  9111 raft_consensus.cc:697] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 23 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:23.970953  9111 consensus_queue.cc:237] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:23.971520  1716 raft_consensus.cc:1217] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Deduplicated request from leader. Original: 18.14->[19.15-19.15]   Dedup: 19.15->[]
I20260709 13:05:23.972193  1716 raft_consensus.cc:1240] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:05:23.973199  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:23.974814  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:23.980474  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:23.981310  1720 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 16.
I20260709 13:05:23.982796  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:23.983837  8370 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.983909  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:23.984445  8370 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:23.987439  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:23.987870  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:23.986034  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.986590  9036 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.991931  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:23.993523  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:23.996234  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:23.996899  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:23.998664  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:23.985282  8986 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.999902  8986 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.001724  1716 raft_consensus.cc:2410] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:05:23.998724  2345 raft_consensus.cc:1240] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:05:24.003012  8941 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:23.988755  8481 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.003924  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:24.004590  1717 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:24.004402  8481 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.004940  9077 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:24.008296  8481 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:24.008665  8370 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:24.010499  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:24.000315  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:24.014123  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:24.015161  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:24.015756  9036 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.017086  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:24.019423  1717 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:24.020181  2313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.032721  9111 consensus_queue.cc:1059] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:24.034363  1612 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:24.034976  8986 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:24.025903  9149 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.034991  8736 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:24.042203  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:24.063843  1715 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:24.064287  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.064631  1643 raft_consensus.cc:1275] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:05:24.065232  1686 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:24.041981  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:24.042945  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:24.043541  9036 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:24.043589  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:24.046450  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:24.068188  1721 raft_consensus.cc:3060] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:05:24.046481  9122 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:24.047828  9020 consensus_queue.cc:1059] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:24.050156  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:24.032971  8370 consensus_queue.cc:1059] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:23.991807  8899 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.061475  1717 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 21.
I20260709 13:05:24.054163  1537 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:24.066747  1713 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:24.065972  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:24.076556  1640 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:24.078161  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:24.079660  8370 raft_consensus.cc:3055] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:24.085577  8370 raft_consensus.cc:740] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:24.084451  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:24.085536  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:24.089912  1644 raft_consensus.cc:2393] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 20: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:24.090792  9036 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:24.089107  8481 raft_consensus.cc:3055] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:24.086889  8899 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.093125  9076 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:24.067562  8826 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:24.073822  9149 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:24.099537  8370 consensus_queue.cc:260] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.135031  8370 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:24.135406  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 47584.7us
I20260709 13:05:24.088347  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:24.101092  9118 log.cc:927] Time spent T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.892s	user 0.008s	sys 0.000s
I20260709 13:05:24.101830  9077 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:24.104049  1537 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:24.063361  8736 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.104764  9124 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:24.110647  9053 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.115335  1538 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:24.118484  1713 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:05:24.111886  8481 raft_consensus.cc:740] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:24.119681  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:24.121124  8899 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:24.105356  9033 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:24.124961  2314 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:24.127355  8756 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.154006  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:24.121485  8986 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 13:05:24.073951  9086 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:24.132489  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:24.087064  9122 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:24.138677  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.164331  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:24.141603  1721 raft_consensus.cc:2468] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 21.
I20260709 13:05:24.165051  1713 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
I20260709 13:05:24.146412  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:24.151576  1715 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:24.131985  8826 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:24.155043  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 85157.6us
I20260709 13:05:24.155596  1538 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:24.168613  8370 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:24.169014  1538 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:24.169265  8370 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.170039  8370 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:24.170867  1538 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:24.156518  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:24.158443  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:24.159328  2344 raft_consensus.cc:2368] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18.
I20260709 13:05:24.160336  2347 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:05:24.137526  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 65467.7us
I20260709 13:05:24.172466  1714 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:05:24.163239  2342 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:05:24.146733  9036 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:05:24.176358  2312 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:24.176205  1538 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:24.176849  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 51969.9us
I20260709 13:05:24.177661  2314 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:24.178002  2312 leader_election.cc:336] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:24.178524  8370 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.174003  1719 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 21.
I20260709 13:05:24.174539  1640 raft_consensus.cc:1275] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:05:24.174595  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:24.152851  9065 raft_consensus.cc:3055] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:05:24.184687  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:24.185335  1713 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 20: Already voted for candidate 066ee0109aba4465ba17fa65e14ef504 in this term.
I20260709 13:05:24.162051  8946 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:24.162810  9086 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:24.163614  9082 consensus_queue.cc:1048] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:05:24.176247  8481 consensus_queue.cc:260] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.210985  8481 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:24.175328  8826 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:24.179209  8899 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:24.204583  2344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:24.193480  8370 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:24.198707  1538 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:24.202149  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:24.214910  2344 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 24.
I20260709 13:05:24.190299  8966 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:24.227222  8966 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.227658  8966 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:24.228861  9122 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:24.214465  9020 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:05:24.231482  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:24.232180  1715 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 16.
I20260709 13:05:24.235384  1538 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:24.179177  9076 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.184064  8941 raft_consensus.cc:2804] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:24.184310  9065 raft_consensus.cc:740] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:24.241212  1717 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 23.
I20260709 13:05:24.233104  1464 catalog_manager.cc:5697] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 14 to 17, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 17 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:24.207089  8756 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:24.211891  9036 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:24.213147  8946 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.241516  1714 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:24.241940  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:24.257541  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:24.258819  2341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:24.190923  9031 raft_consensus.cc:2804] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:24.240677  8370 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:24.251535  1538 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:24.263803  1538 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:24.237867  9144 log.cc:927] Time spent T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.555s	user 0.003s	sys 0.000s
I20260709 13:05:24.263999  8370 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:24.239220  8899 raft_consensus.cc:697] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 21 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:24.239691  8975 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:24.260564  8946 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:24.264483   431 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 1598)
I20260709 13:05:24.270658  8370 raft_consensus.cc:697] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 20 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:24.240497  8941 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.251570  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:24.271545  2312 leader_election.cc:336] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:24.271570  8370 consensus_queue.cc:237] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.273941  2347 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 23.
I20260709 13:05:24.241325  9076 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.272852  8986 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:24.265129  9035 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:24.281049  9030 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:24.283520  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:24.276160  8941 raft_consensus.cc:3060] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:24.290905  1686 leader_election.cc:400] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18.
I20260709 13:05:24.296386  1686 leader_election.cc:403] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:24.297946  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:24.271179  9031 raft_consensus.cc:697] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:24.306339  8942 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:24.299963  9053 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:24.308280  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:24.310947  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:24.311880  9035 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:05:24.312480  9035 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.312942  9035 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:24.313524  9086 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:24.307633  8899 consensus_queue.cc:237] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.318248  1612 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:24.320286  1612 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:24.320770  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:24.289955  8986 raft_consensus.cc:697] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:24.321513  2346 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 24.
I20260709 13:05:24.290271  8975 raft_consensus.cc:697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:24.327463  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:24.303553  9014 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935) with higher term. Message: Invalid argument: T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 17. Current term is 18.
I20260709 13:05:24.326218  9033 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:24.325990  9124 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:24.335074  9146 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:24.356812  9146 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting leader election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:24.357378  9146 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:24.364693  8282 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:24.373361  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:24.373997  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:24.374001  8282 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.375586  8282 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.379887  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:24.381263  1611 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:24.382026  1611 leader_election.cc:336] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:24.382582  1611 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:24.327247  9086 consensus_queue.cc:1048] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.039s
W20260709 13:05:24.383159  1611 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:24.282109  9065 consensus_queue.cc:260] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.388603  8282 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:24.390599  1611 leader_election.cc:336] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:24.402705  9065 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:24.367921  9033 raft_consensus.cc:697] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 23 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:05:24.430837  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 99912.9us
I20260709 13:05:24.436982  1612 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:24.438476  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:24.463165  1612 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:24.466316  2313 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:24.472752  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:24.372123  8975 consensus_queue.cc:237] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.463133  9167 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:24.500415  1612 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:24.464735  9149 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:24.498495  9125 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:24.416795  8986 consensus_queue.cc:237] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.517153  8736 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:24.418139  9031 consensus_queue.cc:237] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:24.496194  9166 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:24.500058  9167 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.530879  8966 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:24.533363  8966 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:24.533736  9167 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:24.527329  9166 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.523316  9076 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:24.568029  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.683866  9166 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:24.684536  8941 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.671264  8899 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.672173  9003 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:24.673211  9165 log.cc:927] Time spent T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.177s	user 0.013s	sys 0.000s
I20260709 13:05:24.525303  9033 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:24.687796  9168 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.147s	user 0.000s	sys 0.009s
I20260709 13:05:24.687907  9146 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:24.716490  1722 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:05:24.717018  1722 pending_rounds.cc:85] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 12
I20260709 13:05:24.717485  1722 pending_rounds.cc:107] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:05:24.717767  1722 raft_consensus.cc:2889] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:24.718466  9021 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.690320  8899 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:24.722508  2312 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:24.691781  8826 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.691916  9003 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.724308  2312 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:24.777510  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 277114us
W20260709 13:05:24.777779  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 55345.9us
I20260709 13:05:24.716837  8941 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:24.719132  9021 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.830983  1643 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 20: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:24.777158  8986 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:24.844364  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 65865.1us
I20260709 13:05:24.845809  1612 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:24.724947  8946 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:24.850155  2312 leader_election.cc:336] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:24.820408  8966 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:24.820822  9173 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:24.863852  8282 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.846527  8798 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:24.864735  8282 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:24.892915  8282 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:24.851344  8986 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.851162  9167 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.881740  8798 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:24.903957  9065 consensus_meta.cc:287] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.010s
W20260709 13:05:24.910916  9035 consensus_meta.cc:287] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.598s	user 0.016s	sys 0.006s
I20260709 13:05:24.906248  8798 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:24.931345  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:24.926278  9183 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:24.933369  9176 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.932598  9146 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:24.966406  9179 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:05:24.972971  9167 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:24.973785  9003 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:24.978387  8946 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:24.981976  8899 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:24.984191  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 540826us
W20260709 13:05:24.984938  1611 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:24.929369  9035 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:24.986651  1639 raft_consensus.cc:2368] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 24. Current term is 25.
I20260709 13:05:24.987706  9035 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 25 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:24.990950  8986 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:24.985394  1611 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:24.973519  9021 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:24.996325  9176 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:24.999127  9176 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:25.001066  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 856105us
I20260709 13:05:25.002302  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:25.002586  1611 leader_election.cc:336] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:25.003078  1611 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:25.003209  9124 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:25.003499  1537 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:25.019034  2346 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 24.
I20260709 13:05:25.020188  1643 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 20 index: 9. (index mismatch)
I20260709 13:05:25.022524  1643 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:25.024551  1643 raft_consensus.cc:1275] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:05:25.021625  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:25.006836  9053 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:25.006263  1464 catalog_manager.cc:5697] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:25.030223  1643 raft_consensus.cc:1275] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:05:25.032105  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:25.034734  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:25.036170  1537 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:25.038894  8941 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:25.040584  9035 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:25.042614  9035 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:25.045455  9035 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:25.038839  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:25.003986  9124 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:25.002491  8826 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:25.034720  9020 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:05:25.054332  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:25.088232  1639 raft_consensus.cc:2368] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 24. Current term is 25.
I20260709 13:05:25.079843  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:25.089422  1643 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:05:25.048168  8986 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:25.037081  9053 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:25.114266  1687 leader_election.cc:400] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 24. Current term is 25.
W20260709 13:05:25.082809  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 44025.9us
W20260709 13:05:25.111827  1611 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:25.121620  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:25.123634  1611 leader_election.cc:336] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:25.124174  1611 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:25.125329  1611 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:25.092947  9003 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:05:25.126864  1611 leader_election.cc:336] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:25.127357  1611 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:25.130970  1611 leader_election.cc:336] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:25.114101  9082 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:25.123124  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 277365us
I20260709 13:05:25.129287  1537 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:25.116492  9174 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:25.135989  1612 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:25.138209  1611 leader_election.cc:336] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:25.120972  9020 raft_consensus.cc:697] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 24 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:25.128187  8826 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:25.107537  9166 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:25.131338  9053 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:25.122920  9087 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:05:25.123255  9093 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
W20260709 13:05:25.171032  1462 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38662 (request call id 212) took 46359 ms (client timeout 15000 ms). Trace:
I20260709 13:05:25.128363  9030 consensus_queue.cc:1048] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:05:25.148883  1612 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:25.151209  1537 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:25.141676  1611 consensus_peers.cc:597] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:25.177785  9030 consensus_queue.cc:1243] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:05:25.180572  1612 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:25.135285  9035 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:25.129631  8912 consensus_queue.cc:1048] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 13:05:25.156896  1464 catalog_manager.cc:5697] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 14 to 17, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 17 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:25.162447  9020 consensus_queue.cc:237] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:25.165227  9166 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:25.147082  9124 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:25.166062  8899 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:25.140278  9082 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:25.187736  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 203880us
I20260709 13:05:25.144973  9174 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:25.204905  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:25.207237  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:25.185750  9053 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:25.187238  9033 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:25.221532  1640 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 22: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:25.217160  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:25.222153  1645 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:25.218760  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:25.219738  1641 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:05:25.220594  1647 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:05:25.220988  1639 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 20: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:25.224720  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:25.226804  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:25.227411  1640 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:25.229794  1720 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:05:25.230274  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:25.230858  2349 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:25.201272  8826 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:25.233855  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:25.235056  1718 raft_consensus.cc:3060] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:25.238721  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:25.240310  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:25.240628  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:25.241067  1714 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
I20260709 13:05:25.241235  1715 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:25.242172  2346 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:05:25.271741   431 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 1598)
I20260709 13:05:25.275928  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:25.278283  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:25.280346  1464 catalog_manager.cc:5697] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 20 to 23, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 23 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:25.200354  8975 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:05:25.244163  9003 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.286139  9082 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:25.287096  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:25.293375  1639 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:05:25.293866  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:25.295178  1644 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:25.364365  9053 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 13:05:25.378157  9003 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:25.379721  1629 tablet_service.cc:1558] Processing DeleteTablet for tablet 59d236b930214b1f91fb55ece6fecd68 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 6 (current committed config index is 9)) from {username='slave'} at 127.0.0.1:47496
W20260709 13:05:25.179206  1462 rpcz_store.cc:269] 0709 13:04:38.811966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.812348 (+   382us) service_pool.cc:224] Handling call
0709 13:05:25.170872 (+46358524us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:19.292740 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:19.293567 (+   827us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:19.293655 (+    88us) write_op.cc:435] Acquired schema lock
0709 13:05:19.293697 (+    42us) tablet.cc:662] Decoding operations
0709 13:05:19.294994 (+  1297us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:19.295126 (+   132us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:19.295155 (+    29us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:05:19.296193 (+  1038us) tablet.cc:701] Row locks acquired
0709 13:05:19.296266 (+    73us) write_op.cc:260] PREPARE: finished
0709 13:05:19.296457 (+   191us) write_op.cc:270] Start()
0709 13:05:19.296650 (+   193us) write_op.cc:276] Timestamp: P: 1783602319296376 usec, L: 0
0709 13:05:19.296691 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 13:05:19.297563 (+   872us) log.cc:844] Serialized 4141 byte log entry
0709 13:05:19.651256 (+353693us) op_driver.cc:464] REPLICATION: finished
0709 13:05:19.961777 (+310521us) write_op.cc:301] APPLY: starting
0709 13:05:19.961893 (+   116us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:20.195015 (+233122us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:20.195161 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:20.391275 (+196114us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:20.391698 (+   423us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 13:05:20.391771 (+    73us) write_op.cc:312] APPLY: finished
0709 13:05:20.392262 (+   491us) log.cc:844] Serialized 97 byte log entry
0709 13:05:20.392496 (+   234us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:20.392648 (+   152us) write_op.cc:454] Released schema lock
0709 13:05:20.392726 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:20.393295 (+   569us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:20.393326 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":576941,"reported_tablets":54,"child_traces":[["op",{"apply.queue_time_us":310166,"apply.run_cpu_time_us":20967,"apply.run_wall_time_us":431490,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45947,"num_ops":10,"prepare.queue_time_us":156019,"prepare.run_cpu_time_us":10074,"prepare.run_wall_time_us":129361,"raft.queue_time_us":177827,"raft.run_cpu_time_us":25525,"raft.run_wall_time_us":174694,"replication_time_us":354340,"thread_start_us":377527,"threads_started":4,"wal-append.queue_time_us":59952,"wal-append.run_cpu_time_us":19609,"wal-append.run_wall_time_us":1994062}]]}
I20260709 13:05:25.367691  1612 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:25.389858  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:25.383677  1647 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
W20260709 13:05:25.299530  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.310988  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:25.312239  9174 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:25.322400  9187 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:05:25.395246  1685 leader_election.cc:336] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.792s (SENT)
I20260709 13:05:25.249696  1640 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:25.352768  2349 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 25.
I20260709 13:05:25.356865  1645 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
W20260709 13:05:25.356957  1611 leader_election.cc:336] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:25.411913  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 55017.5us
I20260709 13:05:25.408140  1685 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:25.358321  1715 raft_consensus.cc:2468] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 20.
W20260709 13:05:25.414522  1685 leader_election.cc:336] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.974s (SENT)
I20260709 13:05:25.363132  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:25.354970  1538 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:25.415897  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 67453.3us
I20260709 13:05:25.355690  9173 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:05:25.354171  8946 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:05:25.398764  1718 raft_consensus.cc:2468] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 20.
I20260709 13:05:25.400246  1714 raft_consensus.cc:3060] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:25.426225  1611 leader_election.cc:336] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:25.405696  9184 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:25.436715  9184 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:25.437193  9184 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:25.437812  1612 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:25.422230  8912 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:25.440237  1714 raft_consensus.cc:2468] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 22.
I20260709 13:05:25.424328  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:25.441867  1647 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:25.381283  9003 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:25.372252  9166 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:25.419486  1645 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:05:25.365808  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:25.432411  1642 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:05:25.417573  1685 leader_election.cc:336] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 1.977s (SENT)
I20260709 13:05:25.433782  8975 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:05:25.447324  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 85884.7us
W20260709 13:05:25.431875  9188 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:05:25.430428  8899 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:25.472919  8899 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.431057  9183 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:05:25.426612  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 296090us
I20260709 13:05:25.432220  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:25.440255  9189 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:05:25.444945  1612 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:05:25.442979  1611 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:25.434160  9172 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:25.376096  1464 catalog_manager.cc:5697] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 10 to 16. New cstate: current_term: 16 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:25.495568  8798 consensus_meta.cc:287] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b: Time spent flushing consensus metadata: real 0.559s	user 0.013s	sys 0.000s
I20260709 13:05:25.423511  9187 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:25.496186  8798 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:25.496236  1464 catalog_manager.cc:5697] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 10 to 14, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 14 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:25.474807  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 45874.7us
I20260709 13:05:25.500574  1538 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:25.477543  1685 leader_election.cc:336] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.090s (SENT)
W20260709 13:05:25.477895  1611 leader_election.cc:336] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:25.501860  1611 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:25.503111  1611 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:25.503159  1612 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:05:25.480002  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:25.511518  1612 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:25.500825  1464 catalog_manager.cc:5697] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 10 to 16, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 16 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:25.474028  9183 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:25.473615  8899 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:25.516779  8912 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:25.517608  8912 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:25.519095  1641 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:25.520180  8912 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:25.493747  8946 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:25.520917  8946 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.521353  8946 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:25.475217  9189 raft_consensus.cc:697] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 25 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:25.474732  1687 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:25.527220  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 52582us
I20260709 13:05:25.493027  9167 raft_consensus.cc:2804] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:05:25.534559  2312 leader_election.cc:336] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:25.498515  9187 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:25.516106  1685 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.130s (SENT)
I20260709 13:05:25.535728  1687 leader_election.cc:400] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 election: Vote denied by peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) with higher term. Message: Invalid argument: T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for earlier term 24. Current term is 25.
W20260709 13:05:25.536532  1685 leader_election.cc:336] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.131s (SENT)
I20260709 13:05:25.538894  2312 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:25.525430  9200 log.cc:927] Time spent T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:05:25.540325  1538 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:25.552918  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:25.539578  9035 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:25.556794  9189 consensus_queue.cc:237] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:25.557746  1464 catalog_manager.cc:5697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:25.559929  1464 catalog_manager.cc:5697] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 12 to 15, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 15 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:25.561902  9185 raft_consensus.cc:2804] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:25.563462  8941 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:05:25.539893  9198 log.cc:927] Time spent T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:05:25.565275  8986 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:25.565591  8595 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:05:25.566202  8986 raft_consensus.cc:697] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:25.567492  8986 consensus_queue.cc:237] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:25.568364  9066 raft_consensus.cc:2804] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:25.539474  9053 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:05:25.570698  1685 leader_election.cc:336] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.175s (SENT)
W20260709 13:05:25.583806  1685 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.181s (SENT)
W20260709 13:05:25.586621  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 52085.5us
I20260709 13:05:25.591799  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:25.600749  2312 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:25.603680  1647 raft_consensus.cc:2468] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 20.
I20260709 13:05:25.605998  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:25.611086  1718 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
I20260709 13:05:25.516165  9183 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:25.612306  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 58725.7us
I20260709 13:05:25.541510  8912 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:25.618126  2314 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:05:25.604220  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 53086.7us
I20260709 13:05:25.585374  9053 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.480559  9166 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:25.591141  9185 raft_consensus.cc:697] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 22 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:25.591812  9066 raft_consensus.cc:697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 20 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:25.591399  9167 raft_consensus.cc:697] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 20 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
W20260709 13:05:25.642299  2312 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:25.591090  8997 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:25.643226  9167 consensus_queue.cc:237] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:25.643004  9185 consensus_queue.cc:237] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:25.630960  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 47242.5us
I20260709 13:05:25.647624  9166 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:25.648466  1685 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.228s (SENT)
I20260709 13:05:25.648962  1685 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:25.637132  1612 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:25.649291  8798 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:25.650369  9123 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:25.651113  9123 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:25.651552  9123 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:25.616775  1464 catalog_manager.cc:5697] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:25.655056  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:25.655853  1718 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 20: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:25.656889  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:25.657475  1717 raft_consensus.cc:3060] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:25.629230  8921 raft_consensus.cc:2804] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:25.659423  8921 raft_consensus.cc:697] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:25.660555  8921 consensus_queue.cc:237] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:25.618024  8912 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.664099  8912 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:25.633347  9053 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:25.667501  9011 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:25.686084  1717 raft_consensus.cc:2468] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 17.
W20260709 13:05:25.701684  2312 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:25.701869  1685 leader_election.cc:336] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Timed out: RequestConsensusVote RPC to 127.0.107.193:39999 timed out after 2.291s (SENT)
I20260709 13:05:25.702677  1685 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:25.706816  1715 raft_consensus.cc:1275] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:05:25.716936  2349 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:05:25.706530  1612 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:25.703948  2312 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:25.697285  9030 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.656787  1464 catalog_manager.cc:5697] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 12 to 14, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 14 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:25.714980  9087 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:25.745860  9176 raft_consensus.cc:2804] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:25.774960  8966 consensus_queue.cc:1048] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 13:05:25.779446  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:25.747462  9030 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:25.793566  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:25.796767  9030 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:25.798720  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:25.798473  8946 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:25.799943  1457 catalog_manager.cc:5028] TS f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): tablet 59d236b930214b1f91fb55ece6fecd68 (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
I20260709 13:05:25.800065  1464 catalog_manager.cc:5697] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 9 to 16. New cstate: current_term: 16 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:25.781360  9053 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:25.801584  8946 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:25.803144  9053 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:25.771853  9035 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
W20260709 13:05:25.817219  1685 consensus_peers.cc:597] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:25.823530  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:25.824857  2349 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 23.
I20260709 13:05:25.827998  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:25.828994  1722 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:25.830505  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:25.831790  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:25.833930  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:25.834018  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:25.834911  1721 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:25.834952  1685 leader_election.cc:336] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:25.838851  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:25.834312  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:25.839936  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:25.840900  1713 raft_consensus.cc:2393] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 22: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:25.843801  1716 raft_consensus.cc:2393] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 20: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:25.843892  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:25.847872  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:25.849411  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:25.849913  1721 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
W20260709 13:05:25.851555  1537 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.851668  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:25.853736  1717 raft_consensus.cc:2393] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 20: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:25.847975  9053 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.855841  9053 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:25.854166  8975 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.858805  8912 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:25.859894  8975 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:25.861754  9199 log.cc:927] Time spent T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.320s	user 0.000s	sys 0.002s
I20260709 13:05:25.851601  8946 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.862366  8975 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:25.840456  9066 consensus_queue.cc:237] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:25.864584  1537 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:25.862545  8946 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:25.868583  9201 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:05:25.869554  8946 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:25.869705  8975 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:25.823796  9176 raft_consensus.cc:697] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:25.871372  9176 consensus_queue.cc:237] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:25.872251  1537 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:25.874121  1464 catalog_manager.cc:5697] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 9 to 13, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 13 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:25.876051  1464 catalog_manager.cc:5697] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 12 to 18, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 18 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:25.858719  9053 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:25.879882  1537 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:25.884759  9184 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:25.877177  8975 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:25.890229  9053 raft_consensus.cc:3060] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:25.891804  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:25.892822  1719 raft_consensus.cc:2393] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 18: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:25.895551  9033 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:25.896503  9033 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:25.899221  9033 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:25.901816  1537 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:25.902698  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:25.886234  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:25.885089  1464 catalog_manager.cc:5697] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 13 to 18, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 18 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:25.902910  8975 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:25.905848  1464 catalog_manager.cc:5697] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 10 to 15. New cstate: current_term: 15 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:25.908769  9184 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:25.883960  1722 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:25.909788  8899 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:25.910630  1537 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:25.910144  1464 catalog_manager.cc:5697] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 9 to 14, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 14 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:25.917021  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:25.918514  1464 catalog_manager.cc:5697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 15 to 16, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 16 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:25.912894  8975 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:25.925820  8975 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.926362  8975 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:25.928237  9035 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.928866  9035 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:25.931003  9035 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:25.919749  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:25.935204  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:25.939468  1722 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
I20260709 13:05:25.940999  1612 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:25.922267  8946 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.940732  8941 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.921576  9053 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:25.936612  8912 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:25.942773  8946 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:25.946882  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:25.947317  8946 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:25.943198  8941 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:25.951380  8941 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:25.942667  9035 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:25.953346  9035 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.954195  9035 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:25.942063  1464 catalog_manager.cc:5697] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 5 to 17. New cstate: current_term: 17 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:25.964310  1722 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:05:25.964152  1464 catalog_manager.cc:5697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 10 to 16. New cstate: current_term: 16 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
W20260709 13:05:25.963006  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 75690.3us
I20260709 13:05:25.912441  8899 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:25.983899  1464 catalog_manager.cc:5697] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 9 to 16, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 16 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:26.002521  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:26.010157  1722 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:26.011282  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:26.012096  1640 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:26.027882  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:26.028847  1639 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:26.030601  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:26.031876  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:26.032688  1642 raft_consensus.cc:2393] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 20: Already voted for candidate cf2f71592be8465fb18e36e15d0b2f89 in this term.
I20260709 13:05:25.956969  9123 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:26.033406  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:26.042997  1641 raft_consensus.cc:2393] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 21: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:26.044109  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:26.044898  1648 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:26.044109  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:26.046120  1646 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:26.050937  1611 consensus_peers.cc:597] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:26.076498  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:26.079205  1645 raft_consensus.cc:2410] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 23 because replica has last-logged OpId of term: 22 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 9.
I20260709 13:05:26.071156  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:26.081904  1722 raft_consensus.cc:2393] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 17: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:26.083026  1640 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 21.
I20260709 13:05:26.086795  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:26.092386  1464 catalog_manager.cc:5697] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 11 to 16, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 16 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:26.043828  8986 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.102962  1719 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:26.069352  8941 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:26.103528  1685 leader_election.cc:336] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:26.106101  1685 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:26.108215  1685 leader_election.cc:336] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:26.109206  1685 leader_election.cc:336] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:26.110895  1464 catalog_manager.cc:5697] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:26.119879  1685 leader_election.cc:336] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:26.120759  1685 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:26.123922  8912 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:05:26.126298  9211 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 13:05:26.133927  1464 catalog_manager.cc:5697] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 13 to 14, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 14 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:26.135998  1464 catalog_manager.cc:5697] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 11 to 17, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 17 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:05:26.137370  2312 consensus_peers.cc:597] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:26.137988  8975 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:26.139294  9123 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:26.141173  8975 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 24 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:26.141439  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:26.142445  2340 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:26.142350  8941 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:26.146039  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:05:26.149921  1685 leader_election.cc:336] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:26.150534  1685 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:26.152005  1685 leader_election.cc:336] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:26.137617  1464 catalog_manager.cc:5697] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 11 to 17, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 17 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:26.070581  8899 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:05:26.145046  8941 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:26.164216  1719 raft_consensus.cc:3055] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:05:26.164968  1719 raft_consensus.cc:740] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:26.158617  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:26.170907  1639 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
W20260709 13:05:26.172701  1685 leader_election.cc:336] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:26.174650  1685 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:26.168278  1719 consensus_queue.cc:260] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:26.177450  1719 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:26.178412  1717 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:05:26.158236  8912 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:26.160403  9030 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:26.183317  1639 raft_consensus.cc:1240] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 20. Current term is 21. Ops: [20.9-20.9]
I20260709 13:05:26.188633  1722 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:05:26.189095  2340 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 24.
I20260709 13:05:26.193682  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:05:26.195793  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 430547us
W20260709 13:05:26.191442  1685 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:26.200323  1640 raft_consensus.cc:1240] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:05:26.202011  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:26.203444  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:26.206125  1685 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:26.207595  1685 leader_election.cc:336] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:26.208768  1685 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 196 similar messages]
W20260709 13:05:26.209931  1685 consensus_peers.cc:597] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:26.204630  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:26.105010  8986 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:26.224948  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:26.210780  1685 consensus_peers.cc:597] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:26.176554  8751 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:26.179663  9176 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:26.239526  1640 raft_consensus.cc:1240] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 25 LEADER]: Rejecting Update request from peer 130bc5ff39b847b1b3204626f284f838 for earlier term 24. Current term is 25. Ops: []
I20260709 13:05:26.239863  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:26.240890  1717 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:26.185606  9087 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:26.235960  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:26.254308  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:26.231709  1648 pending_rounds.cc:85] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 10
I20260709 13:05:26.258430  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:26.259073  1648 pending_rounds.cc:107] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:05:26.259531  1648 raft_consensus.cc:2889] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:26.221519  9014 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:26.268718  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:26.269402  1720 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:05:26.234833  9030 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.283500  9030 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:26.237946  8751 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:26.288221  8751 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:26.291929  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 207889us
I20260709 13:05:26.292930  1538 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:26.278043  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:26.280315  8912 consensus_queue.cc:1059] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:26.252260  9176 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:26.334803  1646 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
W20260709 13:05:26.342396  9219 log.cc:927] Time spent T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:05:26.343977  9209 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:05:26.328862  9014 consensus_queue.cc:1059] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.017s
W20260709 13:05:26.351049   431 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 1598)
I20260709 13:05:26.363468  1717 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:05:26.354773  9030 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:26.460484  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 162874us
W20260709 13:05:26.462296  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 185826us
I20260709 13:05:26.462486  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:26.463404  1719 raft_consensus.cc:2393] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 17: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:05:26.463596  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.464836  1717 raft_consensus.cc:1275] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:05:26.466449  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 213915us
I20260709 13:05:26.457943  9223 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.484097  2340 raft_consensus.cc:1240] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Rejecting Update request from peer f5f4b2707a9647e5bb99e065068dea2b for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 13:05:26.466472  8986 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:26.489496  9221 log.cc:927] Time spent T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:05:26.452795  9035 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:26.460955  9227 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:26.485071  1713 consensus_queue.cc:237] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
W20260709 13:05:26.495682  9218 log.cc:927] Time spent T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:05:26.503592  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:26.494736  1611 consensus_peers.cc:597] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:26.520424  1611 leader_election.cc:336] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:26.523845  1611 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:26.522221  1537 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:26.524642  1717 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:26.526356  1537 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:26.531383  1537 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:26.534081  1537 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:26.500666  8912 raft_consensus.cc:3055] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:26.542135  1537 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:26.491215  9189 consensus_queue.cc:1048] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:05:26.547721  1537 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:26.491957  9223 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:26.530124  9053 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:26.538617  9227 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:26.540225  8941 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:26.511390  9033 consensus_queue.cc:1059] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 13:05:26.548694  9176 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:26.567864  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 274997us
I20260709 13:05:26.568997  1537 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:26.574249  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 95953.5us
I20260709 13:05:26.541440  8912 raft_consensus.cc:740] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:26.538995  8986 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:26.620129  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 99730.1us
I20260709 13:05:26.610551  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:26.613515  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 44631us
W20260709 13:05:26.621354  1611 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:26.622061  1537 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:26.556102  9172 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:26.557235  9226 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:26.639248  1538 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:26.602443  9234 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:05:26.629730  9217 log.cc:927] Time spent T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:05:26.612884  9053 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:26.617534  8899 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:26.630147  9176 raft_consensus.cc:3055] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:05:26.656419  1715 pending_rounds.cc:85] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 16
I20260709 13:05:26.658018  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:26.654892  1537 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:26.656412  1538 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:26.673249  1538 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:26.676370  1715 pending_rounds.cc:107] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 24.17
I20260709 13:05:26.676819  1715 raft_consensus.cc:2889] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:26.677487  1538 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:26.656248  8946 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:26.645385  9228 raft_consensus.cc:3060] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:26.657351  9222 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 13:05:26.660302  9226 raft_consensus.cc:3060] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:26.688355  2312 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:26.690145  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:26.664507  9176 raft_consensus.cc:740] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:26.681802  8899 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:26.697872  8986 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:26.666435  8912 consensus_queue.cc:260] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:26.683027  9234 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:26.680783  8946 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.684831  9205 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:26.662564  9053 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:26.729808  2342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55874 (request call id 666) took 1141 ms. Trace:
I20260709 13:05:26.729934  8912 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:26.732332  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 76027.7us
W20260709 13:05:26.731887  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 120405us
W20260709 13:05:26.736675  2312 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:26.710134  9243 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:26.742408  1611 consensus_peers.cc:597] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:26.715827  8986 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:05:26.734356  2342 rpcz_store.cc:276] 0709 13:05:25.588432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.588602 (+   170us) service_pool.cc:224] Handling call
0709 13:05:25.611314 (+ 22712us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:25.611805 (+   491us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:05:25.961982 (+350177us) log.cc:844] Serialized 14 byte log entry
0709 13:05:26.195177 (+233195us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:26.223359 (+ 28182us) log.cc:844] Serialized 26 byte log entry
0709 13:05:26.223622 (+   263us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:26.223678 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:26.223785 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:26.729359 (+505574us) raft_consensus.cc:1685] finished
0709 13:05:26.729388 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:26.729730 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34904,"prepare.queue_time_us":254835,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":10461,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":34279,"thread_start_us":551156,"threads_started":2,"wal-append.queue_time_us":432276}
I20260709 13:05:26.733573  8946 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:26.733889  9234 raft_consensus.cc:697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 20 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:26.754467  8899 raft_consensus.cc:697] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 21 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:26.736536  9035 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:26.781327  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 76700.5us
I20260709 13:05:26.760296  8899 consensus_queue.cc:237] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:26.749895  9205 raft_consensus.cc:697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 20 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:26.796875  1537 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:26.797071  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:26.805101  1647 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:05:26.793535  9235 log.cc:927] Time spent T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:05:26.806474  9224 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.206s	user 0.002s	sys 0.000s
I20260709 13:05:26.818145  1537 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:26.815023  9242 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
W20260709 13:05:26.829030  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 90645us
I20260709 13:05:26.791743  9223 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:26.795209  8941 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:26.742478  9176 consensus_queue.cc:260] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:26.835978  1611 leader_election.cc:336] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:26.837560  1611 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:26.825608  8975 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:26.825709  9087 raft_consensus.cc:3055] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:26.831533  9262 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:05:26.861763  1686 consensus_peers.cc:597] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ca2582f1941a4c588f8ef51c03dfb492. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:26.862968  9176 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:26.847242  9263 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.023s
W20260709 13:05:26.867461  9258 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:05:26.860705  9087 raft_consensus.cc:740] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:05:26.906962  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.861318  9223 consensus_queue.cc:1059] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:05:26.853508  8941 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:26.901046  9251 log.cc:927] Time spent T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.113s	user 0.000s	sys 0.002s
W20260709 13:05:26.923756  9256 log.cc:927] Time spent T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:05:26.845286  9234 consensus_queue.cc:237] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:26.926656  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 91072.9us
I20260709 13:05:26.930541  9053 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:26.880141  9259 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
W20260709 13:05:26.928076  9245 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.184s	user 0.000s	sys 0.002s
W20260709 13:05:26.944363  1613 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.944697  2342 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 25 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:05:26.864619  9205 consensus_queue.cc:237] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:26.936697  1537 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:26.949545  1537 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:26.939703  8941 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:26.975656  1722 raft_consensus.cc:1275] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:05:26.976145  1720 raft_consensus.cc:1275] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:05:26.978449  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:26.981870  1713 raft_consensus.cc:1275] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:05:26.952966  9031 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:26.971216  9087 consensus_queue.cc:260] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 20.9, Last appended by leader: 9, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:26.990746  9031 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:26.972461  9227 raft_consensus.cc:3060] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:26.992873  9228 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:26.994217  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 158240us
I20260709 13:05:26.961114  8975 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:27.010426  9252 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:26.973034  9123 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:05:27.050907  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 87564us
I20260709 13:05:27.017236  9123 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:27.017935  9030 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.051445  9267 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:05:27.083789  1537 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:05:27.049914  9014 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 13:05:27.080143  9031 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:27.116138  9123 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:27.107223  9265 log.cc:927] Time spent T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.183s	user 0.002s	sys 0.000s
I20260709 13:05:27.124316  9267 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:05:27.125344  9267 raft_consensus.cc:697] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 24 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:27.125782  9166 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:27.126410  9166 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:27.072211  9030 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:27.128759  9166 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:27.142563  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:27.166064  1644 raft_consensus.cc:1275] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:27.167151  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:27.173738  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:27.106444  9223 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:27.186326  1687 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.183050  9227 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:27.187978  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:27.190274  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:27.190390  1639 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:27.191139  1647 raft_consensus.cc:2410] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 24 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 13:05:27.189528  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:27.192795  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:27.193665  2312 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:27.126444  9267 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:27.196867  9261 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.197351  9225 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.218827  9261 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:27.203615  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:27.106635  9275 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:27.211164  1641 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:27.212116  9262 raft_consensus.cc:3055] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:05:27.212237  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.215855  1611 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.217104  1639 raft_consensus.cc:1240] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:27.218712  9273 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.221289  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:27.165518  9030 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:27.251426  1715 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 13:05:27.252138  1715 pending_rounds.cc:85] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 11
I20260709 13:05:27.252586  1715 pending_rounds.cc:107] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 13:05:27.252918  1715 raft_consensus.cc:2889] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:27.219679  1646 raft_consensus.cc:1240] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:05:27.219270  9225 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:27.233105  1611 consensus_peers.cc:597] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:27.254334  1611 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.234161  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:27.235060  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:27.245492  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:27.254810  1611 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:27.256282  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:27.256565  1611 leader_election.cc:336] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.257016  1611 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:27.270357  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 130367us
I20260709 13:05:27.256892  1538 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:27.281610  1717 raft_consensus.cc:1240] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
W20260709 13:05:27.282567  1685 consensus_peers.cc:597] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:27.289088  1537 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:27.289306  1538 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:27.289873  1611 leader_election.cc:336] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:27.291018  1611 consensus_peers.cc:597] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:27.285593  1685 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.294486  1685 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:27.251981  8946 consensus_meta.cc:287] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.502s	user 0.005s	sys 0.009s
I20260709 13:05:27.218725  9227 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:27.246766  9262 raft_consensus.cc:740] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:05:27.303496  1611 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.304136  1611 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:27.304778  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 45403.7us
I20260709 13:05:27.272241  8912 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:05:27.309564  1685 leader_election.cc:336] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.312649  1685 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:27.230819  9176 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.070s
I20260709 13:05:27.216071  9223 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:27.325938  2314 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:27.298203  9234 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.277740  9274 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:27.254805  9275 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:27.299993  8899 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:27.325240  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:27.272753  9243 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:05:27.312805  9239 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:27.310282  9205 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:27.316329  9053 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:27.351619   431 thread.cc:527] Waited for 7000ms trying to join with heartbeat (tid 1598)
I20260709 13:05:27.301234  8975 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:05:27.337893  9031 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:27.304260  8946 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:27.307596  9035 consensus_queue.cc:1048] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:05:27.274178  9273 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:27.354422  9225 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:27.363428  1457 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b82d3b49001741eda872554a7ecb2db3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica f5f4b2707a9647e5bb99e065068dea2b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:27.372442  9223 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:27.323817  8912 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:27.337355  9234 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:27.365039  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 75824.9us
I20260709 13:05:27.309271  9227 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:27.372133  9261 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:27.383411  9226 consensus_meta.cc:287] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504: Time spent flushing consensus metadata: real 0.673s	user 0.015s	sys 0.000s
I20260709 13:05:27.411646  9234 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:27.411948  9226 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:27.412487  9273 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:27.384547  9252 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:27.355320  9262 consensus_queue.cc:260] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:27.413988  9262 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:27.354344  9275 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:27.362965  9205 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:27.419277  8912 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:27.419277  9227 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:27.406306  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 73788.9us
I20260709 13:05:27.374405  8946 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:27.405058  1457 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 54972.1us
I20260709 13:05:27.379599  8975 raft_consensus.cc:493] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.348600  9216 consensus_queue.cc:1048] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.044s
W20260709 13:05:27.429406  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.431877  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 106613us
I20260709 13:05:27.437970  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:27.441078  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:27.443145  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:27.453735  9234 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.454423  9234 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:27.433861  8975 raft_consensus.cc:515] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:27.456753  9234 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:27.456851  8975 leader_election.cc:290] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:27.470957  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 171178us
W20260709 13:05:27.471400  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 161837us
I20260709 13:05:27.473242  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:27.474025  1713 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 17.
I20260709 13:05:27.475132  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:27.475894  1718 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:27.476263  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:27.477487  1685 consensus_peers.cc:597] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:27.478122  1685 leader_election.cc:336] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.478310  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:27.478505  1685 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:27.480916  1537 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:27.482569  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:27.483268  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 179846us
W20260709 13:05:27.484452  1611 consensus_peers.cc:597] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:27.483541  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:27.486110  1715 raft_consensus.cc:2468] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 25.
W20260709 13:05:27.475400  9280 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.127s	user 0.000s	sys 0.002s
W20260709 13:05:27.485400  1611 leader_election.cc:336] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.491462  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:27.488394  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:27.490451  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:27.491623  1611 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:27.488179  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:27.471035  9172 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:27.435364  9223 consensus_queue.cc:1059] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:05:27.502264  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:27.519740  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:27.485880  9274 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:27.487031  8975 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:05:27.519320  1611 leader_election.cc:336] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.475164  8826 consensus_queue.cc:1048] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:05:27.497593  9236 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:27.525612  1611 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:27.479640  9167 consensus_queue.cc:1048] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:05:27.522641  9033 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:27.526062  8975 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.526407  9294 raft_consensus.cc:2804] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:05:27.579633  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:27.574738  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:27.575484  1686 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.578639  1718 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 21.
I20260709 13:05:27.524959  9225 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:27.551506  9255 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:27.546355  9227 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:27.591285  1537 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:27.612975  1718 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 13:05:27.579820  9294 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.613430  1718 pending_rounds.cc:85] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 11
I20260709 13:05:27.580307  8975 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:27.630928  9087 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:27.624972  9225 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:27.637393  9225 raft_consensus.cc:697] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 21 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:27.618052  1718 pending_rounds.cc:107] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:05:27.610736  9239 raft_consensus.cc:2955] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 21 LEADER]: Committing config change with OpId 21.16: config changed from index -1 to 16, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:05:27.618214  9294 raft_consensus.cc:3060] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:05:27.649467  1718 raft_consensus.cc:2889] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:27.661240  9289 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.287s	user 0.004s	sys 0.000s
W20260709 13:05:27.675940  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 370495us
I20260709 13:05:27.683995  9033 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:27.695187  9225 consensus_queue.cc:237] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:27.693107  8966 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:27.693414  9287 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.318s	user 0.002s	sys 0.002s
I20260709 13:05:27.697968  8966 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:27.700676  9285 log.cc:927] Time spent T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.329s	user 0.002s	sys 0.000s
I20260709 13:05:27.728367  1722 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 20 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
W20260709 13:05:27.729821  2312 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.715567  8966 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:27.750329  8946 raft_consensus.cc:3055] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:27.750955  8946 raft_consensus.cc:740] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:27.752159  8946 consensus_queue.cc:260] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:27.754742  9306 ts_tablet_manager.cc:933] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:27.776708  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 297378us
W20260709 13:05:27.778172  1685 consensus_peers.cc:597] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:27.778620  1717 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:05:27.779387  1685 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.780052  1685 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:27.794375  9031 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.791213  9262 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.809845  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:27.796936  8946 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:27.813108  1685 consensus_peers.cc:597] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:27.815115  1685 consensus_peers.cc:597] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:27.817001  1685 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:27.818084  1685 leader_election.cc:336] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.801718  8941 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:05:27.813570  9014 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:27.801266  8975 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:27.824169  8975 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:27.819752  9031 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:27.782202  9267 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:05:27.818475  1685 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:27.838910  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:05:27.839126  1685 leader_election.cc:336] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.839636  1685 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:27.805989  9290 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:05:27.839646  1718 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:27.842338  9309 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:05:27.837852  9030 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:27.844280  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 324971us
W20260709 13:05:27.845145  1611 consensus_peers.cc:597] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:27.845700  1611 leader_election.cc:336] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.846050  1611 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:27.847098  1611 consensus_peers.cc:597] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:27.847581  1611 consensus_peers.cc:597] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:27.848074  1611 consensus_peers.cc:597] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:27.848750  1611 consensus_peers.cc:597] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:27.848678  9031 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:27.849484  9275 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:27.850232  9275 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:05:27.851063  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 274514us
W20260709 13:05:27.853338  9305 log.cc:927] Time spent T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.096s	user 0.003s	sys 0.000s
I20260709 13:05:27.862920  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:27.865607  1611 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.866262  1538 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:27.867977  1611 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:27.875765  1685 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.877967  1685 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:27.877038  1718 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:27.827855  9262 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:27.879539  1537 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:27.857189  9290 consensus_queue.cc:1243] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer 130bc5ff39b847b1b3204626f284f838 log is divergent from this leader: its last log entry 20.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:05:27.876369  9304 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:27.882663  9259 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:27.898649  8975 consensus_queue.cc:1059] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:27.901222  9205 raft_consensus.cc:3055] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:27.907913  9066 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:27.910919  8751 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:27.959024  1641 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:05:27.960506  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:27.961148  2312 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.907064  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:27.964047  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:27.964969  1644 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:27.911823  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:27.967629  9234 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.968377  9234 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:27.912524  9262 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:27.903200  8941 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:27.970888  8941 raft_consensus.cc:697] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:27.972131  8941 consensus_queue.cc:237] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:27.972795  9234 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:27.910599  9261 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.918783  9294 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:27.908034  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:27.974293  1640 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:27.974027  9261 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:27.976536  9261 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:27.933113  2349 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:05:27.936226  1721 pending_rounds.cc:85] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838: Aborting all ops after (but not including) 8
I20260709 13:05:27.937963  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:27.977845  1721 pending_rounds.cc:107] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838: Aborting uncommitted NO_OP operation due to leader change: 20.9
I20260709 13:05:27.978137  1721 raft_consensus.cc:2889] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:27.978698  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:27.938870  1639 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 13:05:27.981757  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:27.981995  9294 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 27 election: Requested vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:27.982582  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:27.982810  1639 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:05:27.984073  1639 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:05:27.938928  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f06439379e24c5ebedb229b007236c2" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:27.985886  9262 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:27.985507  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:05:27.969233  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 25 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:27.989337  1647 raft_consensus.cc:2410] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 for term 25 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 13:05:27.939996  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:27.943992  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 68249.6us
I20260709 13:05:27.991570  1646 raft_consensus.cc:2955] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Committing config change with OpId 21.16: config changed from index -1 to 16, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
W20260709 13:05:27.947495  1611 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.993960  1611 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:27.994987  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 47532.8us
I20260709 13:05:27.895195  9239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:57182 (request call id 0) took 1422 ms. Trace:
I20260709 13:05:27.997890  9172 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:27.997421  9239 rpcz_store.cc:276] 0709 13:05:26.479222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:26.479391 (+   169us) service_pool.cc:224] Handling call
0709 13:05:26.510249 (+ 30858us) log.cc:844] Serialized 506 byte log entry
0709 13:05:26.761851 (+251602us) spinlock_profiling.cc:243] Waited 8.55 ms on lock 0x7b4c00014900. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8247151a20 00007f8247162a29 00007f824716313f 00007f82471272ab 00007f8247154f3c 00007f82471553fe 00007f82473e250b 00007f8248de1381 00007f8248ddf26b 00007f8248de5d53 00007f8248de5cd9
0709 13:05:26.768312 (+  6461us) spinlock_profiling.cc:243] Waited 8.44 ms on lock 0x7b540011aba0. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248dee571 00007f8248e05b26 00007f8248dff49a 00007f8248dfe714 00007f8248ddeeaa 00007f8248de5d53 00007f8248de5cd9 00007f8248de5c69 00007f8248de5c31 00007f8248de4dfd 000000000038f354
0709 13:05:27.893025 (+1124713us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22357,"raft.queue_time_us":84828,"raft.run_cpu_time_us":8461,"raft.run_wall_time_us":350868,"spinlock_wait_cycles":39071232,"thread_start_us":99787,"threads_started":1,"wal-append.queue_time_us":228802}
I20260709 13:05:28.002094  1538 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:28.003156  9294 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:05:28.004211  1538 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:28.004876  9294 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:27.986003  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:28.005880  9294 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:28.009706  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:28.011885  9262 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:05:27.894351  9306 tablet_copy_client.cc:323] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:34553
I20260709 13:05:28.017272  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:28.013586  9066 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:28.017877  9246 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:28.020460  9066 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:28.024158  1537 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:27.941465  9205 raft_consensus.cc:740] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:28.038945  1640 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 21.
I20260709 13:05:28.055756  2314 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:28.058578  1644 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:28.034641  9267 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:28.082314  1538 leader_election.cc:304] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:28.072564  9262 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 13:05:28.086917  9262 consensus_queue.cc:1243] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer f5f4b2707a9647e5bb99e065068dea2b log is divergent from this leader: its last log entry 23.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
W20260709 13:05:28.090461  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 93937.1us
I20260709 13:05:28.095882  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:28.099597  2314 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:28.099988  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:28.043695  9294 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:05:28.097591  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:28.094682  1519 auto_rebalancer.cc:629] Failed to schedule move for tablet 418152c2665a4b04be8a39c62a52f6ca: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:28.097316  9205 consensus_queue.cc:260] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:28.116529  9205 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:28.119807  1519 auto_rebalancer.cc:629] Failed to schedule move for tablet 7e9b1a584a5f430a9a0e7e617064bc2c: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
W20260709 13:05:28.122649  9316 log.cc:927] Time spent T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:05:28.096617  9234 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:05:28.128123  1732 tablet_copy_service.cc:140] P 130bc5ff39b847b1b3204626f284f838: Received BeginTabletCopySession request for tablet ca2582f1941a4c588f8ef51c03dfb492 from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:34078)
I20260709 13:05:28.129014  1732 tablet_copy_service.cc:161] P 130bc5ff39b847b1b3204626f284f838: Beginning new tablet copy session on tablet ca2582f1941a4c588f8ef51c03dfb492 from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:34078: session id = cf2f71592be8465fb18e36e15d0b2f89-ca2582f1941a4c588f8ef51c03dfb492
I20260709 13:05:28.095914  9257 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:28.103871  9263 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.135139  9263 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:28.145856  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 63692.6us
I20260709 13:05:28.146690  1538 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:28.146548  2312 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:28.145911  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 153562us
W20260709 13:05:28.151983  1685 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:28.152343  1685 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:28.163916  9278 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:28.155438  8946 raft_consensus.cc:2749] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:28.155323  9031 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:05:28.228459  9321 log.cc:927] Time spent T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:05:28.241302  1732 tablet_copy_source_session.cc:215] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:05:28.247300  9311 log.cc:927] Time spent T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 13:05:28.189311  9263 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:28.291383  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 138133us
W20260709 13:05:28.294617  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 139489us
W20260709 13:05:28.352444   431 thread.cc:527] Waited for 8000ms trying to join with heartbeat (tid 1598)
W20260709 13:05:28.363322  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 204902us
W20260709 13:05:28.371157  1519 auto_rebalancer.cc:629] Failed to schedule move for tablet baa23f35bd3b4c2d99e3da0ff0f80e97: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:05:28.443481  9318 log.cc:927] Time spent T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:05:28.459511  9306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca2582f1941a4c588f8ef51c03dfb492. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:28.496938  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 350130us
I20260709 13:05:28.512352  9263 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:28.516947  1611 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:28.535274  1611 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:28.571817  9205 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.573062  9205 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:28.577184  9205 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:28.625409  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 108705us
I20260709 13:05:28.625506  9066 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:28.609086  9205 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.641080  9205 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:28.643249  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 127219us
I20260709 13:05:28.644191  9205 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:28.648862  1538 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:28.651844  9228 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:28.636345  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 430884us
I20260709 13:05:28.679682  1538 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:28.681053  9228 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:28.681931  9205 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.683081  9205 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:28.689779  9205 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:28.730470  8826 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.730695  9172 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.737986  9035 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:28.731575  8826 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:28.739198  9035 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:28.731614  9172 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:28.741377  8826 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:28.732376  9184 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.748205  9172 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:28.748014  9184 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:28.746734  9205 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.750010  9205 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:28.750627  9184 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:28.752876  9205 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:28.756678  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 77297.3us
I20260709 13:05:28.757062  9306 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:55888 (request call id 925) took 1181 ms. Trace:
W20260709 13:05:28.782613  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 73876.6us
I20260709 13:05:28.787525  9304 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:28.794643  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:28.797030  9238 raft_consensus.cc:493] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:28.798072  9239 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:28.798970  9239 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:28.799773  1713 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:05:28.801362  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:28.802103  1717 raft_consensus.cc:2468] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:05:28.802644  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:28.798159  9238 raft_consensus.cc:515] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:28.780004  9205 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.812471  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:28.817471  9205 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:28.820428  9205 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:28.811533  9239 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:28.838266  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:28.841210  1648 pending_rounds.cc:85] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b: Aborting all ops after (but not including) 14
I20260709 13:05:28.844898  1640 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:05:28.846195  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:28.846644  9238 leader_election.cc:290] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:28.788720  9304 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:28.827214  9035 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:28.859895  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:28.856868  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:28.850211  1648 pending_rounds.cc:107] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b: Aborting uncommitted NO_OP operation due to leader change: 23.15
I20260709 13:05:28.873831  9228 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.874816  9228 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:28.878094  9228 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:28.867748  1648 raft_consensus.cc:2889] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:28.881872  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:28.764642  9306 rpcz_store.cc:276] 0709 13:05:27.578649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.579179 (+   530us) service_pool.cc:224] Handling call
0709 13:05:27.755553 (+176374us) ts_tablet_manager.cc:934] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
0709 13:05:28.416217 (+660664us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:05:28.576711 (+160494us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:28.576834 (+   123us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:05:28.755359 (+178525us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:05:28.756038 (+   679us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:28.756600 (+   562us) ts_tablet_manager.cc:989] Replica registered.
0709 13:05:28.756911 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":54244,"tablet-copy.queue_time_us":157287,"thread_start_us":195185,"threads_started":2}
I20260709 13:05:28.893438  8946 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.893442  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:28.894124  9306 tablet_copy_client.cc:806] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:28.895049  9306 tablet_copy_client.cc:670] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:28.894687  8946 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:28.897284  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:28.903861  1611 consensus_peers.cc:597] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:28.909266  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:28.915350  8946 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:28.920042  9278 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:28.925019  9288 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.925808  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59ca0c8e4c0491480502ebf944ba503" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:28.926151  9288 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:28.927749  9278 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:28.932353  9278 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:28.937355  1611 consensus_peers.cc:597] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:28.941604  1611 consensus_peers.cc:597] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:28.935856  9288 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:28.941000  8946 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:29.035743  9344 log.cc:927] Time spent T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 13:05:29.036566  8946 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:29.039615  9342 log.cc:927] Time spent T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
W20260709 13:05:29.169251  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 195103us
I20260709 13:05:29.163230  9263 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.179309  9035 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.202853  9263 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:29.230527  9347 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:05:29.206616  9205 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.238521  9205 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.240101  9263 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:29.241225  9205 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:29.211711  9304 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:29.266947  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44424 (request call id 861) took 1321 ms. Trace:
I20260709 13:05:29.267555  1641 rpcz_store.cc:276] 0709 13:05:27.944938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.014749 (+ 69811us) service_pool.cc:224] Handling call
0709 13:05:28.676673 (+661924us) spinlock_profiling.cc:243] Waited 9.03 ms on lock 0x7b5800003098. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d60a22 00007f824ec79c09 00007f824ddd3ee0 00007f824de9d0ab 00007f824de9d131 00007f824de5efc0 00007f824de49929 00007f824de493d2 00007f82473c0bd4
0709 13:05:28.677426 (+   753us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:28.678888 (+  1462us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:28.679465 (+   577us) log.cc:844] Serialized 14 byte log entry
0709 13:05:28.745212 (+ 65747us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:28.768036 (+ 22824us) log.cc:844] Serialized 26 byte log entry
0709 13:05:28.768276 (+   240us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:28.768309 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:28.768399 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:29.261152 (+492753us) raft_consensus.cc:1685] finished
0709 13:05:29.266482 (+  5330us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:29.266710 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1075,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":45,"raft.queue_time_us":4303,"raft.run_cpu_time_us":357,"raft.run_wall_time_us":354,"spinlock_wait_cycles":20772608,"thread_start_us":65143,"threads_started":1,"wal-append.queue_time_us":143502}
I20260709 13:05:29.245814  9290 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:29.287168  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 45617.5us
I20260709 13:05:29.325547  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:29.333801  9290 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.346302  9306 tablet_copy_client.cc:538] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:05:29.353302   431 thread.cc:527] Waited for 9000ms trying to join with heartbeat (tid 1598)
I20260709 13:05:29.360492  9302 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:29.361191  9306 tablet_bootstrap.cc:492] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:05:29.321285  9308 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:29.288201  8946 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:29.288859  9263 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:29.365612  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 396919us
I20260709 13:05:29.361902  9035 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.383991  9227 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:29.384431  1611 leader_election.cc:336] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.384934  1611 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:29.384733  9227 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.387126  9227 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:29.387755  9227 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.388507  9227 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.389950  9066 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:29.393362  9227 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:29.395862  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 98060.5us
I20260709 13:05:29.361490  9302 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.408224  9087 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:29.408603  9227 raft_consensus.cc:493] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.409024  9087 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.409368  9227 raft_consensus.cc:515] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.412487  9227 leader_election.cc:290] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:29.425800  9290 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:29.425941  9087 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:29.426743  9263 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.403453  9308 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.447335  9246 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:05:29.466756  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:29.510898  9263 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:29.534379  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 148004us
I20260709 13:05:29.463020  9227 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 13:05:29.504758  9066 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:29.503906  9087 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:29.480401  9246 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.540879  9290 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.564393  9066 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.565059  9290 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.568146  9290 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:29.571877  9352 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:05:29.575560  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:29.581884  1687 leader_election.cc:304] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:29.582952  9308 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:29.583767  9308 raft_consensus.cc:2749] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:29.584614  9087 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.598104  8751 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:29.599184  8751 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.602094  9353 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:29.602862  9353 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.595105  9288 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.606220  9172 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:29.607174  9302 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:29.607000  9172 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.606269  9288 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:29.608475  9035 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:29.610481  9246 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:29.613381  9288 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:29.607427  8751 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:29.629992  9087 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:05:29.635048  9066 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:29.609537  9172 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:29.624457  2312 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:29.658788  8751 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:29.670753  8751 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.675964  1537 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:29.692051  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.694798  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.695386  9288 raft_consensus.cc:2804] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:29.696033  9288 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.687659  8751 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:29.696520  9288 raft_consensus.cc:3060] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:29.696837  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.698292  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.684320  9308 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.700184  9308 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.702216  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.695832  1647 raft_consensus.cc:2393] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:29.711246  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.712150  1643 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 21.
I20260709 13:05:29.679656  9302 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:29.687696  9087 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:29.713001  9302 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.713325  9087 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.715436  9087 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:29.655607  9035 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.716408  9035 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.718283  9353 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:29.719101  9035 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:29.693362  9290 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:05:29.719458  1642 raft_consensus.cc:2393] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 20: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:29.720068  9308 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:29.588090  9352 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:29.721289  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.715493  9302 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:29.673041  9227 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.724035  9227 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:29.710901  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 55339.6us
W20260709 13:05:29.738212  1611 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.740298  1611 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:29.746204  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:29.748270  1611 leader_election.cc:336] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.748641  1645 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:05:29.748983  1611 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:29.749311  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:29.750557  1611 leader_election.cc:336] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.746325  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.751474  1611 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:29.750993  9035 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:29.750608  8751 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:29.753403  8826 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:29.755035  1611 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.755904  1611 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:29.756821  8826 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:29.761703  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:29.767529  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:29.772357  2312 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.774061  1714 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 23.
I20260709 13:05:29.779685  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.784116  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.785249  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.782094  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:29.761005  9302 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:29.796468  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.797433  1645 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:05:29.798190  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.799903  1646 raft_consensus.cc:2410] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 19 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:05:29.792495  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:29.808729  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 132886us
I20260709 13:05:29.809006  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:29.816411  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.782624  9308 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.814755  1714 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 20.
W20260709 13:05:29.822467  1611 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.793747  9290 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
W20260709 13:05:29.829170  2312 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:29.773458  9352 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:29.833009  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 43953.1us
I20260709 13:05:29.834971  1687 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:29.835832  9246 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:29.836367  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.836431  9246 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:29.836755  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.836756  1687 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:29.837431  9246 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:29.838958  2313 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:29.835790  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.842396  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.842974  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.843066  1645 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:05:29.843411  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.843921  1640 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:29.844631  9349 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:29.845427  9349 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:29.848251  9349 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:29.851783  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.858275  1537 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:29.865847  1537 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:29.875582  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:29.884621  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:29.893461  9239 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:29.907153  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:29.880345  9228 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:29.857439  9302 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:29.923142  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:29.925117  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:29.906769  9334 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:29.909003  9239 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:29.936857  9348 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:29.942709  1720 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:05:29.943221  2342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:29.945129  9302 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:29.897177  9308 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.946025  9239 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:29.947944  2340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:29.948874  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:29.953958  1611 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.893568  9290 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:29.966790  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.964054  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:29.974331  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:29.976269  9308 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:29.949340  9246 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:29.979203  9246 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:29.980509  1713 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 18.
W20260709 13:05:29.982226  9358 log.cc:927] Time spent T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:05:29.984760  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:29.985554  9290 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:29.986248  9228 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.986919  9228 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:29.987317  9227 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:29.995474  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:29.997499  9228 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:29.989379  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:29.998992  1642 raft_consensus.cc:2410] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
W20260709 13:05:29.999528  9371 log.cc:927] Time spent T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.124s	user 0.004s	sys 0.000s
W20260709 13:05:30.003160  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 207554us
W20260709 13:05:30.012378  1611 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:30.017774  2312 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:30.018736  1463 catalog_manager.cc:5697] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 15 to 16, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 16 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:30.021678  2347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:30.024712  9348 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:30.024992  9228 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.025517  9348 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:30.025731  9228 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.027966  9348 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:30.028331  9228 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:30.033512  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 112040us
I20260709 13:05:30.034484  1612 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:30.035565  9355 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:30.036056  9355 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:30.035828  1612 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:30.036856  9355 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:05:30.038321  2312 consensus_peers.cc:597] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:30.041898  1721 raft_consensus.cc:1240] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:30.042234  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:30.043185  8751 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:30.043649  8751 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:30.044052  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:30.044348  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:30.044059  8751 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:30.046136  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:30.047627  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:30.049702  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:30.052153  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:30.053365  1713 raft_consensus.cc:2410] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:05:30.054469  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:30.055316  1719 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:30.058272  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82cbfb23571048409c1bde90d4611c1a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:30.062263  1463 catalog_manager.cc:5697] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 6 to 10, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 10 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:30.075071  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:30.076532  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:30.077361  1719 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:30.068941  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:30.070786  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:30.066910  9288 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.081768  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:30.091193  1686 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:30.093152  9378 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:30.099277  9348 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:30.101009  9263 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.097404  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:30.101711  9263 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.104537  9263 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:30.113293  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:30.117600  1464 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38666 (request call id 218) took 29428 ms (client timeout 15000 ms). Trace:
W20260709 13:05:30.126590  1611 leader_election.cc:336] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:30.127417  1611 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:30.128589  1611 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:30.130046  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 192541us
I20260709 13:05:30.132261  9378 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:30.132855  9378 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.134408  1537 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:30.143481  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:30.144394  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:30.145061  1715 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
I20260709 13:05:30.145183  9349 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:30.137693  9239 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.146930  1713 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:30.137457  9227 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.148085  9227 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.146800  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:30.139549  1538 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:30.153026  1537 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:30.154673  1538 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:30.137082  9228 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.158152  9228 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:30.109500  8751 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.161734  9348 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:30.162213  9348 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.162598  9348 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:30.167110  9349 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:30.170189  1538 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:30.171535  9362 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:30.172276  9362 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:30.172633  9362 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.172974  9362 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:30.173964  9288 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:30.176661  9239 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:30.139603  1463 catalog_manager.cc:5697] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 12 to 16, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 16 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:30.150522  9227 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:30.184295  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:30.188100  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:30.189045  8751 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:30.216369  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 74105us
I20260709 13:05:30.219859  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:30.180474  1463 catalog_manager.cc:5697] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 8 to 14, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 14 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:30.161370  9228 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:30.227560  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:30.223320  1537 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:30.228930  1713 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 18.
I20260709 13:05:30.233572  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:30.237870  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:30.238755  9378 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:30.092909  9355 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.256107  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:30.257216  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:30.125350  1464 rpcz_store.cc:269] 0709 13:05:00.688587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.525892 (+13837305us) service_pool.cc:224] Handling call
0709 13:05:30.117245 (+15591353us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:26.653295 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:26.803510 (+150215us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:26.805908 (+  2398us) write_op.cc:435] Acquired schema lock
0709 13:05:26.814816 (+  8908us) tablet.cc:662] Decoding operations
0709 13:05:26.924836 (+110020us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:26.924909 (+    73us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:26.924944 (+    35us) tablet.cc:685] Acquiring locks for 28 operations
0709 13:05:27.016332 (+ 91388us) tablet.cc:701] Row locks acquired
0709 13:05:27.016349 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:05:27.016544 (+   195us) write_op.cc:270] Start()
0709 13:05:27.016795 (+   251us) write_op.cc:276] Timestamp: P: 1783602327016476 usec, L: 0
0709 13:05:27.016815 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:05:27.094596 (+ 77781us) log.cc:844] Serialized 11057 byte log entry
0709 13:05:27.318127 (+223531us) op_driver.cc:464] REPLICATION: finished
0709 13:05:27.551130 (+233003us) write_op.cc:301] APPLY: starting
0709 13:05:27.574033 (+ 22903us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:27.871420 (+297387us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:27.871557 (+   137us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:28.758609 (+887052us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:28.759522 (+   913us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=56,key_file_lookups=56,delta_file_lookups=56,mrs_lookups=0
0709 13:05:28.759579 (+    57us) write_op.cc:312] APPLY: finished
0709 13:05:28.760545 (+   966us) log.cc:844] Serialized 244 byte log entry
0709 13:05:28.780967 (+ 20422us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:28.882992 (+102025us) write_op.cc:454] Released schema lock
0709 13:05:28.890776 (+  7784us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:28.982498 (+ 91722us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:28.984990 (+  2492us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3064768,"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":200141,"apply.run_cpu_time_us":74472,"apply.run_wall_time_us":1408530,"cfile_cache_hit":114,"cfile_cache_hit_bytes":121223,"num_ops":28,"prepare.queue_time_us":298394,"prepare.run_cpu_time_us":22456,"prepare.run_wall_time_us":544311,"raft.queue_time_us":102512,"raft.run_cpu_time_us":21700,"raft.run_wall_time_us":182407,"replication_time_us":278647,"spinlock_wait_cycles":1018112,"thread_start_us":527227,"threads_started":5,"wal-append.queue_time_us":182187,"wal-append.run_cpu_time_us":30512,"wal-append.run_wall_time_us":2315146}]]}
I20260709 13:05:30.263514  1537 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:30.265625  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:30.281203  9349 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:30.286911  9227 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:30.303151  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:30.306429  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:30.309114  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:30.307300  1642 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 19.
I20260709 13:05:30.313102  1537 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:30.312455  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:30.317306  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:30.311142  1612 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:30.317773  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:30.316213  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 80475.8us
I20260709 13:05:30.321297  1643 raft_consensus.cc:1240] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 23. Current term is 24. Ops: []
I20260709 13:05:30.321286  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:30.325507  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:30.332710  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:30.336274  2312 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:30.336472  9035 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:30.336865  9035 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.337234  9035 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:30.338001  1611 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.342018  1612 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:30.343981  1612 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:30.345108  9176 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:30.346271  9380 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:30.345383  1612 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:30.349882  9066 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:30.350414  9066 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.350847  9066 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:30.353873   431 thread.cc:527] Waited for 10000ms trying to join with heartbeat (tid 1598)
I20260709 13:05:30.357466  9355 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:30.326826  9349 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:30.309335  9263 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:30.369081  1685 leader_election.cc:336] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.369654  1685 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:30.370586  9308 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:30.372414  1685 consensus_peers.cc:597] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:30.374542  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:05:30.374859  1611 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.375458  1715 raft_consensus.cc:2393] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 21: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:30.395900  1538 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:30.319114  9227 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 13:05:30.406649  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:30.410166  1538 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:30.417843  9348 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.420100  9348 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:30.383688  9066 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.425411  9066 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:30.428418  1612 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:30.433694  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 107928us
I20260709 13:05:30.409313  9359 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:30.442205  1537 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:30.443306  9359 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.437605  8826 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:30.443723  9349 consensus_queue.cc:1059] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:05:30.444034  9290 consensus_queue.cc:1059] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:30.439762  9348 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:30.445401  9348 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:30.445468  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:30.446333  1721 raft_consensus.cc:3060] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:05:30.447470  9290 raft_consensus.cc:3055] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:05:30.447954  9290 raft_consensus.cc:740] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
W20260709 13:05:30.439749  1685 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.450059  1685 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:30.448961  9290 consensus_queue.cc:260] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.451563  9308 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:30.443873  9359 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:30.454070  9290 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:30.446197  9348 raft_consensus.cc:3055] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:30.456455  9348 raft_consensus.cc:740] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:30.457568  9348 consensus_queue.cc:260] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.477222  1612 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:30.510195  9372 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:05:30.527549  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 51152.3us
I20260709 13:05:30.529017  1721 raft_consensus.cc:2468] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 21.
W20260709 13:05:30.529747  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 52623.1us
I20260709 13:05:30.531687  1612 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:30.532480  8751 raft_consensus.cc:2804] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:05:30.539574  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 98288us
I20260709 13:05:30.537056  9348 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:30.551541  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 109423us
I20260709 13:05:30.551229  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:05:30.555733  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 58b222526a1a4b52bdcda825e15c46f9 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:30.558043  9362 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.558480  9035 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.560295  9362 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:30.560819  9035 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:30.554919  1537 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:30.542505  8751 raft_consensus.cc:697] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 21 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:30.597874  8751 consensus_queue.cc:237] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.616647  9362 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:30.618161  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 63392.2us
I20260709 13:05:30.619233  1537 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:30.605716  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:30.623662  1721 raft_consensus.cc:3060] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:30.626710  1611 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.624549  9359 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.638132  9263 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:30.629225  9359 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:30.667374  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 48346.9us
I20260709 13:05:30.668448  1537 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:30.669945  9362 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:30.670774  1537 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:30.672166  9362 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:30.677275  1611 consensus_peers.cc:597] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:30.680012  1537 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:30.682961  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:05:30.688501  1611 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.685029  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:30.690042  1642 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:30.689317  1646 raft_consensus.cc:2393] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:30.694429  1537 leader_election.cc:304] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:30.704319  1721 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
W20260709 13:05:30.714233  2312 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:30.715350  2312 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:30.717504  9359 raft_consensus.cc:2749] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:30.718720  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:30.722441  1611 consensus_peers.cc:597] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:30.723748  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:30.726397  1646 raft_consensus.cc:1240] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Rejecting Update request from peer 066ee0109aba4465ba17fa65e14ef504 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:30.733045  1642 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:05:30.727741  1612 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:30.732093  9359 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:30.697861  9348 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:30.734205  9263 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:30.740267  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:30.742003  8751 raft_consensus.cc:2804] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:30.747820  8751 raft_consensus.cc:697] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:30.749027  8751 consensus_queue.cc:237] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.742728  9362 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.755797  9362 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.746025  9263 consensus_queue.cc:1059] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 13:05:30.760360  9263 raft_consensus.cc:3055] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:30.760792  9263 raft_consensus.cc:740] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:30.761696  9263 consensus_queue.cc:260] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.762866  9263 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:30.759282  9388 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:30.773666  9362 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:30.776417  1537 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:30.777386  9388 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:30.778587  2312 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:30.778903  1537 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:30.793872  1537 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:30.790254  1538 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:30.794155  9362 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:30.825825  1537 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:30.810240  9290 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:30.859359  9290 raft_consensus.cc:697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 20 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:30.834383  9288 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:30.838400  9388 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:30.861058  9290 consensus_queue.cc:237] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:30.884832  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 84973.3us
I20260709 13:05:30.911509  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:30.926717  9278 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.927445  9278 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.929881  9278 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:30.912897  9306 log.cc:927] Time spent T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.059s	user 0.004s	sys 0.000s
I20260709 13:05:30.943660  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:30.938133  1611 consensus_peers.cc:597] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:30.992659  9288 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.993429  9288 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:30.997593  9288 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:30.999190  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:31.005538  9306 tablet_bootstrap.cc:492] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=16 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:31.006755  9306 tablet_bootstrap.cc:492] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:05:31.007614  9306 ts_tablet_manager.cc:1403] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 1.648s	user 0.071s	sys 0.024s
I20260709 13:05:31.042207  9306 raft_consensus.cc:359] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:05:31.043812  9306 raft_consensus.cc:740] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
W20260709 13:05:31.044661  1611 consensus_peers.cc:597] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:31.046175  1464 catalog_manager.cc:5697] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 14 to 16, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 16 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:31.044548  9306 consensus_queue.cc:260] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:05:31.058604  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
W20260709 13:05:31.051955  1463 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38650 (request call id 200) took 52555 ms (client timeout 15000 ms). Trace:
I20260709 13:05:31.059571  1718 raft_consensus.cc:2393] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 21: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:31.063464  1464 catalog_manager.cc:5697] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:31.065222  1464 catalog_manager.cc:5697] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 16 to 17, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 17 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:31.066823  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:31.079363  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:31.080400  1720 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:05:31.084010  1718 raft_consensus.cc:2393] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 in current term 19: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:31.137948  1714 raft_consensus.cc:1275] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 13:05:31.059805  1463 rpcz_store.cc:269] 0709 13:04:38.496107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.537330 (+ 41223us) service_pool.cc:224] Handling call
0709 13:05:31.051822 (+52514492us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:30.580252 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:30.580895 (+   643us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:30.580957 (+    62us) write_op.cc:435] Acquired schema lock
0709 13:05:30.580978 (+    21us) tablet.cc:662] Decoding operations
0709 13:05:30.581709 (+   731us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:30.581910 (+   201us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:30.581941 (+    31us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:05:30.582543 (+   602us) tablet.cc:701] Row locks acquired
0709 13:05:30.582590 (+    47us) write_op.cc:260] PREPARE: finished
0709 13:05:30.582789 (+   199us) write_op.cc:270] Start()
0709 13:05:30.582978 (+   189us) write_op.cc:276] Timestamp: P: 1783602330582716 usec, L: 0
0709 13:05:30.583000 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:05:30.617581 (+ 34581us) log.cc:844] Serialized 1701 byte log entry
0709 13:05:30.818874 (+201293us) op_driver.cc:464] REPLICATION: finished
0709 13:05:30.859802 (+ 40928us) write_op.cc:301] APPLY: starting
0709 13:05:30.859923 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:30.863519 (+  3596us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:30.863724 (+   205us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:30.939914 (+ 76190us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:30.940308 (+   394us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 13:05:30.940366 (+    58us) write_op.cc:312] APPLY: finished
0709 13:05:30.940606 (+   240us) log.cc:844] Serialized 49 byte log entry
0709 13:05:30.942342 (+  1736us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:30.942445 (+   103us) write_op.cc:454] Released schema lock
0709 13:05:30.942488 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:30.942947 (+   459us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:30.942976 (+    29us) write_op.cc:454] Released schema lock
0709 13:05:30.975451 (+ 32475us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b1800351b80. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f824ec60de1 00007f824ec6d5b1 00007f8245d98522 00007f8245d9a7c1 00007f8245d9a779 00007f8245d9a709 00007f8245d9a6d1 00007f8245d999cd 000000000038f354 000000000038f199 00007f8245d7b83a
Metrics: {"mutex_wait_us":190556,"reported_tablets":54,"child_traces":[["op",{"apply.queue_time_us":40587,"apply.run_cpu_time_us":9215,"apply.run_wall_time_us":83138,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20855,"num_ops":4,"prepare.queue_time_us":258249,"prepare.run_cpu_time_us":6492,"prepare.run_wall_time_us":126647,"raft.queue_time_us":145392,"raft.run_cpu_time_us":8974,"raft.run_wall_time_us":65465,"replication_time_us":235662,"spinlock_wait_cycles":6653184,"thread_start_us":358692,"threads_started":4,"wal-append.queue_time_us":47532}]]}
I20260709 13:05:31.156493  9372 consensus_queue.cc:1048] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:31.071043  1464 catalog_manager.cc:5697] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 14 to 20, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 20 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:31.218488  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:31.216333  1685 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 207 similar messages]
I20260709 13:05:31.221858  1464 catalog_manager.cc:5697] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 18 to 19, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 19 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:31.240873  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:31.244494  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 102158us
I20260709 13:05:31.247324  1464 catalog_manager.cc:5697] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:31.250910  1464 catalog_manager.cc:5697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 12 to 17, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 17 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:31.252642  1464 catalog_manager.cc:5697] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:31.273343  1464 catalog_manager.cc:5697] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 13 to 15, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 15 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:31.311028  1537 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:31.313864  9290 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.314780  9290 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:31.317739  9388 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.318768  9388 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:31.261675  8751 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:31.323536  1464 catalog_manager.cc:5697] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 14 to 20. New cstate: current_term: 20 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:31.325742  1464 catalog_manager.cc:5697] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 20 to 24. New cstate: current_term: 24 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:31.329063  1464 catalog_manager.cc:5697] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 13 to 16, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 16 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:31.282120  9263 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.335008  1464 catalog_manager.cc:5697] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 17 to 23. New cstate: current_term: 23 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:31.339902  1464 catalog_manager.cc:5697] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 17 to 20. New cstate: current_term: 20 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:31.342046  1464 catalog_manager.cc:5697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 16 to 18, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 18 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
W20260709 13:05:31.334815  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 90915.7us
I20260709 13:05:31.344015  1537 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:31.343820  1464 catalog_manager.cc:5697] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:31.344944  9228 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:31.345466  1537 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:31.334143  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 117160us
W20260709 13:05:31.347190  1685 leader_election.cc:336] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:31.350138  9359 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:31.347791  1685 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:31.354568   431 thread.cc:527] Waited for 11000ms trying to join with heartbeat (tid 1598)
I20260709 13:05:31.351284  1464 catalog_manager.cc:5697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 11 to 18, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 18 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:31.354918  1537 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:31.368091  1464 catalog_manager.cc:5697] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 16 to 20, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 20 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:31.372336  1464 catalog_manager.cc:5697] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 13 to 18, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 18 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:31.332639  9227 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:31.385098  1464 catalog_manager.cc:5697] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 16 to 18, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 18 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:31.395588  1464 catalog_manager.cc:5697] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:31.396739  1464 catalog_manager.cc:5697] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 17 to 19, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 19 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:31.391618  2346 raft_consensus.cc:1217] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 LEARNER]: Deduplicated request from leader. Original: 21.15->[21.16-21.16]   Dedup: 21.16->[]
I20260709 13:05:31.382798  9359 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:31.336252  8751 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:31.407294  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 125954us
I20260709 13:05:31.405032  1464 catalog_manager.cc:5697] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 14 to 18, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 18 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:31.382088  9263 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:31.406953  9246 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:05:31.434146  9306 ts_tablet_manager.cc:1434] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.426s	user 0.005s	sys 0.016s
I20260709 13:05:31.440397  8751 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:31.447037  9263 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:31.470768  9388 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:31.480701  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:31.482316  1537 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:31.482687  1713 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:05:31.482944  1720 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 20.
I20260709 13:05:31.475769  1464 catalog_manager.cc:5697] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 17 to 19, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 19 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:31.489216  9263 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:31.489467  1464 catalog_manager.cc:5697] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:31.495430  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:31.503700  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:31.504205  1716 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:31.500859  9290 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:31.530448  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:31.531888  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:31.538471  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:31.539278  1720 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 22: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:31.545994  9401 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:31.556087  9401 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:31.564694  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:31.545845  9288 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:05:31.566545  1538 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:31.567916  9388 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:31.569705  1538 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:31.571022  9388 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:31.580185  1648 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:31.583297  1538 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:31.585377  1611 consensus_peers.cc:597] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:31.594312  1611 consensus_peers.cc:597] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:31.609932  9388 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:31.606029  8751 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.032s
W20260709 13:05:31.636477  1611 consensus_peers.cc:597] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:31.622319  1732 tablet_copy_service.cc:342] P 130bc5ff39b847b1b3204626f284f838: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-ca2582f1941a4c588f8ef51c03dfb492 received from {username='slave'} at 127.0.0.1:34078
I20260709 13:05:31.647168  1732 tablet_copy_service.cc:434] P 130bc5ff39b847b1b3204626f284f838: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-ca2582f1941a4c588f8ef51c03dfb492 on tablet ca2582f1941a4c588f8ef51c03dfb492 with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:31.617934  9401 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:31.622429  9415 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.030s
W20260709 13:05:31.672753  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 325574us
W20260709 13:05:31.692071  1685 leader_election.cc:336] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:31.698635  1685 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:31.700857  1685 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:31.701671  1685 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:31.707147  1685 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:31.699860  9422 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:31.708590  9302 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:31.716037  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:31.709765  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:31.721518  1685 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:31.724473  9246 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:31.724602  1685 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:31.725520  1685 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:31.731739  1685 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:31.723477  1611 consensus_peers.cc:597] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:31.723352  1640 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 24: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
W20260709 13:05:31.732918  1611 consensus_peers.cc:597] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:31.740602  9422 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:31.742656  1685 leader_election.cc:336] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:31.749748  1685 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:31.753752  1685 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:31.754384  1685 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:31.749130  2312 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:31.757189  1611 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:31.759490  1611 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:31.762359  1685 leader_election.cc:336] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:31.763003  1685 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:31.765473  1685 leader_election.cc:336] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:31.766223  1685 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:31.764309  9422 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:31.781241  1685 leader_election.cc:336] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:31.790824  1685 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:31.779201  1611 consensus_peers.cc:597] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:31.759161  9246 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:31.800384  1685 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:31.802300  1685 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:31.805866  1685 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:31.804944  1611 consensus_peers.cc:597] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:31.890489  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 91808.3us
I20260709 13:05:31.782521  9035 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:31.909467  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 101111us
I20260709 13:05:31.780011  9421 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:31.792887  9278 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:31.805809  9302 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:31.985260  1611 consensus_peers.cc:597] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:31.911646  9308 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:31.990350  1611 consensus_peers.cc:597] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:31.968369  9227 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:32.019610  1685 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:31.997810  1611 consensus_peers.cc:597] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:32.020524  1685 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:32.024767  1685 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:32.025730  1685 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:32.021114  1611 consensus_peers.cc:597] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:32.026814  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 243088us
W20260709 13:05:32.032985  1685 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:32.039646  1685 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:32.049149  1685 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:32.049736  1685 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:32.051793  9422 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:05:32.047374  1611 consensus_peers.cc:597] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:32.048274  9431 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:32.075093  9395 log.cc:927] Time spent T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
W20260709 13:05:32.079546  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 58b222526a1a4b52bdcda825e15c46f9 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 130bc5ff39b847b1b3204626f284f838 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:32.026786  9227 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:32.052563  9308 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:32.061507  9422 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:32.128611  9429 log.cc:927] Time spent T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
W20260709 13:05:32.135231  9428 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.113s	user 0.001s	sys 0.001s
I20260709 13:05:32.140291  9290 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.141359  9290 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:32.144279  9290 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:32.150413  1685 leader_election.cc:336] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:32.150960  1685 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:32.152259  9302 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:32.153916  1685 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:32.158897  9227 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:32.175118  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:32.212809  9362 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.213497  9362 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:32.218142  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:32.221467  1719 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 22.
I20260709 13:05:32.203200  9227 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.224291  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:32.215603  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 177042us
I20260709 13:05:32.228262  9227 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:32.234464  9227 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:32.230409  1687 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:32.246402  9440 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:32.255870  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:32.258976  1537 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:32.260749  1537 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:32.268908  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:32.270035  1720 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 21.
I20260709 13:05:32.277859  1537 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:32.245311  9035 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:32.297910  9227 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:32.298669  9227 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.299304  9227 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:32.260614  9413 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.310756  1640 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 22.
I20260709 13:05:32.275883  9228 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:32.267024  9290 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:05:32.330542  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 52770.6us
W20260709 13:05:32.336082  2312 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:32.338315  9228 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.339267  9228 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:32.330693  9440 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:32.322501  9413 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:32.328157  9290 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.343360  9290 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:32.344000  9413 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:32.347306  9362 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:32.349007  9362 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.349766  9362 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:32.351327  9228 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:32.352898  9362 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:32.354591  9359 raft_consensus.cc:493] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:32.355189   431 thread.cc:527] Waited for 12000ms trying to join with heartbeat (tid 1598)
I20260709 13:05:32.355376  9359 raft_consensus.cc:515] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:32.361498  9359 leader_election.cc:290] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:32.370318  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:32.371315  1714 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 24.
I20260709 13:05:32.371886  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:32.376549  1648 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 21.
W20260709 13:05:32.380661  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 63834.6us
I20260709 13:05:32.385427  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:32.386915  1648 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 24.
I20260709 13:05:32.388144  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:32.397199  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:32.396834  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:32.399606  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:32.410342  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:32.415146  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27aa13d58e6d4987b2e34613996ef421" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:32.419518  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 122104us
I20260709 13:05:32.436723  1537 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
W20260709 13:05:32.439021  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 101881us
I20260709 13:05:32.448838  9288 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:05:32.449450  9288 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.449885  9288 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:32.458894  9439 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.459755  9439 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:32.476306  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:32.477849  1537 leader_election.cc:304] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:32.493948  9415 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.504261  9447 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.505923  9451 raft_consensus.cc:2749] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:05:32.505718  9447 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:32.506294  9227 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:32.504364  9415 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:32.508870  9227 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:32.506968  9439 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:32.514225  9415 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:32.508750  9290 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:32.519172  9447 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:32.520211  9290 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:32.525903  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:32.529436  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:32.530644  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:32.531246  1714 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:32.531777  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:32.534107  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:32.538617  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:32.539283  1716 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:32.545465  1538 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:32.551046  1538 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:32.557425  9362 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:32.550225  9447 raft_consensus.cc:2749] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:32.580098  1611 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:32.588559  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:32.590353  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:32.591647  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:32.593372  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:32.594781  1642 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:32.601169  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:32.602273  1647 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:05:32.604172  1611 consensus_peers.cc:597] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:32.615635  1642 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 22.
I20260709 13:05:32.626777  1538 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:32.610682  9227 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:32.630092  1538 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:32.630133  1716 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 23.
I20260709 13:05:32.640021  9288 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:32.642854  9288 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:32.632462  9447 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:32.646791  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:32.647889  9415 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:32.649554  1538 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:32.673038  9290 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:32.768729  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:32.770390  1720 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:05:32.779023  2312 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:32.768599  1537 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:32.783721  1647 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 23.
W20260709 13:05:32.780625  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 107779us
I20260709 13:05:32.800377  9415 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.801195  9415 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:32.809253  9415 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:32.792793  9288 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:32.786486  9290 raft_consensus.cc:697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 22 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:32.842633  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:32.831307  9288 raft_consensus.cc:697] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 23 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:32.835992  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:32.851155  1640 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:32.819205  1714 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 22.
I20260709 13:05:32.847167  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:32.870258  1640 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 25.
I20260709 13:05:32.850020  9447 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.851305  9359 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.885394  9290 consensus_queue.cc:237] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:32.892177  9278 raft_consensus.cc:1064] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:05:32.881575  9447 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:32.903383  9359 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:32.926450  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:32.930819  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:32.941327  1720 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 066ee0109aba4465ba17fa65e14ef504 in term 25.
W20260709 13:05:32.943406  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 63867us
W20260709 13:05:32.948145  2312 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:32.901340  9447 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:32.959707  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 64432.1us
I20260709 13:05:32.950383  9359 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:32.979414  1537 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838; no voters: 
I20260709 13:05:32.983901  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:32.992969  9447 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.993716  9447 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:32.996208  9447 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:32.989327  9359 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:05:32.954809  9288 consensus_queue.cc:237] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:33.028664  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 83120.5us
I20260709 13:05:33.030946  1538 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:33.032639  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:33.035462  1537 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:33.042815  9278 consensus_queue.cc:237] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 14, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:33.036101  9359 raft_consensus.cc:697] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 25 LEADER]: Becoming Leader. State: Replica: 066ee0109aba4465ba17fa65e14ef504, State: Running, Role: LEADER
I20260709 13:05:33.046682  9359 consensus_queue.cc:237] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:33.048638  9456 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: Append to log took a long time: real 0.080s	user 0.002s	sys 0.000s
I20260709 13:05:33.058034  9227 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:33.098503  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 67663.8us
I20260709 13:05:33.113045  9228 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.114313  9228 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:33.120170  9415 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:33.085597  9227 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.125572  9227 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:33.155361  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:33.163754  9462 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.164423  9462 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:33.117537  9228 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:33.168385  9462 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:33.168690  1611 consensus_peers.cc:597] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:33.183424  9227 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:33.184008  2312 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:33.192503  2312 consensus_peers.cc:597] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:33.196890  1464 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38666 (request call id 219) took 13301 ms (client timeout 15000 ms). Trace:
W20260709 13:05:33.197422  1464 rpcz_store.cc:269] 0709 13:05:19.895427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.275803 (+10380376us) service_pool.cc:224] Handling call
0709 13:05:33.196771 (+2920968us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:31.658223 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:31.658855 (+   632us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:31.658891 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:05:31.658914 (+    23us) tablet.cc:662] Decoding operations
0709 13:05:31.727588 (+ 68674us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:31.727666 (+    78us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:31.727699 (+    33us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:05:31.888636 (+160937us) tablet.cc:701] Row locks acquired
0709 13:05:31.888665 (+    29us) write_op.cc:260] PREPARE: finished
0709 13:05:31.888868 (+   203us) write_op.cc:270] Start()
0709 13:05:31.889057 (+   189us) write_op.cc:276] Timestamp: P: 1783602331888806 usec, L: 0
0709 13:05:31.889087 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:05:31.890111 (+  1024us) log.cc:844] Serialized 9930 byte log entry
0709 13:05:32.321691 (+431580us) op_driver.cc:464] REPLICATION: finished
0709 13:05:32.520382 (+198691us) write_op.cc:301] APPLY: starting
0709 13:05:32.520473 (+    91us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:32.873328 (+352855us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:32.877728 (+  4400us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:33.048618 (+170890us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:33.049082 (+   464us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=50,mrs_lookups=0
0709 13:05:33.049142 (+    60us) write_op.cc:312] APPLY: finished
0709 13:05:33.065085 (+ 15943us) log.cc:844] Serialized 220 byte log entry
0709 13:05:33.066759 (+  1674us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:33.067127 (+   368us) write_op.cc:454] Released schema lock
0709 13:05:33.067244 (+   117us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:33.067833 (+   589us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:33.067868 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4616,"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":198316,"apply.run_cpu_time_us":45095,"apply.run_wall_time_us":547401,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108677,"num_ops":25,"prepare.queue_time_us":89800,"prepare.run_cpu_time_us":8576,"prepare.run_wall_time_us":232538,"replication_time_us":396107,"thread_start_us":158714,"threads_started":2}]]}
W20260709 13:05:33.214226  1611 consensus_peers.cc:597] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:33.219561  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 244455us
I20260709 13:05:33.230068  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:33.230423  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:33.233290  1612 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:33.238623  2343 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 LEARNER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:05:33.244261  9373 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:33.250679  9359 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.251398  9359 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:33.255250  1685 consensus_peers.cc:597] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:33.262063  9227 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.263062  9227 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:33.267468  9227 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:33.253722  9359 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:33.283063  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:33.284723  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:33.287432  1642 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:05:33.287632  9448 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:33.288264  9448 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:33.291218  9448 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:33.293252  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:33.295554  9421 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:33.282801  9401 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:05:33.302662  9449 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:05:33.308460  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:33.310676  1687 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:33.311921  9401 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:33.296305  9421 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:33.324853  1685 consensus_peers.cc:597] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:33.345934  1718 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:05:33.346123  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:33.339691  9421 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:33.355665   431 thread.cc:527] Waited for 13000ms trying to join with heartbeat (tid 1598)
I20260709 13:05:33.357535  9227 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:33.392414  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5c79254e6ff49639cf09f663490dfd7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:33.394199  1685 leader_election.cc:336] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:33.400002  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:33.396183  1685 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:33.411163  9421 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:33.421082  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:33.438380  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:33.440687  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:33.441560  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d17fad9f114b0e98d2af0a989c3185" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:33.441879  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:33.446228  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:33.446576  1645 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 13:05:33.456238  1685 consensus_peers.cc:597] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:33.457293  1685 consensus_peers.cc:597] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:33.467192  1538 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:33.469996  9359 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:33.472756  9401 raft_consensus.cc:2955] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 21 LEADER]: Committing config change with OpId 21.17: config changed from index 16 to 17, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:33.476912  2346 raft_consensus.cc:2955] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Committing config change with OpId 21.17: config changed from index 16 to 17, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:05:33.486904  1685 consensus_peers.cc:597] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:33.512020  1720 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 13:05:33.515470  9462 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:05:33.518728  1685 consensus_peers.cc:597] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:33.520498  2312 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:33.523976  1611 consensus_peers.cc:597] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:33.528546  1538 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:33.531507  1538 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:33.531944  9439 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:33.540717  1538 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:33.539628  9439 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:33.591410  1538 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:33.601406  1538 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:33.598354  9461 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 13:05:33.555459  9462 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:33.608084  9290 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:33.618238  9227 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:33.620574  1685 consensus_peers.cc:597] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:33.641372  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 84436.3us
W20260709 13:05:33.642356  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 96059.4us
I20260709 13:05:33.610041  1538 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:33.645406  9263 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:33.672278  1646 raft_consensus.cc:2955] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Committing config change with OpId 21.17: config changed from index 16 to 17, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:33.649897  9227 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.650213  9439 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:33.684639  1611 consensus_peers.cc:597] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:33.702385  9439 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:33.731935  9359 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.705883  9227 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:33.779481  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 132662us
W20260709 13:05:33.806798  9471 log.cc:927] Time spent T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:05:33.773655  9359 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:33.838904  2312 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:33.848562  2312 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:33.944447  9482 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.950284  9482 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:33.954115  9482 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:33.984005  9439 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:33.986871  9227 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:34.010694  9359 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:34.033012  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:34.066249  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:34.067219  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:34.072492  1714 raft_consensus.cc:2410] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 066ee0109aba4465ba17fa65e14ef504 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
W20260709 13:05:34.093395  1611 consensus_peers.cc:597] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:34.108351  9493 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:34.109077  9493 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:34.111513  1466 catalog_manager.cc:5697] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 16 to 19, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 19 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:34.113960  9485 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.115603  9485 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:34.124538  9493 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:34.125236  9485 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:34.115630  1466 catalog_manager.cc:5697] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 17 to 18, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 18 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:34.143154  9359 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.144019  9359 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:34.145082  1629 tablet_service.cc:1558] Processing DeleteTablet for tablet 59d236b930214b1f91fb55ece6fecd68 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 6 (current committed config index is 9)) from {username='slave'} at 127.0.0.1:47496
W20260709 13:05:34.078229  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 97074.6us
I20260709 13:05:34.152849  1466 catalog_manager.cc:5697] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:05:34.160161  1465 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38662 (request call id 211) took 80917 ms (client timeout 15000 ms). Trace:
W20260709 13:05:34.160729  1465 rpcz_store.cc:269] 0709 13:04:13.242775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.251492 (+  8717us) service_pool.cc:224] Handling call
0709 13:05:34.160040 (+80908548us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":756070,"reported_tablets":54}
I20260709 13:05:34.151576  9415 raft_consensus.cc:493] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.162638  9415 raft_consensus.cc:515] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:34.173612  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 336748us
W20260709 13:05:34.178200  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:34.207775  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49679.4us
I20260709 13:05:34.229689  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:34.215744  1466 catalog_manager.cc:5697] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:34.252087  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:34.253750  1714 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:05:34.259761  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:34.231171  9227 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.275530  1457 catalog_manager.cc:5028] TS f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): tablet 59d236b930214b1f91fb55ece6fecd68 (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
I20260709 13:05:34.249044  9401 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:05:34.327746  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 91170.1us
W20260709 13:05:34.356171   431 thread.cc:527] Waited for 14000ms trying to join with heartbeat (tid 1598)
W20260709 13:05:34.356647  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 51456us
I20260709 13:05:34.324224  9288 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:05:34.348644  9359 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:34.371765  9359 raft_consensus.cc:493] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:34.324009  1457 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 57334.5us
I20260709 13:05:34.345068  9415 leader_election.cc:290] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:34.330035  9401 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:34.316027  9227 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:34.372325  9359 raft_consensus.cc:515] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:34.399127  9288 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.422363  9359 leader_election.cc:290] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:34.428853  9227 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:34.429085  2346 consensus_queue.cc:237] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:34.433748  9288 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:34.436604  9288 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:34.438814  1466 catalog_manager.cc:5697] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 13 to 15, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 15 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:34.447170  9401 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:34.447511  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:34.447964  9475 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:34.448477  9475 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:34.448796  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:34.450856  9475 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:34.470278  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:34.471258  1647 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:05:34.471722  1645 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Refusing update from remote peer 066ee0109aba4465ba17fa65e14ef504: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
W20260709 13:05:34.457736  1685 consensus_peers.cc:597] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:34.473501  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:34.474053  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:34.474959  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:34.464813  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 105938us
I20260709 13:05:34.478135  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:34.483423  1685 consensus_peers.cc:597] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:34.486451  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:34.487787  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:34.489492  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:34.490965  1685 consensus_peers.cc:597] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:34.495929  1685 leader_election.cc:336] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:34.496439  1685 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:34.512123  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380ceca851ad4a84b89cc2e55313d10b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:34.515400  1685 consensus_peers.cc:597] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:34.523499  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:34.523649  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:34.501756  9401 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:34.526533  9359 raft_consensus.cc:493] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.545387  9485 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.562455  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49c94b9e96df4808adbda99815b09c5f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:34.564361  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:34.571872  1611 consensus_peers.cc:597] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:34.564792  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:34.565356  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:34.583488  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:34.593293  1538 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:34.597041  9288 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:34.545610  9359 raft_consensus.cc:515] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:34.608899  9462 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:34.611161  9359 leader_election.cc:290] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:34.576006  9485 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:34.614879  9485 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:34.620062  1537 leader_election.cc:304] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:34.624420  1538 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:34.626066  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 52525.2us
W20260709 13:05:34.630744  1611 consensus_peers.cc:597] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:34.628464  9288 raft_consensus.cc:2749] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:34.638492  9227 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:34.645076  1714 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:05:34.636533  9359 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 13:05:34.651706  9455 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:05:34.659132  1611 consensus_peers.cc:597] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:34.674757  1611 leader_election.cc:336] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:34.675619  1611 consensus_peers.cc:597] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:34.680181  1538 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:34.624610  1537 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:34.686383  9463 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.698375  9463 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:34.699299  2312 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:34.701188  9463 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:34.713280  9455 raft_consensus.cc:2955] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 LEADER]: Committing config change with OpId 19.13: config changed from index 5 to 13, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:34.750948  1714 raft_consensus.cc:2955] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 5 to 13, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:34.775516  9288 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:34.776515  9288 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:34.776281  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 151725us
W20260709 13:05:34.780748  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 94497.6us
W20260709 13:05:34.781896  1685 consensus_peers.cc:597] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:34.786029  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 105989us
W20260709 13:05:34.795447  1685 leader_election.cc:336] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:34.796671  1685 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:34.805215  1612 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:34.808282  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:34.800148  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:34.814749  1538 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:34.814869  1611 consensus_peers.cc:597] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:34.822891  1538 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:34.797578  9488 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:34.824545  1611 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:34.818123  9504 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:34.839149  1611 consensus_peers.cc:597] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:34.829906  9359 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:34.854398  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:34.842979  9290 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:34.876178  1611 consensus_peers.cc:597] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:34.851970  9288 raft_consensus.cc:493] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:34.888216  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 49995.4us
I20260709 13:05:34.903983  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:34.906706  1612 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:34.906926  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2582f1941a4c588f8ef51c03dfb492" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:34.908407  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:34.915545  1643 raft_consensus.cc:1275] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:05:34.922508  1538 leader_election.cc:304] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:34.890141  9288 raft_consensus.cc:515] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:34.931299  9288 leader_election.cc:290] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:34.939760  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:34.918991  1456 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 58b222526a1a4b52bdcda825e15c46f9 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 8)
I20260709 13:05:34.943234  1538 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:34.944722  1538 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:34.946724  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9688a8a5839c4bb0b5a0196390670b14" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:34.952683  1538 leader_election.cc:304] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:34.958316  1537 leader_election.cc:304] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:34.954293  9462 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:34.971554  1538 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:34.972528  9359 raft_consensus.cc:2749] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:34.980293  2312 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:34.971221  9485 raft_consensus.cc:2749] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:35.001520  9480 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:34.994583  9436 consensus_queue.cc:1048] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:05:35.002318  9506 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:34.991567  9486 raft_consensus.cc:2749] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:35.015517  2346 consensus_queue.cc:237] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:35.001411  9479 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:35.060015  9486 raft_consensus.cc:493] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.077788  9486 raft_consensus.cc:515] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:35.083638  9486 leader_election.cc:290] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:35.090240  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:35.086162  9479 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.096689  9479 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:35.099327  9479 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:35.102094  1719 raft_consensus.cc:1275] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:05:35.103578  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:35.106120  9425 consensus_queue.cc:1048] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:35.108647  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:35.089983  1647 raft_consensus.cc:2955] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 5 to 13, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:05:35.113793  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:35.128391  1537 leader_election.cc:304] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:35.130015  9486 raft_consensus.cc:2749] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:35.147553  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b222526a1a4b52bdcda825e15c46f9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:35.147869  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d0b326fa4be595dbd31d3638164d" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:35.153439  9425 raft_consensus.cc:2955] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 LEADER]: Committing config change with OpId 20.14: config changed from index 3 to 14, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:05:35.163157  1611 consensus_peers.cc:597] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:35.178997  1715 raft_consensus.cc:2955] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 3 to 14, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:35.195864  1538 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:35.197301  9479 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:35.251477  1456 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet eca30b43c26f420c965ee25f475581a0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 14)
W20260709 13:05:35.256583  2312 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:35.303514  9520 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.304231  9520 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:35.308030  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:35.310005  9520 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:35.318446  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:35.318084  1467 catalog_manager.cc:5697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:35.338881  2312 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:35.341566  1537 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:35.342751  9520 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:35.356724   431 thread.cc:527] Waited for 15000ms trying to join with heartbeat (tid 1598)
I20260709 13:05:35.362073  1629 tablet_service.cc:1558] Processing DeleteTablet for tablet 59d236b930214b1f91fb55ece6fecd68 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 6 (current committed config index is 9)) from {username='slave'} at 127.0.0.1:47496
I20260709 13:05:35.387112  1457 catalog_manager.cc:5028] TS f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): tablet 59d236b930214b1f91fb55ece6fecd68 (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
W20260709 13:05:35.407089  1466 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38662 (request call id 213) took 36013 ms (client timeout 15000 ms). Trace:
W20260709 13:05:35.423880  1467 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47518 (request call id 101) took 42255 ms (client timeout 15000 ms). Trace:
W20260709 13:05:35.424585  1467 rpcz_store.cc:269] 0709 13:04:53.168374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.170008 (+  1634us) service_pool.cc:224] Handling call
0709 13:05:35.423789 (+42253781us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:35.347456 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:35.348137 (+   681us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:35.348208 (+    71us) write_op.cc:435] Acquired schema lock
0709 13:05:35.348238 (+    30us) tablet.cc:662] Decoding operations
0709 13:05:35.362278 (+ 14040us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:35.362347 (+    69us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:35.362381 (+    34us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:05:35.362597 (+   216us) tablet.cc:701] Row locks acquired
0709 13:05:35.362617 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:05:35.362783 (+   166us) write_op.cc:270] Start()
0709 13:05:35.362918 (+   135us) write_op.cc:276] Timestamp: P: 1783602335362727 usec, L: 0
0709 13:05:35.362942 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:05:35.363784 (+   842us) log.cc:844] Serialized 536 byte log entry
0709 13:05:35.366102 (+  2318us) op_driver.cc:464] REPLICATION: finished
0709 13:05:35.366616 (+   514us) write_op.cc:301] APPLY: starting
0709 13:05:35.366706 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:35.367611 (+   905us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:35.367704 (+    93us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:35.368453 (+   749us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:35.368625 (+   172us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:05:35.368671 (+    46us) write_op.cc:312] APPLY: finished
0709 13:05:35.368924 (+   253us) log.cc:844] Serialized 25 byte log entry
0709 13:05:35.369103 (+   179us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:35.369204 (+   101us) write_op.cc:454] Released schema lock
0709 13:05:35.369248 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:35.374037 (+  4789us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:35.374070 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":30643,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":281,"apply.run_cpu_time_us":2999,"apply.run_wall_time_us":2999,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4182,"num_ops":1,"prepare.queue_time_us":8133,"prepare.run_cpu_time_us":3170,"prepare.run_wall_time_us":16708,"replication_time_us":3011,"thread_start_us":9237,"threads_started":1}]]}
W20260709 13:05:35.413844  1466 rpcz_store.cc:269] 0709 13:04:59.393960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.394267 (+   307us) service_pool.cc:224] Handling call
0709 13:05:35.406990 (+36012723us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:34.550211 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:34.550789 (+   578us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:34.550819 (+    30us) write_op.cc:435] Acquired schema lock
0709 13:05:34.550833 (+    14us) tablet.cc:662] Decoding operations
0709 13:05:34.551424 (+   591us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:34.551519 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:34.551550 (+    31us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:05:34.551950 (+   400us) tablet.cc:701] Row locks acquired
0709 13:05:34.551962 (+    12us) write_op.cc:260] PREPARE: finished
0709 13:05:34.552202 (+   240us) write_op.cc:270] Start()
0709 13:05:34.552533 (+   331us) write_op.cc:276] Timestamp: P: 1783602334552042 usec, L: 0
0709 13:05:34.552560 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:05:34.553228 (+   668us) log.cc:844] Serialized 2142 byte log entry
0709 13:05:34.854863 (+301635us) op_driver.cc:464] REPLICATION: finished
0709 13:05:35.043462 (+188599us) write_op.cc:301] APPLY: starting
0709 13:05:35.057008 (+ 13546us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:35.076755 (+ 19747us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:35.076896 (+   141us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:35.152181 (+ 75285us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:35.152458 (+   277us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 13:05:35.152522 (+    64us) write_op.cc:312] APPLY: finished
0709 13:05:35.152790 (+   268us) log.cc:844] Serialized 57 byte log entry
0709 13:05:35.155503 (+  2713us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:35.155621 (+   118us) write_op.cc:454] Released schema lock
0709 13:05:35.155676 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:35.156187 (+   511us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:35.156225 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":437846,"reported_tablets":54,"spinlock_wait_cycles":100992,"child_traces":[["op",{"apply.queue_time_us":145914,"apply.run_cpu_time_us":10660,"apply.run_wall_time_us":150432,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25037,"num_ops":5,"prepare.queue_time_us":55656,"prepare.run_cpu_time_us":7806,"prepare.run_wall_time_us":218258,"raft.queue_time_us":95920,"raft.run_cpu_time_us":29921,"raft.run_wall_time_us":146425,"replication_time_us":287240,"spinlock_wait_cycles":5504,"thread_start_us":449257,"threads_started":4,"wal-append.queue_time_us":167988}]]}
I20260709 13:05:35.477048  9479 raft_consensus.cc:493] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.477681  9479 raft_consensus.cc:515] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:35.480046  9479 leader_election.cc:290] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:35.485687  1611 consensus_peers.cc:597] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:35.499879  1468 catalog_manager.cc:5697] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 8 to 11, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 11 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:35.506589  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:35.523277  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bddd1ebc9af24fa282580ab028d643e0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:35.526564  1538 leader_election.cc:304] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:35.528992  9462 raft_consensus.cc:2749] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:05:35.633814  1611 consensus_peers.cc:597] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:35.634424  1611 consensus_peers.cc:597] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:35.706586  1468 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38650 (request call id 199) took 93764 ms (client timeout 15000 ms). Trace:
I20260709 13:05:35.712303  1467 catalog_manager.cc:5697] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:35.714030  1467 catalog_manager.cc:5697] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:35.716522  1467 catalog_manager.cc:5697] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 16 to 21, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 21 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:35.718077  1467 catalog_manager.cc:5697] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:35.719861  1467 catalog_manager.cc:5697] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:35.722162  1467 catalog_manager.cc:5697] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 14 to 18, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 18 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
W20260709 13:05:35.707130  1468 rpcz_store.cc:269] 0709 13:04:01.942824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.943451 (+   627us) service_pool.cc:224] Handling call
0709 13:05:35.706462 (+93763011us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:35.581970 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:35.592691 (+ 10721us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:35.592733 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:05:35.592751 (+    18us) tablet.cc:662] Decoding operations
0709 13:05:35.593158 (+   407us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:35.593212 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:35.593241 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:05:35.603952 (+ 10711us) tablet.cc:701] Row locks acquired
0709 13:05:35.604929 (+   977us) write_op.cc:260] PREPARE: finished
0709 13:05:35.608243 (+  3314us) write_op.cc:270] Start()
0709 13:05:35.611929 (+  3686us) write_op.cc:276] Timestamp: P: 1783602335608178 usec, L: 0
0709 13:05:35.611973 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 13:05:35.613102 (+  1129us) log.cc:844] Serialized 541 byte log entry
0709 13:05:35.616663 (+  3561us) op_driver.cc:464] REPLICATION: finished
0709 13:05:35.617223 (+   560us) write_op.cc:301] APPLY: starting
0709 13:05:35.617328 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:35.618325 (+   997us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:35.618539 (+   214us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:35.619532 (+   993us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:35.619725 (+   193us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:05:35.619778 (+    53us) write_op.cc:312] APPLY: finished
0709 13:05:35.620006 (+   228us) log.cc:844] Serialized 25 byte log entry
0709 13:05:35.620202 (+   196us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:35.620353 (+   151us) write_op.cc:454] Released schema lock
0709 13:05:35.620403 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:35.620816 (+   413us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:35.620881 (+    65us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7,"reported_tablets":54,"child_traces":[["op",{"apply.queue_time_us":285,"apply.run_cpu_time_us":3560,"apply.run_wall_time_us":3570,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4182,"num_ops":1,"prepare.queue_time_us":262,"prepare.run_cpu_time_us":5328,"prepare.run_wall_time_us":31475,"replication_time_us":4476}]]}
I20260709 13:05:35.723971  1467 catalog_manager.cc:5697] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 15 to 20, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 20 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:35.729254  1467 catalog_manager.cc:5697] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:35.730882  1467 catalog_manager.cc:5697] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 15 to 18, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 18 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:35.732281  1467 catalog_manager.cc:5697] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 14 to 19. New cstate: current_term: 19 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:35.733562  1467 catalog_manager.cc:5697] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 11 to 22, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 22 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:35.735379  1467 catalog_manager.cc:5697] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 14 to 21, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 21 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:35.738750  1467 catalog_manager.cc:5697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 17 to 22, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 22 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:35.754453  1467 catalog_manager.cc:5697] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 16 to 19, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 19 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:35.756070  1467 catalog_manager.cc:5697] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:35.760291  1467 catalog_manager.cc:5697] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 15 to 19, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 19 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:35.762138  1467 catalog_manager.cc:5697] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 15 to 17, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 17 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:35.763789  1467 catalog_manager.cc:5697] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 18 to 21, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 21 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:35.765457  1467 catalog_manager.cc:5697] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index -1 to 16, term changed from 18 to 21, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195), NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added, now has a pending config: VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193), VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194), VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195), VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 21 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:35.768141  1467 catalog_manager.cc:5697] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 18 to 22, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 22 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:35.770275  1467 catalog_manager.cc:5697] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 12 to 19, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 19 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:35.771662  1467 catalog_manager.cc:5697] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 15 to 17, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 17 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:35.773030  1467 catalog_manager.cc:5697] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 14 to 18, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 18 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:35.774621  1467 catalog_manager.cc:5697] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 20 to 21, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 21 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:35.784821  1467 catalog_manager.cc:5697] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 13 to 17, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 17 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:35.786688  1467 catalog_manager.cc:5697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:35.801731  9486 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.802450  9486 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:35.804895  9486 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:35.812071  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:35.815176  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ee72a7f576482aabe40ef837964708" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:35.795414  1467 catalog_manager.cc:5697] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:35.820346  9486 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.818418  1467 catalog_manager.cc:5697] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:35.820885  1537 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:35.821126  9486 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:35.822400  9479 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:35.825739  1467 catalog_manager.cc:5697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:35.827344  1467 catalog_manager.cc:5697] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:35.834296  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:35.833027  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12ac45e0e48449a98fb89704bae3de27" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:35.838105  1537 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:35.839282  9479 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:35.832485  9486 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:35.835103  1467 catalog_manager.cc:5697] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 14 to 20, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 20 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:35.841794  1467 catalog_manager.cc:5697] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:35.850183  9486 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.852425  9486 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:35.856212  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:35.860668  9486 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:35.860584  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:35.862748  1537 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:35.851948  1467 catalog_manager.cc:5697] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 23 to 25, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 25 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:35.865096  1467 catalog_manager.cc:5697] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 19 to 21, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 21 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:35.866556  9486 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:35.868712  1467 catalog_manager.cc:5697] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 18 to 20, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 20 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
W20260709 13:05:35.873721  2312 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:35.916852  9479 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.917479  9479 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:35.930400  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:35.936820  9479 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:35.941017  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd2b69727c743cfbdc52253e16279ce" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:35.943261  1537 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:35.945324  9479 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:35.948467  2312 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:35.988916  9486 raft_consensus.cc:493] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:35.989578  9486 raft_consensus.cc:515] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:36.059674  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:36.055454  9486 leader_election.cc:290] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:36.073817  1685 consensus_peers.cc:597] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:36.079994  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6f11a07e734ec095c6185363fd6b52" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:36.108386  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 81551.5us
I20260709 13:05:36.117096  9462 raft_consensus.cc:493] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:36.117877  9462 raft_consensus.cc:515] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:36.120332  9462 leader_election.cc:290] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:36.128347  9520 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.129168  9520 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:36.131404  9520 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:36.135425  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:36.139321  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:36.145496  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6485a5991c348019b9779be81656a45" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:36.153276  1611 consensus_peers.cc:597] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:36.154057  1611 consensus_peers.cc:597] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:36.155427  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e59deed37c414a929faeb85bdbaaa406" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:36.158967  1537 leader_election.cc:304] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:36.168732  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 31: this message will repeat every 5th retry.
I20260709 13:05:36.174623  9462 raft_consensus.cc:2749] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:36.193960  1537 leader_election.cc:304] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:36.196079  9529 raft_consensus.cc:2749] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:36.195955  1537 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:36.199050  9486 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:36.203495  1685 consensus_peers.cc:597] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:36.212364  1685 consensus_peers.cc:597] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:36.222301  1685 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 349 similar messages]
W20260709 13:05:36.242584  1685 consensus_peers.cc:597] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:36.253818  1685 consensus_peers.cc:597] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:36.273985  1611 consensus_peers.cc:597] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:36.287923  9520 raft_consensus.cc:493] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.289292  9520 raft_consensus.cc:515] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:36.290076   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:36.295291  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:36.295032  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d236b930214b1f91fb55ece6fecd68" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:36.300638   431 tablet_replica.cc:333] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:36.300891  1537 leader_election.cc:304] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:36.302928   431 raft_consensus.cc:2243] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:36.291853  9520 leader_election.cc:290] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:36.306084  9520 raft_consensus.cc:2749] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:36.314038   431 raft_consensus.cc:2272] T 82cbfb23571048409c1bde90d4611c1a P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:36.318030  1611 consensus_peers.cc:597] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:36.329793  1467 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38666 (request call id 220) took 2877 ms. Trace:
I20260709 13:05:36.330356  1467 rpcz_store.cc:276] 0709 13:05:33.417728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.444083 (+2026355us) service_pool.cc:224] Handling call
0709 13:05:36.294954 (+850871us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:35.982447 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:35.983079 (+   632us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:35.983123 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:05:35.983139 (+    16us) tablet.cc:662] Decoding operations
0709 13:05:35.985874 (+  2735us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:35.985939 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:35.985970 (+    31us) tablet.cc:685] Acquiring locks for 35 operations
0709 13:05:35.988644 (+  2674us) tablet.cc:701] Row locks acquired
0709 13:05:35.988677 (+    33us) write_op.cc:260] PREPARE: finished
0709 13:05:35.988822 (+   145us) write_op.cc:270] Start()
0709 13:05:35.988946 (+   124us) write_op.cc:276] Timestamp: P: 1783602335988773 usec, L: 0
0709 13:05:35.988972 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:05:35.989994 (+  1022us) log.cc:844] Serialized 14068 byte log entry
0709 13:05:35.995319 (+  5325us) op_driver.cc:464] REPLICATION: finished
0709 13:05:35.995837 (+   518us) write_op.cc:301] APPLY: starting
0709 13:05:35.995945 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:36.131219 (+135274us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:36.131349 (+   130us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:36.185223 (+ 53874us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:36.186322 (+  1099us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=70,key_file_lookups=70,delta_file_lookups=70,mrs_lookups=0
0709 13:05:36.186370 (+    48us) write_op.cc:312] APPLY: finished
0709 13:05:36.189402 (+  3032us) log.cc:844] Serialized 300 byte log entry
0709 13:05:36.190253 (+   851us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:36.190531 (+   278us) write_op.cc:454] Released schema lock
0709 13:05:36.190606 (+    75us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:36.191012 (+   406us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:36.191041 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":48,"spinlock_wait_cycles":515968,"child_traces":[["op",{"apply.queue_time_us":288,"apply.run_cpu_time_us":50137,"apply.run_wall_time_us":195138,"cfile_cache_hit":140,"cfile_cache_hit_bytes":146370,"num_ops":35,"prepare.queue_time_us":6055,"prepare.run_cpu_time_us":7912,"prepare.run_wall_time_us":8022,"replication_time_us":6189,"spinlock_wait_cycles":101632}]]}
I20260709 13:05:36.373085   431 tablet_replica.cc:333] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:36.389981   431 raft_consensus.cc:2243] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:36.392863   431 raft_consensus.cc:2272] T aa55479d716d4bc4893f658afa382d73 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.409168  9496 raft_consensus.cc:493] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:36.411128  9496 raft_consensus.cc:515] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } attrs { replace: true } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:36.427762  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:36.379706  1470 catalog_manager.cc:5697] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:36.437674  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca30b43c26f420c965ee25f475581a0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:36.439695  1611 leader_election.cc:336] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:36.440246  1611 leader_election.cc:304] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:36.441416  9504 raft_consensus.cc:2749] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:36.460299  1470 catalog_manager.cc:5697] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 18 to 19, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 19 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:36.462872  2312 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:36.434863  9496 leader_election.cc:290] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:36.480609  1685 consensus_peers.cc:597] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:36.483748  1469 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38650 (request call id 198) took 120366 ms (client timeout 15000 ms). Trace:
W20260709 13:05:36.496583  1469 rpcz_store.cc:269] 0709 13:03:36.133924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.158525 (+ 24601us) service_pool.cc:224] Handling call
0709 13:05:36.483489 (+120324964us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11082,"reported_tablets":54}
W20260709 13:05:36.512907  1685 consensus_peers.cc:597] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:36.550223  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 98191.6us
I20260709 13:05:36.550557  1466 catalog_manager.cc:5697] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 19 to 23, leader changed from 130bc5ff39b847b1b3204626f284f838 (127.0.107.195) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 23 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:36.553197  1466 catalog_manager.cc:5697] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
I20260709 13:05:36.554654  1466 catalog_manager.cc:5697] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index 3 to 14, VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) evicted. New cstate: current_term: 20 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:36.556852  9486 raft_consensus.cc:493] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.557906  9486 raft_consensus.cc:515] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:36.560993  9486 leader_election.cc:290] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:36.556720  1466 catalog_manager.cc:5697] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index 5 to 13, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New cstate: current_term: 19 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:36.569412  1466 catalog_manager.cc:5697] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index 16 to 17, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193), VOTER f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194), VOTER 130bc5ff39b847b1b3204626f284f838 (127.0.107.195), VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 21 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.574571   431 tablet_replica.cc:333] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:36.576232   431 raft_consensus.cc:2243] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:36.580438   431 pending_rounds.cc:70] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504: Trying to abort 1 pending ops.
I20260709 13:05:36.573711  1466 catalog_manager.cc:5697] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 19 to 25, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 25 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } }
W20260709 13:05:36.589200  1611 consensus_peers.cc:597] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:36.587543   431 pending_rounds.cc:77] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504: Aborting op as it isn't in flight: id { term: 17 index: 9 } timestamp: 7305634970771132416 op_type: NO_OP noop_request { }
I20260709 13:05:36.586335  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:36.600265  2312 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:36.615267  1470 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38650 (request call id 201) took 36011 ms (client timeout 15000 ms). Trace:
I20260709 13:05:36.618362   431 raft_consensus.cc:2889] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:36.621383   431 raft_consensus.cc:2272] T 8f06439379e24c5ebedb229b007236c2 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.631642  1629 tablet_service.cc:1558] Processing DeleteTablet for tablet eca30b43c26f420c965ee25f475581a0 with delete_type TABLET_DATA_TOMBSTONED (TS f5f4b2707a9647e5bb99e065068dea2b not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:47496
I20260709 13:05:36.639725  1713 consensus_queue.cc:237] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:36.651533  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 50301.3us
I20260709 13:05:36.650684   431 tablet_replica.cc:333] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
W20260709 13:05:36.616211  1470 rpcz_store.cc:269] 0709 13:05:00.603602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.778260 (+7174658us) service_pool.cc:224] Handling call
0709 13:05:09.035810 (+1257550us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b54002b0054. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8248def090 00007f8248e8df3a 00007f824ec441cc 00007f824ed4237e 00007f824d2032a4 00007f824d203232 00007f824d203161 00007f824d2030dc 00007f824d202392 00007f82471c048c 00007f82471bf8c6
0709 13:05:36.615138 (+27579328us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:36.516001 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:36.516963 (+   962us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:36.517001 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:05:36.517021 (+    20us) tablet.cc:662] Decoding operations
0709 13:05:36.517533 (+   512us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:36.517612 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:36.517668 (+    56us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:05:36.517965 (+   297us) tablet.cc:701] Row locks acquired
0709 13:05:36.517982 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:05:36.518123 (+   141us) write_op.cc:270] Start()
0709 13:05:36.518244 (+   121us) write_op.cc:276] Timestamp: P: 1783602336518072 usec, L: 0
0709 13:05:36.518264 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:05:36.518996 (+   732us) log.cc:844] Serialized 919 byte log entry
0709 13:05:36.531149 (+ 12153us) op_driver.cc:464] REPLICATION: finished
0709 13:05:36.531684 (+   535us) write_op.cc:301] APPLY: starting
0709 13:05:36.531763 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:36.533485 (+  1722us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:36.533576 (+    91us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:36.536500 (+  2924us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:36.536830 (+   330us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:05:36.536964 (+   134us) write_op.cc:312] APPLY: finished
0709 13:05:36.537226 (+   262us) log.cc:844] Serialized 33 byte log entry
0709 13:05:36.537452 (+   226us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:36.537576 (+   124us) write_op.cc:454] Released schema lock
0709 13:05:36.537620 (+    44us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":24846,"reported_tablets":54,"spinlock_wait_cycles":926656896,"child_traces":[["op",{"apply.queue_time_us":260,"apply.run_cpu_time_us":6523,"apply.run_wall_time_us":20488,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8364,"num_ops":2,"prepare.queue_time_us":12622,"prepare.run_cpu_time_us":3365,"prepare.run_wall_time_us":3366,"replication_time_us":12694,"thread_start_us":9058,"threads_started":1}]]}
I20260709 13:05:36.727212  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baba78285345483d9cf6ead6bf20106b" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:36.753383   431 raft_consensus.cc:2243] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:36.754451   431 raft_consensus.cc:2272] T 0c6f11a07e734ec095c6185363fd6b52 P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.765564  2346 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 13:05:36.768980  9533 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.053s
W20260709 13:05:36.851922  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 111525us
I20260709 13:05:36.844192   431 tablet_replica.cc:333] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:36.860783  1642 raft_consensus.cc:1275] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 13:05:36.869987  9544 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.871028  9544 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:36.873926  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 77756.3us
I20260709 13:05:36.858534  9535 tablet_replica.cc:333] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:05:36.883050   431 raft_consensus.cc:2243] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 17 LEADER]: Raft consensus shutting down.
W20260709 13:05:36.938288  1471 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47518 (request call id 99) took 75613 ms (client timeout 15000 ms). Trace:
W20260709 13:05:36.945812  1471 rpcz_store.cc:269] 0709 13:04:21.324903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.329252 (+  4349us) service_pool.cc:224] Handling call
0709 13:04:21.412250 (+ 82998us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b5400310010. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d610b2 00007f824ecdaa89 00007f824ec72da6 00007f824ec44088 00007f824ed4237e 00007f824d2032a4 00007f824d203232 00007f824d203161 00007f824d2030dc
0709 13:05:36.937928 (+75525678us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13742,"reported_tablets":20,"spinlock_wait_cycles":6000512}
I20260709 13:05:36.946431  9535 raft_consensus.cc:2243] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:36.947916  9535 raft_consensus.cc:2272] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.948056   431 raft_consensus.cc:2272] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.961823   431 tablet_replica.cc:333] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:36.963918   431 raft_consensus.cc:2243] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:36.964838   431 pending_rounds.cc:70] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504: Trying to abort 1 pending ops.
I20260709 13:05:36.965168   431 pending_rounds.cc:77] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504: Aborting op as it isn't in flight: id { term: 16 index: 10 } timestamp: 7305634952857915392 op_type: NO_OP noop_request { }
W20260709 13:05:36.967876  9538 log.cc:927] Time spent T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:05:36.965529   431 raft_consensus.cc:2889] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:36.998309   431 raft_consensus.cc:2272] T dbbf62b8237649f584432aa4a9d96ec6 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:37.009837  1538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 150003us
I20260709 13:05:37.023047   431 tablet_replica.cc:333] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:37.028959  1462 catalog_manager.cc:5697] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 13 to 15, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 15 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:37.032240  1462 catalog_manager.cc:5697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 19 to 20, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 20 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:37.025475   431 raft_consensus.cc:2243] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:36.898727  9544 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:37.045855   431 raft_consensus.cc:2272] T bddd1ebc9af24fa282580ab028d643e0 P 066ee0109aba4465ba17fa65e14ef504 [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:37.068418  1611 consensus_peers.cc:597] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:37.073688  1611 consensus_peers.cc:597] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:37.094322  9535 ts_tablet_manager.cc:1916] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:37.098771   431 tablet_replica.cc:333] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:37.110913  9486 raft_consensus.cc:493] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.117239  9486 raft_consensus.cc:515] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:37.120867  9486 leader_election.cc:290] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:37.122946   431 raft_consensus.cc:2243] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.163529  1538 leader_election.cc:304] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:37.168545  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:37.174443  1611 consensus_peers.cc:597] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:37.183012   431 raft_consensus.cc:2272] T a51c23e1c3ae4bdbbb5f21f1607da772 P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.189476  9559 raft_consensus.cc:2749] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:37.196316   431 tablet_replica.cc:333] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:37.208297   431 raft_consensus.cc:2243] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.212818   431 pending_rounds.cc:70] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Trying to abort 1 pending ops.
I20260709 13:05:37.218852   431 pending_rounds.cc:77] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504: Aborting op as it isn't in flight: id { term: 19 index: 12 } timestamp: 7305635072807059456 op_type: NO_OP noop_request { }
I20260709 13:05:37.219406   431 raft_consensus.cc:2889] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:37.219887   431 raft_consensus.cc:2272] T a8d17fad9f114b0e98d2af0a989c3185 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.225821   431 tablet_replica.cc:333] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
W20260709 13:05:37.226856  1462 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47518 (request call id 102) took 21169 ms (client timeout 15000 ms). Trace:
I20260709 13:05:37.226775  9551 consensus_queue.cc:1048] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.002s
I20260709 13:05:37.230654   431 raft_consensus.cc:2243] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.232025   431 raft_consensus.cc:2272] T b82d3b49001741eda872554a7ecb2db3 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.236953  9554 raft_consensus.cc:2955] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 21 LEADER]: Committing config change with OpId 21.18: config changed from index 17 to 18, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:37.242699  2348 raft_consensus.cc:2955] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Committing config change with OpId 21.18: config changed from index 17 to 18, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:05:37.251768  2312 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:37.227860  1462 rpcz_store.cc:269] 0709 13:05:16.128980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.384783 (+9255803us) service_pool.cc:224] Handling call
0709 13:05:37.226739 (+11841956us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:37.045703 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:37.046425 (+   722us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:37.046475 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:05:37.046488 (+    13us) tablet.cc:662] Decoding operations
0709 13:05:37.046959 (+   471us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:37.047023 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:37.047063 (+    40us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:05:37.047442 (+   379us) tablet.cc:701] Row locks acquired
0709 13:05:37.047456 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:05:37.047634 (+   178us) write_op.cc:270] Start()
0709 13:05:37.047772 (+   138us) write_op.cc:276] Timestamp: P: 1783602337047572 usec, L: 0
0709 13:05:37.047806 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:05:37.048868 (+  1062us) log.cc:844] Serialized 919 byte log entry
0709 13:05:37.050839 (+  1971us) op_driver.cc:464] REPLICATION: finished
0709 13:05:37.051288 (+   449us) write_op.cc:301] APPLY: starting
0709 13:05:37.051365 (+    77us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:37.052860 (+  1495us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:37.052950 (+    90us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:37.054252 (+  1302us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:37.054410 (+   158us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:05:37.054454 (+    44us) write_op.cc:312] APPLY: finished
0709 13:05:37.054672 (+   218us) log.cc:844] Serialized 33 byte log entry
0709 13:05:37.055699 (+  1027us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:37.055795 (+    96us) write_op.cc:454] Released schema lock
0709 13:05:37.055843 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:37.056667 (+   824us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:37.056703 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":21989,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":260,"apply.run_cpu_time_us":4113,"apply.run_wall_time_us":4841,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8364,"num_ops":2,"prepare.queue_time_us":1033,"prepare.run_cpu_time_us":3572,"prepare.run_wall_time_us":3572,"replication_time_us":2900,"spinlock_wait_cycles":24832}]]}
I20260709 13:05:37.293857  9555 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:37.298319  9555 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:37.301985  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:37.308827   431 tablet_replica.cc:333] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
W20260709 13:05:37.322508  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 440475us
I20260709 13:05:37.324489  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:37.326145  9555 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:37.327596  9535 ts_tablet_manager.cc:1929] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 20.13
I20260709 13:05:37.337020  9535 log.cc:1199] T eca30b43c26f420c965ee25f475581a0 P f5f4b2707a9647e5bb99e065068dea2b: Deleting WAL directory at /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0/minicluster-data/ts-1-root/wals/eca30b43c26f420c965ee25f475581a0
I20260709 13:05:37.330485   431 raft_consensus.cc:2243] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.339061  1457 catalog_manager.cc:5028] TS f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): tablet eca30b43c26f420c965ee25f475581a0 (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
W20260709 13:05:37.313371  1611 consensus_peers.cc:597] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:37.339939   431 raft_consensus.cc:2272] T 08cf0535992b4db395d567393667593b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.348451  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee0e81a514641709317e983fcf96fe4" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:37.403471  9563 raft_consensus.cc:493] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.404356  9563 raft_consensus.cc:515] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:37.405652   431 tablet_replica.cc:333] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:37.406684  1460 maintenance_manager.cc:419] P 220fdfc07a3c4ce28ca7333445633466: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033368
I20260709 13:05:37.407145  9563 leader_election.cc:290] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:37.407861  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.410753  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:37.416844  9563 raft_consensus.cc:493] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.419369  9563 raft_consensus.cc:515] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:37.423655  9563 leader_election.cc:290] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:37.430001  9520 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.433079  1456 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ca2582f1941a4c588f8ef51c03dfb492 with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:05:37.482333  1456 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 49271.6us
W20260709 13:05:37.453177  2312 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:37.406922   431 raft_consensus.cc:2243] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.482103  9520 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:37.463624  1537 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:37.484241  9486 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:37.485391  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:37.485055  9520 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:37.487622   431 raft_consensus.cc:2272] T ca2582f1941a4c588f8ef51c03dfb492 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.490242  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:37.494333  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:37.499596   431 tablet_replica.cc:333] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:37.499982  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a01faf4c6c64b549e80adf03bba9f51" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:37.501006   431 raft_consensus.cc:2243] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.506860  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca833b0b78d74adc8ca86e91fc94f96f" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 22 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:37.516103  1459 maintenance_manager.cc:643] P 220fdfc07a3c4ce28ca7333445633466: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.108s	user 0.032s	sys 0.008s Metrics: {"bytes_written":6472,"delete_count":0,"lbm_write_time_us":1210,"lbm_writes_lt_1ms":6,"reinsert_count":0,"update_count":264}
I20260709 13:05:37.527240  1460 maintenance_manager.cc:419] P 220fdfc07a3c4ce28ca7333445633466: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.328670
I20260709 13:05:37.533516  9563 raft_consensus.cc:493] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cf2f71592be8465fb18e36e15d0b2f89)
I20260709 13:05:37.534256  9563 raft_consensus.cc:515] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:37.525540  1611 consensus_peers.cc:597] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:37.539302   431 pending_rounds.cc:70] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: Trying to abort 1 pending ops.
I20260709 13:05:37.539707   431 pending_rounds.cc:77] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: Aborting op as it isn't in flight: id { term: 18 index: 13 } timestamp: 7305635087729393664 op_type: NO_OP noop_request { }
I20260709 13:05:37.540117   431 raft_consensus.cc:2889] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:37.545926   431 raft_consensus.cc:2272] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.548214  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:37.552870  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baa23f35bd3b4c2d99e3da0ff0f80e97" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:37.557531  1462 catalog_manager.cc:5697] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index 17 to 18, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New cstate: current_term: 21 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:37.551838  1537 leader_election.cc:304] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:37.566481  1537 raft_consensus.cc:2686] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:05:37.569600   431 tablet_replica.cc:333] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:37.570806   431 raft_consensus.cc:2243] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.571462   431 raft_consensus.cc:2272] T baba78285345483d9cf6ead6bf20106b P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:37.574003  1611 consensus_peers.cc:597] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:37.575825  1611 consensus_peers.cc:597] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:37.579725  1644 raft_consensus.cc:2955] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Committing config change with OpId 21.18: config changed from index 17 to 18, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:37.581987  9563 leader_election.cc:290] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:37.600725  1537 leader_election.cc:304] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:37.612108   431 tablet_replica.cc:333] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:37.613560   431 raft_consensus.cc:2243] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.623034  1537 leader_election.cc:304] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:37.625036   431 raft_consensus.cc:2272] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:37.628721  2312 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 31: this message will repeat every 5th retry.
I20260709 13:05:37.629979  9520 raft_consensus.cc:2749] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:37.641274  9520 raft_consensus.cc:2749] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:37.645210   431 tablet_replica.cc:333] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
W20260709 13:05:37.648926  1455 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 51445.5us
I20260709 13:05:37.650653   431 raft_consensus.cc:2243] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.660111  1537 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:37.662170  9563 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.664578  9486 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:37.665161  9563 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:37.666508   431 pending_rounds.cc:70] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504: Trying to abort 1 pending ops.
I20260709 13:05:37.666884   431 pending_rounds.cc:77] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504: Aborting op as it isn't in flight: id { term: 19 index: 15 } timestamp: 7305635091334078464 op_type: NO_OP noop_request { }
I20260709 13:05:37.667243   431 raft_consensus.cc:2889] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:37.667580   431 raft_consensus.cc:2272] T 9688a8a5839c4bb0b5a0196390670b14 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.675464   431 tablet_replica.cc:333] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:37.679275  9563 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:37.679720   431 raft_consensus.cc:2243] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.681219   431 pending_rounds.cc:70] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Trying to abort 2 pending ops.
I20260709 13:05:37.681571   431 pending_rounds.cc:77] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Aborting op as it isn't in flight: id { term: 23 index: 15 } timestamp: 7305635074237542400 op_type: NO_OP noop_request { }
I20260709 13:05:37.682659   431 raft_consensus.cc:2889] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:37.683023   431 pending_rounds.cc:77] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504: Aborting op as it isn't in flight: id { term: 21 index: 14 } timestamp: 7305635041089744896 op_type: NO_OP noop_request { }
I20260709 13:05:37.683589   431 raft_consensus.cc:2889] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:37.684055   431 raft_consensus.cc:2272] T 380ceca851ad4a84b89cc2e55313d10b P 066ee0109aba4465ba17fa65e14ef504 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.690943  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0764c6a89eca47f39e28feb56e2123c7" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:37.698035  1685 consensus_peers.cc:597] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:37.698942  1685 consensus_peers.cc:597] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:37.702512  1629 tablet_service.cc:1558] Processing DeleteTablet for tablet eca30b43c26f420c965ee25f475581a0 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 14)) from {username='slave'} at 127.0.0.1:47496
I20260709 13:05:37.702872   431 tablet_replica.cc:333] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
W20260709 13:05:37.704767  1685 consensus_peers.cc:597] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:37.706439  1457 catalog_manager.cc:5028] TS f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): tablet eca30b43c26f420c965ee25f475581a0 (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
W20260709 13:05:37.710479  1611 consensus_peers.cc:597] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:37.715155  1685 consensus_peers.cc:597] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:37.717932  1685 consensus_peers.cc:597] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:37.719046  1685 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:37.719677  1685 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:37.721046  9488 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:37.723659  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:37.726688  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:37.727649  9520 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.728361  9520 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:37.730772  9520 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:37.732956   431 raft_consensus.cc:2243] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.733676   431 pending_rounds.cc:70] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: Trying to abort 1 pending ops.
I20260709 13:05:37.733940   431 pending_rounds.cc:77] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504: Aborting op as it isn't in flight: id { term: 14 index: 12 } timestamp: 7305635062053343232 op_type: NO_OP noop_request { }
W20260709 13:05:37.740830  1455 catalog_manager.cc:4755] TS 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 58b222526a1a4b52bdcda825e15c46f9: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:05:37.742789  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:37.742808  1537 leader_election.cc:304] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:37.748754  9520 raft_consensus.cc:2749] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:37.734301   431 raft_consensus.cc:2889] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:37.749574   431 raft_consensus.cc:2272] T 59d236b930214b1f91fb55ece6fecd68 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.758316  1537 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:05:37.694741  1463 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38662 (request call id 214) took 17455 ms (client timeout 15000 ms). Trace:
W20260709 13:05:37.759285  1463 rpcz_store.cc:269] 0709 13:05:20.239365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.156250 (+10916885us) service_pool.cc:224] Handling call
0709 13:05:31.544078 (+387828us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b5400310010. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60b71 00007f8245d610b2 00007f824ecdaa89 00007f824ec72da6 00007f824ec44088 00007f824ed4237e 00007f824d2032a4 00007f824d203232 00007f824d203161 00007f824d2030dc
0709 13:05:31.619041 (+ 74963us) spinlock_profiling.cc:243] Waited 59.1 ms on lock 0x7b5400310010. stack: 00007f8245d66343 00007f8245d66240 00007f82456f51d4 00007f824f433304 00007f824f433299 00007f8245c20db1 00007f8245d60e9c 00007f8245d610f6 00007f824ecdaa89 00007f824ec72da6 00007f824ec44088 00007f824ed4237e 00007f824d2032a4 00007f824d203232 00007f824d203161 00007f824d2030dc
0709 13:05:37.694430 (+6075389us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":54,"spinlock_wait_cycles":184377088}
I20260709 13:05:37.771342  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:37.774418  1538 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:37.766152  9563 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:37.776067   431 tablet_replica.cc:333] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:37.776484  9561 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:37.802636   431 raft_consensus.cc:2243] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.803464   431 raft_consensus.cc:2272] T e87427e2b27d4f89b7953163626fd8e4 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.808480   431 tablet_replica.cc:333] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:37.809625   431 raft_consensus.cc:2243] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 20 LEADER]: Raft consensus shutting down.
W20260709 13:05:37.831911  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:37.832767  9372 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:37.835281  9372 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:37.841580  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:37.842080   431 raft_consensus.cc:2272] T 1d9390db02bf4ea689cbc36b1b946a14 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.844738  1713 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
I20260709 13:05:37.847007  9496 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:37.837541  9372 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:37.850677  9496 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:37.851435  1612 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:37.853055  9496 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:37.853830  9372 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:37.854323  9372 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:37.854490  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:37.854770  9372 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:37.862960  1611 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:37.862093  2312 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 31: this message will repeat every 5th retry.
I20260709 13:05:37.871199   431 tablet_replica.cc:333] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:37.872218   431 raft_consensus.cc:2243] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.873584   431 raft_consensus.cc:2272] T 0764c6a89eca47f39e28feb56e2123c7 P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:37.875545  1611 leader_election.cc:336] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:37.879607  1611 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:37.881357  9496 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:37.924304  1464 catalog_manager.cc:5697] T 3ee0e81a514641709317e983fcf96fe4 P 066ee0109aba4465ba17fa65e14ef504 reported cstate change: term changed from 14 to 18, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 18 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:37.932538   431 tablet_replica.cc:333] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:37.934096   431 raft_consensus.cc:2243] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.934749   431 pending_rounds.cc:70] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: Trying to abort 1 pending ops.
I20260709 13:05:37.935017   431 pending_rounds.cc:77] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504: Aborting op as it isn't in flight: id { term: 22 index: 16 } timestamp: 7305635076954091520 op_type: NO_OP noop_request { }
I20260709 13:05:37.935379   431 raft_consensus.cc:2889] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:37.935725   431 raft_consensus.cc:2272] T c6485a5991c348019b9779be81656a45 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.938898   431 tablet_replica.cc:333] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:37.939894   431 raft_consensus.cc:2243] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.940800   431 raft_consensus.cc:2272] T 27aa13d58e6d4987b2e34613996ef421 P 066ee0109aba4465ba17fa65e14ef504 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.908833  9372 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:37.946717  9372 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:37.956743  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:37.957535  1715 raft_consensus.cc:3060] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:37.961517   431 tablet_replica.cc:333] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:37.962661   431 raft_consensus.cc:2243] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.969270   431 raft_consensus.cc:2272] T 0df07602233941598bee7b19607b2e3b P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.005779   431 tablet_replica.cc:333] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.006878   431 raft_consensus.cc:2243] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.008881  9496 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:38.009593  9496 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:38.012192  9496 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:38.015703  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:38.025431   431 raft_consensus.cc:2272] T 5bd2b69727c743cfbdc52253e16279ce P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.029322   431 tablet_replica.cc:333] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.031944   431 raft_consensus.cc:2243] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.032575   431 raft_consensus.cc:2272] T d747d0b326fa4be595dbd31d3638164d P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.036801   431 tablet_replica.cc:333] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.038213   431 raft_consensus.cc:2243] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.039166   431 raft_consensus.cc:2272] T d3ff300e7f45424ea3bcd7c816082956 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.042493   431 tablet_replica.cc:333] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.044106   431 raft_consensus.cc:2243] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.044795   431 raft_consensus.cc:2272] T e5c79254e6ff49639cf09f663490dfd7 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.048259   431 tablet_replica.cc:333] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.053267   431 raft_consensus.cc:2243] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.055531  9561 raft_consensus.cc:493] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.056131  9561 raft_consensus.cc:515] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:38.058552  9561 leader_election.cc:290] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:38.061826  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:38.062098   431 raft_consensus.cc:2272] T 58b222526a1a4b52bdcda825e15c46f9 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.062099  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2618e77160b44339d8499efcd1e8eca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:38.070043   431 tablet_replica.cc:333] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.025713  1715 raft_consensus.cc:2468] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 20.
I20260709 13:05:38.074354  1612 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:38.071951   431 raft_consensus.cc:2243] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.078087  9496 raft_consensus.cc:2804] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:38.078814  9496 raft_consensus.cc:697] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 20 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:38.079839  9496 consensus_queue.cc:237] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:38.107674  1611 consensus_peers.cc:597] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:38.116886  1464 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38650 (request call id 202) took 16943 ms (client timeout 15000 ms). Trace:
I20260709 13:05:38.119292   431 raft_consensus.cc:2272] T bbc4aafd02974a29b0c9b332b55c978c P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.123106   431 tablet_replica.cc:333] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.123430  9561 raft_consensus.cc:493] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.124686   431 raft_consensus.cc:2243] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.124421  9561 raft_consensus.cc:515] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:38.125543   431 raft_consensus.cc:2272] T 18bdd841c8294b08857684489689676d P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:38.117391  1464 rpcz_store.cc:269] 0709 13:05:21.173204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.201352 (+12028148us) service_pool.cc:224] Handling call
0709 13:05:38.116795 (+4915443us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:37.946624 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:37.947226 (+   602us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:37.947265 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:05:37.947281 (+    16us) tablet.cc:662] Decoding operations
0709 13:05:37.947691 (+   410us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:37.947787 (+    96us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:37.947812 (+    25us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:05:37.948007 (+   195us) tablet.cc:701] Row locks acquired
0709 13:05:37.948024 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:05:37.948313 (+   289us) write_op.cc:270] Start()
0709 13:05:37.948523 (+   210us) write_op.cc:276] Timestamp: P: 1783602337948255 usec, L: 0
0709 13:05:37.948546 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:05:37.949289 (+   743us) log.cc:844] Serialized 536 byte log entry
0709 13:05:37.953051 (+  3762us) op_driver.cc:464] REPLICATION: finished
0709 13:05:37.953402 (+   351us) write_op.cc:301] APPLY: starting
0709 13:05:37.953486 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:37.954247 (+   761us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:37.954345 (+    98us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:37.955088 (+   743us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:37.955230 (+   142us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=4,mrs_lookups=0
0709 13:05:37.955277 (+    47us) write_op.cc:312] APPLY: finished
0709 13:05:37.955482 (+   205us) log.cc:844] Serialized 25 byte log entry
0709 13:05:37.955664 (+   182us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:37.955750 (+    86us) write_op.cc:454] Released schema lock
0709 13:05:37.955796 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:37.956118 (+   322us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:37.956150 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":54,"child_traces":[["op",{"apply.queue_time_us":107,"apply.run_cpu_time_us":2691,"apply.run_wall_time_us":2692,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4182,"num_ops":1,"prepare.queue_time_us":386,"prepare.run_cpu_time_us":2967,"prepare.run_wall_time_us":2968,"replication_time_us":4367}]]}
I20260709 13:05:38.136049  9561 leader_election.cc:290] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:38.165032  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:38.159799  1537 leader_election.cc:304] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:38.112989  1465 catalog_manager.cc:5697] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:38.172073  1465 catalog_manager.cc:5697] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 15 to 16, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 16 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:38.173990  1465 catalog_manager.cc:5697] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 18 to 20, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 20 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:38.182137  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7501558aa93542308c5fc758446387df" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:38.179808  9561 raft_consensus.cc:2749] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:38.189179   431 tablet_replica.cc:333] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.191110  1537 leader_election.cc:304] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:38.192606  9561 raft_consensus.cc:2749] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:38.213454   431 raft_consensus.cc:2243] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.176859  1465 catalog_manager.cc:5697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 20 to 24, leader changed from cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) to 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193). New cstate: current_term: 24 leader_uuid: "066ee0109aba4465ba17fa65e14ef504" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:05:38.220103  1611 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:38.221407   431 raft_consensus.cc:2272] T dcc462cf3919460abb1ef504377a5382 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:38.221684  1611 leader_election.cc:336] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:38.222445  1611 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:38.224689  9549 raft_consensus.cc:2749] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:38.231788  9552 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:38.232496  9552 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:38.234855  9552 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:38.236933  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:38.245543  1611 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:38.246322  1611 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:38.245265  1465 catalog_manager.cc:5697] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 13 to 15, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 15 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:38.249467  9496 raft_consensus.cc:2749] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:38.253883   431 tablet_replica.cc:333] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.280098   431 raft_consensus.cc:2243] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.284662   431 raft_consensus.cc:2272] T 7e9b1a584a5f430a9a0e7e617064bc2c P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:38.282032  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 57924.5us
I20260709 13:05:38.306119   431 tablet_replica.cc:333] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.307608   431 raft_consensus.cc:2243] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.329981   431 raft_consensus.cc:2272] T f0ee72a7f576482aabe40ef837964708 P 066ee0109aba4465ba17fa65e14ef504 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.336500  9555 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:38.337227  9555 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:38.338876   431 tablet_replica.cc:333] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.339432  9555 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:38.340308   431 raft_consensus.cc:2243] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.342585   431 raft_consensus.cc:2272] T 56082a5ff84643f0ba368ceba0524623 P 066ee0109aba4465ba17fa65e14ef504 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:38.352480  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 57303.8us
W20260709 13:05:38.356911  1685 leader_election.cc:336] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:38.358263   431 tablet_replica.cc:333] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.359144   431 raft_consensus.cc:2243] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.359692   431 pending_rounds.cc:70] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Trying to abort 1 pending ops.
I20260709 13:05:38.359941   431 pending_rounds.cc:77] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504: Aborting op as it isn't in flight: id { term: 14 index: 10 } timestamp: 7305634925409378304 op_type: NO_OP noop_request { }
I20260709 13:05:38.360271   431 raft_consensus.cc:2889] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:38.360591   431 raft_consensus.cc:2272] T ca833b0b78d74adc8ca86e91fc94f96f P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:38.364725  1611 consensus_peers.cc:597] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:38.367451  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:38.368013  1645 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:05:38.370199  1687 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:05:38.372643  1611 consensus_peers.cc:597] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:38.374334  9573 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:38.374785  9573 raft_consensus.cc:493] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:05:38.374868  1611 consensus_peers.cc:597] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:38.375212  9573 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:38.381619  9573 raft_consensus.cc:515] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:38.385344   431 tablet_replica.cc:333] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.386399   431 raft_consensus.cc:2243] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:38.386848  1685 leader_election.cc:336] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:38.395859  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51c23e1c3ae4bdbbb5f21f1607da772" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:38.396987  1645 raft_consensus.cc:3060] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:38.410470  9573 leader_election.cc:290] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:38.414444  1645 raft_consensus.cc:2468] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:05:38.418011  1459 maintenance_manager.cc:643] P 220fdfc07a3c4ce28ca7333445633466: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.890s	user 0.113s	sys 0.036s Metrics: {"cfile_cache_miss":9,"cfile_cache_miss_bytes":28109,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":85566,"dirs.run_cpu_time_us":717,"dirs.run_wall_time_us":4507,"lbm_read_time_us":13315,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":24,"lbm_write_time_us":51415,"lbm_writes_10-100_ms":2,"lbm_writes_lt_1ms":10,"peak_mem_usage":304728,"reinsert_count":0,"thread_start_us":77797,"threads_started":2,"update_count":290}
I20260709 13:05:38.421821  1460 maintenance_manager.cc:419] P 220fdfc07a3c4ce28ca7333445633466: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260709 13:05:38.420708  1465 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet 58b222526a1a4b52bdcda825e15c46f9 with cas_config_opid_index 5: aborting the task: latest config opid_index 13; task opid_index 5
I20260709 13:05:38.424034  1465 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47518 (request call id 103) took 6755 ms. Trace:
I20260709 13:05:38.424928  1465 rpcz_store.cc:276] 0709 13:05:31.669234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.161836 (+2492602us) service_pool.cc:224] Handling call
0709 13:05:38.423965 (+4262129us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:38.265979 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:38.266971 (+   992us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:38.267184 (+   213us) write_op.cc:435] Acquired schema lock
0709 13:05:38.267200 (+    16us) tablet.cc:662] Decoding operations
0709 13:05:38.268578 (+  1378us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:38.268641 (+    63us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:38.268675 (+    34us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:05:38.269211 (+   536us) tablet.cc:701] Row locks acquired
0709 13:05:38.269229 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:05:38.269364 (+   135us) write_op.cc:270] Start()
0709 13:05:38.269469 (+   105us) write_op.cc:276] Timestamp: P: 1783602338269323 usec, L: 0
0709 13:05:38.269495 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:05:38.270165 (+   670us) log.cc:844] Serialized 2069 byte log entry
0709 13:05:38.275874 (+  5709us) op_driver.cc:464] REPLICATION: finished
0709 13:05:38.276395 (+   521us) write_op.cc:301] APPLY: starting
0709 13:05:38.276480 (+    85us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:38.284223 (+  7743us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:38.284410 (+   187us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:38.301063 (+ 16653us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:38.301262 (+   199us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 13:05:38.301302 (+    40us) write_op.cc:312] APPLY: finished
0709 13:05:38.301591 (+   289us) log.cc:844] Serialized 57 byte log entry
0709 13:05:38.309109 (+  7518us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:38.311026 (+  1917us) write_op.cc:454] Released schema lock
0709 13:05:38.315798 (+  4772us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:38.316652 (+   854us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:38.316702 (+    50us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":34444,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":309,"apply.run_cpu_time_us":11176,"apply.run_wall_time_us":39992,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25037,"cfile_init":1,"lbm_read_time_us":2784,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":3,"num_ops":5,"prepare.queue_time_us":3526,"prepare.run_cpu_time_us":3847,"prepare.run_wall_time_us":4451,"replication_time_us":6235}]]}
I20260709 13:05:38.424052  1459 maintenance_manager.cc:643] P 220fdfc07a3c4ce28ca7333445633466: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
I20260709 13:05:38.428987  1687 leader_election.cc:304] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:38.430109  9573 raft_consensus.cc:2804] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:38.431007  9573 raft_consensus.cc:697] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:38.431919  9573 consensus_queue.cc:237] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:38.437769   431 raft_consensus.cc:2272] T 15e07cb6db0d46a78611f24c17d04cc9 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:38.475104  2312 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 31: this message will repeat every 5th retry.
I20260709 13:05:38.480443   431 tablet_replica.cc:333] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.510914   431 raft_consensus.cc:2243] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.511943   431 raft_consensus.cc:2272] T e59deed37c414a929faeb85bdbaaa406 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.516024   431 tablet_replica.cc:333] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.520190   431 raft_consensus.cc:2243] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.539382   431 raft_consensus.cc:2272] T e89a11b4204e48568da409c196e33be4 P 066ee0109aba4465ba17fa65e14ef504 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:38.549913  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 45983.2us
I20260709 13:05:38.551438   431 tablet_replica.cc:333] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.552556   431 raft_consensus.cc:2243] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.576824   431 raft_consensus.cc:2272] T 1a01faf4c6c64b549e80adf03bba9f51 P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:38.623749  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 36: this message will repeat every 5th retry.
I20260709 13:05:38.625597   431 tablet_replica.cc:333] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.626752   431 raft_consensus.cc:2243] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.627992   431 raft_consensus.cc:2272] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.630981   431 tablet_replica.cc:333] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.632061   431 raft_consensus.cc:2243] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.638214   431 raft_consensus.cc:2272] T 49c94b9e96df4808adbda99815b09c5f P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.642122   431 tablet_replica.cc:333] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.646490   431 raft_consensus.cc:2243] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.630643  1713 raft_consensus.cc:1275] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:05:38.690660  1538 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:38.758159  1611 consensus_peers.cc:597] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:38.783996  9579 consensus_queue.cc:1048] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:38.858652   431 raft_consensus.cc:2272] T f81336f0bd194664ac95af1a897b4b18 P 066ee0109aba4465ba17fa65e14ef504 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.869707   431 tablet_replica.cc:333] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.874146   431 raft_consensus.cc:2243] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.877099   431 raft_consensus.cc:2272] T 12ac45e0e48449a98fb89704bae3de27 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.883762   431 tablet_replica.cc:333] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.885715   431 raft_consensus.cc:2243] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.887153   431 raft_consensus.cc:2272] T 7501558aa93542308c5fc758446387df P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.938697  9574 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:38.939833  9574 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:38.944940  9574 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:38.949822  1685 leader_election.cc:336] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:38.971035  1465 catalog_manager.cc:5697] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 18 to 19, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 19 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:38.974141  2312 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:38.953879   431 tablet_replica.cc:333] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:38.985991   431 raft_consensus.cc:2243] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.988302   431 pending_rounds.cc:70] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Trying to abort 1 pending ops.
I20260709 13:05:38.993944  9552 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:05:38.995657  1685 consensus_peers.cc:597] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:38.998270   431 pending_rounds.cc:77] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504: Aborting op as it isn't in flight: id { term: 17 index: 9 } timestamp: 7305635087665139712 op_type: NO_OP noop_request { }
I20260709 13:05:39.016049   431 raft_consensus.cc:2889] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:39.016979   431 raft_consensus.cc:2272] T f50422a819a849c0a93a4a07c9af6ba0 P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:39.026050  9552 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.040809  9543 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.042153  9543 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:39.062450  9573 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.063958  9573 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.063988   431 tablet_replica.cc:333] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:39.083102  9556 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.083379  9543 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:39.084259  9556 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.087296  9556 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:39.093001  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:39.095295  1640 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 24.
W20260709 13:05:39.097167  1611 consensus_peers.cc:597] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:39.105183  9584 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.106230  9552 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:39.106451  9584 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.110288  9488 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.111266  9488 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.112928  9584 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:39.114492  9488 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:39.123201  2314 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:39.125248  9582 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:05:39.128288  9582 raft_consensus.cc:493] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
W20260709 13:05:39.100028  1611 consensus_peers.cc:597] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:39.108476  9561 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.118963  9552 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.130877  9561 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.131976  9581 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.133093  9581 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.134578  9561 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:39.129654  9582 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:39.138686   431 raft_consensus.cc:2243] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:39.140801   431 raft_consensus.cc:2272] T dbcb28b8d4104c6fae6682866cc0469d P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:39.141932  9581 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:39.151254  1685 consensus_peers.cc:597] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:39.151692  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:39.151561  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "418152c2665a4b04be8a39c62a52f6ca" candidate_uuid: "066ee0109aba4465ba17fa65e14ef504" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:39.164124  1538 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 066ee0109aba4465ba17fa65e14ef504; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:39.165967  9561 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:39.166069  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:39.167117  1647 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 17.
I20260709 13:05:39.107793  9573 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:39.131127  9552 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.217307   431 tablet_replica.cc:333] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
W20260709 13:05:39.224032  1685 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:39.228925  1685 leader_election.cc:336] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:39.231022  9573 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.231819  9573 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.237452  9573 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:39.238350  1685 leader_election.cc:336] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:39.239758  2312 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:39.241890  1687 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:39.242411  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:39.243618  1713 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 20.
I20260709 13:05:39.242678  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:39.244688  1715 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 19.
I20260709 13:05:39.247610  9552 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:39.248724  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:39.249488  1720 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 20.
I20260709 13:05:39.249562  9573 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:39.250146  9573 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.250543  9573 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:39.205179  9582 raft_consensus.cc:515] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:39.259886  9582 leader_election.cc:290] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 25 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:39.262176  2312 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:39.262773  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08cf0535992b4db395d567393667593b" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:39.265892  1642 raft_consensus.cc:3060] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:05:39.264528  1685 leader_election.cc:336] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:39.278613   431 raft_consensus.cc:2243] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:39.279834   431 raft_consensus.cc:2272] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:39.287182  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:39.288118  1716 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 21.
I20260709 13:05:39.289290  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:39.290287  1643 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:05:39.291929  9573 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.293121  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:39.294365  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:39.299868  1687 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:39.300372  1645 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
W20260709 13:05:39.289237  1685 consensus_peers.cc:597] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:39.301723  1643 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:05:39.300522  1629 tablet_service.cc:1558] Processing DeleteTablet for tablet eca30b43c26f420c965ee25f475581a0 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 14)) from {username='slave'} at 127.0.0.1:47496
I20260709 13:05:39.304629  9573 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:39.295640  1611 leader_election.cc:336] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:39.296471  2312 leader_election.cc:336] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:39.298225  1642 raft_consensus.cc:2468] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 25.
I20260709 13:05:39.313848  9574 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:39.314435  9574 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.314833  9574 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:39.315398  1611 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:39.316207  1611 consensus_peers.cc:597] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:39.317116  1611 consensus_peers.cc:597] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:39.318544  1611 leader_election.cc:336] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:39.305351  1468 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38662 (request call id 215) took 3370 ms. Trace:
I20260709 13:05:39.321897  1468 rpcz_store.cc:276] 0709 13:05:35.934462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.934635 (+   173us) service_pool.cc:224] Handling call
0709 13:05:39.304952 (+3370317us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":217,"reported_tablets":54,"spinlock_wait_cycles":12416}
I20260709 13:05:39.324373  1687 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:39.325757  9488 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:39.326313  1687 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:39.326314  9488 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.327188  9488 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:39.329829   431 tablet_replica.cc:333] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:39.330843  9573 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:39.317018  1612 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:39.333784  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:39.339350  1643 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 21.
W20260709 13:05:39.335688  1611 consensus_peers.cc:597] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:39.338770  1685 leader_election.cc:336] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:39.344630  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:39.345321  1645 raft_consensus.cc:3060] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:39.345223  1612 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
W20260709 13:05:39.343771  1611 leader_election.cc:336] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:39.347306  1687 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:39.348939  9586 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:39.349335  9586 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.349759  9586 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:39.352247  2312 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:39.360286  9586 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.363173   431 raft_consensus.cc:2243] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:39.362996  9581 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:05:39.363617  1455 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 116079us
I20260709 13:05:39.364279  9581 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.364653   431 raft_consensus.cc:2272] T 418152c2665a4b04be8a39c62a52f6ca P 066ee0109aba4465ba17fa65e14ef504 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:39.364799  9581 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:39.368745   431 tablet_replica.cc:333] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:39.363618  9579 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:39.370441  9579 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.370990  9579 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:39.338871  9573 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.373023  9573 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:39.378332  1455 catalog_manager.cc:4755] TS 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 58b222526a1a4b52bdcda825e15c46f9: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:05:39.386179  1642 raft_consensus.cc:1275] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:05:39.369874   431 raft_consensus.cc:2243] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:39.390516   431 raft_consensus.cc:2272] T d2618e77160b44339d8499efcd1e8eca P 066ee0109aba4465ba17fa65e14ef504 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:39.394716  9581 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.396533   431 tablet_replica.cc:333] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:39.397730   431 raft_consensus.cc:2243] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:05:39.398563  1612 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:39.399184   431 raft_consensus.cc:2272] T 6b98c1e123bd4894b2ab4096efb42cd8 P 066ee0109aba4465ba17fa65e14ef504 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:39.400859  9580 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:39.400977  9579 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.401449  9580 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.404105   431 tablet_replica.cc:333] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:39.404781  9580 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:39.405260   431 raft_consensus.cc:2243] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:39.407055   431 raft_consensus.cc:2272] T eca30b43c26f420c965ee25f475581a0 P 066ee0109aba4465ba17fa65e14ef504 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:39.412582  1645 raft_consensus.cc:2468] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:05:39.426237  9579 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:39.427554  9488 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.428390  9581 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:39.428046  9574 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.426592   431 tablet_replica.cc:333] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
W20260709 13:05:39.436547  1685 leader_election.cc:336] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:39.439497  1612 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:39.444792  9581 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:39.445094  2314 leader_election.cc:304] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:39.464226  9552 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.445427  9581 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.465796  9570 raft_consensus.cc:2804] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:05:39.466675  9570 raft_consensus.cc:697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 25 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:05:39.461205  9584 consensus_queue.cc:1048] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:05:39.467793  9570 consensus_queue.cc:237] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:39.470002   431 raft_consensus.cc:2243] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:39.465941  9581 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:39.471472   431 raft_consensus.cc:2272] T adc326ba3c1b4dc7aa89a3a071803a7e P 066ee0109aba4465ba17fa65e14ef504 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:39.465058  9552 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:39.476058  1685 leader_election.cc:336] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:39.477664  9586 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:39.477478  1457 catalog_manager.cc:5028] TS f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): tablet eca30b43c26f420c965ee25f475581a0 (table test-workload [id=3bdc78e1f00f46baa4d8126b52183327]) successfully deleted
I20260709 13:05:39.479774  9552 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:39.483125  1685 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:39.485699  1687 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:39.489530  9488 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:39.485155  9573 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.491292  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:39.463842  9580 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.496904  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:39.486897  9586 raft_consensus.cc:2804] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:39.519424  1643 raft_consensus.cc:2393] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 20: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:39.538205  9594 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.539080  9594 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.541955  9594 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:39.516086  9574 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:39.546293  1687 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:39.526690  9586 raft_consensus.cc:697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:05:39.554003  1611 leader_election.cc:336] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:39.557459  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:39.559437  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
W20260709 13:05:39.556505  1611 leader_election.cc:336] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:39.560518  1642 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 21: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:39.563139   431 tablet_replica.cc:333] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504: stopping tablet replica
I20260709 13:05:39.563972   431 raft_consensus.cc:2243] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:39.565564   431 raft_consensus.cc:2272] T ad4b6776963146fab5ee9487b6684e8c P 066ee0109aba4465ba17fa65e14ef504 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:39.568101  9573 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:39.551429  9594 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:05:39.574726  1611 consensus_peers.cc:597] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:39.582147  1685 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:39.588102  9586 consensus_queue.cc:237] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:39.585582  1611 consensus_peers.cc:597] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:39.624598  9581 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.629168  1687 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:39.634874  9580 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:39.638571  1685 consensus_peers.cc:597] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:39.643311  9573 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.644270  9573 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:39.645699  1685 leader_election.cc:336] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:39.646667  1685 consensus_peers.cc:597] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:39.651626  1685 consensus_peers.cc:597] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:39.655900  9573 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:39.657338  1647 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 22: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:39.658946  1687 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:39.659868  9488 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:39.666074  9573 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:39.671511  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:39.672478  1647 raft_consensus.cc:2393] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 21: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:39.673362  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:39.674090  1643 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 23.
I20260709 13:05:39.677374  9581 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:39.686915  1687 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:39.688522  9594 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:05:39.702667  1685 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:39.704890  1611 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:39.709343  1687 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:39.689071  9556 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.710495  9556 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.711035  9594 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:39.711427  9581 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.712328  9581 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.713608  9556 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:39.715758  9581 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:39.719565  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:39.720103  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:39.720578  1714 raft_consensus.cc:2393] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 20: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:39.720923  1720 raft_consensus.cc:2393] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 21: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:39.721890  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:39.722536  1719 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 23.
I20260709 13:05:39.732360  1687 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:39.774981  1465 catalog_manager.cc:5697] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 24 to 25, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 25 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:39.748021  9594 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:39.786756  9594 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.787241  9594 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:39.787606  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:39.788633  1718 raft_consensus.cc:2393] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 21: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:05:39.796614  1611 consensus_peers.cc:597] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:39.806190  1685 consensus_peers.cc:597] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:39.812724  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 52869.3us
W20260709 13:05:39.814640  1611 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:39.823159  9549 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.824132  9549 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.838188  1468 catalog_manager.cc:5697] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 19 to 20, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 20 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:39.842363  9552 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.843158  9552 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.843674  9549 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:39.844898  9573 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.845777  9573 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.848765  9573 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:39.856559  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:39.857448  9552 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:39.857385  1644 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 25.
W20260709 13:05:39.855294  1611 leader_election.cc:336] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:39.882932  1611 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:39.890384  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:39.890867  9598 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.892074  9598 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:39.894084  1715 raft_consensus.cc:2393] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 22: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:05:39.892086  1611 leader_election.cc:336] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:39.901472  1685 consensus_peers.cc:597] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:39.902671  1644 raft_consensus.cc:1275] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:05:39.903599  1687 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:39.907375  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:39.908178  1644 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:05:39.910395  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:39.911628  1716 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 21.
I20260709 13:05:39.903656  9598 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:39.909822  1685 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:39.913335  9603 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:05:39.916960  9603 raft_consensus.cc:493] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.917449  9603 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:05:39.918112  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:39.919222  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:39.919925  1713 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 20.
I20260709 13:05:39.910424  9543 consensus_queue.cc:1048] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.013s
I20260709 13:05:39.924911  1612 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:39.929792  1612 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:39.930764  9549 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:39.931732  9549 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:39.933037  9594 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:39.937842  1685 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:39.941699  1612 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:39.943220  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:39.949685  9594 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:39.951035  9556 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:39.951644  9556 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.952083  9556 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:39.957183  1612 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
W20260709 13:05:39.957608  1685 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:39.958627  9580 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:39.971578  1612 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:39.972535  9372 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:39.973304  1612 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:39.982120  9556 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:39.986354  1611 consensus_peers.cc:597] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:39.987161  1611 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:39.987768  9552 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:39.988431  9552 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:39.988909  9552 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:39.993626  9556 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:39.983644  1612 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:39.998880  1611 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:40.000007  9556 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:40.000691  9556 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:40.001111  9556 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:40.003425  1687 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:40.004492  9598 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:40.004964  9598 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:40.005363  9598 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:40.011092  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:40.015071  1640 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 24: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:40.017083  9556 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:40.008745  1612 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:40.019011  9549 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:40.020124  9556 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:40.021020  1685 consensus_peers.cc:597] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:40.022693  9603 raft_consensus.cc:515] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:40.023038  9598 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:40.024003  1718 raft_consensus.cc:2393] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 26: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:05:40.025195  9603 leader_election.cc:290] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 26 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:40.029134  1685 leader_election.cc:336] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:40.030802  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:40.031693  1716 raft_consensus.cc:2393] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 24: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:05:40.033397  1685 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:40.034888  1687 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:40.036228  9573 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:05:40.040016  1644 consensus_queue.cc:237] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:05:40.038396  9598 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:40.052232  1612 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:40.057166  9549 raft_consensus.cc:3060] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:05:40.057585  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:40.059051  1716 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f5f4b2707a9647e5bb99e065068dea2b in current term 21: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
W20260709 13:05:40.060717  2312 consensus_peers.cc:597] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:40.066186  9549 raft_consensus.cc:2749] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:05:40.070518  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81336f0bd194664ac95af1a897b4b18" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:40.081527  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:40.082923  1642 raft_consensus.cc:2393] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 130bc5ff39b847b1b3204626f284f838 in current term 21: Already voted for candidate f5f4b2707a9647e5bb99e065068dea2b in this term.
I20260709 13:05:40.085740  1647 raft_consensus.cc:1275] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:40.104717  1645 raft_consensus.cc:2468] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 26.
W20260709 13:05:40.131131  1611 consensus_peers.cc:597] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:40.137599  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 43741us
W20260709 13:05:40.134307  1685 consensus_peers.cc:597] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:40.147598  9552 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:40.138655  1611 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:40.166183  1611 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:40.167495  9556 raft_consensus.cc:2749] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:05:40.175521  9552 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:40.177064  1464 catalog_manager.cc:5697] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 17 to 18, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 18 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:40.182214  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e9b1a584a5f430a9a0e7e617064bc2c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:40.220657  1717 raft_consensus.cc:3060] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:40.189442  1716 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:05:40.202740  9598 consensus_queue.cc:1048] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:40.205200  1687 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:40.227434  1687 leader_election.cc:304] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:40.227736  9573 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:05:40.212877  1685 consensus_peers.cc:597] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:40.229347  9488 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:40.232409  9488 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:40.234915  9488 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
I20260709 13:05:40.229504  9573 raft_consensus.cc:2804] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:05:40.241498  9573 raft_consensus.cc:697] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 26 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:40.242697  9573 consensus_queue.cc:237] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:40.246928  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:40.247720  1647 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 21.
W20260709 13:05:40.260704  1685 leader_election.cc:336] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:40.262995  9556 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:05:40.264869  1717 raft_consensus.cc:2468] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 22.
W20260709 13:05:40.272598  2312 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:40.280714  2312 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:40.281153  1611 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:40.283746  1611 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:40.287708  9556 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:05:40.283319  2312 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:40.297693  1612 consensus_peers.cc:597] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e87427e2b27d4f89b7953163626fd8e4. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:40.318626  1685 consensus_peers.cc:597] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:40.319684  1685 consensus_peers.cc:597] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:40.322925  1612 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:40.327028  9614 raft_consensus.cc:2804] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:40.327570  9614 raft_consensus.cc:697] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 22 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:40.328533  9614 consensus_queue.cc:237] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:40.354976  1687 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:40.356041  9573 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:40.356597  9573 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:40.357074  9573 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:40.364001  9552 raft_consensus.cc:2955] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 LEADER]: Committing config change with OpId 19.15: config changed from index -1 to 15, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:05:40.367028  1716 raft_consensus.cc:2955] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Committing config change with OpId 19.15: config changed from index -1 to 15, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:05:40.367879  1464 catalog_manager.cc:5697] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 25 to 26, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 26 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:40.397862  1468 catalog_manager.cc:5697] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 21 to 22, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 22 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:40.377620  9573 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:40.434079  9573 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999)
W20260709 13:05:40.437707  1685 leader_election.cc:336] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:40.449517  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:40.450392  1647 raft_consensus.cc:3060] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:40.461134  1611 consensus_peers.cc:597] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:40.466976  1455 catalog_manager.cc:4755] TS 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 58b222526a1a4b52bdcda825e15c46f9: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
W20260709 13:05:40.468538  1611 consensus_peers.cc:597] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:40.469431  1611 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:40.472684  1464 catalog_manager.cc:5697] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: config changed from index -1 to 15, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 19 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:05:40.483227  1647 raft_consensus.cc:2468] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 22.
I20260709 13:05:40.509653  9603 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:40.509750  1687 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:40.510550  9603 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:40.511595  9573 raft_consensus.cc:2804] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:40.512602  9573 raft_consensus.cc:697] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 22 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:40.513839  9603 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:40.513792  9573 consensus_queue.cc:237] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:40.534961  1611 consensus_peers.cc:597] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:40.535567  1611 consensus_peers.cc:597] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:40.536229  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:40.564668  9625 ts_tablet_manager.cc:933] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:40.566558  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:40.566913  1464 catalog_manager.cc:5697] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 21 to 22, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 22 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:40.598109  1611 consensus_peers.cc:597] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:40.605549  9625 tablet_copy_client.cc:323] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:44293
W20260709 13:05:40.609485  1685 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:40.610169  1685 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:40.609455  1657 tablet_copy_service.cc:140] P f5f4b2707a9647e5bb99e065068dea2b: Received BeginTabletCopySession request for tablet e87427e2b27d4f89b7953163626fd8e4 from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:44424)
I20260709 13:05:40.612311  1657 tablet_copy_service.cc:161] P f5f4b2707a9647e5bb99e065068dea2b: Beginning new tablet copy session on tablet e87427e2b27d4f89b7953163626fd8e4 from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:44424: session id = cf2f71592be8465fb18e36e15d0b2f89-e87427e2b27d4f89b7953163626fd8e4
I20260709 13:05:40.623271  9573 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:40.631289  1657 tablet_copy_source_session.cc:215] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:05:40.635769  1519 auto_rebalancer.cc:629] Failed to schedule move for tablet e89a11b4204e48568da409c196e33be4: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:40.636343  9625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e87427e2b27d4f89b7953163626fd8e4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:40.639932  1519 auto_rebalancer.cc:629] Failed to schedule move for tablet 0df07602233941598bee7b19607b2e3b: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:40.648437  1647 consensus_queue.cc:237] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:05:40.655984  9552 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:40.656966  9552 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:40.660349  9552 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:40.662531  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:40.663405  1611 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:40.663647  1713 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 22.
I20260709 13:05:40.666728  1612 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:40.668224  9614 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:40.669164  9614 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 066ee0109aba4465ba17fa65e14ef504)
I20260709 13:05:40.669806  9614 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:40.680598  9625 tablet_copy_client.cc:806] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:40.681823  9625 tablet_copy_client.cc:670] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:40.687572  9625 tablet_copy_client.cc:538] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:40.696973  9614 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:40.730459  1713 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:05:40.736460  1612 consensus_peers.cc:597] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b -> Peer cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935): Couldn't send request to peer cf2f71592be8465fb18e36e15d0b2f89. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: adc326ba3c1b4dc7aa89a3a071803a7e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:40.740116  9616 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:05:40.748075  9614 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:40.779629  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b98c1e123bd4894b2ab4096efb42cd8" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:40.780309  1716 raft_consensus.cc:3060] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:40.796339  1716 raft_consensus.cc:2468] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 23.
W20260709 13:05:40.810649  1685 consensus_peers.cc:597] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:40.824471  9625 tablet_bootstrap.cc:492] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
I20260709 13:05:40.826575  1640 raft_consensus.cc:1275] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 26 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:05:40.828490  9584 consensus_queue.cc:1048] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:05:40.847615  1611 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:05:40.848395  1611 consensus_peers.cc:597] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:40.851744  1612 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:40.854545  9579 raft_consensus.cc:2804] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:40.855587  9579 raft_consensus.cc:697] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 23 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:40.856966  9579 consensus_queue.cc:237] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:40.872740  1713 raft_consensus.cc:1275] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:05:40.873744  9616 raft_consensus.cc:2955] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Committing config change with OpId 18.14: config changed from index -1 to 14, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:05:40.884670  1713 raft_consensus.cc:2955] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Committing config change with OpId 18.14: config changed from index -1 to 14, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } } }
I20260709 13:05:40.892030  9631 consensus_queue.cc:1048] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:05:40.932191  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:40.943632  2312 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:40.947696  1685 consensus_peers.cc:597] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:40.949483  1611 consensus_peers.cc:597] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:41.012257  1647 raft_consensus.cc:1275] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:05:41.018827  9603 consensus_queue.cc:1048] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:05:41.064127  1464 catalog_manager.cc:5697] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 22 to 23, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 23 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:41.066531  1464 catalog_manager.cc:5697] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: config changed from index -1 to 14, NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) added. New cstate: current_term: 18 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:41.113570  2312 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:41.167593  1611 consensus_peers.cc:597] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:41.168193  1611 consensus_peers.cc:597] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:41.196141  9647 ts_tablet_manager.cc:933] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89: Initiating tablet copy from peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:41.199465  1611 consensus_peers.cc:597] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:41.210768  1611 consensus_peers.cc:597] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:41.226020  9648 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.226656  9648 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:41.230434  9648 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:41.230381  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:41.231630  1685 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111) [suppressed 425 similar messages]
I20260709 13:05:41.231676  1640 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 20.
I20260709 13:05:41.235030  1687 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
W20260709 13:05:41.238411  1685 leader_election.cc:336] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:41.240305  9648 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:41.240829  9648 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.243548  9648 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:41.255947  9648 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:41.258419  9648 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:41.261350  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:41.262161  1640 raft_consensus.cc:3060] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:41.264245  9647 tablet_copy_client.cc:323] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:44293
I20260709 13:05:41.266356  1657 tablet_copy_service.cc:140] P f5f4b2707a9647e5bb99e065068dea2b: Received BeginTabletCopySession request for tablet adc326ba3c1b4dc7aa89a3a071803a7e from peer cf2f71592be8465fb18e36e15d0b2f89 ({username='slave'} at 127.0.0.1:44424)
I20260709 13:05:41.267100  1657 tablet_copy_service.cc:161] P f5f4b2707a9647e5bb99e065068dea2b: Beginning new tablet copy session on tablet adc326ba3c1b4dc7aa89a3a071803a7e from peer cf2f71592be8465fb18e36e15d0b2f89 at {username='slave'} at 127.0.0.1:44424: session id = cf2f71592be8465fb18e36e15d0b2f89-adc326ba3c1b4dc7aa89a3a071803a7e
W20260709 13:05:41.272562  1685 leader_election.cc:336] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:41.283663  1640 raft_consensus.cc:2468] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 21.
I20260709 13:05:41.286958  1687 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:41.289180  9584 raft_consensus.cc:2804] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:41.289834  9584 raft_consensus.cc:697] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 21 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:05:41.291344  1611 consensus_peers.cc:597] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:41.291332  9584 consensus_queue.cc:237] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:41.295895  1657 tablet_copy_source_session.cc:215] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:41.301039  9647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adc326ba3c1b4dc7aa89a3a071803a7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:41.307585  1713 raft_consensus.cc:1275] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:05:41.314507  9579 consensus_queue.cc:1048] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:05:41.329948  9625 log.cc:927] Time spent T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89: Append to log took a long time: real 0.094s	user 0.002s	sys 0.000s
W20260709 13:05:41.378707  1685 consensus_peers.cc:597] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:41.381824  9625 tablet_bootstrap.cc:492] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=0 applied=15 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:41.398845  9625 tablet_bootstrap.cc:492] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:05:41.399669  9625 ts_tablet_manager.cc:1403] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 0.576s	user 0.052s	sys 0.014s
I20260709 13:05:41.407753  9625 raft_consensus.cc:359] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:05:41.408826  9625 raft_consensus.cc:740] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:05:41.416096  1465 catalog_manager.cc:5697] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 19 to 21, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 21 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:41.417712  9584 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.409595  9625 consensus_queue.cc:260] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:05:41.438153  9584 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:41.440239  9625 ts_tablet_manager.cc:1434] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.000s
I20260709 13:05:41.441219  9584 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:41.426786  2312 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:05:41.443390  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:41.444175  1640 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 21.
I20260709 13:05:41.432019  9647 tablet_copy_client.cc:806] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:41.445250  9647 tablet_copy_client.cc:670] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:41.445787  1687 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:41.456528  9648 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:41.457041  9648 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.457419  9648 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:41.463209  1657 tablet_copy_service.cc:342] P f5f4b2707a9647e5bb99e065068dea2b: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-e87427e2b27d4f89b7953163626fd8e4 received from {username='slave'} at 127.0.0.1:44424
I20260709 13:05:41.463828  1657 tablet_copy_service.cc:434] P f5f4b2707a9647e5bb99e065068dea2b: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-e87427e2b27d4f89b7953163626fd8e4 on tablet e87427e2b27d4f89b7953163626fd8e4 with peer cf2f71592be8465fb18e36e15d0b2f89
W20260709 13:05:41.492522  1611 consensus_peers.cc:597] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:05:41.498291  9640 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.510053  9640 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:41.512315  9640 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:41.489581  9648 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:41.514175  1685 leader_election.cc:336] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:41.514978  9648 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:41.517552  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:41.517876  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:41.518288  1643 raft_consensus.cc:3060] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:41.518689  1647 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 21.
I20260709 13:05:41.520907  1687 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:41.529749  9647 tablet_copy_client.cc:538] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:41.538496  2348 raft_consensus.cc:1217] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 LEARNER]: Deduplicated request from leader. Original: 19.14->[19.15-19.15]   Dedup: 19.15->[]
I20260709 13:05:41.533703  9648 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:41.539211  9648 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.539567  9648 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:41.544481  1643 raft_consensus.cc:2468] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 22.
W20260709 13:05:41.551295  1685 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:41.551994  9647 tablet_bootstrap.cc:492] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap starting.
W20260709 13:05:41.552213  1685 leader_election.cc:336] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:41.559056  1687 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:41.560294  9640 raft_consensus.cc:2804] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:41.561054  9640 raft_consensus.cc:697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 22 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:41.562043  9640 consensus_queue.cc:237] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:41.572985  1611 consensus_peers.cc:597] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:41.604763  1468 catalog_manager.cc:5697] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 20 to 22, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 22 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:41.618530  9648 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:41.636404  1685 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:41.638384  9584 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.639948  9584 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:41.642943  9584 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:41.647046  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:41.647847  1643 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 22.
I20260709 13:05:41.650563  1687 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:41.651944  9584 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:41.652403  9584 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.652745  9584 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:05:41.654069  1685 leader_election.cc:336] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:41.693274  9648 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:41.697392  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:41.698114  1643 raft_consensus.cc:3060] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:41.705945  1643 raft_consensus.cc:2468] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 22.
I20260709 13:05:41.700630  9584 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:41.719161  1687 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:41.719951  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:41.720544  1643 raft_consensus.cc:3060] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:41.721131  9648 raft_consensus.cc:2804] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:41.722028  9648 raft_consensus.cc:697] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 22 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:05:41.720214  1611 consensus_peers.cc:597] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:41.724553  1685 consensus_peers.cc:597] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:41.725113  1685 leader_election.cc:336] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:41.727783  1643 raft_consensus.cc:2468] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 23.
I20260709 13:05:41.725759  9584 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:41.729773  1687 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:41.723080  9648 consensus_queue.cc:237] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:41.731662  9640 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.732105  9644 raft_consensus.cc:2804] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:41.732213  9640 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:41.732831  9644 raft_consensus.cc:697] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 23 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:41.734683  9640 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:41.734298  9644 consensus_queue.cc:237] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:41.750854  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:41.751709  1647 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 24.
I20260709 13:05:41.761376  1687 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:41.766443  9584 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:05:41.770151  9584 raft_consensus.cc:493] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.772558  9584 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:05:41.773382  1685 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:05:41.783721   431 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 1594)
I20260709 13:05:41.804175  1468 catalog_manager.cc:5697] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 20 to 22, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 22 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:41.810720  9584 raft_consensus.cc:515] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:41.840312  9584 leader_election.cc:290] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 25 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:41.847152  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56082a5ff84643f0ba368ceba0524623" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b"
I20260709 13:05:41.848197  1647 raft_consensus.cc:3060] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:05:41.799643  1685 consensus_peers.cc:597] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:41.863242  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 63671.1us
W20260709 13:05:41.881122  1611 consensus_peers.cc:597] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:41.901428  1611 consensus_peers.cc:597] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:41.909853  2312 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:05:41.914924  1640 raft_consensus.cc:1275] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:05:41.922734  9644 consensus_queue.cc:1048] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:05:41.915426  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 51040.5us
I20260709 13:05:41.927873  1468 catalog_manager.cc:5697] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 21 to 23, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 23 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:41.916563  9647 tablet_bootstrap.cc:492] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:41.931559  9647 tablet_bootstrap.cc:492] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89: Bootstrap complete.
I20260709 13:05:41.932621  9647 ts_tablet_manager.cc:1403] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89: Time spent bootstrapping tablet: real 0.381s	user 0.057s	sys 0.005s
I20260709 13:05:41.944684  9632 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.945501  9632 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:41.948175  9632 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:41.950513  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:41.951354  1713 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 21.
I20260709 13:05:41.953835  1612 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:41.955986  9616 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:41.956431  9616 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.956779  9616 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:41.964428  9616 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:41.975170  9616 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 election: Requested vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:41.980715  9647 raft_consensus.cc:359] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:05:41.982069  9647 raft_consensus.cc:740] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Initialized, Role: LEARNER
I20260709 13:05:41.986452  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ee6d32e41946dc9cf53a3d62cdb9ae" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:05:41.987179  1715 raft_consensus.cc:3060] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:41.920130  1647 raft_consensus.cc:2468] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 25.
I20260709 13:05:41.994436  9579 raft_consensus.cc:1064] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
I20260709 13:05:41.997437  9579 consensus_queue.cc:237] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 13, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:42.009773  1685 leader_election.cc:336] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:42.042811  2343 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 LEARNER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:05:42.043884  1687 leader_election.cc:304] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504
I20260709 13:05:42.044369  1713 raft_consensus.cc:1275] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:05:42.045867  9644 raft_consensus.cc:2804] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:05:42.046631  9644 raft_consensus.cc:697] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 25 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:05:42.047777  9644 consensus_queue.cc:237] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:42.026939  1611 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:41.983224  9647 consensus_queue.cc:260] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: true } }
I20260709 13:05:42.055191  9647 ts_tablet_manager.cc:1434] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89: Time spent starting tablet: real 0.122s	user 0.006s	sys 0.004s
I20260709 13:05:42.034461  1715 raft_consensus.cc:2468] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5f4b2707a9647e5bb99e065068dea2b in term 22.
W20260709 13:05:42.058008  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 48270.1us
W20260709 13:05:42.062599  2312 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:42.067267  9579 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:05:42.068542  1612 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b; no voters: 
I20260709 13:05:42.071609  1657 tablet_copy_service.cc:342] P f5f4b2707a9647e5bb99e065068dea2b: Request end of tablet copy session cf2f71592be8465fb18e36e15d0b2f89-adc326ba3c1b4dc7aa89a3a071803a7e received from {username='slave'} at 127.0.0.1:44424
I20260709 13:05:42.072268  1657 tablet_copy_service.cc:434] P f5f4b2707a9647e5bb99e065068dea2b: ending tablet copy session cf2f71592be8465fb18e36e15d0b2f89-adc326ba3c1b4dc7aa89a3a071803a7e on tablet adc326ba3c1b4dc7aa89a3a071803a7e with peer cf2f71592be8465fb18e36e15d0b2f89
I20260709 13:05:42.078321  9616 raft_consensus.cc:2804] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:42.079424  9616 raft_consensus.cc:697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 22 LEADER]: Becoming Leader. State: Replica: f5f4b2707a9647e5bb99e065068dea2b, State: Running, Role: LEADER
I20260709 13:05:42.079790  9632 consensus_queue.cc:1048] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:05:42.081104  9616 consensus_queue.cc:237] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:42.099663  1685 consensus_peers.cc:597] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:42.112649  1611 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:05:42.145259  1685 consensus_peers.cc:597] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:42.153949  1643 raft_consensus.cc:1275] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:05:42.155820  9644 consensus_queue.cc:1048] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:42.166015  1468 catalog_manager.cc:5697] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b reported cstate change: term changed from 20 to 22, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194). New cstate: current_term: 22 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:42.170974  1611 consensus_peers.cc:597] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:42.184377  1465 catalog_manager.cc:5697] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 23 to 25, leader changed from 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 25 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:42.189491  1611 consensus_peers.cc:597] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:42.190158  1611 consensus_peers.cc:597] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:42.194489  1645 raft_consensus.cc:1275] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:05:42.196915  9584 consensus_queue.cc:1048] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:05:42.198163  1640 raft_consensus.cc:1275] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:05:42.203760  9584 consensus_queue.cc:1048] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:05:42.232132  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 88274us
I20260709 13:05:42.237870  2343 raft_consensus.cc:1217] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEARNER]: Deduplicated request from leader. Original: 18.13->[18.14-18.14]   Dedup: 18.14->[]
I20260709 13:05:42.235397  9641 raft_consensus.cc:2955] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 LEADER]: Committing config change with OpId 19.16: config changed from index 15 to 16, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:42.239668  2348 raft_consensus.cc:2955] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Committing config change with OpId 19.16: config changed from index 15 to 16, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:05:42.279585  1685 consensus_peers.cc:597] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:42.280242  1685 consensus_peers.cc:597] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:42.280741  1685 consensus_peers.cc:597] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:42.299475  1685 consensus_peers.cc:597] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:42.314585  1468 catalog_manager.cc:5697] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: config changed from index 15 to 16, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 19 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:42.365007  1715 raft_consensus.cc:2955] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Committing config change with OpId 19.16: config changed from index 15 to 16, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:05:42.413792  1685 consensus_peers.cc:597] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:42.453537  1685 consensus_peers.cc:597] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:42.454042  1685 consensus_peers.cc:597] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:42.539559  1643 raft_consensus.cc:1275] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 13:05:42.549182  1611 consensus_peers.cc:597] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:42.550623  9644 consensus_queue.cc:1048] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:05:42.559156  2312 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:42.564555  1455 catalog_manager.cc:4755] TS 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet ca2582f1941a4c588f8ef51c03dfb492: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:05:42.577082  1685 consensus_peers.cc:597] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:42.602167  1611 consensus_peers.cc:597] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:42.658047  1611 consensus_peers.cc:597] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:42.666241  2312 consensus_peers.cc:597] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:42.684034  1715 raft_consensus.cc:1275] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:05:42.687852  9641 consensus_queue.cc:1048] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:05:42.701078  1685 consensus_peers.cc:597] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:42.742107  9579 raft_consensus.cc:1064] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: attempting to promote NON_VOTER cf2f71592be8465fb18e36e15d0b2f89 to VOTER
W20260709 13:05:42.744607  1685 consensus_peers.cc:597] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:42.745277  9579 consensus_queue.cc:237] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 12, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:42.776278  1685 consensus_peers.cc:597] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:42.784296   431 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 1594)
I20260709 13:05:42.791224  2348 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEARNER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:05:42.793700  1716 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:05:42.797739  9632 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:05:42.799749  9614 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:05:42.870769  1611 consensus_peers.cc:597] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:42.871958  1611 consensus_peers.cc:597] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:42.979624  1685 consensus_peers.cc:597] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:42.980268  9674 log.cc:927] Time spent T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: Append to log took a long time: real 0.116s	user 0.000s	sys 0.004s
W20260709 13:05:42.984344  2312 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:42.984970  2312 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:43.008284  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 134627us
W20260709 13:05:43.022248  1685 consensus_peers.cc:597] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:43.051749  1611 consensus_peers.cc:597] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:43.097503  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:43.143774  9641 raft_consensus.cc:2955] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Committing config change with OpId 18.15: config changed from index 14 to 15, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:05:43.154328  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:05:43.166381  2346 raft_consensus.cc:2955] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Committing config change with OpId 18.15: config changed from index 14 to 15, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:43.231112  1468 catalog_manager.cc:5697] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: config changed from index 14 to 15, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 18 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:43.258589  1713 raft_consensus.cc:2955] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Committing config change with OpId 18.15: config changed from index 14 to 15, cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } attrs { replace: true } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:05:43.279929  1611 consensus_peers.cc:597] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:43.282205  1685 consensus_peers.cc:597] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:43.314990  1685 consensus_peers.cc:597] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:43.339835  1611 consensus_peers.cc:597] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:43.412066  1642 consensus_queue.cc:237] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:43.424716  2346 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:05:43.427323  1713 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Refusing update from remote peer f5f4b2707a9647e5bb99e065068dea2b: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:05:43.429468  9552 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:05:43.430919  2312 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:05:43.431283  9552 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:05:43.453445  9616 raft_consensus.cc:2955] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Committing config change with OpId 18.16: config changed from index 15 to 16, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:43.461505  2346 raft_consensus.cc:2955] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Committing config change with OpId 18.16: config changed from index 15 to 16, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:43.464887  1716 raft_consensus.cc:2955] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Committing config change with OpId 18.16: config changed from index 15 to 16, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
I20260709 13:05:43.516978  1457 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet adc326ba3c1b4dc7aa89a3a071803a7e with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:05:43.516959  2312 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:43.534442  1611 consensus_peers.cc:597] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:05:43.536999  1465 catalog_manager.cc:5697] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: config changed from index 15 to 16, VOTER 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193) evicted. New cstate: current_term: 18 leader_uuid: "f5f4b2707a9647e5bb99e065068dea2b" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:05:43.613111  1611 consensus_peers.cc:597] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:43.624910  1685 consensus_peers.cc:597] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:43.650429  1455 catalog_manager.cc:4755] TS 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet adc326ba3c1b4dc7aa89a3a071803a7e: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:05:43.652702  1611 consensus_peers.cc:597] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:43.746309  1611 consensus_peers.cc:597] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:43.765889  1611 consensus_peers.cc:597] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:43.784870   431 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 1594)
W20260709 13:05:43.826982  1456 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b82d3b49001741eda872554a7ecb2db3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 130bc5ff39b847b1b3204626f284f838 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 20 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } } }
W20260709 13:05:44.018790  2312 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:44.065469  1611 consensus_peers.cc:597] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:44.180433  1611 consensus_peers.cc:597] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:44.201625  2314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 68363.7us
W20260709 13:05:44.224208  1611 consensus_peers.cc:597] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:44.225745  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 91542.8us
W20260709 13:05:44.327813  1611 consensus_peers.cc:597] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:44.409701  1685 consensus_peers.cc:597] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:44.442873  1685 consensus_peers.cc:597] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:44.452445  1685 consensus_peers.cc:597] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:44.539677  2312 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:44.540336  2312 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:44.556551  1611 consensus_peers.cc:597] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:44.627912  1611 consensus_peers.cc:597] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:44.630786  1685 consensus_peers.cc:597] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:44.669188  1611 consensus_peers.cc:597] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:44.742748  1685 consensus_peers.cc:597] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:44.750496  1611 consensus_peers.cc:597] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:44.785573   431 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 1594)
W20260709 13:05:44.860350  1685 consensus_peers.cc:597] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:44.864409  1685 consensus_peers.cc:597] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:44.865200  1611 consensus_peers.cc:597] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:44.865302  1685 consensus_peers.cc:597] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:44.884357  1611 consensus_peers.cc:597] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:44.904177  1685 consensus_peers.cc:597] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:44.945607  1455 catalog_manager.cc:4755] TS 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet adc326ba3c1b4dc7aa89a3a071803a7e: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:05:45.086620  1685 consensus_peers.cc:597] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:45.129287  1685 consensus_peers.cc:597] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:45.149549  2312 consensus_peers.cc:597] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:45.150607  2312 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:45.149791  1685 consensus_peers.cc:597] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:45.172631  1685 consensus_peers.cc:597] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:45.182204  2312 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:45.216202  1685 consensus_peers.cc:597] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:45.235356  1685 consensus_peers.cc:597] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:45.255276  1685 consensus_peers.cc:597] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:45.540714  1611 consensus_peers.cc:597] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:45.559935  2312 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:45.585882  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:45.632186  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 56473.9us
W20260709 13:05:45.711493  1611 consensus_peers.cc:597] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:45.712857  1611 consensus_peers.cc:597] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:45.740401  1685 consensus_peers.cc:597] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:45.742034  1685 consensus_peers.cc:597] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:45.757501  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:45.786263   431 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 1594)
W20260709 13:05:45.830421  1685 consensus_peers.cc:597] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:45.899264  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 68875.4us
W20260709 13:05:45.908409  1611 consensus_peers.cc:597] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:45.941118  2312 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:45.941123  1611 consensus_peers.cc:597] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:45.942965  1611 consensus_peers.cc:597] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:46.054168  1455 catalog_manager.cc:4755] TS 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet adc326ba3c1b4dc7aa89a3a071803a7e: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:05:46.114727  2312 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:46.135499  1611 consensus_peers.cc:597] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:46.213682  1611 consensus_peers.cc:597] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:46.224594  1611 consensus_peers.cc:597] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:46.233896  1685 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111) [suppressed 521 similar messages]
W20260709 13:05:46.263192  1685 consensus_peers.cc:597] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:46.355042  1611 consensus_peers.cc:597] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:46.494987  2312 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:46.641832  2312 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:46.682443  1611 consensus_peers.cc:597] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:46.710127  1611 consensus_peers.cc:597] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:46.789728   431 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 1594)
W20260709 13:05:46.830956  1611 consensus_peers.cc:597] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:46.947398  1685 consensus_peers.cc:597] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:46.971165  1685 consensus_peers.cc:597] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:47.032099  2312 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:47.107854  1685 consensus_peers.cc:597] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:47.114573  1611 consensus_peers.cc:597] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:47.203586  1611 consensus_peers.cc:597] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:47.213759  1685 consensus_peers.cc:597] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.279258  1685 consensus_peers.cc:597] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.336510  1685 consensus_peers.cc:597] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:47.350158  1685 consensus_peers.cc:597] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.370482  1685 consensus_peers.cc:597] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:47.403892  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 50253.9us
W20260709 13:05:47.434774  1685 consensus_peers.cc:597] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.455497  1611 consensus_peers.cc:597] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:47.467065  1611 consensus_peers.cc:597] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:47.521827  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 55064.8us
W20260709 13:05:47.539389  1685 consensus_peers.cc:597] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:47.547425  1685 consensus_peers.cc:597] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.548135  1685 consensus_peers.cc:597] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.627718  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 62871.7us
W20260709 13:05:47.732643  2312 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:47.790480   431 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 1594)
W20260709 13:05:47.834836  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 54916.4us
W20260709 13:05:47.837524  2312 consensus_peers.cc:597] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:47.838616  2312 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:47.846194  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 48859.5us
W20260709 13:05:47.853839  1611 consensus_peers.cc:597] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:47.854521  1611 consensus_peers.cc:597] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:47.861907  1611 consensus_peers.cc:597] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:47.862591  1685 consensus_peers.cc:597] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.898726  1685 consensus_peers.cc:597] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.912477  1611 consensus_peers.cc:597] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.922132  1611 consensus_peers.cc:597] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:47.933733  1685 consensus_peers.cc:597] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:48.182025  1455 catalog_manager.cc:4755] TS 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet adc326ba3c1b4dc7aa89a3a071803a7e: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:05:48.206036  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:48.210093  1611 consensus_peers.cc:597] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:48.216622  2312 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:48.259675  1685 consensus_peers.cc:597] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:48.272560  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:48.298393  1685 consensus_peers.cc:597] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:48.334635  1685 consensus_peers.cc:597] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:48.335692  1685 consensus_peers.cc:597] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:48.339807  1611 consensus_peers.cc:597] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:48.396493  2312 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:48.453459  1611 consensus_peers.cc:597] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:48.528319  1611 consensus_peers.cc:597] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:48.541917  1611 consensus_peers.cc:597] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:48.676759  1685 consensus_peers.cc:597] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:48.684028  2312 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:48.748586  2312 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:48.775732  1611 consensus_peers.cc:597] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:48.791172   431 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 1594)
W20260709 13:05:48.914980  1611 consensus_peers.cc:597] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:49.217713  2312 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:49.273993  1685 consensus_peers.cc:597] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:49.341113  1611 consensus_peers.cc:597] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:49.546072  1611 consensus_peers.cc:597] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:49.551899  1611 consensus_peers.cc:597] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:49.617268  1611 consensus_peers.cc:597] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:49.624078  1611 consensus_peers.cc:597] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:49.629599  1685 consensus_peers.cc:597] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:49.655730  1611 consensus_peers.cc:597] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:49.686223  1685 consensus_peers.cc:597] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:49.709849  1611 consensus_peers.cc:597] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:49.724398  1611 consensus_peers.cc:597] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:49.738967  2312 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:49.739830  1685 consensus_peers.cc:597] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:49.793205   431 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 1594)
W20260709 13:05:49.924086  1685 consensus_peers.cc:597] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:49.939903  1685 consensus_peers.cc:597] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:49.949615  1685 consensus_peers.cc:597] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:49.976938  1685 consensus_peers.cc:597] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:49.982394  1611 consensus_peers.cc:597] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:50.034229  1685 consensus_peers.cc:597] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.035110  1685 consensus_peers.cc:597] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:50.098596  2312 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:50.140642  1685 consensus_peers.cc:597] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.147317  1685 consensus_peers.cc:597] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.279218  1611 consensus_peers.cc:597] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:50.280527  2312 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:50.308352  2312 consensus_peers.cc:597] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:50.386997  1611 consensus_peers.cc:597] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.459792  1685 consensus_peers.cc:597] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.474797  1611 consensus_peers.cc:597] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:50.492910  1685 consensus_peers.cc:597] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.554122  1611 consensus_peers.cc:597] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:50.618614  1685 consensus_peers.cc:597] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.662351  1685 consensus_peers.cc:597] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:50.705399  1685 consensus_peers.cc:597] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.792066  1611 consensus_peers.cc:597] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:50.793990   431 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 1594)
W20260709 13:05:50.808827  1685 consensus_peers.cc:597] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:50.823559  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:50.827605  2312 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:50.860671  1611 consensus_peers.cc:597] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:50.897789  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:50.976948  2312 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:51.089732  1685 consensus_peers.cc:597] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:51.141906  1611 consensus_peers.cc:597] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:51.148645  1611 consensus_peers.cc:597] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:51.171672  2312 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:51.193013  1611 consensus_peers.cc:597] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:51.227757  1685 consensus_peers.cc:597] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:51.238600  2312 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:51.246850  1611 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111) [suppressed 497 similar messages]
W20260709 13:05:51.296840  1611 consensus_peers.cc:597] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:51.349714  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 52945.3us
W20260709 13:05:51.537935  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 44727.1us
W20260709 13:05:51.543413  1611 consensus_peers.cc:597] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:51.635991  1685 consensus_peers.cc:597] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:51.692750  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 56800.3us
W20260709 13:05:51.794606   431 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 1594)
W20260709 13:05:51.928963  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 67338.2us
W20260709 13:05:51.961023  2312 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:52.082911  1685 consensus_peers.cc:597] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:52.086710  1685 consensus_peers.cc:597] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:52.085388  1611 consensus_peers.cc:597] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.159236  1611 consensus_peers.cc:597] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:52.212754  2312 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:52.243985  1611 consensus_peers.cc:597] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:52.311923  1685 consensus_peers.cc:597] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.317126  1611 consensus_peers.cc:597] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:52.318668  1611 consensus_peers.cc:597] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:52.354655  1455 catalog_manager.cc:4755] TS 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet adc326ba3c1b4dc7aa89a3a071803a7e: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:05:52.371163  1611 consensus_peers.cc:597] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:52.397419  1685 consensus_peers.cc:597] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.405383  1685 consensus_peers.cc:597] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:52.498397  1611 consensus_peers.cc:597] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:52.513407  1685 consensus_peers.cc:597] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.546070  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 84156.1us
W20260709 13:05:52.548440  1611 consensus_peers.cc:597] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:52.549103  1611 consensus_peers.cc:597] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:52.574515  1685 consensus_peers.cc:597] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:52.683148  1611 consensus_peers.cc:597] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:52.687474  1685 consensus_peers.cc:597] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:52.699137  2312 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:52.702381  1685 consensus_peers.cc:597] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.750723  1685 consensus_peers.cc:597] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.751278  1685 consensus_peers.cc:597] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.795217   431 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 1594)
W20260709 13:05:52.875252  1611 consensus_peers.cc:597] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:52.891716  2312 consensus_peers.cc:597] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:52.954550  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 43536.5us
W20260709 13:05:53.013845  2312 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:53.046187  1685 consensus_peers.cc:597] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:53.095266  1685 consensus_peers.cc:597] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:53.105628  1685 consensus_peers.cc:597] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:53.120644  2312 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:53.131601  1685 consensus_peers.cc:597] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:53.210930  1611 consensus_peers.cc:597] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:53.229614  1611 consensus_peers.cc:597] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:53.236774  1611 consensus_peers.cc:597] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:53.270516  1685 consensus_peers.cc:597] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:53.387434  1685 consensus_peers.cc:597] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:53.425858  2312 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:53.443459  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:53.531064  2312 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:53.564466  1685 consensus_peers.cc:597] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:53.619585  1685 consensus_peers.cc:597] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:53.621978  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:53.646696  1611 consensus_peers.cc:597] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:53.647246  2312 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:53.653062  1611 consensus_peers.cc:597] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:53.678007  1611 consensus_peers.cc:597] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:53.734756  1611 consensus_peers.cc:597] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:53.796232   431 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 1594)
W20260709 13:05:53.811419  1611 consensus_peers.cc:597] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:53.895452  1611 consensus_peers.cc:597] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:54.152153  1611 consensus_peers.cc:597] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:54.196272  1685 consensus_peers.cc:597] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:54.219887  1611 consensus_peers.cc:597] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:54.329885  1611 consensus_peers.cc:597] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:54.423391  2312 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:54.496169  1685 consensus_peers.cc:597] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:54.577827  1685 consensus_peers.cc:597] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:54.588016  1685 consensus_peers.cc:597] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:54.667385  2312 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:54.669724  1611 consensus_peers.cc:597] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:54.727090  1611 consensus_peers.cc:597] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:54.795902  1685 consensus_peers.cc:597] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:54.798576   431 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 1594)
W20260709 13:05:54.889353  1685 consensus_peers.cc:597] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:54.890266  1685 consensus_peers.cc:597] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:54.954873  1455 catalog_manager.cc:4755] TS 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 58b222526a1a4b52bdcda825e15c46f9: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:05:55.585842  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 695504us
W20260709 13:05:55.625928  2312 consensus_peers.cc:597] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:55.635613  2312 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:55.652690  2312 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:55.770857  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 99372.8us
W20260709 13:05:55.800036   431 thread.cc:527] Waited for 15000ms trying to join with diag-logger (tid 1594)
W20260709 13:05:56.086354  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 177741us
W20260709 13:05:56.240386  1502 debug-util.cc:398] Leaking SignalData structure 0x7b0800358700 after lost signal to thread 434
W20260709 13:05:56.241809  1502 debug-util.cc:398] Leaking SignalData structure 0x7b080035e2c0 after lost signal to thread 1505
W20260709 13:05:56.243310  1502 debug-util.cc:398] Leaking SignalData structure 0x7b08002f0dc0 after lost signal to thread 1672
W20260709 13:05:56.244374  1502 debug-util.cc:398] Leaking SignalData structure 0x7b0800333080 after lost signal to thread 1746
W20260709 13:05:56.245980  1502 debug-util.cc:398] Leaking SignalData structure 0x7b08004071e0 after lost signal to thread 2373
W20260709 13:05:56.297768  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 588470us
I20260709 13:05:56.301903  9750 raft_consensus.cc:493] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:56.303380  9750 raft_consensus.cc:515] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:56.302292  9759 raft_consensus.cc:493] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:56.332811  9759 raft_consensus.cc:515] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:56.335886  9759 leader_election.cc:290] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:56.342257  2312 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111) [suppressed 406 similar messages]
I20260709 13:05:56.352015  9750 leader_election.cc:290] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:56.354354  9772 raft_consensus.cc:493] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:56.354969  9772 raft_consensus.cc:515] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:56.357291  9772 leader_election.cc:290] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:56.353367  9750 raft_consensus.cc:493] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:56.364920  9750 raft_consensus.cc:515] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:56.374670  9774 raft_consensus.cc:493] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:56.375695  9774 raft_consensus.cc:515] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:56.367637  9750 leader_election.cc:290] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:56.399834  9774 leader_election.cc:290] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:56.412582  9774 raft_consensus.cc:493] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:56.413151  9774 raft_consensus.cc:515] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:05:56.416545  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 644309us
I20260709 13:05:56.422657  9776 raft_consensus.cc:493] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:56.423393  9776 raft_consensus.cc:515] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:56.441390  9776 leader_election.cc:290] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:56.439244  9774 leader_election.cc:290] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:05:56.509498  2312 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:56.551919  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 59565.1us
W20260709 13:05:56.554392  1611 consensus_peers.cc:597] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:56.555075  1611 consensus_peers.cc:597] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:56.555562  1611 consensus_peers.cc:597] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:56.556855  1611 consensus_peers.cc:597] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:56.557890  1611 consensus_peers.cc:597] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:56.572619  1686 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39935, user_credentials={real_user=slave}} blocked reactor thread for 79630.7us
W20260709 13:05:56.574250  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 50951.4us
W20260709 13:05:56.582079  1685 consensus_peers.cc:597] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:56.587185  1685 consensus_peers.cc:597] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:05:56.601475  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50422a819a849c0a93a4a07c9af6ba0" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:56.602434  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97987f9622cb4f52a1b74b4ee6bfdfdb" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:56.603838  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08853c4c3f0b40ddb5a0c34f0c2ee95a" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:56.604048  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df07602233941598bee7b19607b2e3b" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:56.606096  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e89a11b4204e48568da409c196e33be4" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:56.607429  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9390db02bf4ea689cbc36b1b946a14" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:56.608189  1685 consensus_peers.cc:597] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:56.609011  1685 consensus_peers.cc:597] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:56.609855  1685 consensus_peers.cc:597] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:05:56.601718  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbc4aafd02974a29b0c9b332b55c978c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:56.628073  1611 leader_election.cc:336] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:56.628696  1611 leader_election.cc:304] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:56.629678  9771 raft_consensus.cc:2749] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:56.629781  1611 leader_election.cc:336] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:05:56.630357  1611 leader_election.cc:336] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:56.630769  1611 leader_election.cc:304] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:56.631440  9771 raft_consensus.cc:2749] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:56.631691  1611 leader_election.cc:336] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:56.632153  1611 leader_election.cc:304] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:56.632993  9771 raft_consensus.cc:2749] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:56.633244  1611 leader_election.cc:336] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:56.633721  1611 leader_election.cc:304] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:56.633955  1612 leader_election.cc:304] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:56.634509  9772 raft_consensus.cc:2749] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:56.634728  1611 leader_election.cc:336] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:56.635171  1611 leader_election.cc:304] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:56.635289  9772 raft_consensus.cc:2749] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:56.636145  9775 raft_consensus.cc:2749] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:56.670081  2312 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:56.671324  1611 leader_election.cc:336] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:56.671872  1611 leader_election.cc:304] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:56.680088  9775 raft_consensus.cc:2749] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:56.802793   431 thread.cc:527] Waited for 16000ms trying to join with diag-logger (tid 1594)
W20260709 13:05:56.835569  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:56.856927  2312 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:56.875658  2312 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:57.035296  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 222029us
W20260709 13:05:57.060844  1685 consensus_peers.cc:597] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:57.106385  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:05:57.118079  9770 raft_consensus.cc:493] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
W20260709 13:05:57.119321  1685 consensus_peers.cc:597] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:05:57.118897  9770 raft_consensus.cc:515] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:57.131964  9782 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:57.133033  9781 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:57.138729  9779 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:57.211659  9785 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:57.139581  9779 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:05:57.239472  1685 consensus_peers.cc:597] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:05:57.133888  9782 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:57.212608  9785 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:57.151894  9781 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:57.230899  9784 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:57.239140  9792 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:57.277667  9796 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:57.280464  9789 raft_consensus.cc:493] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 130bc5ff39b847b1b3204626f284f838)
I20260709 13:05:57.281183  9796 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:57.281765  9789 raft_consensus.cc:515] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:57.284575  9789 leader_election.cc:290] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:57.285030  9779 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:57.285707  9792 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:57.286597  9785 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:57.288072  9792 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:57.290795  9800 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:57.291549  9800 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:57.295300  9770 leader_election.cc:290] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:05:57.296038  9784 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:57.299131  9784 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:57.303676  9782 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:57.306715  9800 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:57.318607  9781 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:05:57.319296  9785 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:57.319880  9785 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:57.320385  9794 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:05:57.325834  1685 consensus_peers.cc:597] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:05:57.321059  9794 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:57.333443  9794 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:57.335321  9785 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:57.329103  9796 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:05:57.354018  1611 leader_election.cc:336] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:05:57.355681  1611 consensus_peers.cc:597] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:57.356503  1611 consensus_peers.cc:597] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:57.358014  1611 consensus_peers.cc:597] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:57.362561  1685 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:05:57.360927  1611 consensus_peers.cc:597] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:57.364766  1685 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:05:57.365603  1685 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:57.368451  9796 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:57.372259  9796 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:05:57.374421  9796 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:05:57.377563  1685 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:57.379752  2346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:57.382201  2343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:57.383859  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:57.384142  1685 consensus_peers.cc:597] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:57.384840  1685 leader_election.cc:336] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:05:57.385368  1611 consensus_peers.cc:597] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:05:57.385492  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:05:57.390517  9796 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:05:57.396656  9796 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:05:57.401610  9796 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:05:57.409289  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbcb28b8d4104c6fae6682866cc0469d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:57.409861  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18bdd841c8294b08857684489689676d" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:57.411504  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:57.411948  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e07cb6db0d46a78611f24c17d04cc9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:57.413501  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc462cf3919460abb1ef504377a5382" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:57.414238  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:57.415364  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:57.420403  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ff300e7f45424ea3bcd7c816082956" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:57.420840  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:05:57.414331  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa55479d716d4bc4893f658afa382d73" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:57.425925  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:57.432266  2370 debug-util.cc:398] Leaking SignalData structure 0x7b08003d1460 after lost signal to thread 434
W20260709 13:05:57.433719  2370 debug-util.cc:398] Leaking SignalData structure 0x7b08003d04a0 after lost signal to thread 1505
W20260709 13:05:57.435315  2370 debug-util.cc:398] Leaking SignalData structure 0x7b080030a7e0 after lost signal to thread 1672
W20260709 13:05:57.436856  2370 debug-util.cc:398] Leaking SignalData structure 0x7b08000c4520 after lost signal to thread 1746
I20260709 13:05:57.441318  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
I20260709 13:05:57.442761  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbf62b8237649f584432aa4a9d96ec6" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:57.446873  2370 debug-util.cc:398] Leaking SignalData structure 0x7b080028dca0 after lost signal to thread 2373
I20260709 13:05:57.450199  1687 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:57.451467  9796 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:57.462607  1687 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:57.463618  9796 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:57.471906  1687 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:57.473150  9796 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:57.474885  1687 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:57.475888  9796 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:57.475919  1685 leader_election.cc:336] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:05:57.478207  1685 leader_election.cc:336] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:57.478901  1687 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:57.479395  1685 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:57.479960  9796 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:57.480593  1687 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:57.480911  9796 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:57.483650  9796 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:57.508651  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4b6776963146fab5ee9487b6684e8c" candidate_uuid: "f5f4b2707a9647e5bb99e065068dea2b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:05:57.518599  1685 consensus_peers.cc:597] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:57.519194  1685 leader_election.cc:336] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:57.519598  1685 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:57.522264  9806 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:57.535652  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 62453.1us
W20260709 13:05:57.536255  2312 leader_election.cc:336] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:05:57.536628  2312 leader_election.cc:304] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:57.538125  9805 raft_consensus.cc:2749] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:57.546299  1686 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:57.547482  9796 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:57.549371  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:57.550320  9796 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:05:57.552407  1686 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:57.553185  9796 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:57.556819  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:05:57.565599  1611 consensus_peers.cc:597] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:05:57.577478  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d3b49001741eda872554a7ecb2db3" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5f4b2707a9647e5bb99e065068dea2b" is_pre_election: true
W20260709 13:05:57.585973  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 64024.8us
I20260709 13:05:57.617624  1686 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:05:57.627421  9806 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:57.641001  1685 consensus_peers.cc:597] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:57.641736  1685 consensus_peers.cc:597] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:05:57.656054  1612 leader_election.cc:304] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5f4b2707a9647e5bb99e065068dea2b; no voters: 066ee0109aba4465ba17fa65e14ef504, 130bc5ff39b847b1b3204626f284f838
I20260709 13:05:57.660912  9791 raft_consensus.cc:2749] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:57.795583  1611 consensus_peers.cc:597] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:57.803562   431 thread.cc:527] Waited for 17000ms trying to join with diag-logger (tid 1594)
W20260709 13:05:57.857846  1611 consensus_peers.cc:597] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:57.857698  2312 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:57.864864  1611 consensus_peers.cc:597] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:57.887300  2312 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:57.944904  1743 debug-util.cc:398] Leaking SignalData structure 0x7b08003951c0 after lost signal to thread 434
W20260709 13:05:57.946336  1743 debug-util.cc:398] Leaking SignalData structure 0x7b08003b13e0 after lost signal to thread 1505
W20260709 13:05:57.947916  1743 debug-util.cc:398] Leaking SignalData structure 0x7b08001ce6a0 after lost signal to thread 1672
W20260709 13:05:57.949103  1743 debug-util.cc:398] Leaking SignalData structure 0x7b0800084920 after lost signal to thread 1746
W20260709 13:05:57.950428  1743 debug-util.cc:398] Leaking SignalData structure 0x7b080009d740 after lost signal to thread 2373
W20260709 13:05:58.000020  1611 consensus_peers.cc:597] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:58.099165  1611 consensus_peers.cc:597] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:58.198468  1685 consensus_peers.cc:597] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:58.270635  1611 consensus_peers.cc:597] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:58.375597  1611 consensus_peers.cc:597] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:58.388276  2312 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:05:58.434615  1685 consensus_peers.cc:597] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:58.499150  1685 consensus_peers.cc:597] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:58.557525  2312 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:05:58.599648  1685 consensus_peers.cc:597] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:58.711898  2312 consensus_peers.cc:597] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:58.739252  1685 consensus_peers.cc:597] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:58.792618  1685 consensus_peers.cc:597] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:58.803685  1611 consensus_peers.cc:597] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:58.804171   431 thread.cc:527] Waited for 18000ms trying to join with diag-logger (tid 1594)
W20260709 13:05:58.832281  1611 consensus_peers.cc:597] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:58.832944  1611 consensus_peers.cc:597] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:58.839203  1611 consensus_peers.cc:597] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:58.847484  1611 consensus_peers.cc:597] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:58.850564  1685 consensus_peers.cc:597] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:58.939841  2312 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:59.013593  1685 consensus_peers.cc:597] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:59.059407  1611 consensus_peers.cc:597] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:59.102187  2312 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:05:59.235153  1611 consensus_peers.cc:597] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:59.317379  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 53174.7us
W20260709 13:05:59.344147  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:59.352900  1685 consensus_peers.cc:597] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:59.396839  1685 consensus_peers.cc:597] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:59.415205  1685 consensus_peers.cc:597] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:59.467438  1685 consensus_peers.cc:597] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:59.530148  1685 consensus_peers.cc:597] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:59.584151  2312 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:59.613950  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:05:59.649436  1611 consensus_peers.cc:597] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:59.666553  1685 consensus_peers.cc:597] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:59.686244  1685 consensus_peers.cc:597] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:59.745713  1685 consensus_peers.cc:597] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:59.764611  1611 consensus_peers.cc:597] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:59.793466  1611 consensus_peers.cc:597] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:59.796613  1611 consensus_peers.cc:597] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:59.804881   431 thread.cc:527] Waited for 19000ms trying to join with diag-logger (tid 1594)
W20260709 13:05:59.822223  2312 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:59.925202  1611 consensus_peers.cc:597] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:59.932121  1611 consensus_peers.cc:597] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:59.956070  1685 consensus_peers.cc:597] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:00.126341  1685 consensus_peers.cc:597] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:00.126860  1685 consensus_peers.cc:597] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:00.228648  1685 consensus_peers.cc:597] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:00.265837  1611 consensus_peers.cc:597] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:00.274652  1611 consensus_peers.cc:597] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:00.296823  1611 consensus_peers.cc:597] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:00.395084  1611 consensus_peers.cc:597] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:00.522188  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 51810.8us
W20260709 13:06:00.539072  2312 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:00.546931  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 49420.1us
W20260709 13:06:00.548280  1611 consensus_peers.cc:597] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:00.606779  2312 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:00.644444  1455 catalog_manager.cc:4755] TS 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet adc326ba3c1b4dc7aa89a3a071803a7e: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:00.692327  1685 consensus_peers.cc:597] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:00.805680   431 thread.cc:527] Waited for 20000ms trying to join with diag-logger (tid 1594)
W20260709 13:06:00.815861  2312 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:00.869944  2312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 54143.7us
W20260709 13:06:00.907238  1611 consensus_peers.cc:597] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:00.934738  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 66338.8us
W20260709 13:06:00.995169  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 54581.8us
W20260709 13:06:01.023636  1685 consensus_peers.cc:597] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:01.032569  1611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 67478.3us
W20260709 13:06:01.148079  2312 consensus_peers.cc:597] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:01.152715  1685 consensus_peers.cc:597] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:01.179752  1685 consensus_peers.cc:597] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:01.201246  1685 consensus_peers.cc:597] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:01.207412  2312 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:01.282795  1611 consensus_peers.cc:597] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:01.328075  2312 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:01.371821  1685 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111) [suppressed 495 similar messages]
W20260709 13:06:01.376773  1685 consensus_peers.cc:597] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:01.441977  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 65295.2us
W20260709 13:06:01.460435  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 113199us
W20260709 13:06:01.487058  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 57094.8us
W20260709 13:06:01.523631  1611 consensus_peers.cc:597] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:01.576746  1685 consensus_peers.cc:597] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:01.630189  1611 consensus_peers.cc:597] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:01.631580  1685 consensus_peers.cc:597] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:01.632203  1611 consensus_peers.cc:597] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:01.689334  2312 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:01.760610  1685 consensus_peers.cc:597] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:01.766294  1611 consensus_peers.cc:597] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:01.782337  1611 consensus_peers.cc:597] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:01.784130  1611 consensus_peers.cc:597] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:01.823086  1685 consensus_peers.cc:597] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:01.847287  1685 consensus_peers.cc:597] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:01.861189   431 thread.cc:527] Waited for 21000ms trying to join with diag-logger (tid 1594)
W20260709 13:06:01.899427  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:01.904094  1611 consensus_peers.cc:597] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:01.985857  1685 consensus_peers.cc:597] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:02.028373  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 76332.7us
W20260709 13:06:02.103972  1611 consensus_peers.cc:597] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:02.104720  1611 consensus_peers.cc:597] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:02.149574  1685 consensus_peers.cc:597] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:02.175386  1685 consensus_peers.cc:597] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:02.178105  1685 consensus_peers.cc:597] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:02.194410  1611 consensus_peers.cc:597] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:02.194734  2312 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:02.220711  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 102375us
W20260709 13:06:02.316923  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:02.380352  1685 consensus_peers.cc:597] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:02.396377  1611 consensus_peers.cc:597] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:02.589511  1611 consensus_peers.cc:597] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:02.604698  2312 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:02.623654  1685 consensus_peers.cc:597] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:02.662254  1685 consensus_peers.cc:597] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:02.861761   431 thread.cc:527] Waited for 22000ms trying to join with diag-logger (tid 1594)
W20260709 13:06:02.884215  1685 consensus_peers.cc:597] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:02.884996  1685 consensus_peers.cc:597] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:02.885803  1612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 105358us
W20260709 13:06:02.928339  2312 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:02.985522  1611 consensus_peers.cc:597] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:02.986198  1611 consensus_peers.cc:597] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:02.999971  2312 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:03.020910  1611 consensus_peers.cc:597] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:03.028824  1611 consensus_peers.cc:597] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:03.075726  1611 consensus_peers.cc:597] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:03.089324  1611 consensus_peers.cc:597] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:03.168596  1669 debug-util.cc:398] Leaking SignalData structure 0x7b080036f240 after lost signal to thread 434
W20260709 13:06:03.170162  1669 debug-util.cc:398] Leaking SignalData structure 0x7b08003122c0 after lost signal to thread 1505
W20260709 13:06:03.173588  1669 debug-util.cc:398] Leaking SignalData structure 0x7b08000c3ca0 after lost signal to thread 1672
W20260709 13:06:03.183569  1669 debug-util.cc:398] Leaking SignalData structure 0x7b080024eb40 after lost signal to thread 1746
W20260709 13:06:03.187393  1669 debug-util.cc:398] Leaking SignalData structure 0x7b0800123a20 after lost signal to thread 2373
W20260709 13:06:03.358963  1685 consensus_peers.cc:597] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:03.361728  1685 consensus_peers.cc:597] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:03.527931  1611 consensus_peers.cc:597] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:03.565303  1685 consensus_peers.cc:597] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:03.603276  2312 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:03.644202  1611 consensus_peers.cc:597] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:03.676293  1685 consensus_peers.cc:597] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:03.775630  1685 consensus_peers.cc:597] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:03.849655  2312 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:03.853268  1685 consensus_peers.cc:597] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:03.866133   431 thread.cc:527] Waited for 23000ms trying to join with diag-logger (tid 1594)
W20260709 13:06:03.866842  1611 consensus_peers.cc:597] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:03.890110  2312 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:03.942294  1611 consensus_peers.cc:597] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:03.946647  1611 consensus_peers.cc:597] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:03.972278  1611 consensus_peers.cc:597] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:04.005744  1685 consensus_peers.cc:597] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:04.012087  2312 consensus_peers.cc:597] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:04.017553  1611 consensus_peers.cc:597] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:04.115135  1611 consensus_peers.cc:597] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:04.179104  1685 consensus_peers.cc:597] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:04.198385  1611 consensus_peers.cc:597] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:04.248446  2312 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:04.373363  1685 consensus_peers.cc:597] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:04.402911  1685 consensus_peers.cc:597] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:04.473785  1685 consensus_peers.cc:597] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:04.611714  1685 consensus_peers.cc:597] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:04.615135  1611 consensus_peers.cc:597] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:04.672441  1611 consensus_peers.cc:597] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:04.696610  1685 consensus_peers.cc:597] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:04.751262  1611 consensus_peers.cc:597] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:04.763546  1685 consensus_peers.cc:597] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:04.764397  1685 consensus_peers.cc:597] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:04.775327  2312 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:04.796794  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:04.799252  1611 consensus_peers.cc:597] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:04.871755  1685 consensus_peers.cc:597] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:06:04.906342   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:06:05.005160  2312 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:05.013290  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:05.078229  1611 consensus_peers.cc:597] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:05.320911  1685 consensus_peers.cc:597] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:05.369074  1611 consensus_peers.cc:597] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:06:05.385144   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:06:05.395568  1685 consensus_peers.cc:597] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:05.405067  1687 consensus_peers.cc:597] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:05.418452  2314 consensus_peers.cc:597] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:05.437137  1687 consensus_peers.cc:597] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:05.437865  2312 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:05.443303  2314 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:05.445097  1687 consensus_peers.cc:597] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:05.454495  1687 consensus_peers.cc:597] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:05.455402  1687 consensus_peers.cc:597] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:05.460326  1687 consensus_peers.cc:597] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:05.465729  1687 consensus_peers.cc:597] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:05.472153  1685 consensus_peers.cc:597] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:05.489766  1611 consensus_peers.cc:597] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:05.499819  1685 consensus_peers.cc:597] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:05.515338  1687 consensus_peers.cc:597] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:05.518187  2312 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:05.548548  1611 consensus_peers.cc:597] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:06:05.575407   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:05.580425   431 tablet_replica.cc:333] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.582674   431 raft_consensus.cc:2243] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.586352   431 raft_consensus.cc:2272] T 82cbfb23571048409c1bde90d4611c1a P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:05.590430  2314 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:05.591900   431 tablet_replica.cc:333] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
W20260709 13:06:05.591204  2314 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:05.593282   431 raft_consensus.cc:2243] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:05.594769   431 raft_consensus.cc:2272] T aa55479d716d4bc4893f658afa382d73 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.599064   431 tablet_replica.cc:333] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.600378   431 raft_consensus.cc:2243] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.601624   431 raft_consensus.cc:2272] T b7b0d191c5dc46468a777fba0a666b32 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.606019   431 tablet_replica.cc:333] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
W20260709 13:06:05.607291  2314 consensus_peers.cc:597] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:05.608017   431 raft_consensus.cc:2243] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.609125   431 raft_consensus.cc:2272] T bddd1ebc9af24fa282580ab028d643e0 P f5f4b2707a9647e5bb99e065068dea2b [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.612882   431 tablet_replica.cc:333] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
W20260709 13:06:05.620479  1611 consensus_peers.cc:597] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:06:05.623930   431 raft_consensus.cc:2243] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.624948   431 raft_consensus.cc:2272] T 8f06439379e24c5ebedb229b007236c2 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.634732   431 tablet_replica.cc:333] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.639463   431 raft_consensus.cc:2243] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.640605   431 raft_consensus.cc:2272] T 97987f9622cb4f52a1b74b4ee6bfdfdb P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.644729   431 tablet_replica.cc:333] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.645628   431 raft_consensus.cc:2243] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:05.646906   431 raft_consensus.cc:2272] T a8d17fad9f114b0e98d2af0a989c3185 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.650270   431 tablet_replica.cc:333] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.651345   431 raft_consensus.cc:2243] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.652711   431 raft_consensus.cc:2272] T b82d3b49001741eda872554a7ecb2db3 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.656136   431 tablet_replica.cc:333] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
W20260709 13:06:05.665964  1687 consensus_peers.cc:597] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:05.672043  1687 consensus_peers.cc:597] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:05.672631   431 raft_consensus.cc:2243] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.674491   431 raft_consensus.cc:2272] T 08cf0535992b4db395d567393667593b P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.682242   431 tablet_replica.cc:333] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.683418   431 raft_consensus.cc:2243] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.686455   431 raft_consensus.cc:2272] T a51c23e1c3ae4bdbbb5f21f1607da772 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.697422   431 tablet_replica.cc:333] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.699911   431 raft_consensus.cc:2243] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:05.704950  1687 consensus_peers.cc:597] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:05.706707   431 raft_consensus.cc:2272] T ca2582f1941a4c588f8ef51c03dfb492 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.710131   431 tablet_replica.cc:333] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.711311   431 raft_consensus.cc:2243] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.712179   431 raft_consensus.cc:2272] T 3ee0e81a514641709317e983fcf96fe4 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:05.719372  2314 consensus_peers.cc:597] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:05.725251  1687 consensus_peers.cc:597] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:05.730965  1687 consensus_peers.cc:597] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:05.732676  1687 consensus_peers.cc:597] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:05.734728   431 tablet_replica.cc:333] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.735793   431 raft_consensus.cc:2243] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.736768   431 raft_consensus.cc:2272] T baba78285345483d9cf6ead6bf20106b P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.740346   431 tablet_replica.cc:333] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.745592   431 raft_consensus.cc:2243] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.746721   431 raft_consensus.cc:2272] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.752272   431 tablet_replica.cc:333] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.754369   431 raft_consensus.cc:2243] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 19 LEADER]: Raft consensus shutting down.
W20260709 13:06:05.758884  1685 consensus_peers.cc:597] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:05.762822  2314 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:05.772743  1687 consensus_peers.cc:597] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:05.781526  1687 consensus_peers.cc:597] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:05.782874  1611 consensus_peers.cc:597] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 13:06:05.785156   431 raft_consensus.cc:2272] T dbbf62b8237649f584432aa4a9d96ec6 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.789541   431 tablet_replica.cc:333] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.790517   431 raft_consensus.cc:2243] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:06:05.791745   431 raft_consensus.cc:2272] T 45c609109d4a4c40a3e74ff0917c7958 P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.794617   431 tablet_replica.cc:333] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.795467   431 raft_consensus.cc:2243] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:05.796669   431 raft_consensus.cc:2272] T e7a5f5eedbd1406788cafbad6a95a4c7 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:05.811142  1687 consensus_peers.cc:597] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:05.815236   431 tablet_replica.cc:333] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.816463   431 raft_consensus.cc:2243] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.817332   431 raft_consensus.cc:2272] T 0764c6a89eca47f39e28feb56e2123c7 P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.820452   431 tablet_replica.cc:333] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.821286   431 raft_consensus.cc:2243] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.822341   431 raft_consensus.cc:2272] T 1d9390db02bf4ea689cbc36b1b946a14 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.826522   431 tablet_replica.cc:333] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.832263   431 raft_consensus.cc:2243] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.833058   431 raft_consensus.cc:2272] T f59ca0c8e4c0491480502ebf944ba503 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.835894   431 tablet_replica.cc:333] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.836808   431 raft_consensus.cc:2243] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 LEADER]: Raft consensus shutting down.
W20260709 13:06:05.844842  1687 consensus_peers.cc:597] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:05.840648   431 raft_consensus.cc:2272] T e87427e2b27d4f89b7953163626fd8e4 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.850469   431 tablet_replica.cc:333] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.856657   431 raft_consensus.cc:2243] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:05.858023   431 raft_consensus.cc:2272] T 7ae86dda8d6f4caf8f08a1607327d9f9 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.861229   431 tablet_replica.cc:333] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.862020   431 raft_consensus.cc:2243] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.862773   431 raft_consensus.cc:2272] T 0df07602233941598bee7b19607b2e3b P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.866166   431 tablet_replica.cc:333] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.866997   431 raft_consensus.cc:2243] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Raft consensus shutting down.
W20260709 13:06:05.870877  1687 consensus_peers.cc:597] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:05.871644   431 raft_consensus.cc:2272] T d747d0b326fa4be595dbd31d3638164d P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.874838   431 tablet_replica.cc:333] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.875741   431 raft_consensus.cc:2243] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:05.877285   431 raft_consensus.cc:2272] T d3ff300e7f45424ea3bcd7c816082956 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.880110   431 tablet_replica.cc:333] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.880941   431 raft_consensus.cc:2243] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:05.882179   431 raft_consensus.cc:2272] T e5c79254e6ff49639cf09f663490dfd7 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.884999   431 tablet_replica.cc:333] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.885898   431 raft_consensus.cc:2243] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.886636   431 raft_consensus.cc:2272] T 58b222526a1a4b52bdcda825e15c46f9 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.889578   431 tablet_replica.cc:333] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.890372   431 raft_consensus.cc:2243] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:06:05.894670   431 raft_consensus.cc:2272] T 5bd2b69727c743cfbdc52253e16279ce P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.899523   431 tablet_replica.cc:333] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.903039   431 raft_consensus.cc:2243] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.905401   431 raft_consensus.cc:2272] T bbc4aafd02974a29b0c9b332b55c978c P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.922694   431 tablet_replica.cc:333] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.923615   431 raft_consensus.cc:2243] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.924367   431 raft_consensus.cc:2272] T 56082a5ff84643f0ba368ceba0524623 P f5f4b2707a9647e5bb99e065068dea2b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.927963   431 tablet_replica.cc:333] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.928767   431 raft_consensus.cc:2243] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:05.929662   431 raft_consensus.cc:2272] T ca833b0b78d74adc8ca86e91fc94f96f P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.932484   431 tablet_replica.cc:333] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.933301   431 raft_consensus.cc:2243] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:05.934883   431 raft_consensus.cc:2272] T 15e07cb6db0d46a78611f24c17d04cc9 P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:05.936709  1685 consensus_peers.cc:597] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:05.945402  2314 consensus_peers.cc:597] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:05.949705   431 tablet_replica.cc:333] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.952147   431 raft_consensus.cc:2243] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:05.953486   431 raft_consensus.cc:2272] T dcc462cf3919460abb1ef504377a5382 P f5f4b2707a9647e5bb99e065068dea2b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.960821   431 tablet_replica.cc:333] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.963760   431 raft_consensus.cc:2243] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:05.966068   431 raft_consensus.cc:2272] T 7e9b1a584a5f430a9a0e7e617064bc2c P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:05.974074  1687 consensus_peers.cc:597] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:05.982004   431 tablet_replica.cc:333] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.982913   431 raft_consensus.cc:2243] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:06:05.984064   431 raft_consensus.cc:2272] T 18bdd841c8294b08857684489689676d P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:05.986951   431 tablet_replica.cc:333] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:05.997493   431 raft_consensus.cc:2243] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:06.010576   431 raft_consensus.cc:2272] T e59deed37c414a929faeb85bdbaaa406 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.013348   431 tablet_replica.cc:333] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:06.014184   431 raft_consensus.cc:2243] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:06.015002   431 raft_consensus.cc:2272] T e89a11b4204e48568da409c196e33be4 P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.017591   431 tablet_replica.cc:333] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:06.018371   431 raft_consensus.cc:2243] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:06.019076   431 raft_consensus.cc:2272] T 1a01faf4c6c64b549e80adf03bba9f51 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.021787   431 tablet_replica.cc:333] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:06.027626   431 raft_consensus.cc:2243] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:06.028721   431 raft_consensus.cc:2272] T f81336f0bd194664ac95af1a897b4b18 P f5f4b2707a9647e5bb99e065068dea2b [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.041849   431 tablet_replica.cc:333] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:06.042711   431 raft_consensus.cc:2243] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:06.044080   431 raft_consensus.cc:2272] T 12ac45e0e48449a98fb89704bae3de27 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:06.048215  1685 consensus_peers.cc:597] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:06:06.048558   431 tablet_replica.cc:333] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:06.051584   431 raft_consensus.cc:2243] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:06.058663   431 raft_consensus.cc:2272] T 7501558aa93542308c5fc758446387df P f5f4b2707a9647e5bb99e065068dea2b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.062688   431 tablet_replica.cc:333] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:06.063519   431 raft_consensus.cc:2243] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:06.064373   431 raft_consensus.cc:2272] T f50422a819a849c0a93a4a07c9af6ba0 P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:06.073995  1685 consensus_peers.cc:597] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:06:06.085753   431 tablet_replica.cc:333] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:06.086688   431 raft_consensus.cc:2243] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:06:06.088510  1611 consensus_peers.cc:597] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:06:06.088876   431 raft_consensus.cc:2272] T 48ee6d32e41946dc9cf53a3d62cdb9ae P f5f4b2707a9647e5bb99e065068dea2b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.109876   431 tablet_replica.cc:333] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:06.110754   431 raft_consensus.cc:2243] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:06.112208  1611 consensus_peers.cc:597] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 13:06:06.112440   431 raft_consensus.cc:2272] T 49c94b9e96df4808adbda99815b09c5f P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.119481   431 tablet_replica.cc:333] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:06.120260   431 raft_consensus.cc:2243] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:06:06.121524   431 raft_consensus.cc:2272] T f0ee72a7f576482aabe40ef837964708 P f5f4b2707a9647e5bb99e065068dea2b [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.127310   431 tablet_replica.cc:333] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:06.128154   431 raft_consensus.cc:2243] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:06:06.129448   431 raft_consensus.cc:2272] T 6b98c1e123bd4894b2ab4096efb42cd8 P f5f4b2707a9647e5bb99e065068dea2b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.136861   431 tablet_replica.cc:333] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:06.141014   431 raft_consensus.cc:2243] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:06.142331   431 raft_consensus.cc:2272] T adc326ba3c1b4dc7aa89a3a071803a7e P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.151605   431 tablet_replica.cc:333] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:06.152385   431 raft_consensus.cc:2243] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:06.153048   431 raft_consensus.cc:2272] T d2618e77160b44339d8499efcd1e8eca P f5f4b2707a9647e5bb99e065068dea2b [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.155649   431 tablet_replica.cc:333] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:06.162571   431 raft_consensus.cc:2243] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:06.163904   431 raft_consensus.cc:2272] T dbcb28b8d4104c6fae6682866cc0469d P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:06.164798  2312 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
I20260709 13:06:06.172278   431 tablet_replica.cc:333] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:06.173125   431 raft_consensus.cc:2243] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 20 LEADER]: Raft consensus shutting down.
W20260709 13:06:06.173599  1685 consensus_peers.cc:597] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:06.174093  1685 consensus_peers.cc:597] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:06:06.181030   431 raft_consensus.cc:2272] T baa23f35bd3b4c2d99e3da0ff0f80e97 P f5f4b2707a9647e5bb99e065068dea2b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.183944   431 tablet_replica.cc:333] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:06.184708   431 raft_consensus.cc:2243] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:06:06.185989   431 raft_consensus.cc:2272] T 418152c2665a4b04be8a39c62a52f6ca P f5f4b2707a9647e5bb99e065068dea2b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.190333   431 tablet_replica.cc:333] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b: stopping tablet replica
I20260709 13:06:06.198395   431 raft_consensus.cc:2243] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:06.199309   431 raft_consensus.cc:2272] T ad4b6776963146fab5ee9487b6684e8c P f5f4b2707a9647e5bb99e065068dea2b [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:06.394815  2314 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 529 similar messages]
W20260709 13:06:06.891325  2314 connection.cc:769] client connection to 127.0.107.194:44293 send error: Network error: sendmsg error: Broken pipe (error 32)
W20260709 13:06:06.893038  1685 consensus_peers.cc:597] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:06.895516  1685 consensus_peers.cc:597] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:06.902506  1685 consensus_peers.cc:597] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:06:06.918927  9838 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:06.921370  9847 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:06.926241  9860 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:06.926712  9847 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:06.931031  9847 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:06.924705  2312 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
I20260709 13:06:06.931797  9838 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:06.943559  9838 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:06:06.921484  9854 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:06:06.949468  1687 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
W20260709 13:06:06.950320  1687 leader_election.cc:336] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:06.926887  9860 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:06:06.951704  1685 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:06.952256  1685 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:06:06.953267  1685 leader_election.cc:336] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:06.953919  9838 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:06.950014  9854 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:06:06.959728  2314 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:06.957909  1685 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:06.964771  9847 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:06.966166  9854 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
W20260709 13:06:06.967731  1687 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
W20260709 13:06:06.968446  1685 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:06.968883  1685 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:06.970137  9838 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:06.970252  9854 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:06.970932  9838 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:06.973166  9838 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:06.974292  2312 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:06.977149  2312 consensus_peers.cc:597] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:06.979524  1687 leader_election.cc:336] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
W20260709 13:06:06.979875  2312 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:06.986006  1685 leader_election.cc:336] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:06.986696  1685 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:06.987634  9854 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:06.989892  9860 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:06:06.996942  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:06:06.997272  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:06:06.997882  2348 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:06:06.998286  1713 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19.
W20260709 13:06:07.002826  2312 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:07.003077  9838 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.003767  2312 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:07.005085  9845 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:07.004103  9838 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:06:07.008767  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
W20260709 13:06:07.012311  1687 leader_election.cc:336] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:07.012861  1687 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: cf2f71592be8465fb18e36e15d0b2f89, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:07.013919  9854 raft_consensus.cc:2749] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:07.014994  9838 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:06:07.016247  9838 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.016737  9838 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:07.019784  9838 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:06:07.021539  9838 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.022939  9838 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:06:07.024492  1687 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
W20260709 13:06:07.025116  1685 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:07.025776  1685 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:07.026638  9854 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:07.026985  9838 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:07.030220  1687 leader_election.cc:336] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
W20260709 13:06:07.033732  1685 leader_election.cc:336] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:07.034186  1685 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:07.034999  9838 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:07.071136  9845 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.071908  9845 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:06:07.076220  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:06:07.078737  9845 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:06:07.079294  2314 leader_election.cc:336] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:07.079541  9838 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.079847  2314 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:07.080154  9838 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:07.081204  9845 raft_consensus.cc:2749] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:07.084824  9838 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:07.087805  1687 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
W20260709 13:06:07.088625  1685 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:07.089094  1685 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:07.089915  9838 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:07.125319  9838 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.126132  9838 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:06:07.130214  9838 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:06:07.130371  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:06:07.131096  2348 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
W20260709 13:06:07.133689  1687 leader_election.cc:336] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:07.134770  1686 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:07.139878  9838 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:07.140388  9838 raft_consensus.cc:493] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.141029  9838 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:07.148691  9838 raft_consensus.cc:515] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:06:07.152099  9838 leader_election.cc:290] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:06:07.152448  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7a5f5eedbd1406788cafbad6a95a4c7" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:06:07.153184  2348 raft_consensus.cc:3060] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:07.156612  1687 leader_election.cc:336] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:07.160833  2348 raft_consensus.cc:2468] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:06:07.162130  1686 leader_election.cc:304] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:07.163131  9838 raft_consensus.cc:2804] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:07.164137  9838 raft_consensus.cc:697] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
I20260709 13:06:07.165302  9838 consensus_queue.cc:237] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:06:07.166836  9854 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.167433  9854 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:07.169552  9847 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.170560  9845 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.170322  9847 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:06:07.171118  9845 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:06:07.172773  9847 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:06:07.173326  9845 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:06:07.176157  1685 leader_election.cc:336] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:07.169720  9854 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:06:07.177682  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:06:07.178398  2348 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 25.
I20260709 13:06:07.179505  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:06:07.179672  2314 leader_election.cc:336] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:07.180543  9854 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:06:07.181001  9854 raft_consensus.cc:493] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.181348  9854 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:06:07.183697  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
W20260709 13:06:07.188102  1687 leader_election.cc:336] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:07.188666  1687 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:07.190210  9854 raft_consensus.cc:515] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:06:07.191545  1713 raft_consensus.cc:2393] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf2f71592be8465fb18e36e15d0b2f89 in current term 26: Already voted for candidate 130bc5ff39b847b1b3204626f284f838 in this term.
I20260709 13:06:07.194092  9847 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:06:07.195771  1687 leader_election.cc:336] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:07.198925  2313 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:07.200134  9845 raft_consensus.cc:3060] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:06:07.205264  9838 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.206029  9838 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:07.208307  9845 raft_consensus.cc:2749] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:06:07.213605  9838 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:07.214303  1685 leader_election.cc:336] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:07.217739  1687 leader_election.cc:336] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:07.220948  9854 leader_election.cc:290] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 26 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:06:07.222375  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c609109d4a4c40a3e74ff0917c7958" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:06:07.224677  1464 catalog_manager.cc:5697] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 18 to 19, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 19 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:07.225521  1687 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:07.230171  9838 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:07.235750  2348 raft_consensus.cc:2468] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 26.
I20260709 13:06:07.241007  1686 leader_election.cc:304] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:06:07.242120  9838 raft_consensus.cc:2804] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:06:07.242699  9838 raft_consensus.cc:697] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 26 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:06:07.243301  1687 leader_election.cc:336] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:07.243716  9838 consensus_queue.cc:237] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:06:07.255273  9838 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.255904  9838 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:07.258064  9838 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:07.258699  1685 consensus_peers.cc:597] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:06:07.260769  9845 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.261463  9845 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:06:07.263609  9845 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:06:07.265086  1685 leader_election.cc:336] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:07.269196  2314 leader_election.cc:336] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:07.269459  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:06:07.270128  9845 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.272449  9845 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:06:07.271126  1687 leader_election.cc:336] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:07.275808  1687 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:07.276773  9854 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:07.279682  2313 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 130bc5ff39b847b1b3204626f284f838, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:07.280488  9860 raft_consensus.cc:2749] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:07.287639  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:06:07.288425  1713 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 22.
I20260709 13:06:07.292043  2313 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:06:07.292954  9845 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:06:07.293568  9845 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:07.294888  9845 raft_consensus.cc:493] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.295410  9838 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.299420  9838 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:07.302137  9851 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.302670  9851 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:07.304839  9854 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.305527  9854 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:07.308991  9838 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:06:07.309823  9853 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.312824  9853 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:07.313313  9854 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:06:07.314181  9845 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:07.315362  9853 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:06:07.311523  9851 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:07.313508  1685 leader_election.cc:336] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:07.320094  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:06:07.329867  1464 catalog_manager.cc:5697] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 25 to 26, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 26 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:07.316237  2314 leader_election.cc:336] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
W20260709 13:06:07.333956  1685 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:07.334921  1685 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:07.343204  1687 leader_election.cc:336] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:07.343809  1687 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:07.344563  9838 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:07.347028  1685 leader_election.cc:336] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:07.347770  1685 consensus_peers.cc:597] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:07.344805  1687 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:07.348611  1687 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:07.349355  9838 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:07.352573  9845 raft_consensus.cc:515] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:06:07.355202  9845 leader_election.cc:290] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
W20260709 13:06:07.361774  1687 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:07.362524  1687 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:07.363567  9838 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:07.367726  2312 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:06:07.393941  9838 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.394552  9838 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:06:07.396891  9838 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:06:07.400609  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:06:07.401278  2348 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 18.
I20260709 13:06:07.402369  1686 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
W20260709 13:06:07.403357  2314 leader_election.cc:336] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:07.404192  9854 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:07.405012  9854 raft_consensus.cc:493] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.405400  9854 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:07.407855  9851 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.408452  9851 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:07.410838  9851 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:07.413018  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 51285.3us
W20260709 13:06:07.415481  1687 leader_election.cc:336] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:07.417769  1687 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:07.420279  9838 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:07.427330  1685 consensus_peers.cc:597] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:06:07.430241  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae86dda8d6f4caf8f08a1607327d9f9" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838"
I20260709 13:06:07.430928  1715 raft_consensus.cc:3060] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:07.448030  9853 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.448647  9853 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:06:07.450363  1685 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:07.450724  9853 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:07.450973  1685 consensus_peers.cc:597] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:06:07.451958  9854 raft_consensus.cc:515] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:06:07.839445  9854 leader_election.cc:290] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Requested vote from peers f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:06:07.453292  1715 raft_consensus.cc:2468] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 23.
I20260709 13:06:07.454617  9869 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.843354  9869 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:07.844779  2348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adc326ba3c1b4dc7aa89a3a071803a7e" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89"
I20260709 13:06:07.845412  2348 raft_consensus.cc:3060] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:07.624612  9838 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.846031  9838 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:06:07.737517  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 286556us
W20260709 13:06:07.738312  1669 debug-util.cc:398] Leaking SignalData structure 0x7b08002955a0 after lost signal to thread 434
W20260709 13:06:07.852864  1669 debug-util.cc:398] Leaking SignalData structure 0x7b08000efa80 after lost signal to thread 1505
I20260709 13:06:07.798894  9857 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
W20260709 13:06:07.831236  1687 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
W20260709 13:06:07.853614  1669 debug-util.cc:398] Leaking SignalData structure 0x7b08000e9800 after lost signal to thread 1672
I20260709 13:06:07.867794  2349 raft_consensus.cc:1275] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
W20260709 13:06:07.868499  1669 debug-util.cc:398] Leaking SignalData structure 0x7b0800383e00 after lost signal to thread 1746
I20260709 13:06:07.854104  2348 raft_consensus.cc:2468] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:06:07.861075  9853 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:07.869683  9870 consensus_queue.cc:1048] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:06:07.858111  9857 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:06:07.872476  1686 leader_election.cc:304] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: 
I20260709 13:06:07.873134  9857 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:07.858666  1687 consensus_peers.cc:597] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:07.875990  9870 raft_consensus.cc:2804] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:07.876897  9870 raft_consensus.cc:697] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Becoming Leader. State: Replica: 130bc5ff39b847b1b3204626f284f838, State: Running, Role: LEADER
W20260709 13:06:07.877218  1687 leader_election.cc:336] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
W20260709 13:06:07.878087  1687 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:07.877853  9870 consensus_queue.cc:237] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:06:07.878512  1687 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:06:07.878918  2314 leader_election.cc:336] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:07.879405  9851 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:07.869609  9853 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:06:07.885977  2312 leader_election.cc:336] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:07.886508  2312 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:07.887526  9872 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:07.890699  9869 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:06:07.903760  9838 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:06:07.906672  9853 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:07.910533  1669 debug-util.cc:398] Leaking SignalData structure 0x7b08001db2a0 after lost signal to thread 2373
W20260709 13:06:07.913254   431 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 1669)
I20260709 13:06:07.914830  2313 leader_election.cc:304] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838, cf2f71592be8465fb18e36e15d0b2f89; no voters: f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:07.940689  9871 raft_consensus.cc:2804] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:06:07.962476  1687 leader_election.cc:336] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
W20260709 13:06:07.963426  1687 leader_election.cc:336] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
W20260709 13:06:07.964131  1687 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
W20260709 13:06:07.964694  1687 leader_election.cc:336] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:07.961493  1464 catalog_manager.cc:5697] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 reported cstate change: term changed from 18 to 19, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to 130bc5ff39b847b1b3204626f284f838 (127.0.107.195). New cstate: current_term: 19 leader_uuid: "130bc5ff39b847b1b3204626f284f838" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:07.968070  9871 raft_consensus.cc:697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 LEADER]: Becoming Leader. State: Replica: cf2f71592be8465fb18e36e15d0b2f89, State: Running, Role: LEADER
I20260709 13:06:07.972828  9871 consensus_queue.cc:237] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:06:07.978477  2313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34553, user_credentials={real_user=slave}} blocked reactor thread for 48479.3us
W20260709 13:06:07.991046  1685 consensus_peers.cc:597] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:06:08.011024  2348 raft_consensus.cc:1275] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 26 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
W20260709 13:06:08.014765  1685 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:08.019052  1685 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:08.015166  9874 consensus_queue.cc:1048] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.005s
I20260709 13:06:08.031387  1465 catalog_manager.cc:5697] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 reported cstate change: term changed from 22 to 23, leader changed from f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194) to cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196). New cstate: current_term: 23 leader_uuid: "cf2f71592be8465fb18e36e15d0b2f89" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:08.033824  1685 consensus_peers.cc:597] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:08.035570  1685 consensus_peers.cc:597] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:08.036434  1685 leader_election.cc:336] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:08.036845  1685 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:08.037891  9847 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:08.037799  9874 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:06:08.042780  1685 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:08.043337  1685 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:08.044095  9874 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:06:08.046073  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20260709 13:06:08.050978  1687 consensus_peers.cc:597] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:08.054358  2312 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:08.058022  1685 leader_election.cc:336] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:08.063333  1685 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:08.065892  9874 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:08.216709  2314 consensus_peers.cc:597] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:08.297572  2312 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:08.300522  2314 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:08.301229  2314 consensus_peers.cc:597] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:08.303627  2314 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:08.347050  2314 consensus_peers.cc:597] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:08.418897  2314 consensus_peers.cc:597] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:08.489554  2314 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:08.623245  2312 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:06:08.792282  9890 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:08.793102  9890 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:08.795778  9890 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:06:08.805382  9894 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:08.806327  9894 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:08.808483  9894 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:06:08.823734  9891 raft_consensus.cc:493] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:08.824536  9891 raft_consensus.cc:515] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:08.828066  9891 leader_election.cc:290] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:06:08.825412  9896 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:08.828967  9896 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:06:08.837010  9896 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), cf2f71592be8465fb18e36e15d0b2f89 (127.0.107.196:39935)
I20260709 13:06:08.855598  1720 raft_consensus.cc:1275] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Refusing update from remote peer cf2f71592be8465fb18e36e15d0b2f89: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 13:06:08.875088  1685 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:08.894215  1685 consensus_peers.cc:597] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:08.895565  1685 leader_election.cc:336] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:08.896333  1685 consensus_peers.cc:597] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:08.896919  1685 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:08.898873  9899 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:08.899654  9899 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:08.903952  9899 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:08.905138  1685 leader_election.cc:336] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:08.913849   431 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 1669)
W20260709 13:06:08.920029  1687 consensus_peers.cc:597] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:08.920622  1687 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:08.921018  1687 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:06:08.923305  1502 debug-util.cc:398] Leaking SignalData structure 0x7b08003025e0 after lost signal to thread 434
W20260709 13:06:08.924791  1502 debug-util.cc:398] Leaking SignalData structure 0x7b0800202600 after lost signal to thread 1505
W20260709 13:06:08.930765  1502 debug-util.cc:398] Leaking SignalData structure 0x7b0800172ce0 after lost signal to thread 1746
W20260709 13:06:08.932054  1687 consensus_peers.cc:597] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:08.932365  1502 debug-util.cc:398] Leaking SignalData structure 0x7b0800173300 after lost signal to thread 2373
W20260709 13:06:08.932595  1687 consensus_peers.cc:597] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:08.933130  1687 consensus_peers.cc:597] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:08.933754  1687 consensus_peers.cc:597] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:08.934329  1687 leader_election.cc:336] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:08.934844  1687 leader_election.cc:304] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:08.938655  9894 raft_consensus.cc:2749] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:08.943538  1687 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:08.944093  1687 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:06:08.944718  1687 consensus_peers.cc:597] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:08.938256  9899 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:08.946879  9894 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:08.964219  9871 consensus_queue.cc:1048] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:06:08.965481  2314 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:08.972891  2349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "130bc5ff39b847b1b3204626f284f838" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cf2f71592be8465fb18e36e15d0b2f89" is_pre_election: true
I20260709 13:06:08.973690  2349 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130bc5ff39b847b1b3204626f284f838 in term 19.
I20260709 13:06:08.983528  2349 raft_consensus.cc:1275] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Refusing update from remote peer 130bc5ff39b847b1b3204626f284f838: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
W20260709 13:06:08.983686  1685 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:08.989710  9894 consensus_queue.cc:1048] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:06:09.003381  1687 leader_election.cc:336] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:09.003981  1687 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:09.004930  9894 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:09.005131  1687 consensus_peers.cc:597] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:09.013994  9899 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:09.014869  9899 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:09.018287  9899 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:09.019490  1687 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:09.020047  1687 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:09.021304  9899 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:09.025882  1687 consensus_peers.cc:597] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:09.026115  1685 leader_election.cc:336] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:09.038597  1687 consensus_peers.cc:597] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.039216  1687 consensus_peers.cc:597] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.040050  1687 consensus_peers.cc:597] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.040800  1687 consensus_peers.cc:597] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:09.048434  9899 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:09.049088  9899 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:09.051481  9899 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:09.056118  1687 leader_election.cc:336] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:09.058593  1687 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:06:09.059301  1685 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:09.059417  9899 raft_consensus.cc:2749] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:06:09.071686  1687 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:09.072420  1687 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:09.073391  9899 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:09.090592  9899 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:09.091207  9899 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:09.093398  9899 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:09.102919  1685 leader_election.cc:336] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:09.108155  1687 leader_election.cc:336] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:09.113607  9899 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:09.114756  9899 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:09.118870  9899 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:06:09.124401  1687 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:06:09.124900  1685 leader_election.cc:336] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:09.125437  9899 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:06:09.135593  1687 leader_election.cc:336] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:09.143362  1687 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:09.144484  9894 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:09.145375  2314 consensus_peers.cc:597] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:09.174108  9894 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:09.174731  9894 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:09.177000  9894 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:09.183701  1687 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
W20260709 13:06:09.194324  1685 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:09.200120  1685 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:09.206597  9897 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:06:09.212643  1687 consensus_peers.cc:597] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.236016  1685 consensus_peers.cc:597] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:09.237588  2312 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20260709 13:06:09.245950  1685 consensus_peers.cc:597] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:06:09.248306  9894 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:09.248932  9894 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:09.251529  9894 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:09.252753  2312 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20260709 13:06:09.269750  1685 consensus_peers.cc:597] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:09.270391  1685 leader_election.cc:336] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:09.290783  1685 consensus_peers.cc:597] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:09.296418  1687 consensus_peers.cc:597] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.311683  1687 leader_election.cc:336] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:09.312095  1687 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:09.313030  9897 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:09.313297  1687 consensus_peers.cc:597] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.318236  1687 consensus_peers.cc:597] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.367235  1685 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:39999, user_credentials={real_user=slave}} blocked reactor thread for 47385.7us
W20260709 13:06:09.368327  1687 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:44293, user_credentials={real_user=slave}} blocked reactor thread for 48396.9us
I20260709 13:06:09.395928  9891 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:09.406046  9891 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:09.418692  9891 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:09.432937  2312 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20260709 13:06:09.454056  1685 consensus_peers.cc:597] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:09.455087  1685 leader_election.cc:336] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:09.450814  2312 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
I20260709 13:06:09.459306  9911 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:09.469130  9911 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:09.472128  9911 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:09.473255  1685 consensus_peers.cc:597] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:09.477430  1685 consensus_peers.cc:597] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:09.495571  1685 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:09.513561  1687 consensus_peers.cc:597] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.535171  1687 consensus_peers.cc:597] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.535953  1687 consensus_peers.cc:597] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.536550  1687 leader_election.cc:336] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:09.537015  1687 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:09.538004  9911 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:09.540524  9911 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:09.541401  9911 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:09.545722  9912 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:09.551327  9912 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:09.554356  9912 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:09.550311  1685 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:09.543897  9911 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:09.563369  1685 leader_election.cc:336] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:09.543234  1687 consensus_peers.cc:597] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.573719  1687 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:09.574406  1687 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:09.575347  9911 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:09.579195  9911 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:09.580116  9911 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:09.582991  9911 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:09.589478  1685 leader_election.cc:336] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:09.592792  1687 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:09.593386  1687 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:09.594319  9911 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:06:09.594535  1687 leader_election.cc:336] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:09.595232  1687 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:09.596241  9911 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:09.601472  1687 leader_election.cc:336] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:09.602066  1687 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:09.602897  9911 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:09.748580  1685 consensus_peers.cc:597] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:09.806926  1743 debug-util.cc:398] Leaking SignalData structure 0x7b0800225380 after lost signal to thread 434
W20260709 13:06:09.809134  1743 debug-util.cc:398] Leaking SignalData structure 0x7b08002bd360 after lost signal to thread 1505
W20260709 13:06:09.811384  1743 debug-util.cc:398] Leaking SignalData structure 0x7b0800223f60 after lost signal to thread 1746
W20260709 13:06:09.813585  1743 debug-util.cc:398] Leaking SignalData structure 0x7b08000b6980 after lost signal to thread 2373
W20260709 13:06:09.885061  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:09.915114   431 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 1669)
W20260709 13:06:09.919186  1685 consensus_peers.cc:597] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:06:09.930675  9911 raft_consensus.cc:493] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:09.931504  9911 raft_consensus.cc:515] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:09.934194  9911 leader_election.cc:290] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:09.935931  1685 consensus_peers.cc:597] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:06:09.947119  9911 raft_consensus.cc:493] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:09.947978  9911 raft_consensus.cc:515] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:09.950501  9911 leader_election.cc:290] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:06:09.960258  9911 raft_consensus.cc:493] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:09.961328  9911 raft_consensus.cc:515] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:09.964434  9911 leader_election.cc:290] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:09.968209  1687 consensus_peers.cc:597] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.968997  1685 leader_election.cc:336] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:09.985076  1685 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:10.002812  2312 consensus_peers.cc:597] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:10.004853  1685 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:10.007786  1687 leader_election.cc:336] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:10.008980  1687 leader_election.cc:304] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:10.013118  9871 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:10.013917  9871 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:06:10.016594  9911 raft_consensus.cc:2749] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:10.021529  2312 leader_election.cc:336] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:10.028131  2314 leader_election.cc:336] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:10.028584  2314 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:10.029695  9914 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:10.029742  9871 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:10.054402  1687 leader_election.cc:336] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:10.055035  1687 leader_election.cc:304] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:10.067000  9911 raft_consensus.cc:2749] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:06:10.067278  1687 leader_election.cc:336] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
W20260709 13:06:10.069903  1685 consensus_peers.cc:597] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:06:10.067835  1687 leader_election.cc:304] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:10.073148  9911 raft_consensus.cc:2749] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:06:10.207705  1685 consensus_peers.cc:597] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:10.218317  1685 consensus_peers.cc:597] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:10.250886  2312 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:10.272673  1685 consensus_peers.cc:597] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:10.333930  2314 consensus_peers.cc:597] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:10.389659  1685 consensus_peers.cc:597] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:10.418928  2312 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:10.564893  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 96: this message will repeat every 5th retry.
W20260709 13:06:10.665788  1685 consensus_peers.cc:597] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:10.702684  1685 consensus_peers.cc:597] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:10.711833  2312 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:10.741110  2314 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:10.784869  2314 consensus_peers.cc:597] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:10.831826  2314 consensus_peers.cc:597] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:10.837692  1687 consensus_peers.cc:597] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:10.843626  1685 consensus_peers.cc:597] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:06:10.861711  9916 raft_consensus.cc:493] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:10.862598  9916 raft_consensus.cc:515] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:10.866281  9916 leader_election.cc:290] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:10.875790  1687 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
W20260709 13:06:10.876974  1685 leader_election.cc:336] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:10.877367  1685 leader_election.cc:304] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:10.878870  9916 raft_consensus.cc:2749] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:10.916425   431 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 1669)
W20260709 13:06:10.920742  2314 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:10.980341  1687 consensus_peers.cc:597] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:11.039141  9916 raft_consensus.cc:493] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:11.039921  9916 raft_consensus.cc:515] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:11.043630  9916 leader_election.cc:290] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:11.051398  1687 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
W20260709 13:06:11.052296  1685 leader_election.cc:336] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:11.052752  1685 leader_election.cc:304] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:11.053617  9916 raft_consensus.cc:2749] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:11.107579  9916 raft_consensus.cc:493] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:11.108314  9916 raft_consensus.cc:515] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:06:11.117964  1685 leader_election.cc:336] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:11.118894  1687 leader_election.cc:336] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:11.119333  1687 leader_election.cc:304] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:11.120273  9918 raft_consensus.cc:2749] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:11.121352  9916 leader_election.cc:290] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:06:11.140789  9916 raft_consensus.cc:493] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:11.141537  9916 raft_consensus.cc:515] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:06:11.151692  1685 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:11.159726  9916 leader_election.cc:290] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:11.168591  1687 leader_election.cc:336] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:11.169143  1687 leader_election.cc:304] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:11.170396  9916 raft_consensus.cc:2749] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:06:11.194171  2314 consensus_peers.cc:597] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:11.243279  9916 raft_consensus.cc:493] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:11.243971  9916 raft_consensus.cc:515] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:06:11.250675  2314 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:11.251094  9916 leader_election.cc:290] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:11.259851  1685 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:11.262147  2312 consensus_peers.cc:597] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:11.290956  1687 leader_election.cc:336] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:11.294981  1687 leader_election.cc:304] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:11.301826  9916 raft_consensus.cc:2749] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:06:11.313050  1685 consensus_peers.cc:597] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:11.301436  1687 consensus_peers.cc:597] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:11.318439  1687 consensus_peers.cc:597] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:11.319013  1687 consensus_peers.cc:597] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:11.319588  1687 consensus_peers.cc:597] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:11.330910  1687 consensus_peers.cc:597] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:11.370657  1687 consensus_peers.cc:597] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:11.387384  1687 consensus_peers.cc:597] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:11.388175  1687 consensus_peers.cc:597] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.391378  1687 consensus_peers.cc:597] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:11.397872  1455 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111) [suppressed 644 similar messages]
W20260709 13:06:11.398769  1455 catalog_manager.cc:4755] TS 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 58b222526a1a4b52bdcda825e15c46f9: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:11.421193  2314 consensus_peers.cc:597] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:11.424469  1685 consensus_peers.cc:597] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:11.433416  2314 consensus_peers.cc:597] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:11.472754  9918 raft_consensus.cc:493] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:11.473403  9918 raft_consensus.cc:515] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:11.475690  9918 leader_election.cc:290] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:11.484261  1685 leader_election.cc:336] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:11.485535  1687 leader_election.cc:336] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:11.485989  1687 leader_election.cc:304] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:11.487888  9916 raft_consensus.cc:2749] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:11.489588  1687 consensus_peers.cc:597] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:11.513106  9918 raft_consensus.cc:493] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:11.513816  9918 raft_consensus.cc:515] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:11.515947  9918 leader_election.cc:290] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:11.523131  1687 leader_election.cc:336] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
W20260709 13:06:11.524186  1685 leader_election.cc:336] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:11.524605  1685 leader_election.cc:304] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:11.525678  9916 raft_consensus.cc:2749] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:11.538398  9916 raft_consensus.cc:493] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:11.538918  9916 raft_consensus.cc:515] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:11.547405  9919 raft_consensus.cc:493] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:11.548437  9916 leader_election.cc:290] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:06:11.548228  9919 raft_consensus.cc:515] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } attrs { replace: true } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
W20260709 13:06:11.551643  1685 leader_election.cc:336] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:11.552466  1687 consensus_peers.cc:597] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:11.554378  9919 leader_election.cc:290] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293), 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553)
I20260709 13:06:11.556005  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e87427e2b27d4f89b7953163626fd8e4" candidate_uuid: "cf2f71592be8465fb18e36e15d0b2f89" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "130bc5ff39b847b1b3204626f284f838" is_pre_election: true
I20260709 13:06:11.557142  1719 raft_consensus.cc:2468] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2f71592be8465fb18e36e15d0b2f89 in term 19.
W20260709 13:06:11.559358  1687 leader_election.cc:336] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:11.559934  1687 leader_election.cc:304] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:11.562083  9916 raft_consensus.cc:2749] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:11.564517  2312 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:11.565409  2314 leader_election.cc:336] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:11.565981  2314 leader_election.cc:304] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:11.566830  9919 raft_consensus.cc:2749] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:11.636401  1687 consensus_peers.cc:597] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:11.642246  9916 raft_consensus.cc:493] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:11.642935  9916 raft_consensus.cc:515] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
W20260709 13:06:11.658972  1685 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:11.680850  9918 raft_consensus.cc:493] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:11.681435  9918 raft_consensus.cc:515] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:11.683748  9918 leader_election.cc:290] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:06:11.685128  9916 leader_election.cc:290] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:11.687523  1685 consensus_peers.cc:597] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:11.694234  2312 consensus_peers.cc:597] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 96: this message will repeat every 5th retry.
W20260709 13:06:11.706591  1685 consensus_peers.cc:597] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:11.720022  1685 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:11.729213  1687 consensus_peers.cc:597] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:11.729979  1687 leader_election.cc:336] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:11.730487  1687 leader_election.cc:304] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:11.738597  9918 raft_consensus.cc:2749] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:06:11.740186  1687 leader_election.cc:336] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:11.740723  1687 leader_election.cc:304] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:11.741616  9918 raft_consensus.cc:2749] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:06:11.796147  1687 consensus_peers.cc:597] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:11.797624  9918 raft_consensus.cc:493] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:11.799851  9918 raft_consensus.cc:515] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:11.802501  9918 leader_election.cc:290] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:11.822440  1685 leader_election.cc:336] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:11.831285  1687 consensus_peers.cc:597] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:11.874763  1685 consensus_peers.cc:597] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:06:11.908826  9901 raft_consensus.cc:493] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:11.909595  9901 raft_consensus.cc:515] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "cf2f71592be8465fb18e36e15d0b2f89" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39935 } attrs { promote: false } }
I20260709 13:06:11.911803  9901 leader_election.cc:290] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:11.921260  2312 leader_election.cc:336] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:11.927448  2314 leader_election.cc:336] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:11.928126  2314 leader_election.cc:304] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf2f71592be8465fb18e36e15d0b2f89; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:11.929131  9901 raft_consensus.cc:2749] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:06:11.908825  1687 consensus_peers.cc:597] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:11.930900  1687 leader_election.cc:336] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:11.931429  1687 leader_election.cc:304] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:11.934420  9918 raft_consensus.cc:2749] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:11.956710   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:06:11.957732  2312 consensus_peers.cc:597] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20260709 13:06:11.958382  2312 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 96: this message will repeat every 5th retry.
W20260709 13:06:11.993893  1687 consensus_peers.cc:597] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:11.994524  1687 consensus_peers.cc:597] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:12.000396  1685 consensus_peers.cc:597] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:12.001068  1685 consensus_peers.cc:597] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:12.026309  1685 consensus_peers.cc:597] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:06:12.040552  9916 raft_consensus.cc:493] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:12.041181  9916 raft_consensus.cc:515] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:12.043195  9916 leader_election.cc:290] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
W20260709 13:06:12.043993  1685 consensus_peers.cc:597] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:12.045476  1687 consensus_peers.cc:597] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:12.049690  1685 leader_election.cc:336] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
W20260709 13:06:12.050952  1687 leader_election.cc:336] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:12.051446  1687 leader_election.cc:304] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:12.052904  9916 raft_consensus.cc:2749] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:12.096346  2312 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 96: this message will repeat every 5th retry.
W20260709 13:06:12.111115  1685 consensus_peers.cc:597] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:06:12.141886   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 13:06:12.197860   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:12.198963   431 tablet_replica.cc:333] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.199874   431 raft_consensus.cc:2243] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.200830   431 raft_consensus.cc:2272] T a8d17fad9f114b0e98d2af0a989c3185 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.203217   431 tablet_replica.cc:333] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.203876   431 raft_consensus.cc:2243] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.204468   431 raft_consensus.cc:2272] T 0c6f11a07e734ec095c6185363fd6b52 P 130bc5ff39b847b1b3204626f284f838 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.206876   431 tablet_replica.cc:333] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.207473   431 raft_consensus.cc:2243] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.208281   431 raft_consensus.cc:2272] T aa55479d716d4bc4893f658afa382d73 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.210397   431 tablet_replica.cc:333] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.211125   431 raft_consensus.cc:2243] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.212235   431 raft_consensus.cc:2272] T 82cbfb23571048409c1bde90d4611c1a P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.215356   431 tablet_replica.cc:333] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.216212   431 raft_consensus.cc:2243] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.217566   431 raft_consensus.cc:2272] T 8f06439379e24c5ebedb229b007236c2 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.220105   431 tablet_replica.cc:333] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.220995   431 raft_consensus.cc:2243] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.221776   431 raft_consensus.cc:2272] T dbbf62b8237649f584432aa4a9d96ec6 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.224712   431 tablet_replica.cc:333] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.225411   431 raft_consensus.cc:2243] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.226718   431 raft_consensus.cc:2272] T b82d3b49001741eda872554a7ecb2db3 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.227890  9916 raft_consensus.cc:493] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:12.228463  9916 raft_consensus.cc:515] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:12.230180   431 tablet_replica.cc:333] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.231098   431 raft_consensus.cc:2243] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.231981   431 raft_consensus.cc:2272] T b7b0d191c5dc46468a777fba0a666b32 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:12.233218  1687 consensus_peers.cc:597] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:12.234268  9916 leader_election.cc:290] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:06:12.235054   431 tablet_replica.cc:333] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.236019   431 raft_consensus.cc:2243] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 17 LEADER]: Raft consensus shutting down.
W20260709 13:06:12.236653  1685 leader_election.cc:336] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:12.237478   431 raft_consensus.cc:2272] T baba78285345483d9cf6ead6bf20106b P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:12.239521  1687 leader_election.cc:336] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:12.239950  1687 leader_election.cc:304] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
I20260709 13:06:12.240366   431 tablet_replica.cc:333] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.240736  9916 raft_consensus.cc:2749] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:12.241271   431 raft_consensus.cc:2243] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.242727   431 raft_consensus.cc:2272] T 08853c4c3f0b40ddb5a0c34f0c2ee95a P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.246270   431 tablet_replica.cc:333] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
W20260709 13:06:12.246949  2313 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Couldn't send request to peer 130bc5ff39b847b1b3204626f284f838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:12.247114   431 raft_consensus.cc:2243] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.247890   431 raft_consensus.cc:2272] T 9688a8a5839c4bb0b5a0196390670b14 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.250455   431 tablet_replica.cc:333] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.251155   431 raft_consensus.cc:2243] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.252545   431 raft_consensus.cc:2272] T a51c23e1c3ae4bdbbb5f21f1607da772 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.255421   431 tablet_replica.cc:333] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
W20260709 13:06:12.255766  1687 consensus_peers.cc:597] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:12.256369   431 raft_consensus.cc:2243] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.257342   431 raft_consensus.cc:2272] T 380ceca851ad4a84b89cc2e55313d10b P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.260318   431 tablet_replica.cc:333] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.261252   431 raft_consensus.cc:2243] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 16 LEADER]: Raft consensus shutting down.
W20260709 13:06:12.262040  2313 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Couldn't send request to peer 130bc5ff39b847b1b3204626f284f838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:12.262992   431 raft_consensus.cc:2272] T 59d236b930214b1f91fb55ece6fecd68 P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:12.265086  2313 consensus_peers.cc:597] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Couldn't send request to peer 130bc5ff39b847b1b3204626f284f838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:12.266013   431 tablet_replica.cc:333] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.266736   431 raft_consensus.cc:2243] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.267939   431 raft_consensus.cc:2272] T e7a5f5eedbd1406788cafbad6a95a4c7 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.270969   431 tablet_replica.cc:333] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
W20260709 13:06:12.271399  2313 consensus_peers.cc:597] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Couldn't send request to peer 130bc5ff39b847b1b3204626f284f838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:12.271853   431 raft_consensus.cc:2243] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.273298   431 raft_consensus.cc:2272] T 1d9390db02bf4ea689cbc36b1b946a14 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.275635   431 tablet_replica.cc:333] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.276453   431 raft_consensus.cc:2243] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.277275   431 raft_consensus.cc:2272] T c6485a5991c348019b9779be81656a45 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.279702   431 tablet_replica.cc:333] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.280471   431 raft_consensus.cc:2243] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.281376   431 raft_consensus.cc:2272] T f59ca0c8e4c0491480502ebf944ba503 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.284044   431 tablet_replica.cc:333] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.284654   431 raft_consensus.cc:2243] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.285712   431 raft_consensus.cc:2272] T e87427e2b27d4f89b7953163626fd8e4 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.288026   431 tablet_replica.cc:333] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.288679   431 raft_consensus.cc:2243] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.289412   431 raft_consensus.cc:2272] T 7ae86dda8d6f4caf8f08a1607327d9f9 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.292150   431 tablet_replica.cc:333] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.293031   431 raft_consensus.cc:2243] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.294394   431 raft_consensus.cc:2272] T 45c609109d4a4c40a3e74ff0917c7958 P 130bc5ff39b847b1b3204626f284f838 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.297405   431 tablet_replica.cc:333] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.298273   431 raft_consensus.cc:2243] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.298935   431 raft_consensus.cc:2272] T d747d0b326fa4be595dbd31d3638164d P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:12.300686  2313 consensus_peers.cc:597] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Couldn't send request to peer 130bc5ff39b847b1b3204626f284f838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:12.301892   431 tablet_replica.cc:333] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.302735   431 raft_consensus.cc:2243] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.303674   431 raft_consensus.cc:2272] T d3ff300e7f45424ea3bcd7c816082956 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:12.303808  2313 consensus_peers.cc:597] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Couldn't send request to peer 130bc5ff39b847b1b3204626f284f838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:12.306529   431 tablet_replica.cc:333] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.307513   431 raft_consensus.cc:2243] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.308539   431 raft_consensus.cc:2272] T e5c79254e6ff49639cf09f663490dfd7 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.311503   431 tablet_replica.cc:333] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.312402   431 raft_consensus.cc:2243] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.313477   431 raft_consensus.cc:2272] T 58b222526a1a4b52bdcda825e15c46f9 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.316725   431 tablet_replica.cc:333] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.317518   431 raft_consensus.cc:2243] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.317780  9916 raft_consensus.cc:493] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5f4b2707a9647e5bb99e065068dea2b)
I20260709 13:06:12.318619   431 raft_consensus.cc:2272] T 27aa13d58e6d4987b2e34613996ef421 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.318536  9916 raft_consensus.cc:515] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "066ee0109aba4465ba17fa65e14ef504" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39999 } } peers { permanent_uuid: "f5f4b2707a9647e5bb99e065068dea2b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44293 } } peers { permanent_uuid: "130bc5ff39b847b1b3204626f284f838" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34553 } }
I20260709 13:06:12.321516   431 tablet_replica.cc:333] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.321537  9916 leader_election.cc:290] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999), f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293)
I20260709 13:06:12.322502   431 raft_consensus.cc:2243] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.323904   431 raft_consensus.cc:2272] T 0df07602233941598bee7b19607b2e3b P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.326969   431 tablet_replica.cc:333] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.327857   431 raft_consensus.cc:2243] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:12.328234  1685 leader_election.cc:336] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:12.328552   431 raft_consensus.cc:2272] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:12.328847  1687 leader_election.cc:336] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111)
I20260709 13:06:12.329285  1687 leader_election.cc:304] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 130bc5ff39b847b1b3204626f284f838; no voters: 066ee0109aba4465ba17fa65e14ef504, f5f4b2707a9647e5bb99e065068dea2b
W20260709 13:06:12.329948  1687 raft_consensus.cc:2686] T 5bd2b69727c743cfbdc52253e16279ce P 130bc5ff39b847b1b3204626f284f838: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:06:12.331866   431 tablet_replica.cc:333] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.332960   431 raft_consensus.cc:2243] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.334287   431 raft_consensus.cc:2272] T bbc4aafd02974a29b0c9b332b55c978c P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.336525   431 tablet_replica.cc:333] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.337257   431 raft_consensus.cc:2243] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.338441   431 raft_consensus.cc:2272] T 56082a5ff84643f0ba368ceba0524623 P 130bc5ff39b847b1b3204626f284f838 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.341362   431 tablet_replica.cc:333] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.342089   431 raft_consensus.cc:2243] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.343694   431 raft_consensus.cc:2272] T ca833b0b78d74adc8ca86e91fc94f96f P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.345824   431 tablet_replica.cc:333] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.346434   431 raft_consensus.cc:2243] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.347186   431 raft_consensus.cc:2272] T 18bdd841c8294b08857684489689676d P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.349324   431 tablet_replica.cc:333] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.349938   431 raft_consensus.cc:2243] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.350502   431 raft_consensus.cc:2272] T dcc462cf3919460abb1ef504377a5382 P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.352941   431 tablet_replica.cc:333] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.353765   431 raft_consensus.cc:2243] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.354650   431 raft_consensus.cc:2272] T 7e9b1a584a5f430a9a0e7e617064bc2c P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.357724   431 tablet_replica.cc:333] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.358538   431 raft_consensus.cc:2243] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:06:12.359005  2312 consensus_peers.cc:597] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:06:12.359879   431 raft_consensus.cc:2272] T ca2582f1941a4c588f8ef51c03dfb492 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.362005   431 tablet_replica.cc:333] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.362573   431 raft_consensus.cc:2243] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.363291   431 raft_consensus.cc:2272] T 15e07cb6db0d46a78611f24c17d04cc9 P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.365300   431 tablet_replica.cc:333] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.365942   431 raft_consensus.cc:2243] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.367089   431 raft_consensus.cc:2272] T e89a11b4204e48568da409c196e33be4 P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.369414   431 tablet_replica.cc:333] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.369998   431 raft_consensus.cc:2243] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.370918   431 raft_consensus.cc:2272] T 1a01faf4c6c64b549e80adf03bba9f51 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.373468   431 tablet_replica.cc:333] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.374073   431 raft_consensus.cc:2243] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.374888   431 raft_consensus.cc:2272] T 48ee6d32e41946dc9cf53a3d62cdb9ae P 130bc5ff39b847b1b3204626f284f838 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.376912   431 tablet_replica.cc:333] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.377465   431 raft_consensus.cc:2243] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.378660   431 raft_consensus.cc:2272] T 49c94b9e96df4808adbda99815b09c5f P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.380527   431 tablet_replica.cc:333] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.381134   431 raft_consensus.cc:2243] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.382249   431 raft_consensus.cc:2272] T f81336f0bd194664ac95af1a897b4b18 P 130bc5ff39b847b1b3204626f284f838 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.384331   431 tablet_replica.cc:333] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.384979   431 raft_consensus.cc:2243] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.385524   431 raft_consensus.cc:2272] T 12ac45e0e48449a98fb89704bae3de27 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.387702   431 tablet_replica.cc:333] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.388422   431 raft_consensus.cc:2243] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.389787   431 raft_consensus.cc:2272] T 7501558aa93542308c5fc758446387df P 130bc5ff39b847b1b3204626f284f838 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.392062   431 tablet_replica.cc:333] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.392916   431 raft_consensus.cc:2243] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.394248   431 raft_consensus.cc:2272] T f50422a819a849c0a93a4a07c9af6ba0 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.396941   431 tablet_replica.cc:333] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.397547   431 raft_consensus.cc:2243] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.398689   431 raft_consensus.cc:2272] T 97987f9622cb4f52a1b74b4ee6bfdfdb P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.400862   431 tablet_replica.cc:333] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.401590   431 raft_consensus.cc:2243] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.402395   431 raft_consensus.cc:2272] T e59deed37c414a929faeb85bdbaaa406 P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.404980   431 tablet_replica.cc:333] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
W20260709 13:06:12.405170  2313 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Couldn't send request to peer 130bc5ff39b847b1b3204626f284f838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:12.405884   431 raft_consensus.cc:2243] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.406786   431 raft_consensus.cc:2272] T 6b98c1e123bd4894b2ab4096efb42cd8 P 130bc5ff39b847b1b3204626f284f838 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.409124   431 tablet_replica.cc:333] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.409770   431 raft_consensus.cc:2243] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.410656   431 raft_consensus.cc:2272] T eca30b43c26f420c965ee25f475581a0 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.412724   431 tablet_replica.cc:333] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.413264   431 raft_consensus.cc:2243] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.414784   431 raft_consensus.cc:2272] T adc326ba3c1b4dc7aa89a3a071803a7e P 130bc5ff39b847b1b3204626f284f838 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.416908   431 tablet_replica.cc:333] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.417411   431 raft_consensus.cc:2243] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.418480   431 raft_consensus.cc:2272] T ad4b6776963146fab5ee9487b6684e8c P 130bc5ff39b847b1b3204626f284f838 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.420542   431 tablet_replica.cc:333] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.421034   431 raft_consensus.cc:2243] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.421473   431 raft_consensus.cc:2272] T 418152c2665a4b04be8a39c62a52f6ca P 130bc5ff39b847b1b3204626f284f838 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.423372   431 tablet_replica.cc:333] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.423894   431 raft_consensus.cc:2243] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.424571   431 raft_consensus.cc:2272] T dbcb28b8d4104c6fae6682866cc0469d P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.427362   431 tablet_replica.cc:333] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838: stopping tablet replica
I20260709 13:06:12.428013   431 raft_consensus.cc:2243] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.428886   431 raft_consensus.cc:2272] T baa23f35bd3b4c2d99e3da0ff0f80e97 P 130bc5ff39b847b1b3204626f284f838 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:12.429827  2313 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Couldn't send request to peer 130bc5ff39b847b1b3204626f284f838. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:12.533831  2313 consensus_peers.cc:597] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Couldn't send request to peer 130bc5ff39b847b1b3204626f284f838. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:34553: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:12.574028  2313 consensus_peers.cc:597] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): Couldn't send request to peer 130bc5ff39b847b1b3204626f284f838. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:34553: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:12.722180  2312 consensus_peers.cc:597] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20260709 13:06:12.725920  2314 consensus_peers.cc:597] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:12.743148  2312 consensus_peers.cc:597] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:12.800283  2312 consensus_peers.cc:597] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 101: this message will repeat every 5th retry.
W20260709 13:06:12.911832  2312 consensus_peers.cc:597] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): Couldn't send request to peer 066ee0109aba4465ba17fa65e14ef504. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
I20260709 13:06:13.074393   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
W20260709 13:06:13.133268  2314 consensus_peers.cc:597] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:06:13.180747   431 tablet_server.cc:179] TabletServer@127.0.107.196:0 shutting down...
W20260709 13:06:13.232586  2314 consensus_peers.cc:597] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 -> Peer f5f4b2707a9647e5bb99e065068dea2b (127.0.107.194:44293): Couldn't send request to peer f5f4b2707a9647e5bb99e065068dea2b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:44293: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:06:13.238637   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:13.239460   431 tablet_replica.cc:333] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.240408   431 raft_consensus.cc:2243] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.241056   431 raft_consensus.cc:2272] T adc326ba3c1b4dc7aa89a3a071803a7e P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.244028   431 tablet_replica.cc:333] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.244833   431 raft_consensus.cc:2243] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.245426   431 raft_consensus.cc:2272] T e87427e2b27d4f89b7953163626fd8e4 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.248000   431 tablet_replica.cc:333] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.248620   431 raft_consensus.cc:2243] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.249243   431 raft_consensus.cc:2272] T b82d3b49001741eda872554a7ecb2db3 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.251588   431 tablet_replica.cc:333] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.252193   431 raft_consensus.cc:2243] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.253022   431 raft_consensus.cc:2272] T ca2582f1941a4c588f8ef51c03dfb492 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.256719   431 tablet_replica.cc:333] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.257465   431 raft_consensus.cc:2243] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.258256   431 raft_consensus.cc:2272] T 59d236b930214b1f91fb55ece6fecd68 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.260528   431 tablet_replica.cc:333] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.261130   431 raft_consensus.cc:2243] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.262017   431 raft_consensus.cc:2272] T 58b222526a1a4b52bdcda825e15c46f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.264169   431 tablet_replica.cc:333] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.264707   431 raft_consensus.cc:2243] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.265597   431 raft_consensus.cc:2272] T eca30b43c26f420c965ee25f475581a0 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.268074   431 tablet_replica.cc:333] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.268692   431 raft_consensus.cc:2243] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.269307   431 raft_consensus.cc:2272] T f0ee72a7f576482aabe40ef837964708 P cf2f71592be8465fb18e36e15d0b2f89 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.271512   431 tablet_replica.cc:333] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.272220   431 raft_consensus.cc:2243] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.273166   431 raft_consensus.cc:2272] T 380ceca851ad4a84b89cc2e55313d10b P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.275857   431 tablet_replica.cc:333] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.276538   431 raft_consensus.cc:2243] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.277855   431 raft_consensus.cc:2272] T 08cf0535992b4db395d567393667593b P cf2f71592be8465fb18e36e15d0b2f89 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.280731   431 tablet_replica.cc:333] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.281307   431 raft_consensus.cc:2243] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.282222   431 raft_consensus.cc:2272] T 0c6f11a07e734ec095c6185363fd6b52 P cf2f71592be8465fb18e36e15d0b2f89 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.284477   431 tablet_replica.cc:333] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.285010   431 raft_consensus.cc:2243] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.285881   431 raft_consensus.cc:2272] T 27aa13d58e6d4987b2e34613996ef421 P cf2f71592be8465fb18e36e15d0b2f89 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.287979   431 tablet_replica.cc:333] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.288511   431 raft_consensus.cc:2243] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.289407   431 raft_consensus.cc:2272] T b7b0d191c5dc46468a777fba0a666b32 P cf2f71592be8465fb18e36e15d0b2f89 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.291419   431 tablet_replica.cc:333] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.291939   431 raft_consensus.cc:2243] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.292742   431 raft_consensus.cc:2272] T 9688a8a5839c4bb0b5a0196390670b14 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.294728   431 tablet_replica.cc:333] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.295245   431 raft_consensus.cc:2243] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.296175   431 raft_consensus.cc:2272] T d2618e77160b44339d8499efcd1e8eca P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.298225   431 tablet_replica.cc:333] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.298765   431 raft_consensus.cc:2243] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.299723   431 raft_consensus.cc:2272] T bddd1ebc9af24fa282580ab028d643e0 P cf2f71592be8465fb18e36e15d0b2f89 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.301731   431 tablet_replica.cc:333] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.302208   431 raft_consensus.cc:2243] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.302999   431 raft_consensus.cc:2272] T f59ca0c8e4c0491480502ebf944ba503 P cf2f71592be8465fb18e36e15d0b2f89 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.304859   431 tablet_replica.cc:333] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.305330   431 raft_consensus.cc:2243] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.305908   431 raft_consensus.cc:2272] T e7a5f5eedbd1406788cafbad6a95a4c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.307786   431 tablet_replica.cc:333] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.308274   431 raft_consensus.cc:2243] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.309211   431 raft_consensus.cc:2272] T c6485a5991c348019b9779be81656a45 P cf2f71592be8465fb18e36e15d0b2f89 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.311266   431 tablet_replica.cc:333] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.311861   431 raft_consensus.cc:2243] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.312790   431 raft_consensus.cc:2272] T 7ae86dda8d6f4caf8f08a1607327d9f9 P cf2f71592be8465fb18e36e15d0b2f89 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.314853   431 tablet_replica.cc:333] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.315328   431 raft_consensus.cc:2243] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.315894   431 raft_consensus.cc:2272] T 45c609109d4a4c40a3e74ff0917c7958 P cf2f71592be8465fb18e36e15d0b2f89 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.318028   431 tablet_replica.cc:333] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.318481   431 raft_consensus.cc:2243] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.319801   431 raft_consensus.cc:2272] T 0764c6a89eca47f39e28feb56e2123c7 P cf2f71592be8465fb18e36e15d0b2f89 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.322134   431 tablet_replica.cc:333] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89: stopping tablet replica
I20260709 13:06:13.322736   431 raft_consensus.cc:2243] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.323596   431 raft_consensus.cc:2272] T 3ee0e81a514641709317e983fcf96fe4 P cf2f71592be8465fb18e36e15d0b2f89 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.710443   431 tablet_server.cc:196] TabletServer@127.0.107.196:0 shutdown complete.
I20260709 13:06:13.760102   431 master.cc:562] Master@127.0.107.254:38281 shutting down...
W20260709 13:06:13.809113  1519 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:06:13.809413  1519 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:06:13.809670  1519 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:06:16.614343  1456 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.195:34553: connect: Connection refused (error 111) [suppressed 196 similar messages]
W20260709 13:06:16.618307  1456 catalog_manager.cc:4755] TS 130bc5ff39b847b1b3204626f284f838 (127.0.107.195:34553): ChangeConfig:REMOVE_PEER RPC failed for tablet b82d3b49001741eda872554a7ecb2db3: Network error: Client connection negotiation failed: client connection to 127.0.107.195:34553: connect: Connection refused (error 111)
W20260709 13:06:44.196120  1455 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260709 13:06:44.200093  1455 catalog_manager.cc:4755] TS 066ee0109aba4465ba17fa65e14ef504 (127.0.107.193:39999): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 58b222526a1a4b52bdcda825e15c46f9: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39999: connect: Connection refused (error 111)
I20260709 13:06:46.989104   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:46.990821   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:46.991302   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 220fdfc07a3c4ce28ca7333445633466: stopping tablet replica
I20260709 13:06:47.259048   431 master.cc:584] Master@127.0.107.254:38281 shutdown complete.
I20260709 13:06:47.321862   431 test_util.cc:182] -----------------------------------------------
I20260709 13:06:47.322263   431 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602054858684-431-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (325758 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:06:47.326877   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:43521
I20260709 13:06:47.328226   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:47.334769  9930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:47.335036  9931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:47.335976  9933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:47.337846   431 server_base.cc:1061] running on GCE node
I20260709 13:06:47.343760   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:47.344028   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:47.344147   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602407344136 us; error 0 us; skew 500 ppm
I20260709 13:06:47.344980   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:47.348657   431 webserver.cc:533] Webserver started at http://127.0.107.254:38915/ using document root <none> and password file <none>
I20260709 13:06:47.349179   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:47.349345   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:47.349588   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:47.351096   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/master-0-root/instance:
uuid: "0b93df6119c0475a83c010dafc5ff2bd"
format_stamp: "Formatted at 2026-07-09 13:06:47 on dist-test-slave-mxd3"
I20260709 13:06:47.356513   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:06:47.360756  9938 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:47.361869   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:47.362263   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/master-0-root
uuid: "0b93df6119c0475a83c010dafc5ff2bd"
format_stamp: "Formatted at 2026-07-09 13:06:47 on dist-test-slave-mxd3"
I20260709 13:06:47.362782   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:47.383740   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:47.385586   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:47.438030   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:43521
I20260709 13:06:47.438158  9989 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:43521 every 8 connection(s)
I20260709 13:06:47.443228  9990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.458554  9990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd: Bootstrap starting.
I20260709 13:06:47.466602  9990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.471737  9990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd: No bootstrap required, opened a new log
I20260709 13:06:47.474172  9990 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b93df6119c0475a83c010dafc5ff2bd" member_type: VOTER }
I20260709 13:06:47.474619  9990 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.474869  9990 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b93df6119c0475a83c010dafc5ff2bd, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.475567  9990 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b93df6119c0475a83c010dafc5ff2bd" member_type: VOTER }
I20260709 13:06:47.476032  9990 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:47.476260  9990 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:47.476550  9990 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:47.481781  9990 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b93df6119c0475a83c010dafc5ff2bd" member_type: VOTER }
I20260709 13:06:47.482404  9990 leader_election.cc:304] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b93df6119c0475a83c010dafc5ff2bd; no voters: 
I20260709 13:06:47.483696  9990 leader_election.cc:290] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:47.484346  9993 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:47.486156  9993 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd [term 1 LEADER]: Becoming Leader. State: Replica: 0b93df6119c0475a83c010dafc5ff2bd, State: Running, Role: LEADER
I20260709 13:06:47.486982  9993 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b93df6119c0475a83c010dafc5ff2bd" member_type: VOTER }
I20260709 13:06:47.487169  9990 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:47.494868  9994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b93df6119c0475a83c010dafc5ff2bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b93df6119c0475a83c010dafc5ff2bd" member_type: VOTER } }
I20260709 13:06:47.495749  9994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:47.495499  9995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b93df6119c0475a83c010dafc5ff2bd. Latest consensus state: current_term: 1 leader_uuid: "0b93df6119c0475a83c010dafc5ff2bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b93df6119c0475a83c010dafc5ff2bd" member_type: VOTER } }
I20260709 13:06:47.496438  9995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:47.498725 10003 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:47.505187 10003 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:47.506474   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:47.515443 10003 catalog_manager.cc:1383] Generated new cluster ID: c047ed56b2e5492b8098ef0f384dd8a6
I20260709 13:06:47.515697 10003 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:47.542820 10003 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:47.544286 10003 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:47.559265 10003 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd: Generated new TSK 0
I20260709 13:06:47.559978 10003 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:47.573513   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:47.581306 10011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:47.582137 10012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:47.583811 10014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:47.585480   431 server_base.cc:1061] running on GCE node
I20260709 13:06:47.588150   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:47.588364   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:47.588598   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602407588579 us; error 0 us; skew 500 ppm
I20260709 13:06:47.589283   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:47.591900   431 webserver.cc:533] Webserver started at http://127.0.107.193:38361/ using document root <none> and password file <none>
I20260709 13:06:47.592443   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:47.592653   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:47.592944   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:47.594264   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-0-root/instance:
uuid: "efa338cd132c457896b11e08d7ea787d"
format_stamp: "Formatted at 2026-07-09 13:06:47 on dist-test-slave-mxd3"
I20260709 13:06:47.599326   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:06:47.603196 10019 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:47.604230   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:06:47.604611   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-0-root
uuid: "efa338cd132c457896b11e08d7ea787d"
format_stamp: "Formatted at 2026-07-09 13:06:47 on dist-test-slave-mxd3"
I20260709 13:06:47.604919   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:47.626382   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:47.627800   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:47.629545   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:47.632210   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:47.632499   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:47.632767   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:47.632970   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:47.682763   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:42425
I20260709 13:06:47.682972 10081 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:42425 every 8 connection(s)
I20260709 13:06:47.687938   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:47.696583 10086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:47.700656 10087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:47.702603 10089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:47.703665   431 server_base.cc:1061] running on GCE node
I20260709 13:06:47.707226   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:47.707480   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:47.707734   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602407707712 us; error 0 us; skew 500 ppm
I20260709 13:06:47.708474   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:47.710148 10082 heartbeater.cc:344] Connected to a master server at 127.0.107.254:43521
I20260709 13:06:47.710644 10082 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:47.711623 10082 heartbeater.cc:507] Master 127.0.107.254:43521 requested a full tablet report, sending...
I20260709 13:06:47.712146   431 webserver.cc:533] Webserver started at http://127.0.107.194:36779/ using document root <none> and password file <none>
I20260709 13:06:47.712688   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:47.712881   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:47.713205   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:47.714519   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-1-root/instance:
uuid: "575e44d297f045a389903d65a471d548"
format_stamp: "Formatted at 2026-07-09 13:06:47 on dist-test-slave-mxd3"
I20260709 13:06:47.714558  9955 ts_manager.cc:194] Registered new tserver with Master: efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:47.716977  9955 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51784
I20260709 13:06:47.719578   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:06:47.723759 10094 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:47.724720   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:06:47.725142   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-1-root
uuid: "575e44d297f045a389903d65a471d548"
format_stamp: "Formatted at 2026-07-09 13:06:47 on dist-test-slave-mxd3"
I20260709 13:06:47.725453   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:47.755779   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:47.757084   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:47.758810   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:47.761585   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:47.761909   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:06:47.762328   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:47.762727   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:47.806946   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:33415
I20260709 13:06:47.806990 10156 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:33415 every 8 connection(s)
I20260709 13:06:47.813177   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:47.820969 10160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:47.821463 10161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:47.825881 10157 heartbeater.cc:344] Connected to a master server at 127.0.107.254:43521
I20260709 13:06:47.826567 10157 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:47.827816 10157 heartbeater.cc:507] Master 127.0.107.254:43521 requested a full tablet report, sending...
W20260709 13:06:47.828147 10163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:47.828485   431 server_base.cc:1061] running on GCE node
I20260709 13:06:47.829769   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:47.830067   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:47.830297   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602407830278 us; error 0 us; skew 500 ppm
I20260709 13:06:47.830664  9955 ts_manager.cc:194] Registered new tserver with Master: 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:47.831122   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:47.833283  9955 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51788
I20260709 13:06:47.834718   431 webserver.cc:533] Webserver started at http://127.0.107.195:40281/ using document root <none> and password file <none>
I20260709 13:06:47.835439   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:47.835700   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:47.835913   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:47.837688   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-2-root/instance:
uuid: "8fe33410d51a469e81be6edc602d7d78"
format_stamp: "Formatted at 2026-07-09 13:06:47 on dist-test-slave-mxd3"
I20260709 13:06:47.845088   431 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:47.850700 10168 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:47.852085   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:06:47.852568   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-2-root
uuid: "8fe33410d51a469e81be6edc602d7d78"
format_stamp: "Formatted at 2026-07-09 13:06:47 on dist-test-slave-mxd3"
I20260709 13:06:47.852970   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:47.865927   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:47.867311   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:47.868981   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:47.871927   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:47.872254   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:47.872565   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:47.872756   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:47.919040   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:40419
I20260709 13:06:47.919179 10230 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:40419 every 8 connection(s)
I20260709 13:06:47.924041   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:47.932348 10234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:47.932567 10235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:47.936637 10237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:47.937453   431 server_base.cc:1061] running on GCE node
I20260709 13:06:47.937832 10231 heartbeater.cc:344] Connected to a master server at 127.0.107.254:43521
I20260709 13:06:47.938393 10231 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:47.938661   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:47.938936   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:47.939127   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602407939109 us; error 0 us; skew 500 ppm
I20260709 13:06:47.939424 10231 heartbeater.cc:507] Master 127.0.107.254:43521 requested a full tablet report, sending...
I20260709 13:06:47.939821   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:47.941820  9955 ts_manager.cc:194] Registered new tserver with Master: 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:47.942870   431 webserver.cc:533] Webserver started at http://127.0.107.196:43231/ using document root <none> and password file <none>
I20260709 13:06:47.943414   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:47.943610   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:47.943548  9955 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51790
I20260709 13:06:47.944039   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:47.945367   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-3-root/instance:
uuid: "50999113203c482b9b9300caa485f154"
format_stamp: "Formatted at 2026-07-09 13:06:47 on dist-test-slave-mxd3"
I20260709 13:06:47.949962   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:06:47.953218 10242 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:47.954133   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:06:47.954450   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-3-root
uuid: "50999113203c482b9b9300caa485f154"
format_stamp: "Formatted at 2026-07-09 13:06:47 on dist-test-slave-mxd3"
I20260709 13:06:47.954785   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:47.967540   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:47.968865   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:47.970559   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:47.972872   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:47.973062   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:47.973366   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:47.973515   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:48.016824   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.196:40841
I20260709 13:06:48.017030 10304 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.196:40841 every 8 connection(s)
I20260709 13:06:48.031777 10305 heartbeater.cc:344] Connected to a master server at 127.0.107.254:43521
I20260709 13:06:48.032181 10305 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:48.032933 10305 heartbeater.cc:507] Master 127.0.107.254:43521 requested a full tablet report, sending...
I20260709 13:06:48.034828  9955 ts_manager.cc:194] Registered new tserver with Master: 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:48.035590   431 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.015237794s
I20260709 13:06:48.036185  9955 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51804
I20260709 13:06:48.065609  9955 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51818:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:06:48.316017 10047 tablet_service.cc:1511] Processing CreateTablet for tablet 04676809490049a5923174641a3c1137 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:06:48.317256 10045 tablet_service.cc:1511] Processing CreateTablet for tablet 80b0c1b6201b46d5b1f46ccad59503b7 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:06:48.317992 10047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04676809490049a5923174641a3c1137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.318742 10045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80b0c1b6201b46d5b1f46ccad59503b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.320317 10044 tablet_service.cc:1511] Processing CreateTablet for tablet aa03c270e7364d769700b877d171588e (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:06:48.322222 10044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa03c270e7364d769700b877d171588e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.322983 10046 tablet_service.cc:1511] Processing CreateTablet for tablet 83a64aca789f4ea8a4925aa79dcb0235 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:06:48.324431 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a64aca789f4ea8a4925aa79dcb0235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.327227 10043 tablet_service.cc:1511] Processing CreateTablet for tablet 3fa9341e7fbe49e1a832721bfa4abff3 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:06:48.328756 10043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fa9341e7fbe49e1a832721bfa4abff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.332693 10042 tablet_service.cc:1511] Processing CreateTablet for tablet 5461abd09b074193b3d03df659fa2ced (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:06:48.334200 10042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5461abd09b074193b3d03df659fa2ced. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.339578 10196 tablet_service.cc:1511] Processing CreateTablet for tablet 04676809490049a5923174641a3c1137 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:06:48.340348 10194 tablet_service.cc:1511] Processing CreateTablet for tablet 56df6e7bddfd4832b31ccc4bb8b3521b (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:06:48.341234 10196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04676809490049a5923174641a3c1137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.342880 10192 tablet_service.cc:1511] Processing CreateTablet for tablet 5461abd09b074193b3d03df659fa2ced (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:06:48.344394 10192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5461abd09b074193b3d03df659fa2ced. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.346879 10191 tablet_service.cc:1511] Processing CreateTablet for tablet e959733d8ef94ea19aad472fcf384371 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:06:48.348580 10191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e959733d8ef94ea19aad472fcf384371. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.348160 10041 tablet_service.cc:1511] Processing CreateTablet for tablet 52ffbaf3a54145c2a791c4a67dbdf64a (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:06:48.346880 10195 tablet_service.cc:1511] Processing CreateTablet for tablet 83a64aca789f4ea8a4925aa79dcb0235 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:06:48.351241 10194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56df6e7bddfd4832b31ccc4bb8b3521b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.348935 10193 tablet_service.cc:1511] Processing CreateTablet for tablet aa03c270e7364d769700b877d171588e (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:06:48.352159 10190 tablet_service.cc:1511] Processing CreateTablet for tablet 52ffbaf3a54145c2a791c4a67dbdf64a (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:06:48.353611 10190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52ffbaf3a54145c2a791c4a67dbdf64a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.356251 10041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52ffbaf3a54145c2a791c4a67dbdf64a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.357375 10193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa03c270e7364d769700b877d171588e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.362623 10195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a64aca789f4ea8a4925aa79dcb0235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.394368 10324 tablet_bootstrap.cc:492] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:48.403992 10044 tablet_service.cc:1511] Processing CreateTablet for tablet 1f3c90bb2ac14652a45cdcccba25f97c (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:06:48.405619 10044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f3c90bb2ac14652a45cdcccba25f97c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.411281 10325 tablet_bootstrap.cc:492] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:48.411610 10324 tablet_bootstrap.cc:654] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.411499 10270 tablet_service.cc:1511] Processing CreateTablet for tablet 04676809490049a5923174641a3c1137 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:06:48.413126 10270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04676809490049a5923174641a3c1137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.417770 10267 tablet_service.cc:1511] Processing CreateTablet for tablet 3fa9341e7fbe49e1a832721bfa4abff3 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:06:48.419373 10267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fa9341e7fbe49e1a832721bfa4abff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.419000 10269 tablet_service.cc:1511] Processing CreateTablet for tablet 80b0c1b6201b46d5b1f46ccad59503b7 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:06:48.420465 10269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80b0c1b6201b46d5b1f46ccad59503b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.421734 10266 tablet_service.cc:1511] Processing CreateTablet for tablet 5461abd09b074193b3d03df659fa2ced (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:06:48.423249 10266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5461abd09b074193b3d03df659fa2ced. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.422866 10268 tablet_service.cc:1511] Processing CreateTablet for tablet 56df6e7bddfd4832b31ccc4bb8b3521b (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:06:48.424079 10325 tablet_bootstrap.cc:654] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.424304 10268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56df6e7bddfd4832b31ccc4bb8b3521b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.425513 10265 tablet_service.cc:1511] Processing CreateTablet for tablet e959733d8ef94ea19aad472fcf384371 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:06:48.427065 10265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e959733d8ef94ea19aad472fcf384371. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.431061 10264 tablet_service.cc:1511] Processing CreateTablet for tablet 1f3c90bb2ac14652a45cdcccba25f97c (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:06:48.432596 10264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f3c90bb2ac14652a45cdcccba25f97c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.490051 10324 tablet_bootstrap.cc:492] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:48.490629 10324 ts_tablet_manager.cc:1403] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.097s	user 0.020s	sys 0.017s
I20260709 13:06:48.494028 10324 raft_consensus.cc:359] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:48.494855 10324 raft_consensus.cc:385] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.495162 10324 raft_consensus.cc:740] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.498394 10328 tablet_bootstrap.cc:492] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:48.499522 10195 tablet_service.cc:1511] Processing CreateTablet for tablet b626433d4aec4288976ffc448a79bb68 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
W20260709 13:06:48.500161 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:48.500896 10195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b626433d4aec4288976ffc448a79bb68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.504626 10324 consensus_queue.cc:260] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:48.506124 10325 tablet_bootstrap.cc:492] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:48.506795 10325 ts_tablet_manager.cc:1403] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.096s	user 0.010s	sys 0.022s
I20260709 13:06:48.510074 10325 raft_consensus.cc:359] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:48.510814 10325 raft_consensus.cc:385] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.511118 10325 raft_consensus.cc:740] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.511291 10121 tablet_service.cc:1511] Processing CreateTablet for tablet 80b0c1b6201b46d5b1f46ccad59503b7 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:06:48.513329 10121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80b0c1b6201b46d5b1f46ccad59503b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.515220 10264 tablet_service.cc:1511] Processing CreateTablet for tablet b626433d4aec4288976ffc448a79bb68 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:06:48.515362 10120 tablet_service.cc:1511] Processing CreateTablet for tablet 56df6e7bddfd4832b31ccc4bb8b3521b (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:06:48.516577 10264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b626433d4aec4288976ffc448a79bb68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.518735 10119 tablet_service.cc:1511] Processing CreateTablet for tablet aa03c270e7364d769700b877d171588e (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:06:48.522208 10118 tablet_service.cc:1511] Processing CreateTablet for tablet 3fa9341e7fbe49e1a832721bfa4abff3 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:06:48.523872 10328 tablet_bootstrap.cc:654] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.525688 10117 tablet_service.cc:1511] Processing CreateTablet for tablet e959733d8ef94ea19aad472fcf384371 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:06:48.528334 10116 tablet_service.cc:1511] Processing CreateTablet for tablet 52ffbaf3a54145c2a791c4a67dbdf64a (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:06:48.528178 10122 tablet_service.cc:1511] Processing CreateTablet for tablet 83a64aca789f4ea8a4925aa79dcb0235 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:06:48.530023 10325 consensus_queue.cc:260] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:48.533699 10115 tablet_service.cc:1511] Processing CreateTablet for tablet 1f3c90bb2ac14652a45cdcccba25f97c (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:06:48.535907 10116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52ffbaf3a54145c2a791c4a67dbdf64a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.538174 10122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a64aca789f4ea8a4925aa79dcb0235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.539198 10115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f3c90bb2ac14652a45cdcccba25f97c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.539952 10119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa03c270e7364d769700b877d171588e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.542006 10118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fa9341e7fbe49e1a832721bfa4abff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.546615 10120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56df6e7bddfd4832b31ccc4bb8b3521b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.548542 10114 tablet_service.cc:1511] Processing CreateTablet for tablet c7dfde2e205846058c75fc205443d330 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:06:48.550806 10114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7dfde2e205846058c75fc205443d330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.554641 10046 tablet_service.cc:1511] Processing CreateTablet for tablet b626433d4aec4288976ffc448a79bb68 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:06:48.560997 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b626433d4aec4288976ffc448a79bb68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.570034 10117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e959733d8ef94ea19aad472fcf384371. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.570578 10191 tablet_service.cc:1511] Processing CreateTablet for tablet c7dfde2e205846058c75fc205443d330 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:06:48.572464 10191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7dfde2e205846058c75fc205443d330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.587231 10268 tablet_service.cc:1511] Processing CreateTablet for tablet c7dfde2e205846058c75fc205443d330 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:06:48.589242 10268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7dfde2e205846058c75fc205443d330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.616349 10044 tablet_service.cc:1511] Processing CreateTablet for tablet 6d2f27feb83847298256dd63dafcd35c (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:06:48.617785 10044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d2f27feb83847298256dd63dafcd35c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.622134 10113 tablet_service.cc:1511] Processing CreateTablet for tablet 6d2f27feb83847298256dd63dafcd35c (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:06:48.623600 10113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d2f27feb83847298256dd63dafcd35c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.630278 10082 heartbeater.cc:499] Master 127.0.107.254:43521 was elected leader, sending a full tablet report...
I20260709 13:06:48.638281 10231 heartbeater.cc:499] Master 127.0.107.254:43521 was elected leader, sending a full tablet report...
I20260709 13:06:48.641121 10195 tablet_service.cc:1511] Processing CreateTablet for tablet 6d2f27feb83847298256dd63dafcd35c (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:06:48.638285 10330 raft_consensus.cc:493] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.654747 10195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d2f27feb83847298256dd63dafcd35c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.654389 10330 raft_consensus.cc:515] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:48.654409 10325 ts_tablet_manager.cc:1434] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.147s	user 0.007s	sys 0.005s
I20260709 13:06:48.662528 10328 tablet_bootstrap.cc:492] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:48.663095 10328 ts_tablet_manager.cc:1403] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.165s	user 0.011s	sys 0.011s
I20260709 13:06:48.665863 10325 tablet_bootstrap.cc:492] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:48.666930 10330 leader_election.cc:290] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:48.666276 10328 raft_consensus.cc:359] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:48.654385 10264 tablet_service.cc:1511] Processing CreateTablet for tablet 75ed1b358949418a9d300990f8f81652 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:06:48.672115 10264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75ed1b358949418a9d300990f8f81652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.674916 10042 tablet_service.cc:1511] Processing CreateTablet for tablet 75ed1b358949418a9d300990f8f81652 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
W20260709 13:06:48.676146 10232 tablet.cc:2406] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:06:48.676352 10042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75ed1b358949418a9d300990f8f81652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.680238 10041 tablet_service.cc:1511] Processing CreateTablet for tablet c7d966d297f346da9e984aa2e181590a (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:06:48.682736 10329 raft_consensus.cc:493] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.684415 10045 tablet_service.cc:1511] Processing CreateTablet for tablet 5f64c791e7fb4a79bd498140ef0b0943 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:06:48.685285 10329 raft_consensus.cc:515] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:48.684612 10192 tablet_service.cc:1511] Processing CreateTablet for tablet 375f151edb2245daa6d1b06187f21b79 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:06:48.685868 10268 tablet_service.cc:1511] Processing CreateTablet for tablet 375f151edb2245daa6d1b06187f21b79 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:06:48.687024 10192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 375f151edb2245daa6d1b06187f21b79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.687235 10268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 375f151edb2245daa6d1b06187f21b79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.688143 10329 leader_election.cc:290] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:48.690057 10324 ts_tablet_manager.cc:1434] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.199s	user 0.050s	sys 0.048s
I20260709 13:06:48.687371 10194 tablet_service.cc:1511] Processing CreateTablet for tablet 5f64c791e7fb4a79bd498140ef0b0943 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:06:48.697475 10194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f64c791e7fb4a79bd498140ef0b0943. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.697115 10193 tablet_service.cc:1511] Processing CreateTablet for tablet c7d966d297f346da9e984aa2e181590a (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:06:48.668826 10328 raft_consensus.cc:385] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.681625 10041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7d966d297f346da9e984aa2e181590a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.703161 10193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7d966d297f346da9e984aa2e181590a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.702778 10121 tablet_service.cc:1511] Processing CreateTablet for tablet 75ed1b358949418a9d300990f8f81652 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:06:48.712453 10114 tablet_service.cc:1511] Processing CreateTablet for tablet 375f151edb2245daa6d1b06187f21b79 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:06:48.722061 10045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f64c791e7fb4a79bd498140ef0b0943. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.724648 10121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75ed1b358949418a9d300990f8f81652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.734005 10265 tablet_service.cc:1511] Processing CreateTablet for tablet c7d966d297f346da9e984aa2e181590a (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:06:48.735399 10265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7d966d297f346da9e984aa2e181590a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.701960 10328 raft_consensus.cc:740] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.746984 10328 consensus_queue.cc:260] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:48.758071 10043 tablet_service.cc:1511] Processing CreateTablet for tablet 50256706bf3a4bf29b64ea4ea6583da3 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:06:48.758018 10117 tablet_service.cc:1511] Processing CreateTablet for tablet 5f64c791e7fb4a79bd498140ef0b0943 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:06:48.759820 10043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50256706bf3a4bf29b64ea4ea6583da3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.764359 10191 tablet_service.cc:1511] Processing CreateTablet for tablet b41d6a8939d7411e901d88f31d2bdf38 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:06:48.765781 10191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b41d6a8939d7411e901d88f31d2bdf38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.765951 10047 tablet_service.cc:1511] Processing CreateTablet for tablet b41d6a8939d7411e901d88f31d2bdf38 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:06:48.766121 10268 tablet_service.cc:1511] Processing CreateTablet for tablet 50256706bf3a4bf29b64ea4ea6583da3 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:06:48.767237 10047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b41d6a8939d7411e901d88f31d2bdf38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.767465 10268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50256706bf3a4bf29b64ea4ea6583da3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.755283 10324 tablet_bootstrap.cc:492] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:48.774706 10114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 375f151edb2245daa6d1b06187f21b79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.781993 10333 tablet_bootstrap.cc:492] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:48.792112 10325 tablet_bootstrap.cc:654] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.799425 10190 tablet_service.cc:1511] Processing CreateTablet for tablet d9cb1393c4f549f0bc9b5e82121b543a (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:06:48.800889 10190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9cb1393c4f549f0bc9b5e82121b543a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.817159 10305 heartbeater.cc:499] Master 127.0.107.254:43521 was elected leader, sending a full tablet report...
I20260709 13:06:48.817096 10338 raft_consensus.cc:493] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.819286 10338 raft_consensus.cc:515] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:48.825918 10266 tablet_service.cc:1511] Processing CreateTablet for tablet b41d6a8939d7411e901d88f31d2bdf38 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:06:48.827359 10266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b41d6a8939d7411e901d88f31d2bdf38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.832261 10196 tablet_service.cc:1511] Processing CreateTablet for tablet 7f0446529cd446d0ba97ee860c6ae221 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:06:48.834350 10196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f0446529cd446d0ba97ee860c6ae221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.838261 10157 heartbeater.cc:499] Master 127.0.107.254:43521 was elected leader, sending a full tablet report...
I20260709 13:06:48.843029 10040 tablet_service.cc:1511] Processing CreateTablet for tablet 7f0446529cd446d0ba97ee860c6ae221 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:06:48.844647 10040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f0446529cd446d0ba97ee860c6ae221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.846614 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04676809490049a5923174641a3c1137" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:48.846660 10264 tablet_service.cc:1511] Processing CreateTablet for tablet d9cb1393c4f549f0bc9b5e82121b543a (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:06:48.851761 10264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9cb1393c4f549f0bc9b5e82121b543a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:48.852034 10169 leader_election.cc:343] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.853276 10117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f64c791e7fb4a79bd498140ef0b0943. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.778076 10119 tablet_service.cc:1511] Processing CreateTablet for tablet 50256706bf3a4bf29b64ea4ea6583da3 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:06:48.861447 10324 tablet_bootstrap.cc:654] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.835021 10269 tablet_service.cc:1511] Processing CreateTablet for tablet 5c136ae5f6aa4055a44f8aec8883b02a (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:06:48.865181 10269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c136ae5f6aa4055a44f8aec8883b02a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.865800 10057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04676809490049a5923174641a3c1137" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:48.866653 10057 raft_consensus.cc:2468] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:48.841311 10039 tablet_service.cc:1511] Processing CreateTablet for tablet 5c136ae5f6aa4055a44f8aec8883b02a (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:06:48.868083 10039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c136ae5f6aa4055a44f8aec8883b02a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.868582 10172 leader_election.cc:304] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 50999113203c482b9b9300caa485f154
I20260709 13:06:48.869611 10330 raft_consensus.cc:2804] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:48.870028 10330 raft_consensus.cc:493] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.870352 10330 raft_consensus.cc:3060] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.799444 10115 tablet_service.cc:1511] Processing CreateTablet for tablet d9cb1393c4f549f0bc9b5e82121b543a (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:06:48.886960 10115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9cb1393c4f549f0bc9b5e82121b543a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.902320 10119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50256706bf3a4bf29b64ea4ea6583da3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.907300 10122 tablet_service.cc:1511] Processing CreateTablet for tablet 7f0446529cd446d0ba97ee860c6ae221 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:06:48.909287 10122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f0446529cd446d0ba97ee860c6ae221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.912616 10338 leader_election.cc:290] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:48.929226 10118 tablet_service.cc:1511] Processing CreateTablet for tablet 5c136ae5f6aa4055a44f8aec8883b02a (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:06:48.930760 10118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c136ae5f6aa4055a44f8aec8883b02a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.936376 10333 tablet_bootstrap.cc:654] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.930671 10193 tablet_service.cc:1511] Processing CreateTablet for tablet e66a35e846da4843bda7db1f5fca14b6 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:06:48.945735 10193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e66a35e846da4843bda7db1f5fca14b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.951716 10325 tablet_bootstrap.cc:492] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:48.952418 10325 ts_tablet_manager.cc:1403] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.287s	user 0.018s	sys 0.019s
I20260709 13:06:48.957010 10325 raft_consensus.cc:359] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:48.958015 10325 raft_consensus.cc:385] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.960542 10206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04676809490049a5923174641a3c1137" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:48.951642 10328 ts_tablet_manager.cc:1434] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.288s	user 0.010s	sys 0.031s
I20260709 13:06:48.964246 10324 tablet_bootstrap.cc:492] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:48.964640 10325 raft_consensus.cc:740] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.964818 10324 ts_tablet_manager.cc:1403] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.260s	user 0.012s	sys 0.020s
I20260709 13:06:48.966686 10325 consensus_queue.cc:260] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:48.969318 10324 raft_consensus.cc:359] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:48.971812 10324 raft_consensus.cc:385] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.972172 10324 raft_consensus.cc:740] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.973091 10324 consensus_queue.cc:260] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:48.975301 10324 ts_tablet_manager.cc:1434] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:06:48.976341 10324 tablet_bootstrap.cc:492] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:48.977778 10325 ts_tablet_manager.cc:1434] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.001s
I20260709 13:06:48.971313 10328 tablet_bootstrap.cc:492] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:48.981540 10325 tablet_bootstrap.cc:492] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:48.991652 10041 tablet_service.cc:1511] Processing CreateTablet for tablet e66a35e846da4843bda7db1f5fca14b6 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:06:48.993098 10041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e66a35e846da4843bda7db1f5fca14b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.993716 10113 tablet_service.cc:1511] Processing CreateTablet for tablet 90eeb5566a2f4fa4af640a137fe578df (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:06:48.999027 10113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90eeb5566a2f4fa4af640a137fe578df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.000026 10268 tablet_service.cc:1511] Processing CreateTablet for tablet e66a35e846da4843bda7db1f5fca14b6 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:06:49.001603 10268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e66a35e846da4843bda7db1f5fca14b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.006875 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04676809490049a5923174641a3c1137" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
W20260709 13:06:49.008679 10020 leader_election.cc:343] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.026790 10194 tablet_service.cc:1511] Processing CreateTablet for tablet 90eeb5566a2f4fa4af640a137fe578df (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:06:49.028240 10194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90eeb5566a2f4fa4af640a137fe578df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.032514 10324 tablet_bootstrap.cc:654] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.042483 10265 tablet_service.cc:1511] Processing CreateTablet for tablet 90eeb5566a2f4fa4af640a137fe578df (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:06:49.043934 10265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90eeb5566a2f4fa4af640a137fe578df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.054294 10328 tablet_bootstrap.cc:654] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.062088 10330 raft_consensus.cc:515] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:49.062786 10039 tablet_service.cc:1511] Processing CreateTablet for tablet 19d98d0921054d6a932a308e4adb1937 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:06:49.063730 10206 raft_consensus.cc:2393] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa338cd132c457896b11e08d7ea787d in current term 1: Already voted for candidate 8fe33410d51a469e81be6edc602d7d78 in this term.
I20260709 13:06:49.064245 10039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19d98d0921054d6a932a308e4adb1937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.065775 10057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04676809490049a5923174641a3c1137" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:49.066409 10057 raft_consensus.cc:3060] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.069613 10020 leader_election.cc:304] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa338cd132c457896b11e08d7ea787d; no voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:49.075677 10325 tablet_bootstrap.cc:654] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.076437 10333 tablet_bootstrap.cc:492] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:49.077008 10333 ts_tablet_manager.cc:1403] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.296s	user 0.016s	sys 0.008s
I20260709 13:06:49.078042 10057 raft_consensus.cc:2468] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:49.078590 10329 raft_consensus.cc:2749] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:49.079380 10172 leader_election.cc:304] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:49.080477 10331 raft_consensus.cc:2804] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.084219 10269 tablet_service.cc:1511] Processing CreateTablet for tablet 39e4808ea4be41f483e08e83162c59c8 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:06:49.086252 10115 tablet_service.cc:1511] Processing CreateTablet for tablet 19d98d0921054d6a932a308e4adb1937 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:06:49.088662 10115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19d98d0921054d6a932a308e4adb1937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.087643 10269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39e4808ea4be41f483e08e83162c59c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.087266 10333 raft_consensus.cc:359] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:49.101264 10333 raft_consensus.cc:385] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.101894 10333 raft_consensus.cc:740] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.102623 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04676809490049a5923174641a3c1137" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
W20260709 13:06:49.104212 10169 leader_election.cc:343] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.104895 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fa9341e7fbe49e1a832721bfa4abff3" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:49.101322 10330 leader_election.cc:290] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
W20260709 13:06:49.106973 10246 leader_election.cc:343] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.106824 10191 tablet_service.cc:1511] Processing CreateTablet for tablet 19d98d0921054d6a932a308e4adb1937 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:06:49.100973 10047 tablet_service.cc:1511] Processing CreateTablet for tablet 39e4808ea4be41f483e08e83162c59c8 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:06:49.116880 10193 tablet_service.cc:1511] Processing CreateTablet for tablet ccaebf4041f14bada42fd7122f34419c (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:06:49.119014 10047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39e4808ea4be41f483e08e83162c59c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.121279 10268 tablet_service.cc:1511] Processing CreateTablet for tablet ccaebf4041f14bada42fd7122f34419c (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:06:49.122634 10268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccaebf4041f14bada42fd7122f34419c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.124405 10114 tablet_service.cc:1511] Processing CreateTablet for tablet 3666fc7cbb0d4f698fb337ad4c238c38 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:06:49.125764 10114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3666fc7cbb0d4f698fb337ad4c238c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.126219 10190 tablet_service.cc:1511] Processing CreateTablet for tablet 3666fc7cbb0d4f698fb337ad4c238c38 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:06:49.127686 10045 tablet_service.cc:1511] Processing CreateTablet for tablet ccaebf4041f14bada42fd7122f34419c (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:06:49.130196 10324 tablet_bootstrap.cc:492] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:49.130793 10324 ts_tablet_manager.cc:1403] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.155s	user 0.013s	sys 0.005s
I20260709 13:06:49.130604 10266 tablet_service.cc:1511] Processing CreateTablet for tablet 3666fc7cbb0d4f698fb337ad4c238c38 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:06:49.131078 10119 tablet_service.cc:1511] Processing CreateTablet for tablet 39e4808ea4be41f483e08e83162c59c8 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:06:49.132441 10119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39e4808ea4be41f483e08e83162c59c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.114281 10333 consensus_queue.cc:260] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:49.132333 10266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3666fc7cbb0d4f698fb337ad4c238c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.135933 10040 tablet_service.cc:1511] Processing CreateTablet for tablet 1ec007b8bb1e4c69a49a9ba8aeb56eee (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:06:49.138091 10122 tablet_service.cc:1511] Processing CreateTablet for tablet 1ec007b8bb1e4c69a49a9ba8aeb56eee (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:06:49.139510 10122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ec007b8bb1e4c69a49a9ba8aeb56eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.139665 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fa9341e7fbe49e1a832721bfa4abff3" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:49.140328 10191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19d98d0921054d6a932a308e4adb1937. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:49.141602 10243 leader_election.cc:343] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.145828 10243 leader_election.cc:304] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:49.144151 10045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccaebf4041f14bada42fd7122f34419c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.149564 10341 raft_consensus.cc:2749] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:49.150934 10040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ec007b8bb1e4c69a49a9ba8aeb56eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.152854 10196 tablet_service.cc:1511] Processing CreateTablet for tablet 1ec007b8bb1e4c69a49a9ba8aeb56eee (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:06:49.159337 10324 raft_consensus.cc:359] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:49.160550 10324 raft_consensus.cc:385] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.165427 10264 tablet_service.cc:1511] Processing CreateTablet for tablet 7dd93f389d32493b9dcc10e602f452da (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:06:49.166877 10264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dd93f389d32493b9dcc10e602f452da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.169023 10190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3666fc7cbb0d4f698fb337ad4c238c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.170717 10046 tablet_service.cc:1511] Processing CreateTablet for tablet 7dd93f389d32493b9dcc10e602f452da (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:06:49.178277 10196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ec007b8bb1e4c69a49a9ba8aeb56eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.182241 10193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccaebf4041f14bada42fd7122f34419c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.160992 10324 raft_consensus.cc:740] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.144430 10331 raft_consensus.cc:697] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:06:49.185213 10331 consensus_queue.cc:237] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:49.186959 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dd93f389d32493b9dcc10e602f452da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.198057 10118 tablet_service.cc:1511] Processing CreateTablet for tablet 7dd93f389d32493b9dcc10e602f452da (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:06:49.199486 10118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dd93f389d32493b9dcc10e602f452da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.201591 10325 tablet_bootstrap.cc:492] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:49.202173 10325 ts_tablet_manager.cc:1403] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.221s	user 0.018s	sys 0.002s
I20260709 13:06:49.205430 10194 tablet_service.cc:1511] Processing CreateTablet for tablet bc1d8020dcd34e63b72a7de2db565ff6 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:06:49.205440 10325 raft_consensus.cc:359] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:49.212190 10325 raft_consensus.cc:385] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.212612 10325 raft_consensus.cc:740] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.212285 10194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc1d8020dcd34e63b72a7de2db565ff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.215385 10325 consensus_queue.cc:260] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:49.217764 10325 ts_tablet_manager.cc:1434] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.002s
I20260709 13:06:49.217785 10042 tablet_service.cc:1511] Processing CreateTablet for tablet bc1d8020dcd34e63b72a7de2db565ff6 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:06:49.219537 10042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc1d8020dcd34e63b72a7de2db565ff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.228223 10349 raft_consensus.cc:493] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.228863 10349 raft_consensus.cc:515] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:49.238848 10333 ts_tablet_manager.cc:1434] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.162s	user 0.018s	sys 0.015s
I20260709 13:06:49.239151 10268 tablet_service.cc:1511] Processing CreateTablet for tablet bc1d8020dcd34e63b72a7de2db565ff6 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:06:49.240620 10268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc1d8020dcd34e63b72a7de2db565ff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.241297 10325 tablet_bootstrap.cc:492] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:49.246039 10333 tablet_bootstrap.cc:492] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:49.250588 10195 tablet_service.cc:1511] Processing CreateTablet for tablet 0529bf107d07493eb0984c4cbb9ea01e (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:06:49.251940 10195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0529bf107d07493eb0984c4cbb9ea01e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.205986 10324 consensus_queue.cc:260] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:49.264997 10329 raft_consensus.cc:493] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.265520 10329 raft_consensus.cc:515] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:49.266120 10349 leader_election.cc:290] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:49.266428 10269 tablet_service.cc:1511] Processing CreateTablet for tablet 0529bf107d07493eb0984c4cbb9ea01e (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:06:49.267839 10269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0529bf107d07493eb0984c4cbb9ea01e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.270455 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fa9341e7fbe49e1a832721bfa4abff3" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:49.271152 10280 raft_consensus.cc:2468] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 0.
I20260709 13:06:49.272643 10329 leader_election.cc:290] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:49.274619 10020 leader_election.cc:304] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:49.276185 10324 ts_tablet_manager.cc:1434] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.145s	user 0.006s	sys 0.005s
I20260709 13:06:49.280087 10328 tablet_bootstrap.cc:492] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:49.280758 10328 ts_tablet_manager.cc:1403] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.315s	user 0.013s	sys 0.008s
I20260709 13:06:49.284843 10328 raft_consensus.cc:359] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:49.285553 10328 raft_consensus.cc:385] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.285919 10328 raft_consensus.cc:740] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.287204 10328 consensus_queue.cc:260] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:49.281364 10041 tablet_service.cc:1511] Processing CreateTablet for tablet c21c65d190c541ab8e2d6ae0d4be85e7 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:06:49.290190 10328 ts_tablet_manager.cc:1434] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260709 13:06:49.291311 10328 tablet_bootstrap.cc:492] T 04676809490049a5923174641a3c1137 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:49.292825 10329 raft_consensus.cc:2804] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:49.293294 10329 raft_consensus.cc:493] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.293670 10329 raft_consensus.cc:3060] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.301359 10329 raft_consensus.cc:515] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:49.304198 10329 leader_election.cc:290] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Requested vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:49.305972 10324 tablet_bootstrap.cc:492] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:49.308295 10347 raft_consensus.cc:493] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.309208 10041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c21c65d190c541ab8e2d6ae0d4be85e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.312208 10347 raft_consensus.cc:515] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:49.315348 10347 leader_election.cc:290] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:49.318415 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df6e7bddfd4832b31ccc4bb8b3521b" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:49.323100 10354 raft_consensus.cc:493] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.323117 10279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fa9341e7fbe49e1a832721bfa4abff3" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:49.327879 10325 tablet_bootstrap.cc:654] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.328205 10279 raft_consensus.cc:3060] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.323956 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fa9341e7fbe49e1a832721bfa4abff3" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:49.324601 10190 tablet_service.cc:1511] Processing CreateTablet for tablet c21c65d190c541ab8e2d6ae0d4be85e7 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:06:49.325950 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fa9341e7fbe49e1a832721bfa4abff3" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:49.330778 10190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c21c65d190c541ab8e2d6ae0d4be85e7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:49.327396 10169 leader_election.cc:343] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:49.331240 10020 leader_election.cc:343] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.321442  9954 catalog_manager.cc:5697] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:49.337931 10354 raft_consensus.cc:515] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:49.340440 10354 leader_election.cc:290] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:49.342701 10266 tablet_service.cc:1511] Processing CreateTablet for tablet 7364742faea3470694e1a23f151f1b57 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:06:49.344135 10266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7364742faea3470694e1a23f151f1b57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.347085 10333 tablet_bootstrap.cc:654] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.342698 10045 tablet_service.cc:1511] Processing CreateTablet for tablet 7364742faea3470694e1a23f151f1b57 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:06:49.351573 10045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7364742faea3470694e1a23f151f1b57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.360468 10328 tablet_bootstrap.cc:654] T 04676809490049a5923174641a3c1137 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:49.335644 10020 leader_election.cc:343] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.362748 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa03c270e7364d769700b877d171588e" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
W20260709 13:06:49.364364 10020 leader_election.cc:343] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.365775 10269 tablet_service.cc:1511] Processing CreateTablet for tablet 0981226de91f4f9e96cf2e8b926aaf19 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:06:49.367763 10206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa03c270e7364d769700b877d171588e" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:49.369684 10279 raft_consensus.cc:2468] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:06:49.370625 10195 tablet_service.cc:1511] Processing CreateTablet for tablet 0981226de91f4f9e96cf2e8b926aaf19 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
W20260709 13:06:49.371727 10020 leader_election.cc:343] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.384420 10114 tablet_service.cc:1511] Processing CreateTablet for tablet c21c65d190c541ab8e2d6ae0d4be85e7 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:06:49.385819 10269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0981226de91f4f9e96cf2e8b926aaf19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.388482 10119 tablet_service.cc:1511] Processing CreateTablet for tablet 0529bf107d07493eb0984c4cbb9ea01e (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:06:49.390638 10020 leader_election.cc:304] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa338cd132c457896b11e08d7ea787d; no voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:49.392031 10020 leader_election.cc:304] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 575e44d297f045a389903d65a471d548
I20260709 13:06:49.393121 10354 raft_consensus.cc:2749] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:49.393837 10354 raft_consensus.cc:2804] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.394158 10114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c21c65d190c541ab8e2d6ae0d4be85e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.394250 10354 raft_consensus.cc:697] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:06:49.395203 10354 consensus_queue.cc:237] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:49.398350 10119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0529bf107d07493eb0984c4cbb9ea01e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.400652 10195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0981226de91f4f9e96cf2e8b926aaf19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.411645 10118 tablet_service.cc:1511] Processing CreateTablet for tablet 7364742faea3470694e1a23f151f1b57 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:06:49.412580 10206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e959733d8ef94ea19aad472fcf384371" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:49.413859 10118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7364742faea3470694e1a23f151f1b57. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:49.416056 10095 leader_election.cc:343] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.443905 10328 tablet_bootstrap.cc:492] T 04676809490049a5923174641a3c1137 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:49.443866 10047 tablet_service.cc:1511] Processing CreateTablet for tablet 0981226de91f4f9e96cf2e8b926aaf19 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:06:49.444643 10328 ts_tablet_manager.cc:1403] T 04676809490049a5923174641a3c1137 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.154s	user 0.015s	sys 0.003s
I20260709 13:06:49.445539 10047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0981226de91f4f9e96cf2e8b926aaf19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.447044 10324 tablet_bootstrap.cc:654] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.447989 10328 raft_consensus.cc:359] T 04676809490049a5923174641a3c1137 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:49.448781 10328 raft_consensus.cc:385] T 04676809490049a5923174641a3c1137 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.449101 10328 raft_consensus.cc:740] T 04676809490049a5923174641a3c1137 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.428463 10279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e959733d8ef94ea19aad472fcf384371" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:49.443885 10325 tablet_bootstrap.cc:492] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:49.450289 10325 ts_tablet_manager.cc:1403] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.224s	user 0.012s	sys 0.001s
W20260709 13:06:49.462478 10095 leader_election.cc:343] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.472400 10341 raft_consensus.cc:493] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.490074 10095 leader_election.cc:304] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:49.490217 10341 raft_consensus.cc:515] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:49.491071 10349 raft_consensus.cc:2749] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:49.464445 10328 consensus_queue.cc:260] T 04676809490049a5923174641a3c1137 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:49.492545 10341 leader_election.cc:290] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:49.493453 10122 tablet_service.cc:1511] Processing CreateTablet for tablet dc4e649e35ee49a2a224ecc9f962980a (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:06:49.495052 10122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc4e649e35ee49a2a224ecc9f962980a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.495741 10057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3c90bb2ac14652a45cdcccba25f97c" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:49.496989 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3c90bb2ac14652a45cdcccba25f97c" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
W20260709 13:06:49.498548 10243 leader_election.cc:343] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:49.499742 10246 leader_election.cc:343] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.500212 10246 leader_election.cc:304] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d
W20260709 13:06:49.503590 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:49.504041 10325 raft_consensus.cc:359] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:49.504786 10325 raft_consensus.cc:385] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.505074 10325 raft_consensus.cc:740] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.505867 10325 consensus_queue.cc:260] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:49.507771 10325 ts_tablet_manager.cc:1434] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.000s
I20260709 13:06:49.508193 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df6e7bddfd4832b31ccc4bb8b3521b" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:49.508878 10325 tablet_bootstrap.cc:492] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
W20260709 13:06:49.509673 10169 leader_election.cc:343] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.509788 10341 raft_consensus.cc:2749] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:49.510213 10169 leader_election.cc:304] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548
I20260709 13:06:49.511473 10347 raft_consensus.cc:2749] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:49.519539 10333 tablet_bootstrap.cc:492] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:49.543238 10328 ts_tablet_manager.cc:1434] T 04676809490049a5923174641a3c1137 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.098s	user 0.004s	sys 0.007s
I20260709 13:06:49.534487 10333 ts_tablet_manager.cc:1403] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.295s	user 0.027s	sys 0.034s
I20260709 13:06:49.545684 10190 tablet_service.cc:1511] Processing CreateTablet for tablet dc4e649e35ee49a2a224ecc9f962980a (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:06:49.547142 10328 tablet_bootstrap.cc:492] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:49.547150 10190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc4e649e35ee49a2a224ecc9f962980a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.552752 10325 tablet_bootstrap.cc:654] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.560269 10324 tablet_bootstrap.cc:492] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:49.560946 10324 ts_tablet_manager.cc:1403] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.255s	user 0.041s	sys 0.031s
I20260709 13:06:49.562211 10333 raft_consensus.cc:359] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:49.564114 10324 raft_consensus.cc:359] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:49.565111 10333 raft_consensus.cc:385] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.566705 10333 raft_consensus.cc:740] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.567080 10328 tablet_bootstrap.cc:654] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.567487 10333 consensus_queue.cc:260] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:49.566298 10324 raft_consensus.cc:385] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.569326 10333 ts_tablet_manager.cc:1434] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.002s
I20260709 13:06:49.569661 10324 raft_consensus.cc:740] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.571194 10324 consensus_queue.cc:260] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:49.609973 10347 raft_consensus.cc:493] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.611617 10325 tablet_bootstrap.cc:492] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:49.615192 10325 ts_tablet_manager.cc:1403] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.107s	user 0.017s	sys 0.000s
I20260709 13:06:49.611675 10347 raft_consensus.cc:515] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:49.617476  9955 catalog_manager.cc:5697] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 0 to 1, leader changed from <none> to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:49.630805 10324 ts_tablet_manager.cc:1434] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.069s	user 0.007s	sys 0.004s
I20260709 13:06:49.573694 10333 tablet_bootstrap.cc:492] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:49.635852 10324 tablet_bootstrap.cc:492] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:49.635231 10266 tablet_service.cc:1511] Processing CreateTablet for tablet dc4e649e35ee49a2a224ecc9f962980a (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:06:49.624487 10057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52ffbaf3a54145c2a791c4a67dbdf64a" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:49.624972 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52ffbaf3a54145c2a791c4a67dbdf64a" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:49.628944 10325 raft_consensus.cc:359] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:49.639096 10325 raft_consensus.cc:385] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.639603 10266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc4e649e35ee49a2a224ecc9f962980a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:49.640282 10172 leader_election.cc:343] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:49.641971 10169 leader_election.cc:343] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.642542 10169 leader_election.cc:304] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:49.643693 10331 raft_consensus.cc:2749] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:49.646536 10333 tablet_bootstrap.cc:654] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.623652 10347 leader_election.cc:290] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:49.648685 10347 consensus_queue.cc:1048] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:49.648708 10041 tablet_service.cc:1511] Processing CreateTablet for tablet d139f1fbdd384830bc7ee062ab3d281c (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:06:49.651005 10041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d139f1fbdd384830bc7ee062ab3d281c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.659575 10122 tablet_service.cc:1511] Processing CreateTablet for tablet d139f1fbdd384830bc7ee062ab3d281c (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:06:49.661696 10122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d139f1fbdd384830bc7ee062ab3d281c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.665067 10328 tablet_bootstrap.cc:492] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:49.665944 10328 ts_tablet_manager.cc:1403] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.119s	user 0.013s	sys 0.003s
I20260709 13:06:49.667904 10190 tablet_service.cc:1511] Processing CreateTablet for tablet d139f1fbdd384830bc7ee062ab3d281c (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:06:49.639531 10325 raft_consensus.cc:740] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.670217 10190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d139f1fbdd384830bc7ee062ab3d281c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.673195 10269 tablet_service.cc:1511] Processing CreateTablet for tablet 75aced19d8c94fc78e5510b592bb01be (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:06:49.674755 10269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75aced19d8c94fc78e5510b592bb01be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.678644 10328 raft_consensus.cc:359] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:49.679960 10328 raft_consensus.cc:385] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.680348 10328 raft_consensus.cc:740] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.680071 10325 consensus_queue.cc:260] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:49.686339 10279 raft_consensus.cc:3060] T 04676809490049a5923174641a3c1137 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.681510 10328 consensus_queue.cc:260] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:49.711393 10328 ts_tablet_manager.cc:1434] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.045s	user 0.012s	sys 0.001s
I20260709 13:06:49.713099 10328 tablet_bootstrap.cc:492] T 5461abd09b074193b3d03df659fa2ced P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:49.713918 10325 ts_tablet_manager.cc:1434] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.098s	user 0.010s	sys 0.007s
I20260709 13:06:49.715456 10325 tablet_bootstrap.cc:492] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:49.726552 10047 tablet_service.cc:1511] Processing CreateTablet for tablet 75aced19d8c94fc78e5510b592bb01be (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:06:49.724371 10333 tablet_bootstrap.cc:492] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:49.727906 10333 ts_tablet_manager.cc:1403] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.154s	user 0.028s	sys 0.008s
I20260709 13:06:49.728188 10047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75aced19d8c94fc78e5510b592bb01be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.739074 10119 tablet_service.cc:1511] Processing CreateTablet for tablet 75aced19d8c94fc78e5510b592bb01be (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:06:49.740631 10333 raft_consensus.cc:359] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:49.741811 10333 raft_consensus.cc:385] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.742125 10333 raft_consensus.cc:740] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.743005 10119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75aced19d8c94fc78e5510b592bb01be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.742925 10333 consensus_queue.cc:260] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:49.743330 10331 consensus_queue.cc:1048] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:49.750696 10349 raft_consensus.cc:493] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.751358 10349 raft_consensus.cc:515] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:49.755687 10206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7dfde2e205846058c75fc205443d330" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:49.756114 10325 tablet_bootstrap.cc:654] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.759182 10278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7dfde2e205846058c75fc205443d330" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:49.745528 10333 ts_tablet_manager.cc:1434] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.003s
I20260709 13:06:49.761318 10333 tablet_bootstrap.cc:492] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548: Bootstrap starting.
W20260709 13:06:49.761415 10095 leader_election.cc:343] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:49.762316 10095 leader_election.cc:343] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.762705 10095 leader_election.cc:304] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:49.763954 10349 leader_election.cc:290] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:49.764545 10349 raft_consensus.cc:2749] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:49.765853 10324 tablet_bootstrap.cc:654] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.768770 10333 tablet_bootstrap.cc:654] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.774057 10325 tablet_bootstrap.cc:492] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:49.774631 10325 ts_tablet_manager.cc:1403] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.060s	user 0.007s	sys 0.007s
I20260709 13:06:49.778285 10325 raft_consensus.cc:359] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:49.780563 10325 raft_consensus.cc:385] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.788360 10325 raft_consensus.cc:740] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.807855 10328 tablet_bootstrap.cc:654] T 5461abd09b074193b3d03df659fa2ced P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.809890 10325 consensus_queue.cc:260] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:49.812355 10325 ts_tablet_manager.cc:1434] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.037s	user 0.001s	sys 0.006s
I20260709 13:06:49.813582 10325 tablet_bootstrap.cc:492] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:49.819954 10333 tablet_bootstrap.cc:492] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:49.820506 10333 ts_tablet_manager.cc:1403] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.059s	user 0.011s	sys 0.003s
I20260709 13:06:49.822463 10325 tablet_bootstrap.cc:654] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.824319 10333 raft_consensus.cc:359] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:49.825140 10333 raft_consensus.cc:385] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.825446 10333 raft_consensus.cc:740] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.826233 10047 tablet_service.cc:1511] Processing CreateTablet for tablet d242deba350e42cb8e2b623027a26516 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:06:49.827603 10047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d242deba350e42cb8e2b623027a26516. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.828228 10269 tablet_service.cc:1511] Processing CreateTablet for tablet d242deba350e42cb8e2b623027a26516 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:06:49.828773 10324 tablet_bootstrap.cc:492] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:49.829306 10324 ts_tablet_manager.cc:1403] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.194s	user 0.002s	sys 0.019s
I20260709 13:06:49.829993 10333 consensus_queue.cc:260] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:49.830991 10195 tablet_service.cc:1511] Processing CreateTablet for tablet cded7cb005f64f9c894f568140883a77 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:06:49.832362 10195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cded7cb005f64f9c894f568140883a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.832793 10333 ts_tablet_manager.cc:1434] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.002s
I20260709 13:06:49.832429 10324 raft_consensus.cc:359] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:49.833335 10324 raft_consensus.cc:385] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.827039 10190 tablet_service.cc:1511] Processing CreateTablet for tablet d242deba350e42cb8e2b623027a26516 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:06:49.833758 10324 raft_consensus.cc:740] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.834803 10190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d242deba350e42cb8e2b623027a26516. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.834597 10324 consensus_queue.cc:260] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:49.835462 10266 tablet_service.cc:1511] Processing CreateTablet for tablet cded7cb005f64f9c894f568140883a77 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:06:49.835896 10122 tablet_service.cc:1511] Processing CreateTablet for tablet 0e003c3e14c2444b937f547eef84b672 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:06:49.837055 10324 ts_tablet_manager.cc:1434] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:06:49.837371 10122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e003c3e14c2444b937f547eef84b672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.838322 10324 tablet_bootstrap.cc:492] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:49.842610 10041 tablet_service.cc:1511] Processing CreateTablet for tablet 0e003c3e14c2444b937f547eef84b672 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:06:49.829487 10269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d242deba350e42cb8e2b623027a26516. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.844540 10041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e003c3e14c2444b937f547eef84b672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.848033 10266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cded7cb005f64f9c894f568140883a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.848937 10324 tablet_bootstrap.cc:654] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.859794 10193 tablet_service.cc:1511] Processing CreateTablet for tablet 0e003c3e14c2444b937f547eef84b672 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:06:49.861606 10193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e003c3e14c2444b937f547eef84b672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.863741 10333 tablet_bootstrap.cc:492] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548: Bootstrap starting.
W20260709 13:06:49.873726 10020 consensus_peers.cc:597] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:49.875384 10324 tablet_bootstrap.cc:492] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:49.880384 10119 tablet_service.cc:1511] Processing CreateTablet for tablet cded7cb005f64f9c894f568140883a77 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:06:49.881922 10119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cded7cb005f64f9c894f568140883a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.884027 10328 tablet_bootstrap.cc:492] T 5461abd09b074193b3d03df659fa2ced P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:49.884624 10328 ts_tablet_manager.cc:1403] T 5461abd09b074193b3d03df659fa2ced P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.172s	user 0.015s	sys 0.010s
I20260709 13:06:49.887934 10328 raft_consensus.cc:359] T 5461abd09b074193b3d03df659fa2ced P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:49.889281 10324 ts_tablet_manager.cc:1403] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.051s	user 0.012s	sys 0.002s
I20260709 13:06:49.892561 10333 tablet_bootstrap.cc:654] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.901889 10264 tablet_service.cc:1511] Processing CreateTablet for tablet c78325f9138849749ada9825bfe45cd8 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:06:49.903796 10264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c78325f9138849749ada9825bfe45cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.888854 10328 raft_consensus.cc:385] T 5461abd09b074193b3d03df659fa2ced P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.894109 10045 tablet_service.cc:1511] Processing CreateTablet for tablet d8d642d68d234d668cb67cb81b80556e (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:06:49.904886 10328 raft_consensus.cc:740] T 5461abd09b074193b3d03df659fa2ced P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.896893 10269 tablet_service.cc:1511] Processing CreateTablet for tablet d8d642d68d234d668cb67cb81b80556e (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:06:49.905946 10045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8d642d68d234d668cb67cb81b80556e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.906325 10328 consensus_queue.cc:260] T 5461abd09b074193b3d03df659fa2ced P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:49.900557 10196 tablet_service.cc:1511] Processing CreateTablet for tablet c78325f9138849749ada9825bfe45cd8 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:06:49.907302 10269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8d642d68d234d668cb67cb81b80556e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.896207 10114 tablet_service.cc:1511] Processing CreateTablet for tablet d8d642d68d234d668cb67cb81b80556e (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:06:49.909049 10354 raft_consensus.cc:493] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.909475 10196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c78325f9138849749ada9825bfe45cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.910431 10328 ts_tablet_manager.cc:1434] T 5461abd09b074193b3d03df659fa2ced P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.002s
I20260709 13:06:49.914660 10354 raft_consensus.cc:515] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:49.915799 10114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8d642d68d234d668cb67cb81b80556e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.917156 10354 leader_election.cc:290] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:49.920042 10206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52ffbaf3a54145c2a791c4a67dbdf64a" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:49.921113 10206 raft_consensus.cc:2468] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 0.
I20260709 13:06:49.933693 10046 tablet_service.cc:1511] Processing CreateTablet for tablet c78325f9138849749ada9825bfe45cd8 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:06:49.935952 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c78325f9138849749ada9825bfe45cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.919683 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52ffbaf3a54145c2a791c4a67dbdf64a" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:49.939139 10020 leader_election.cc:304] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:49.940593 10354 raft_consensus.cc:2804] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:49.941059 10354 raft_consensus.cc:493] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:49.940989 10020 leader_election.cc:343] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.942338 10354 raft_consensus.cc:3060] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.943554 10328 tablet_bootstrap.cc:492] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:49.947187 10377 raft_consensus.cc:493] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.947834 10377 raft_consensus.cc:515] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:49.951378 10328 tablet_bootstrap.cc:654] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.953702 10377 leader_election.cc:290] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:49.953254 10118 tablet_service.cc:1511] Processing CreateTablet for tablet 9f7da1cc13f64f56b02defc2e119c507 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:06:49.955173 10118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f7da1cc13f64f56b02defc2e119c507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.964648 10324 raft_consensus.cc:359] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:49.965540 10324 raft_consensus.cc:385] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.967543 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80b0c1b6201b46d5b1f46ccad59503b7" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:49.976421 10280 raft_consensus.cc:2468] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 0.
I20260709 13:06:49.973109 10324 raft_consensus.cc:740] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.988024 10333 tablet_bootstrap.cc:492] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:49.988679 10333 ts_tablet_manager.cc:1403] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.125s	user 0.020s	sys 0.023s
I20260709 13:06:49.989400 10095 leader_election.cc:304] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:06:49.990293 10349 raft_consensus.cc:2804] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:49.990756 10349 raft_consensus.cc:493] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.991178 10349 raft_consensus.cc:3060] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.992713 10325 tablet_bootstrap.cc:492] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:49.993273 10325 ts_tablet_manager.cc:1403] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.180s	user 0.005s	sys 0.026s
I20260709 13:06:49.994372 10045 tablet_service.cc:1511] Processing CreateTablet for tablet 1f060317dcbd4f0db70f83fd18320215 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:06:49.996951 10045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f060317dcbd4f0db70f83fd18320215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.999747 10325 raft_consensus.cc:359] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:50.000643 10325 raft_consensus.cc:385] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.001286 10325 raft_consensus.cc:740] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.997386 10333 raft_consensus.cc:359] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:50.002219 10122 tablet_service.cc:1511] Processing CreateTablet for tablet 1f060317dcbd4f0db70f83fd18320215 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:06:50.002333 10268 tablet_service.cc:1511] Processing CreateTablet for tablet 9f7da1cc13f64f56b02defc2e119c507 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:06:50.002945 10333 raft_consensus.cc:385] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.003388 10333 raft_consensus.cc:740] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.003901 10268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f7da1cc13f64f56b02defc2e119c507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.004364 10333 consensus_queue.cc:260] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:50.006335 10379 raft_consensus.cc:493] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.006167 10325 consensus_queue.cc:260] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:50.006954 10379 raft_consensus.cc:515] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:50.000356 10191 tablet_service.cc:1511] Processing CreateTablet for tablet 9f7da1cc13f64f56b02defc2e119c507 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:06:49.974238 10377 raft_consensus.cc:493] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.003909 10122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f060317dcbd4f0db70f83fd18320215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.016212 10324 consensus_queue.cc:260] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:50.029417 10191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f7da1cc13f64f56b02defc2e119c507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.033751 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a64aca789f4ea8a4925aa79dcb0235" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:50.034909 10349 raft_consensus.cc:515] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
W20260709 13:06:50.035351 10169 leader_election.cc:343] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.013901 10377 raft_consensus.cc:515] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:50.049546 10349 leader_election.cc:290] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Requested vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:50.050220 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80b0c1b6201b46d5b1f46ccad59503b7" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:50.050865 10280 raft_consensus.cc:3060] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.052175 10325 ts_tablet_manager.cc:1434] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.059s	user 0.012s	sys 0.002s
I20260709 13:06:50.054785 10377 leader_election.cc:290] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:50.056030 10206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa03c270e7364d769700b877d171588e" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
W20260709 13:06:50.057446 10095 leader_election.cc:343] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.062649 10190 tablet_service.cc:1511] Processing CreateTablet for tablet 1f060317dcbd4f0db70f83fd18320215 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:06:50.064080 10190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f060317dcbd4f0db70f83fd18320215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.053465 10325 tablet_bootstrap.cc:492] T aa03c270e7364d769700b877d171588e P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:50.065057 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80b0c1b6201b46d5b1f46ccad59503b7" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:50.066906 10280 raft_consensus.cc:2468] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
I20260709 13:06:50.067215 10056 raft_consensus.cc:2468] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 0.
I20260709 13:06:50.068171 10095 leader_election.cc:304] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:06:50.068459 10057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80b0c1b6201b46d5b1f46ccad59503b7" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:50.069764 10057 raft_consensus.cc:3060] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.070043 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa03c270e7364d769700b877d171588e" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:50.070742 10056 raft_consensus.cc:2468] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 0.
I20260709 13:06:50.071303 10333 ts_tablet_manager.cc:1434] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.082s	user 0.012s	sys 0.003s
I20260709 13:06:50.072592 10098 leader_election.cc:304] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d; no voters: 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:50.069113 10377 raft_consensus.cc:493] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.080778 10377 raft_consensus.cc:515] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:50.082231 10328 tablet_bootstrap.cc:492] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:50.082753 10328 ts_tablet_manager.cc:1403] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.140s	user 0.018s	sys 0.027s
I20260709 13:06:50.084194 10206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52ffbaf3a54145c2a791c4a67dbdf64a" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:50.084446 10347 raft_consensus.cc:493] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.084856 10206 raft_consensus.cc:2468] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 0.
I20260709 13:06:50.084887 10347 raft_consensus.cc:515] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:50.086282 10095 leader_election.cc:304] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:50.086938 10377 leader_election.cc:290] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:50.087165 10347 leader_election.cc:290] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:50.088290 10057 raft_consensus.cc:2468] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
I20260709 13:06:50.089162 10354 raft_consensus.cc:515] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:50.090117 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461abd09b074193b3d03df659fa2ced" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:50.093430 10056 raft_consensus.cc:2468] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:50.095534 10206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52ffbaf3a54145c2a791c4a67dbdf64a" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:50.095664 10172 leader_election.cc:304] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:50.096215 10206 raft_consensus.cc:3060] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.096736 10347 raft_consensus.cc:2804] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.097234 10347 raft_consensus.cc:493] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.097663 10347 raft_consensus.cc:3060] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.098803 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52ffbaf3a54145c2a791c4a67dbdf64a" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:50.099749 10055 raft_consensus.cc:2393] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 575e44d297f045a389903d65a471d548 in current term 1: Already voted for candidate efa338cd132c457896b11e08d7ea787d in this term.
I20260709 13:06:50.100066 10389 raft_consensus.cc:2804] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.100512 10389 raft_consensus.cc:493] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.100878 10389 raft_consensus.cc:3060] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.078613 10349 raft_consensus.cc:2804] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.092531 10328 raft_consensus.cc:359] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:50.094137 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52ffbaf3a54145c2a791c4a67dbdf64a" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:50.111033 10131 raft_consensus.cc:3060] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.112329 10206 raft_consensus.cc:2468] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:06:50.113739 10020 leader_election.cc:304] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:50.113847 10328 raft_consensus.cc:385] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.117745 10331 raft_consensus.cc:493] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.133810 10328 raft_consensus.cc:740] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.134042 10331 raft_consensus.cc:515] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:50.091212 10354 leader_election.cc:290] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:50.123052 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461abd09b074193b3d03df659fa2ced" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:50.136083 10268 tablet_service.cc:1511] Processing CreateTablet for tablet 2258980c3a7540dea8ae11850194e183 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:06:50.123741 10333 tablet_bootstrap.cc:492] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:50.150513 10379 leader_election.cc:290] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:50.125164 10325 tablet_bootstrap.cc:654] T aa03c270e7364d769700b877d171588e P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.127053 10131 raft_consensus.cc:2468] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:06:50.162392 10377 raft_consensus.cc:2764] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:06:50.168555 10268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2258980c3a7540dea8ae11850194e183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.170637 10280 raft_consensus.cc:2468] T 5461abd09b074193b3d03df659fa2ced P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:50.175168 10331 leader_election.cc:290] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:50.176275 10279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e959733d8ef94ea19aad472fcf384371" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:50.179165 10388 raft_consensus.cc:2804] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.174443 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a64aca789f4ea8a4925aa79dcb0235" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:50.179440 10328 consensus_queue.cc:260] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:50.169392 10349 raft_consensus.cc:697] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Becoming Leader. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
W20260709 13:06:50.170185 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:40419, user_credentials={real_user=slave}} blocked reactor thread for 83982.9us
I20260709 13:06:50.193763 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e959733d8ef94ea19aad472fcf384371" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:50.194494 10131 raft_consensus.cc:2468] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
W20260709 13:06:50.183001 10172 leader_election.cc:343] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.173849 10389 raft_consensus.cc:515] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
W20260709 13:06:50.189107 10169 leader_election.cc:343] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.198204 10169 leader_election.cc:304] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 50999113203c482b9b9300caa485f154
I20260709 13:06:50.191247 10347 raft_consensus.cc:515] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:50.200608 10331 raft_consensus.cc:2804] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.202934 10331 raft_consensus.cc:493] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.203151 10389 leader_election.cc:290] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:50.203270 10331 raft_consensus.cc:3060] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.200716 10172 leader_election.cc:304] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:50.204865 10379 raft_consensus.cc:2749] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.206367 10206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa03c270e7364d769700b877d171588e" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:50.207212 10347 leader_election.cc:290] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Requested vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:50.208775 10046 tablet_service.cc:1511] Processing CreateTablet for tablet 2258980c3a7540dea8ae11850194e183 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:06:50.208930 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461abd09b074193b3d03df659fa2ced" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:50.209764 10280 raft_consensus.cc:3060] T 5461abd09b074193b3d03df659fa2ced P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.210750 10333 tablet_bootstrap.cc:654] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:50.180562 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:40419, user_credentials={real_user=slave}} blocked reactor thread for 66037us
I20260709 13:06:50.213930 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2258980c3a7540dea8ae11850194e183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.217976 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa03c270e7364d769700b877d171588e" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:50.218592 10056 raft_consensus.cc:3060] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.202390 10388 raft_consensus.cc:697] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:06:50.228574 10122 tablet_service.cc:1511] Processing CreateTablet for tablet 2258980c3a7540dea8ae11850194e183 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:06:50.231271 10122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2258980c3a7540dea8ae11850194e183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.231329 10388 consensus_queue.cc:237] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:50.233165 10393 consensus_queue.cc:1048] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:50.236608 10331 raft_consensus.cc:515] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:50.238868 10268 tablet_service.cc:1511] Processing CreateTablet for tablet f660e208f31341f0bc602253c996258c (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:06:50.242893 10268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f660e208f31341f0bc602253c996258c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.243467 10056 raft_consensus.cc:2468] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
I20260709 13:06:50.251659 10331 leader_election.cc:290] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:50.252172 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e959733d8ef94ea19aad472fcf384371" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:50.252708 10279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e959733d8ef94ea19aad472fcf384371" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:50.255604 10324 ts_tablet_manager.cc:1434] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.348s	user 0.011s	sys 0.008s
I20260709 13:06:50.256369 10349 consensus_queue.cc:237] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:50.257337 10098 leader_election.cc:304] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d; no voters: 
W20260709 13:06:50.266065 10095 leader_election.cc:343] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.262758 10131 raft_consensus.cc:3060] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.268900 10280 raft_consensus.cc:2468] T 5461abd09b074193b3d03df659fa2ced P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:50.270401 10389 raft_consensus.cc:2804] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.270939 10389 raft_consensus.cc:697] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Becoming Leader. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:06:50.270390 10045 tablet_service.cc:1511] Processing CreateTablet for tablet f660e208f31341f0bc602253c996258c (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:06:50.271768 10389 consensus_queue.cc:237] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:50.273511 10045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f660e208f31341f0bc602253c996258c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.273998 10196 tablet_service.cc:1511] Processing CreateTablet for tablet f660e208f31341f0bc602253c996258c (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:06:50.276188 10196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f660e208f31341f0bc602253c996258c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:50.286139 10169 leader_election.cc:343] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.287328 10324 tablet_bootstrap.cc:492] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:50.286984 10328 ts_tablet_manager.cc:1434] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.204s	user 0.008s	sys 0.006s
I20260709 13:06:50.290850 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461abd09b074193b3d03df659fa2ced" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:50.305474  9951 catalog_manager.cc:5697] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 0 to 1, leader changed from <none> to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.312305 10056 raft_consensus.cc:3060] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.296053 10169 leader_election.cc:304] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:50.297465 10325 tablet_bootstrap.cc:492] T aa03c270e7364d769700b877d171588e P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:50.314813 10325 ts_tablet_manager.cc:1403] T aa03c270e7364d769700b877d171588e P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.262s	user 0.032s	sys 0.032s
I20260709 13:06:50.302608 10122 tablet_service.cc:1511] Processing CreateTablet for tablet a0265654cefc40a5937d7e194f03dd5c (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:06:50.303324 10333 tablet_bootstrap.cc:492] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:50.318635 10333 ts_tablet_manager.cc:1403] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.201s	user 0.025s	sys 0.032s
I20260709 13:06:50.307297 10328 tablet_bootstrap.cc:492] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:50.308800 10131 raft_consensus.cc:2468] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:50.317823 10331 raft_consensus.cc:2804] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.320199 10331 raft_consensus.cc:697] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:06:50.321228 10331 consensus_queue.cc:237] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:50.323136 10333 raft_consensus.cc:359] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:50.323621 10190 tablet_service.cc:1511] Processing CreateTablet for tablet a0265654cefc40a5937d7e194f03dd5c (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:06:50.324884 10190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0265654cefc40a5937d7e194f03dd5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.325253 10056 raft_consensus.cc:2468] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:50.323932 10333 raft_consensus.cc:385] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.318351 10325 raft_consensus.cc:359] T aa03c270e7364d769700b877d171588e P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:50.328123 10122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0265654cefc40a5937d7e194f03dd5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.329133 10325 raft_consensus.cc:385] T aa03c270e7364d769700b877d171588e P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.337018  9952 catalog_manager.cc:5697] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 reported cstate change: term changed from 0 to 1, leader changed from <none> to 575e44d297f045a389903d65a471d548 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "575e44d297f045a389903d65a471d548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.339123 10324 tablet_bootstrap.cc:654] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.329957 10325 raft_consensus.cc:740] T aa03c270e7364d769700b877d171588e P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.344655 10325 consensus_queue.cc:260] T aa03c270e7364d769700b877d171588e P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:50.348287 10169 leader_election.cc:304] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 50999113203c482b9b9300caa485f154
I20260709 13:06:50.348464 10268 tablet_service.cc:1511] Processing CreateTablet for tablet a0265654cefc40a5937d7e194f03dd5c (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:06:50.349184 10347 raft_consensus.cc:2804] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.349759 10268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0265654cefc40a5937d7e194f03dd5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.350677 10119 tablet_service.cc:1511] Processing CreateTablet for tablet d6c60f07656843aeac5d469a6533aaaa (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:06:50.351682 10325 ts_tablet_manager.cc:1434] T aa03c270e7364d769700b877d171588e P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.037s	user 0.002s	sys 0.004s
I20260709 13:06:50.352073 10119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6c60f07656843aeac5d469a6533aaaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.352749 10325 tablet_bootstrap.cc:492] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:50.349668 10347 raft_consensus.cc:697] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:06:50.354660 10347 consensus_queue.cc:237] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:50.355513 10046 tablet_service.cc:1511] Processing CreateTablet for tablet d6c60f07656843aeac5d469a6533aaaa (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:06:50.359619 10047 tablet_service.cc:1511] Processing CreateTablet for tablet 1f4dd5b2c17747058bab6588c63b9d72 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:06:50.326776 10333 raft_consensus.cc:740] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.358031 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6c60f07656843aeac5d469a6533aaaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.382710 10047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f4dd5b2c17747058bab6588c63b9d72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.387647 10193 tablet_service.cc:1511] Processing CreateTablet for tablet d6c60f07656843aeac5d469a6533aaaa (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:06:50.389262 10193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6c60f07656843aeac5d469a6533aaaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.389417 10269 tablet_service.cc:1511] Processing CreateTablet for tablet 1f4dd5b2c17747058bab6588c63b9d72 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:06:50.390697 10269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f4dd5b2c17747058bab6588c63b9d72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.395426 10191 tablet_service.cc:1511] Processing CreateTablet for tablet 48dcd8683f004a61808243f33d859911 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:06:50.395527 10333 consensus_queue.cc:260] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:50.397236 10191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48dcd8683f004a61808243f33d859911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.398650 10333 ts_tablet_manager.cc:1434] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.080s	user 0.007s	sys 0.001s
I20260709 13:06:50.399958 10333 tablet_bootstrap.cc:492] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:50.399798  9954 catalog_manager.cc:5697] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.413443 10122 tablet_service.cc:1511] Processing CreateTablet for tablet 1f4dd5b2c17747058bab6588c63b9d72 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:06:50.414915 10122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f4dd5b2c17747058bab6588c63b9d72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.423905 10264 tablet_service.cc:1511] Processing CreateTablet for tablet 48dcd8683f004a61808243f33d859911 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:06:50.425278 10264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48dcd8683f004a61808243f33d859911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.438215 10324 tablet_bootstrap.cc:492] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:50.438850 10324 ts_tablet_manager.cc:1403] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.153s	user 0.021s	sys 0.031s
I20260709 13:06:50.442559 10324 raft_consensus.cc:359] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:50.443396 10324 raft_consensus.cc:385] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.443768 10324 raft_consensus.cc:740] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.444828 10324 consensus_queue.cc:260] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:50.446029 10328 tablet_bootstrap.cc:654] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.447789 10324 ts_tablet_manager.cc:1434] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:06:50.449172 10324 tablet_bootstrap.cc:492] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:50.452808 10045 tablet_service.cc:1511] Processing CreateTablet for tablet 48dcd8683f004a61808243f33d859911 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:06:50.454522 10045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48dcd8683f004a61808243f33d859911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.461349 10333 tablet_bootstrap.cc:654] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.476672 10325 tablet_bootstrap.cc:654] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.477561 10268 tablet_service.cc:1511] Processing CreateTablet for tablet 925f342702b542288fd34463d0585a5b (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:06:50.478817 10268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 925f342702b542288fd34463d0585a5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.483254 10041 tablet_service.cc:1511] Processing CreateTablet for tablet abca9c95a1334ebc97985eedf46fc01f (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:06:50.485081 10041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abca9c95a1334ebc97985eedf46fc01f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.472694 10118 tablet_service.cc:1511] Processing CreateTablet for tablet 925f342702b542288fd34463d0585a5b (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:06:50.487016 10118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 925f342702b542288fd34463d0585a5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.488989 10347 raft_consensus.cc:493] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.489535 10347 raft_consensus.cc:515] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:50.493736 10347 leader_election.cc:290] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:50.495421 10278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b626433d4aec4288976ffc448a79bb68" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:50.495878 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b626433d4aec4288976ffc448a79bb68" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
W20260709 13:06:50.497013 10169 leader_election.cc:343] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:50.497965 10172 leader_election.cc:343] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.498503 10172 leader_election.cc:304] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:50.499967 10347 raft_consensus.cc:2749] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.503172 10046 tablet_service.cc:1511] Processing CreateTablet for tablet 64ea79b0bb13439c9384cba78627c40f (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:06:50.509461 10333 tablet_bootstrap.cc:492] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:50.510109 10333 ts_tablet_manager.cc:1403] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.110s	user 0.018s	sys 0.000s
I20260709 13:06:50.513834 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64ea79b0bb13439c9384cba78627c40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.513192 10042 tablet_service.cc:1511] Processing CreateTablet for tablet 4457dec3dec94c1dbac542a298184d0f (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:06:50.519670 10042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4457dec3dec94c1dbac542a298184d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.520527 10333 raft_consensus.cc:359] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:50.521862 10333 raft_consensus.cc:385] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.522333 10333 raft_consensus.cc:740] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.523332 10347 raft_consensus.cc:493] T aa03c270e7364d769700b877d171588e P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.524017 10347 raft_consensus.cc:515] T aa03c270e7364d769700b877d171588e P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:50.523587 10333 consensus_queue.cc:260] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:50.528836 10269 tablet_service.cc:1511] Processing CreateTablet for tablet 64ea79b0bb13439c9384cba78627c40f (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:06:50.529438 10191 tablet_service.cc:1511] Processing CreateTablet for tablet abca9c95a1334ebc97985eedf46fc01f (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:06:50.529992 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa03c270e7364d769700b877d171588e" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:50.530944 10056 raft_consensus.cc:2393] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fe33410d51a469e81be6edc602d7d78 in current term 1: Already voted for candidate 575e44d297f045a389903d65a471d548 in this term.
I20260709 13:06:50.531193 10191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abca9c95a1334ebc97985eedf46fc01f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.530210 10269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64ea79b0bb13439c9384cba78627c40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.532542 10122 tablet_service.cc:1511] Processing CreateTablet for tablet abca9c95a1334ebc97985eedf46fc01f (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:06:50.475301 10193 tablet_service.cc:1511] Processing CreateTablet for tablet 925f342702b542288fd34463d0585a5b (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:06:50.533697 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa03c270e7364d769700b877d171588e" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:50.534498 10193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 925f342702b542288fd34463d0585a5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.535881 10169 leader_election.cc:304] T aa03c270e7364d769700b877d171588e P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:50.536029 10119 tablet_service.cc:1511] Processing CreateTablet for tablet 64ea79b0bb13439c9384cba78627c40f (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:06:50.537292 10331 raft_consensus.cc:3060] T aa03c270e7364d769700b877d171588e P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.537758 10122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abca9c95a1334ebc97985eedf46fc01f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.539811 10119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64ea79b0bb13439c9384cba78627c40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.541771 10347 leader_election.cc:290] T aa03c270e7364d769700b877d171588e P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:50.547855 10266 tablet_service.cc:1511] Processing CreateTablet for tablet 4457dec3dec94c1dbac542a298184d0f (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:06:50.548713 10114 tablet_service.cc:1511] Processing CreateTablet for tablet 1249fe82a5ce4bd7b416e6bb46be8f79 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:06:50.549331 10266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4457dec3dec94c1dbac542a298184d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.550056 10114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1249fe82a5ce4bd7b416e6bb46be8f79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.550200 10196 tablet_service.cc:1511] Processing CreateTablet for tablet 4457dec3dec94c1dbac542a298184d0f (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:06:50.551774 10196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4457dec3dec94c1dbac542a298184d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.556097 10324 tablet_bootstrap.cc:654] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.573930 10349 raft_consensus.cc:493] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.599493 10331 raft_consensus.cc:2749] T aa03c270e7364d769700b877d171588e P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.602362 10362 raft_consensus.cc:493] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.602973 10362 raft_consensus.cc:515] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:50.606690 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df6e7bddfd4832b31ccc4bb8b3521b" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:50.607944 10333 ts_tablet_manager.cc:1434] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.097s	user 0.010s	sys 0.000s
W20260709 13:06:50.608865 10243 leader_election.cc:343] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.609298 10333 tablet_bootstrap.cc:492] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:50.574698 10349 raft_consensus.cc:515] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:50.619326 10328 tablet_bootstrap.cc:492] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:50.619947 10328 ts_tablet_manager.cc:1403] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.313s	user 0.011s	sys 0.003s
I20260709 13:06:50.626694 10190 tablet_service.cc:1511] Processing CreateTablet for tablet 1249fe82a5ce4bd7b416e6bb46be8f79 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:06:50.627341 10349 leader_election.cc:290] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:50.628010 10190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1249fe82a5ce4bd7b416e6bb46be8f79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.628494 10278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3c90bb2ac14652a45cdcccba25f97c" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:50.629218 10278 raft_consensus.cc:2468] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 0.
I20260709 13:06:50.630407 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3c90bb2ac14652a45cdcccba25f97c" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:50.631057 10056 raft_consensus.cc:2468] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 0.
I20260709 13:06:50.633005 10098 leader_election.cc:304] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:50.648198 10268 tablet_service.cc:1511] Processing CreateTablet for tablet 1249fe82a5ce4bd7b416e6bb46be8f79 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:06:50.648953 10045 tablet_service.cc:1511] Processing CreateTablet for tablet ce414a4edf504d78b1b1e83498a5cfeb (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:06:50.649797 10268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1249fe82a5ce4bd7b416e6bb46be8f79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.650112 10045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce414a4edf504d78b1b1e83498a5cfeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.637151 10349 raft_consensus.cc:2804] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.657855 10349 raft_consensus.cc:493] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.658310 10349 raft_consensus.cc:3060] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.638252 10362 leader_election.cc:290] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:50.665444 10328 raft_consensus.cc:359] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:50.666275 10328 raft_consensus.cc:385] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.666678 10328 raft_consensus.cc:740] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.667637 10328 consensus_queue.cc:260] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:50.670205 10324 tablet_bootstrap.cc:492] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:50.670801 10324 ts_tablet_manager.cc:1403] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.222s	user 0.016s	sys 0.039s
I20260709 13:06:50.679780 10349 raft_consensus.cc:515] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:50.679908  9952 catalog_manager.cc:5697] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 reported cstate change: term changed from 0 to 1, leader changed from <none> to 575e44d297f045a389903d65a471d548 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "575e44d297f045a389903d65a471d548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.687449 10114 tablet_service.cc:1511] Processing CreateTablet for tablet ce414a4edf504d78b1b1e83498a5cfeb (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:06:50.689010 10114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce414a4edf504d78b1b1e83498a5cfeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.690301 10324 raft_consensus.cc:359] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:50.691001 10324 raft_consensus.cc:385] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.691231 10324 raft_consensus.cc:740] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.692410 10325 tablet_bootstrap.cc:492] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:50.692904 10325 ts_tablet_manager.cc:1403] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.340s	user 0.014s	sys 0.021s
I20260709 13:06:50.696000 10196 tablet_service.cc:1511] Processing CreateTablet for tablet ce414a4edf504d78b1b1e83498a5cfeb (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:06:50.697309 10196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce414a4edf504d78b1b1e83498a5cfeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.707231 10389 consensus_queue.cc:1048] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:50.706411 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:50.718377 10324 consensus_queue.cc:260] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:50.720960 10328 ts_tablet_manager.cc:1434] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.101s	user 0.010s	sys 0.001s
I20260709 13:06:50.722213 10328 tablet_bootstrap.cc:492] T c7dfde2e205846058c75fc205443d330 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:50.728709 10325 raft_consensus.cc:359] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:50.729552 10325 raft_consensus.cc:385] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.729620 10324 ts_tablet_manager.cc:1434] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.002s
I20260709 13:06:50.729921 10325 raft_consensus.cc:740] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.730494 10328 tablet_bootstrap.cc:654] T c7dfde2e205846058c75fc205443d330 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.731200 10324 tablet_bootstrap.cc:492] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:50.730794 10325 consensus_queue.cc:260] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:50.733625  9955 catalog_manager.cc:5697] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.737464 10388 raft_consensus.cc:493] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.738039 10388 raft_consensus.cc:515] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:50.740154 10388 leader_election.cc:290] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:50.741997 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a64aca789f4ea8a4925aa79dcb0235" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:50.742022 10206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a64aca789f4ea8a4925aa79dcb0235" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:50.743767 10206 raft_consensus.cc:2468] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 0.
I20260709 13:06:50.745518 10349 leader_election.cc:290] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Requested vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:50.746104 10278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3c90bb2ac14652a45cdcccba25f97c" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:50.746722 10278 raft_consensus.cc:3060] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.747087 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3c90bb2ac14652a45cdcccba25f97c" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:50.747736 10055 raft_consensus.cc:3060] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.752025 10333 tablet_bootstrap.cc:654] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.746731 10020 leader_election.cc:304] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
W20260709 13:06:50.755581 10020 leader_election.cc:343] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.755499 10122 tablet_service.cc:1511] Processing CreateTablet for tablet 4af4e546f1084d8cb71469fbf3886792 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:06:50.758783 10325 ts_tablet_manager.cc:1434] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.066s	user 0.007s	sys 0.001s
I20260709 13:06:50.760069 10325 tablet_bootstrap.cc:492] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:50.761829 10331 raft_consensus.cc:493] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.762533 10331 raft_consensus.cc:515] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:50.757441 10122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af4e546f1084d8cb71469fbf3886792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.766512 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7dfde2e205846058c75fc205443d330" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:50.769408 10131 raft_consensus.cc:2468] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:50.768661 10388 raft_consensus.cc:2804] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.770697 10388 raft_consensus.cc:493] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.770941 10278 raft_consensus.cc:2468] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
I20260709 13:06:50.770818 10193 tablet_service.cc:1511] Processing CreateTablet for tablet a0ea0c11178d43888a5596f49ffc7a7e (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:06:50.771926 10169 leader_election.cc:304] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:50.772226 10193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0ea0c11178d43888a5596f49ffc7a7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.772836 10347 raft_consensus.cc:2804] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.773212 10347 raft_consensus.cc:493] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.773506 10388 raft_consensus.cc:3060] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.773597 10347 raft_consensus.cc:3060] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.774569 10268 tablet_service.cc:1511] Processing CreateTablet for tablet 4af4e546f1084d8cb71469fbf3886792 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:06:50.775708 10268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af4e546f1084d8cb71469fbf3886792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.778678 10325 tablet_bootstrap.cc:654] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.781672 10095 leader_election.cc:304] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:06:50.785899 10389 consensus_queue.cc:1048] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:06:50.783550 10349 raft_consensus.cc:2804] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.791271 10331 leader_election.cc:290] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:50.795277 10042 tablet_service.cc:1511] Processing CreateTablet for tablet a0ea0c11178d43888a5596f49ffc7a7e (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:06:50.796952 10042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0ea0c11178d43888a5596f49ffc7a7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.797904 10349 raft_consensus.cc:697] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Becoming Leader. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:06:50.799269 10349 consensus_queue.cc:237] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:50.783599 10266 tablet_service.cc:1511] Processing CreateTablet for tablet a0ea0c11178d43888a5596f49ffc7a7e (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:06:50.802858 10266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0ea0c11178d43888a5596f49ffc7a7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.806859 10379 consensus_queue.cc:1048] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:50.772853 10041 tablet_service.cc:1511] Processing CreateTablet for tablet 4af4e546f1084d8cb71469fbf3886792 (DEFAULT_TABLE table=test-workload [id=af71821d7e9545ff9d8be73a7f3a9e6c]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:06:50.813890 10041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af4e546f1084d8cb71469fbf3886792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.812804 10347 raft_consensus.cc:515] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:50.820417 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7dfde2e205846058c75fc205443d330" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:50.829345 10324 tablet_bootstrap.cc:654] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:50.830595 10169 leader_election.cc:343] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.831647 10354 consensus_queue.cc:1048] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:50.850190 10055 raft_consensus.cc:2468] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
I20260709 13:06:50.861368 10328 tablet_bootstrap.cc:492] T c7dfde2e205846058c75fc205443d330 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:50.869467 10328 ts_tablet_manager.cc:1403] T c7dfde2e205846058c75fc205443d330 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.147s	user 0.020s	sys 0.011s
I20260709 13:06:50.871496 10354 consensus_queue.cc:1048] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:50.862648 10420 consensus_queue.cc:1048] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:50.868207 10331 consensus_queue.cc:1048] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:06:50.850977 10098 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42425, user_credentials={real_user=slave}} blocked reactor thread for 45269.8us
I20260709 13:06:50.874883 10328 raft_consensus.cc:359] T c7dfde2e205846058c75fc205443d330 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:50.878831 10388 raft_consensus.cc:515] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:50.873306 10347 leader_election.cc:290] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:50.881055 10324 tablet_bootstrap.cc:492] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:50.894158 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7dfde2e205846058c75fc205443d330" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:50.896767 10328 raft_consensus.cc:385] T c7dfde2e205846058c75fc205443d330 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.902839 10324 ts_tablet_manager.cc:1403] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.172s	user 0.009s	sys 0.020s
I20260709 13:06:50.902853 10328 raft_consensus.cc:740] T c7dfde2e205846058c75fc205443d330 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.908416 10328 consensus_queue.cc:260] T c7dfde2e205846058c75fc205443d330 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:50.910555 10328 ts_tablet_manager.cc:1434] T c7dfde2e205846058c75fc205443d330 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 13:06:50.910749 10420 raft_consensus.cc:493] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.911576 10328 tablet_bootstrap.cc:492] T e959733d8ef94ea19aad472fcf384371 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:50.911234 10420 raft_consensus.cc:515] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:50.918876 10324 raft_consensus.cc:359] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:50.919837 10333 tablet_bootstrap.cc:492] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:50.913435 10420 leader_election.cc:290] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:50.920620 10333 ts_tablet_manager.cc:1403] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.312s	user 0.019s	sys 0.012s
I20260709 13:06:50.920673 10324 raft_consensus.cc:385] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.922923 10324 raft_consensus.cc:740] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.915319 10388 leader_election.cc:290] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:50.916115 10277 raft_consensus.cc:3060] T c7dfde2e205846058c75fc205443d330 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.928997 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a64aca789f4ea8a4925aa79dcb0235" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:50.928840 10333 raft_consensus.cc:359] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:50.929656 10333 raft_consensus.cc:385] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.929433 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7dfde2e205846058c75fc205443d330" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:50.929960 10333 raft_consensus.cc:740] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.930202 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df6e7bddfd4832b31ccc4bb8b3521b" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:50.930684 10333 consensus_queue.cc:260] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:50.930980 10204 raft_consensus.cc:2468] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:50.932467 10243 leader_election.cc:304] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 575e44d297f045a389903d65a471d548
I20260709 13:06:50.933226 10129 raft_consensus.cc:3060] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.933722 10362 raft_consensus.cc:2804] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.934127 10362 raft_consensus.cc:493] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.934461 10362 raft_consensus.cc:3060] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.933362 10333 ts_tablet_manager.cc:1434] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:06:50.941229 10333 tablet_bootstrap.cc:492] T 56df6e7bddfd4832b31ccc4bb8b3521b P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:50.941495 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a64aca789f4ea8a4925aa79dcb0235" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:50.940992 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fa9341e7fbe49e1a832721bfa4abff3" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:50.942121 10204 raft_consensus.cc:3060] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.944885  9952 catalog_manager.cc:5697] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 reported cstate change: term changed from 0 to 1, leader changed from <none> to 575e44d297f045a389903d65a471d548 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "575e44d297f045a389903d65a471d548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:50.925946 10324 consensus_queue.cc:260] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:50.930215 10130 raft_consensus.cc:3060] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.951557 10331 consensus_queue.cc:1048] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:50.957887 10324 ts_tablet_manager.cc:1434] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.001s
I20260709 13:06:50.961982 10132 raft_consensus.cc:3060] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.962132 10279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fa9341e7fbe49e1a832721bfa4abff3" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:50.977237 10430 raft_consensus.cc:493] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:50.980801 10419 log.cc:927] Time spent T aa03c270e7364d769700b877d171588e P 8fe33410d51a469e81be6edc602d7d78: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
W20260709 13:06:50.994438 10169 consensus_peers.cc:597] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 -> Peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Couldn't send request to peer 50999113203c482b9b9300caa485f154. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:50.988718 10095 leader_election.cc:304] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:50.999814 10129 raft_consensus.cc:2468] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:06:51.006062 10324 tablet_bootstrap.cc:492] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:51.013477 10333 tablet_bootstrap.cc:654] T 56df6e7bddfd4832b31ccc4bb8b3521b P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.028090 10277 raft_consensus.cc:2468] T c7dfde2e205846058c75fc205443d330 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:51.033201 10130 raft_consensus.cc:2468] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:51.040333 10020 leader_election.cc:304] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:51.010154 10362 raft_consensus.cc:515] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:51.057504 10388 raft_consensus.cc:493] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.071547 10388 raft_consensus.cc:515] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:51.057713 10169 leader_election.cc:304] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:51.061095 10354 raft_consensus.cc:2804] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.074462 10331 raft_consensus.cc:2804] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.074687 10354 raft_consensus.cc:697] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:06:51.075019 10331 raft_consensus.cc:697] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:06:51.075505 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df6e7bddfd4832b31ccc4bb8b3521b" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:51.075644 10354 consensus_queue.cc:237] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:51.076424 10203 raft_consensus.cc:3060] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.073962 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df6e7bddfd4832b31ccc4bb8b3521b" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:51.073989 10388 leader_election.cc:290] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:51.078101 10362 leader_election.cc:290] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:51.068152 10349 consensus_queue.cc:1048] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:06:51.078660 10243 leader_election.cc:343] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.079222 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d2f27feb83847298256dd63dafcd35c" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:51.075829 10331 consensus_queue.cc:237] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:51.010861 10430 raft_consensus.cc:515] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:51.084559 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d2f27feb83847298256dd63dafcd35c" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:51.090433 10430 leader_election.cc:290] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:51.091058 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b626433d4aec4288976ffc448a79bb68" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:51.091933 10202 raft_consensus.cc:2468] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:51.093770 10243 leader_election.cc:304] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
W20260709 13:06:51.094141 10020 leader_election.cc:343] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.095173 10430 raft_consensus.cc:2804] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.095692 10430 raft_consensus.cc:493] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.096109 10430 raft_consensus.cc:3060] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:51.104637 10422 log.cc:927] Time spent T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:06:51.107811 10204 raft_consensus.cc:2468] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:06:51.067049 10328 tablet_bootstrap.cc:654] T e959733d8ef94ea19aad472fcf384371 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.152819 10430 raft_consensus.cc:515] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
W20260709 13:06:51.150148 10020 leader_election.cc:343] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.153839 10020 leader_election.cc:304] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa338cd132c457896b11e08d7ea787d; no voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:51.160060 10057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b626433d4aec4288976ffc448a79bb68" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:51.190786 10333 tablet_bootstrap.cc:492] T 56df6e7bddfd4832b31ccc4bb8b3521b P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:51.191483 10333 ts_tablet_manager.cc:1403] T 56df6e7bddfd4832b31ccc4bb8b3521b P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.251s	user 0.018s	sys 0.008s
I20260709 13:06:51.160866 10354 raft_consensus.cc:2749] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:51.194486 10246 leader_election.cc:343] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.195070 10325 tablet_bootstrap.cc:492] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:51.195710 10325 ts_tablet_manager.cc:1403] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.436s	user 0.010s	sys 0.045s
I20260709 13:06:51.205552 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b626433d4aec4288976ffc448a79bb68" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:51.206378 10205 raft_consensus.cc:3060] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.208976 10324 tablet_bootstrap.cc:654] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.213157 10203 raft_consensus.cc:2468] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:51.216984 10333 raft_consensus.cc:359] T 56df6e7bddfd4832b31ccc4bb8b3521b P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:51.226966 10333 raft_consensus.cc:385] T 56df6e7bddfd4832b31ccc4bb8b3521b P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.227612 10333 raft_consensus.cc:740] T 56df6e7bddfd4832b31ccc4bb8b3521b P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.243155 10333 consensus_queue.cc:260] T 56df6e7bddfd4832b31ccc4bb8b3521b P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
W20260709 13:06:51.240826 10172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42425, user_credentials={real_user=slave}} blocked reactor thread for 71502.4us
I20260709 13:06:51.252107 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b626433d4aec4288976ffc448a79bb68" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:51.249809 10430 leader_election.cc:290] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:51.257277 10328 tablet_bootstrap.cc:492] T e959733d8ef94ea19aad472fcf384371 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:51.257992 10328 ts_tablet_manager.cc:1403] T e959733d8ef94ea19aad472fcf384371 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.347s	user 0.017s	sys 0.004s
W20260709 13:06:51.260442 10246 leader_election.cc:343] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.247574 10243 leader_election.cc:304] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 575e44d297f045a389903d65a471d548
I20260709 13:06:51.262535  9955 catalog_manager.cc:5697] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:51.273494 10333 ts_tablet_manager.cc:1434] T 56df6e7bddfd4832b31ccc4bb8b3521b P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.082s	user 0.008s	sys 0.001s
I20260709 13:06:51.282146 10325 raft_consensus.cc:359] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:51.264755 10430 raft_consensus.cc:2804] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.269074 10205 raft_consensus.cc:2468] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:51.287910 10243 leader_election.cc:304] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:06:51.283088 10328 raft_consensus.cc:359] T e959733d8ef94ea19aad472fcf384371 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:51.288986 10328 raft_consensus.cc:385] T e959733d8ef94ea19aad472fcf384371 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.263111 10349 consensus_queue.cc:1048] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:06:51.290398 10328 raft_consensus.cc:740] T e959733d8ef94ea19aad472fcf384371 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.272550  9952 catalog_manager.cc:5697] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 0 to 1, leader changed from <none> to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:51.284101 10325 raft_consensus.cc:385] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.292899 10325 raft_consensus.cc:740] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.291960 10430 raft_consensus.cc:697] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:06:51.293493 10362 raft_consensus.cc:2804] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.293670 10325 consensus_queue.cc:260] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:51.294229 10362 raft_consensus.cc:697] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:06:51.291347 10328 consensus_queue.cc:260] T e959733d8ef94ea19aad472fcf384371 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:51.294658 10430 consensus_queue.cc:237] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:51.295130 10362 consensus_queue.cc:237] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:51.296449 10328 ts_tablet_manager.cc:1434] T e959733d8ef94ea19aad472fcf384371 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.001s
I20260709 13:06:51.297505 10328 tablet_bootstrap.cc:492] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:51.302911 10333 tablet_bootstrap.cc:492] T 5f64c791e7fb4a79bd498140ef0b0943 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:51.306049 10420 raft_consensus.cc:2749] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.306468 10325 ts_tablet_manager.cc:1434] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.110s	user 0.007s	sys 0.005s
I20260709 13:06:51.307682 10325 tablet_bootstrap.cc:492] T 375f151edb2245daa6d1b06187f21b79 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:51.338168 10324 tablet_bootstrap.cc:492] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:51.338842 10324 ts_tablet_manager.cc:1403] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.350s	user 0.019s	sys 0.000s
I20260709 13:06:51.343289 10325 tablet_bootstrap.cc:654] T 375f151edb2245daa6d1b06187f21b79 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.342763 10324 raft_consensus.cc:359] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:51.344198 10324 raft_consensus.cc:385] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.344388 10324 raft_consensus.cc:740] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.352219 10333 tablet_bootstrap.cc:654] T 5f64c791e7fb4a79bd498140ef0b0943 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.353627 10328 tablet_bootstrap.cc:654] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.352681  9954 catalog_manager.cc:5697] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:51.357681 10324 consensus_queue.cc:260] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:51.370441 10280 raft_consensus.cc:3060] T e959733d8ef94ea19aad472fcf384371 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.383311 10421 consensus_queue.cc:1048] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:06:51.385058 10354 raft_consensus.cc:493] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.388934 10354 raft_consensus.cc:515] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:51.392204 10354 leader_election.cc:290] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:51.402577 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d966d297f346da9e984aa2e181590a" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:51.405019 10324 ts_tablet_manager.cc:1434] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.066s	user 0.009s	sys 0.000s
I20260709 13:06:51.403353 10205 raft_consensus.cc:2468] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 0.
I20260709 13:06:51.415403 10020 leader_election.cc:304] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:51.415719 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d966d297f346da9e984aa2e181590a" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:51.417629 10388 raft_consensus.cc:2804] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.418272 10388 raft_consensus.cc:493] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.418751 10388 raft_consensus.cc:3060] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.424355 10324 tablet_bootstrap.cc:492] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
W20260709 13:06:51.424583 10020 leader_election.cc:343] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.436333 10324 tablet_bootstrap.cc:654] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.434465  9952 catalog_manager.cc:5697] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:51.438194 10325 tablet_bootstrap.cc:492] T 375f151edb2245daa6d1b06187f21b79 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:51.438469 10333 tablet_bootstrap.cc:492] T 5f64c791e7fb4a79bd498140ef0b0943 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:51.438699 10328 tablet_bootstrap.cc:492] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:51.438793 10325 ts_tablet_manager.cc:1403] T 375f151edb2245daa6d1b06187f21b79 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.131s	user 0.012s	sys 0.017s
I20260709 13:06:51.439244 10328 ts_tablet_manager.cc:1403] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.142s	user 0.014s	sys 0.016s
I20260709 13:06:51.438947 10333 ts_tablet_manager.cc:1403] T 5f64c791e7fb4a79bd498140ef0b0943 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.146s	user 0.044s	sys 0.041s
I20260709 13:06:51.448910 10325 raft_consensus.cc:359] T 375f151edb2245daa6d1b06187f21b79 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:51.449088 10333 raft_consensus.cc:359] T 5f64c791e7fb4a79bd498140ef0b0943 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:51.449815 10325 raft_consensus.cc:385] T 375f151edb2245daa6d1b06187f21b79 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.449824 10333 raft_consensus.cc:385] T 5f64c791e7fb4a79bd498140ef0b0943 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.450376 10325 raft_consensus.cc:740] T 375f151edb2245daa6d1b06187f21b79 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.450392 10333 raft_consensus.cc:740] T 5f64c791e7fb4a79bd498140ef0b0943 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.450584 10328 raft_consensus.cc:359] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:51.452142 10328 raft_consensus.cc:385] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.452347 10325 consensus_queue.cc:260] T 375f151edb2245daa6d1b06187f21b79 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:51.452380 10333 consensus_queue.cc:260] T 5f64c791e7fb4a79bd498140ef0b0943 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:51.451304 10388 raft_consensus.cc:515] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:51.455261 10333 ts_tablet_manager.cc:1434] T 5f64c791e7fb4a79bd498140ef0b0943 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.002s
I20260709 13:06:51.452507 10328 raft_consensus.cc:740] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.456647 10328 consensus_queue.cc:260] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:51.459205 10333 tablet_bootstrap.cc:492] T 75ed1b358949418a9d300990f8f81652 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:51.460726 10328 ts_tablet_manager.cc:1434] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:06:51.462293 10328 tablet_bootstrap.cc:492] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:51.467743 10333 tablet_bootstrap.cc:654] T 75ed1b358949418a9d300990f8f81652 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.472061 10388 leader_election.cc:290] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:51.474246 10325 ts_tablet_manager.cc:1434] T 375f151edb2245daa6d1b06187f21b79 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 13:06:51.475432 10325 tablet_bootstrap.cc:492] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:51.479112 10328 tablet_bootstrap.cc:654] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.480216 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d966d297f346da9e984aa2e181590a" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:51.481133 10205 raft_consensus.cc:3060] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.481819 10325 tablet_bootstrap.cc:654] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.487583 10388 consensus_queue.cc:1048] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:51.487972 10328 tablet_bootstrap.cc:492] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:51.490065 10328 ts_tablet_manager.cc:1403] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.000s
I20260709 13:06:51.491861 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d966d297f346da9e984aa2e181590a" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:51.492334 10325 tablet_bootstrap.cc:492] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:51.492796 10325 ts_tablet_manager.cc:1403] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.018s	user 0.006s	sys 0.006s
I20260709 13:06:51.492871 10328 raft_consensus.cc:359] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
W20260709 13:06:51.493395 10020 leader_election.cc:343] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.495654 10333 tablet_bootstrap.cc:492] T 75ed1b358949418a9d300990f8f81652 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:51.496425 10333 ts_tablet_manager.cc:1403] T 75ed1b358949418a9d300990f8f81652 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.037s	user 0.025s	sys 0.000s
I20260709 13:06:51.495656 10325 raft_consensus.cc:359] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:51.493588 10328 raft_consensus.cc:385] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.498018 10328 raft_consensus.cc:740] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.498803 10328 consensus_queue.cc:260] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:51.500851 10328 ts_tablet_manager.cc:1434] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 13:06:51.501760 10328 tablet_bootstrap.cc:492] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:51.503726 10430 raft_consensus.cc:493] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.504261 10430 raft_consensus.cc:515] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:51.504807 10429 consensus_queue.cc:1048] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:51.506472 10430 leader_election.cc:290] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:51.496893 10325 raft_consensus.cc:385] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.508567 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "375f151edb2245daa6d1b06187f21b79" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:51.509735 10325 raft_consensus.cc:740] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
W20260709 13:06:51.510957 10243 leader_election.cc:343] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.510732 10325 consensus_queue.cc:260] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:51.511904 10206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "375f151edb2245daa6d1b06187f21b79" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:51.512596 10206 raft_consensus.cc:2468] T 375f151edb2245daa6d1b06187f21b79 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:51.513865 10243 leader_election.cc:304] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 575e44d297f045a389903d65a471d548
I20260709 13:06:51.514600 10328 tablet_bootstrap.cc:654] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.515012 10430 raft_consensus.cc:2804] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.515424 10430 raft_consensus.cc:493] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.515750 10430 raft_consensus.cc:3060] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.510701 10333 raft_consensus.cc:359] T 75ed1b358949418a9d300990f8f81652 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:51.518244 10333 raft_consensus.cc:385] T 75ed1b358949418a9d300990f8f81652 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.518651 10333 raft_consensus.cc:740] T 75ed1b358949418a9d300990f8f81652 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.519599 10333 consensus_queue.cc:260] T 75ed1b358949418a9d300990f8f81652 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:51.522153 10333 ts_tablet_manager.cc:1434] T 75ed1b358949418a9d300990f8f81652 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.002s
I20260709 13:06:51.523494 10333 tablet_bootstrap.cc:492] T 6d2f27feb83847298256dd63dafcd35c P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:51.530042 10324 tablet_bootstrap.cc:492] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:51.530498 10324 ts_tablet_manager.cc:1403] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.107s	user 0.011s	sys 0.001s
I20260709 13:06:51.532011 10325 ts_tablet_manager.cc:1434] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.002s
I20260709 13:06:51.533136 10325 tablet_bootstrap.cc:492] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:51.535368 10333 tablet_bootstrap.cc:654] T 6d2f27feb83847298256dd63dafcd35c P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.517953 10205 raft_consensus.cc:2468] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:06:51.538349 10324 raft_consensus.cc:359] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:51.539165 10324 raft_consensus.cc:385] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.539369 10324 raft_consensus.cc:740] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.540926 10324 consensus_queue.cc:260] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:51.543254 10324 ts_tablet_manager.cc:1434] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:06:51.544302 10324 tablet_bootstrap.cc:492] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:51.544706 10325 tablet_bootstrap.cc:654] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.550626 10324 tablet_bootstrap.cc:654] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.552522 10020 leader_election.cc:304] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 50999113203c482b9b9300caa485f154
I20260709 13:06:51.554222 10328 tablet_bootstrap.cc:492] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:51.551721 10430 raft_consensus.cc:515] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:51.554850 10328 ts_tablet_manager.cc:1403] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.053s	user 0.008s	sys 0.005s
I20260709 13:06:51.555718 10388 raft_consensus.cc:2804] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.556258 10388 raft_consensus.cc:697] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:06:51.557097 10388 consensus_queue.cc:237] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:51.557904 10328 raft_consensus.cc:359] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:51.559557 10328 raft_consensus.cc:385] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.560073 10328 raft_consensus.cc:740] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.560813 10328 consensus_queue.cc:260] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:51.562675 10328 ts_tablet_manager.cc:1434] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 13:06:51.563858 10328 tablet_bootstrap.cc:492] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:51.566587 10430 leader_election.cc:290] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:51.568114 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "375f151edb2245daa6d1b06187f21b79" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:51.569198 10333 tablet_bootstrap.cc:492] T 6d2f27feb83847298256dd63dafcd35c P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:51.569700 10333 ts_tablet_manager.cc:1403] T 6d2f27feb83847298256dd63dafcd35c P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.046s	user 0.017s	sys 0.003s
W20260709 13:06:51.569700 10243 leader_election.cc:343] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.570026 10328 tablet_bootstrap.cc:654] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.571583 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "375f151edb2245daa6d1b06187f21b79" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:51.572208 10205 raft_consensus.cc:3060] T 375f151edb2245daa6d1b06187f21b79 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.573048 10333 raft_consensus.cc:359] T 6d2f27feb83847298256dd63dafcd35c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:51.574007 10333 raft_consensus.cc:385] T 6d2f27feb83847298256dd63dafcd35c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.574362 10333 raft_consensus.cc:740] T 6d2f27feb83847298256dd63dafcd35c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.583639 10325 tablet_bootstrap.cc:492] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:51.584400 10325 ts_tablet_manager.cc:1403] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.051s	user 0.016s	sys 0.000s
I20260709 13:06:51.588059 10325 raft_consensus.cc:359] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:51.588994 10325 raft_consensus.cc:385] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.590247 10457 raft_consensus.cc:493] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.590968 10457 raft_consensus.cc:515] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:51.594801 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64c791e7fb4a79bd498140ef0b0943" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
W20260709 13:06:51.596585 10172 leader_election.cc:343] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.598649 10457 leader_election.cc:290] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:51.599309 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64c791e7fb4a79bd498140ef0b0943" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:51.580463 10333 consensus_queue.cc:260] T 6d2f27feb83847298256dd63dafcd35c P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:51.600080 10132 raft_consensus.cc:2468] T 5f64c791e7fb4a79bd498140ef0b0943 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:51.601550 10169 leader_election.cc:304] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:06:51.603299 10347 raft_consensus.cc:2804] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.603740 10347 raft_consensus.cc:493] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.604099 10347 raft_consensus.cc:3060] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.637831 10333 ts_tablet_manager.cc:1434] T 6d2f27feb83847298256dd63dafcd35c P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.068s	user 0.018s	sys 0.018s
I20260709 13:06:51.639285 10333 tablet_bootstrap.cc:492] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:51.640030 10205 raft_consensus.cc:2468] T 375f151edb2245daa6d1b06187f21b79 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:51.641429 10243 leader_election.cc:304] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 575e44d297f045a389903d65a471d548
I20260709 13:06:51.642904 10430 raft_consensus.cc:2804] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.643513 10430 raft_consensus.cc:697] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:06:51.644352 10430 consensus_queue.cc:237] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:51.646761 10333 tablet_bootstrap.cc:654] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.646771  9950 catalog_manager.cc:5697] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 0 to 1, leader changed from <none> to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:51.648602 10347 raft_consensus.cc:515] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:51.589347 10325 raft_consensus.cc:740] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.650890 10325 consensus_queue.cc:260] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:51.654925 10347 leader_election.cc:290] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:51.655592 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64c791e7fb4a79bd498140ef0b0943" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:51.658969 10328 tablet_bootstrap.cc:492] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:51.659698 10328 ts_tablet_manager.cc:1403] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.096s	user 0.040s	sys 0.031s
I20260709 13:06:51.658973 10324 tablet_bootstrap.cc:492] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:51.661406 10324 ts_tablet_manager.cc:1403] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.117s	user 0.017s	sys 0.024s
W20260709 13:06:51.661547 10172 leader_election.cc:343] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.663205 10328 raft_consensus.cc:359] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:51.664101 10333 tablet_bootstrap.cc:492] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:51.664999 10333 ts_tablet_manager.cc:1403] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.001s
I20260709 13:06:51.664453 10328 raft_consensus.cc:385] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.667557 10328 raft_consensus.cc:740] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.668474 10328 consensus_queue.cc:260] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:51.671424 10324 raft_consensus.cc:359] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:51.672235 10324 raft_consensus.cc:385] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.672529 10324 raft_consensus.cc:740] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.674116 10457 raft_consensus.cc:493] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.675179 10457 raft_consensus.cc:515] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:51.678936 10324 consensus_queue.cc:260] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:51.679275 10347 consensus_queue.cc:1048] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:06:51.681046 10457 leader_election.cc:290] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:51.682020 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d2f27feb83847298256dd63dafcd35c" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:51.682781 10055 raft_consensus.cc:2468] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:51.684626 10333 raft_consensus.cc:359] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:51.684990 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64c791e7fb4a79bd498140ef0b0943" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:51.685420 10333 raft_consensus.cc:385] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.685706 10132 raft_consensus.cc:3060] T 5f64c791e7fb4a79bd498140ef0b0943 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.685863 10333 raft_consensus.cc:740] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.686168 10324 ts_tablet_manager.cc:1434] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.002s
I20260709 13:06:51.687310 10324 tablet_bootstrap.cc:492] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:51.687561 10325 ts_tablet_manager.cc:1434] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.103s	user 0.009s	sys 0.001s
I20260709 13:06:51.689052 10325 tablet_bootstrap.cc:492] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:51.690078 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d2f27feb83847298256dd63dafcd35c" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:51.690794 10131 raft_consensus.cc:2468] T 6d2f27feb83847298256dd63dafcd35c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:51.694947 10169 leader_election.cc:304] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:51.696017 10347 raft_consensus.cc:2804] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.696379 10347 raft_consensus.cc:493] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.696686 10347 raft_consensus.cc:3060] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.701246 10420 raft_consensus.cc:493] T 56df6e7bddfd4832b31ccc4bb8b3521b P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.702026 10420 raft_consensus.cc:515] T 56df6e7bddfd4832b31ccc4bb8b3521b P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:51.704319 10347 raft_consensus.cc:515] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:51.706838 10328 ts_tablet_manager.cc:1434] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.047s	user 0.010s	sys 0.000s
I20260709 13:06:51.706457 10457 consensus_queue.cc:1048] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:51.707937 10328 tablet_bootstrap.cc:492] T d9cb1393c4f549f0bc9b5e82121b543a P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:51.693886 10333 consensus_queue.cc:260] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:51.713696 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d2f27feb83847298256dd63dafcd35c" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:51.714381 10055 raft_consensus.cc:3060] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.714493 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df6e7bddfd4832b31ccc4bb8b3521b" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:51.715329 10205 raft_consensus.cc:2393] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 575e44d297f045a389903d65a471d548 in current term 1: Already voted for candidate 50999113203c482b9b9300caa485f154 in this term.
I20260709 13:06:51.705122 10420 leader_election.cc:290] T 56df6e7bddfd4832b31ccc4bb8b3521b P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:51.718057 10278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df6e7bddfd4832b31ccc4bb8b3521b" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:51.723196 10095 leader_election.cc:304] T 56df6e7bddfd4832b31ccc4bb8b3521b P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:51.728133 10423 raft_consensus.cc:3060] T 56df6e7bddfd4832b31ccc4bb8b3521b P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.732645 10362 raft_consensus.cc:493] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.733207 10362 raft_consensus.cc:515] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:51.734514 10132 raft_consensus.cc:2468] T 5f64c791e7fb4a79bd498140ef0b0943 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:51.734450  9950 catalog_manager.cc:5697] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:51.736248 10324 tablet_bootstrap.cc:654] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.738492 10057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ed1b358949418a9d300990f8f81652" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:51.738793 10169 leader_election.cc:304] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:06:51.742473 10057 raft_consensus.cc:2468] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:51.742650 10362 leader_election.cc:290] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:51.744153 10055 raft_consensus.cc:2468] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:51.744655 10246 leader_election.cc:304] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:51.743559 10347 leader_election.cc:290] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:51.746862 10333 ts_tablet_manager.cc:1434] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.082s	user 0.005s	sys 0.002s
I20260709 13:06:51.745576 10172 leader_election.cc:304] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:51.751520 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d2f27feb83847298256dd63dafcd35c" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:51.752319 10130 raft_consensus.cc:3060] T 6d2f27feb83847298256dd63dafcd35c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.745796 10457 raft_consensus.cc:2804] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.747933 10362 raft_consensus.cc:2804] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.753402 10457 raft_consensus.cc:697] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:06:51.753798 10362 raft_consensus.cc:493] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.754362 10362 raft_consensus.cc:3060] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.754448 10457 consensus_queue.cc:237] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:51.758270 10333 tablet_bootstrap.cc:492] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:51.760764 10423 raft_consensus.cc:2749] T 56df6e7bddfd4832b31ccc4bb8b3521b P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.765821 10430 consensus_queue.cc:1048] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:51.769938 10347 raft_consensus.cc:2804] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.781533 10325 tablet_bootstrap.cc:654] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.772360 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ed1b358949418a9d300990f8f81652" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:51.786721 10333 tablet_bootstrap.cc:654] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.783700 10347 raft_consensus.cc:697] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:06:51.784772 10132 raft_consensus.cc:2468] T 75ed1b358949418a9d300990f8f81652 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:51.790762 10362 raft_consensus.cc:515] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:51.795125 10130 raft_consensus.cc:2468] T 6d2f27feb83847298256dd63dafcd35c P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:51.796254 10347 consensus_queue.cc:237] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:51.798923 10430 consensus_queue.cc:1048] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:51.804131 10328 tablet_bootstrap.cc:654] T d9cb1393c4f549f0bc9b5e82121b543a P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.814601  9952 catalog_manager.cc:5697] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:51.823704 10430 consensus_queue.cc:1048] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:51.826913 10362 leader_election.cc:290] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
W20260709 13:06:51.832777 10246 consensus_peers.cc:597] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:51.847702 10325 tablet_bootstrap.cc:492] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:51.848268 10325 ts_tablet_manager.cc:1403] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.159s	user 0.013s	sys 0.002s
I20260709 13:06:51.857048 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ed1b358949418a9d300990f8f81652" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:51.857793 10055 raft_consensus.cc:3060] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:51.857946 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:51.860045 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ed1b358949418a9d300990f8f81652" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:51.866780 10132 raft_consensus.cc:3060] T 75ed1b358949418a9d300990f8f81652 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.880115 10324 tablet_bootstrap.cc:492] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:51.880707 10324 ts_tablet_manager.cc:1403] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.194s	user 0.024s	sys 0.010s
I20260709 13:06:51.883491 10324 raft_consensus.cc:359] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:51.884174 10324 raft_consensus.cc:385] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.884450 10324 raft_consensus.cc:740] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.885139 10324 consensus_queue.cc:260] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:51.886886 10055 raft_consensus.cc:2468] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:51.888239 10246 leader_election.cc:304] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:51.889482 10430 raft_consensus.cc:2804] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.889977 10430 raft_consensus.cc:697] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:06:51.890822 10430 consensus_queue.cc:237] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:51.895263 10324 ts_tablet_manager.cc:1434] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:06:51.896523 10324 tablet_bootstrap.cc:492] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:51.909101 10324 tablet_bootstrap.cc:654] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.920558 10325 raft_consensus.cc:359] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:51.921537 10325 raft_consensus.cc:385] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.921943 10325 raft_consensus.cc:740] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.922925 10325 consensus_queue.cc:260] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:51.925865 10325 ts_tablet_manager.cc:1434] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.077s	user 0.009s	sys 0.000s
I20260709 13:06:51.927119 10325 tablet_bootstrap.cc:492] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:51.933892 10325 tablet_bootstrap.cc:654] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.938822 10132 raft_consensus.cc:2468] T 75ed1b358949418a9d300990f8f81652 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:51.943851 10333 tablet_bootstrap.cc:492] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:51.944516 10333 ts_tablet_manager.cc:1403] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.187s	user 0.006s	sys 0.021s
I20260709 13:06:51.948364 10324 tablet_bootstrap.cc:492] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:51.948901 10324 ts_tablet_manager.cc:1403] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.053s	user 0.014s	sys 0.001s
I20260709 13:06:51.947853 10333 raft_consensus.cc:359] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:51.949900 10333 raft_consensus.cc:385] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.950253 10333 raft_consensus.cc:740] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.951345 10333 consensus_queue.cc:260] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:51.951414 10324 raft_consensus.cc:359] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:51.952235 10324 raft_consensus.cc:385] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.952572 10324 raft_consensus.cc:740] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.956956 10324 consensus_queue.cc:260] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:51.953528 10333 ts_tablet_manager.cc:1434] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:06:51.959657 10333 tablet_bootstrap.cc:492] T 7f0446529cd446d0ba97ee860c6ae221 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:51.959525 10362 raft_consensus.cc:493] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.965886 10362 raft_consensus.cc:515] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:51.970243 10362 leader_election.cc:290] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:51.970578 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d966d297f346da9e984aa2e181590a" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:51.971131 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d966d297f346da9e984aa2e181590a" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:51.971462 10205 raft_consensus.cc:2393] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50999113203c482b9b9300caa485f154 in current term 1: Already voted for candidate efa338cd132c457896b11e08d7ea787d in this term.
I20260709 13:06:51.972826 10246 leader_election.cc:304] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:51.973974 10324 ts_tablet_manager.cc:1434] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:06:51.974080 10362 raft_consensus.cc:3060] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.980329 10324 tablet_bootstrap.cc:492] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:51.985069  9950 catalog_manager.cc:5697] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:51.987092 10362 raft_consensus.cc:2749] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.991551 10429 consensus_queue.cc:1048] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:51.993057 10333 tablet_bootstrap.cc:654] T 7f0446529cd446d0ba97ee860c6ae221 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.998744 10429 consensus_queue.cc:1048] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.008594 10324 tablet_bootstrap.cc:654] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.018471 10333 tablet_bootstrap.cc:492] T 7f0446529cd446d0ba97ee860c6ae221 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:52.028251 10429 raft_consensus.cc:493] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.029019 10429 raft_consensus.cc:515] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:52.031674 10429 leader_election.cc:290] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:52.033560 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50256706bf3a4bf29b64ea4ea6583da3" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:52.034454 10280 raft_consensus.cc:2468] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 0.
I20260709 13:06:52.034494 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50256706bf3a4bf29b64ea4ea6583da3" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
W20260709 13:06:52.036676 10020 leader_election.cc:343] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.037982 10020 leader_election.cc:304] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 575e44d297f045a389903d65a471d548
I20260709 13:06:52.039621 10333 ts_tablet_manager.cc:1403] T 7f0446529cd446d0ba97ee860c6ae221 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.080s	user 0.013s	sys 0.005s
I20260709 13:06:52.044435 10333 raft_consensus.cc:359] T 7f0446529cd446d0ba97ee860c6ae221 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.025204 10324 tablet_bootstrap.cc:492] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:52.046636 10324 ts_tablet_manager.cc:1403] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.067s	user 0.015s	sys 0.001s
I20260709 13:06:52.045571 10429 raft_consensus.cc:2804] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.049525 10429 raft_consensus.cc:493] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.050868 10324 raft_consensus.cc:359] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.051543 10324 raft_consensus.cc:385] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.051561 10429 raft_consensus.cc:3060] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.051909 10324 raft_consensus.cc:740] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.052872 10324 consensus_queue.cc:260] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.051637 10333 raft_consensus.cc:385] T 7f0446529cd446d0ba97ee860c6ae221 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.053793 10333 raft_consensus.cc:740] T 7f0446529cd446d0ba97ee860c6ae221 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.054560 10333 consensus_queue.cc:260] T 7f0446529cd446d0ba97ee860c6ae221 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.057114 10324 ts_tablet_manager.cc:1434] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:06:52.062940 10324 tablet_bootstrap.cc:492] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:52.066298 10333 ts_tablet_manager.cc:1434] T 7f0446529cd446d0ba97ee860c6ae221 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:06:52.066416 10328 tablet_bootstrap.cc:492] T d9cb1393c4f549f0bc9b5e82121b543a P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:52.070039 10328 ts_tablet_manager.cc:1403] T d9cb1393c4f549f0bc9b5e82121b543a P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.362s	user 0.054s	sys 0.090s
I20260709 13:06:52.071447 10325 tablet_bootstrap.cc:492] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:52.071918 10325 ts_tablet_manager.cc:1403] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.145s	user 0.054s	sys 0.040s
I20260709 13:06:52.072685 10328 raft_consensus.cc:359] T d9cb1393c4f549f0bc9b5e82121b543a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:52.073416 10328 raft_consensus.cc:385] T d9cb1393c4f549f0bc9b5e82121b543a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.073854 10328 raft_consensus.cc:740] T d9cb1393c4f549f0bc9b5e82121b543a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.074595 10328 consensus_queue.cc:260] T d9cb1393c4f549f0bc9b5e82121b543a P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:52.074723 10325 raft_consensus.cc:359] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.075553 10325 raft_consensus.cc:385] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.075858 10325 raft_consensus.cc:740] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.076826 10328 ts_tablet_manager.cc:1434] T d9cb1393c4f549f0bc9b5e82121b543a P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:52.077939 10333 tablet_bootstrap.cc:492] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:52.078567 10328 tablet_bootstrap.cc:492] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:52.076586 10325 consensus_queue.cc:260] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.083801 10347 raft_consensus.cc:493] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.084345 10347 raft_consensus.cc:515] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.086187 10324 tablet_bootstrap.cc:654] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.086536 10347 leader_election.cc:290] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
W20260709 13:06:52.087397 10243 consensus_peers.cc:597] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:52.088968 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f0446529cd446d0ba97ee860c6ae221" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:52.088733 10429 raft_consensus.cc:515] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:52.089666 10055 raft_consensus.cc:2468] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:52.090759 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f0446529cd446d0ba97ee860c6ae221" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:52.090907 10429 leader_election.cc:290] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Requested vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:52.091347 10132 raft_consensus.cc:2468] T 7f0446529cd446d0ba97ee860c6ae221 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:52.083916 10325 ts_tablet_manager.cc:1434] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:06:52.092732 10325 tablet_bootstrap.cc:492] T d9cb1393c4f549f0bc9b5e82121b543a P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:52.093878 10278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50256706bf3a4bf29b64ea4ea6583da3" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:52.094579 10328 tablet_bootstrap.cc:654] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.095454 10169 leader_election.cc:304] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:52.096308 10347 raft_consensus.cc:2804] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.096815 10347 raft_consensus.cc:493] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.094504 10278 raft_consensus.cc:3060] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.097267 10347 raft_consensus.cc:3060] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.098814 10325 tablet_bootstrap.cc:654] T d9cb1393c4f549f0bc9b5e82121b543a P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.102080 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50256706bf3a4bf29b64ea4ea6583da3" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
W20260709 13:06:52.109135 10020 leader_election.cc:343] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.111150  9953 catalog_manager.cc:5697] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.112627 10278 raft_consensus.cc:2468] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:06:52.114900 10347 raft_consensus.cc:515] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.084316 10333 tablet_bootstrap.cc:654] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.117584 10328 tablet_bootstrap.cc:492] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:52.118376 10020 leader_election.cc:304] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 575e44d297f045a389903d65a471d548
I20260709 13:06:52.120266 10429 raft_consensus.cc:2804] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.121031 10429 raft_consensus.cc:697] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:06:52.122288 10324 tablet_bootstrap.cc:492] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:52.123283 10324 ts_tablet_manager.cc:1403] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.061s	user 0.006s	sys 0.014s
I20260709 13:06:52.122289 10429 consensus_queue.cc:237] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:52.126073 10433 raft_consensus.cc:493] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.118369 10328 ts_tablet_manager.cc:1403] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.007s
I20260709 13:06:52.129220 10324 raft_consensus.cc:359] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.126845 10433 raft_consensus.cc:515] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.131060 10324 raft_consensus.cc:385] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.131402 10324 raft_consensus.cc:740] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.132128 10324 consensus_queue.cc:260] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.132570 10433 leader_election.cc:290] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:52.133179 10333 tablet_bootstrap.cc:492] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:52.130303 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f0446529cd446d0ba97ee860c6ae221" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:52.134449 10055 raft_consensus.cc:3060] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.134677 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b626433d4aec4288976ffc448a79bb68" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:52.137041 10324 ts_tablet_manager.cc:1434] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.003s
I20260709 13:06:52.138460 10324 tablet_bootstrap.cc:492] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:52.139513 10325 tablet_bootstrap.cc:492] T d9cb1393c4f549f0bc9b5e82121b543a P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:52.139943 10278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b626433d4aec4288976ffc448a79bb68" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:52.140221 10333 ts_tablet_manager.cc:1403] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.062s	user 0.016s	sys 0.000s
I20260709 13:06:52.141813 10020 leader_election.cc:304] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa338cd132c457896b11e08d7ea787d; no voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:52.140055 10325 ts_tablet_manager.cc:1403] T d9cb1393c4f549f0bc9b5e82121b543a P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.048s	user 0.011s	sys 0.023s
I20260709 13:06:52.142719 10433 raft_consensus.cc:2749] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.144155 10333 raft_consensus.cc:359] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:52.144965 10333 raft_consensus.cc:385] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.145272 10333 raft_consensus.cc:740] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.145357 10325 raft_consensus.cc:359] T d9cb1393c4f549f0bc9b5e82121b543a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:52.146165 10325 raft_consensus.cc:385] T d9cb1393c4f549f0bc9b5e82121b543a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.146530 10325 raft_consensus.cc:740] T d9cb1393c4f549f0bc9b5e82121b543a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.146647 10324 tablet_bootstrap.cc:654] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.147126 10055 raft_consensus.cc:2468] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:52.147364 10325 consensus_queue.cc:260] T d9cb1393c4f549f0bc9b5e82121b543a P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:52.139969 10328 raft_consensus.cc:359] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:52.148722 10328 raft_consensus.cc:385] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.149035 10328 raft_consensus.cc:740] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.150211 10347 leader_election.cc:290] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:52.150763 10328 consensus_queue.cc:260] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:52.151513 10172 leader_election.cc:304] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:52.153265 10328 ts_tablet_manager.cc:1434] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:06:52.153419 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f0446529cd446d0ba97ee860c6ae221" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:52.154062 10132 raft_consensus.cc:3060] T 7f0446529cd446d0ba97ee860c6ae221 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.156165 10325 ts_tablet_manager.cc:1434] T d9cb1393c4f549f0bc9b5e82121b543a P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:06:52.158586 10328 tablet_bootstrap.cc:492] T b41d6a8939d7411e901d88f31d2bdf38 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:52.159674 10325 tablet_bootstrap.cc:492] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:52.160121 10347 raft_consensus.cc:2804] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.163477 10132 raft_consensus.cc:2468] T 7f0446529cd446d0ba97ee860c6ae221 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:52.164278 10347 raft_consensus.cc:697] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:06:52.171934 10333 consensus_queue.cc:260] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:52.174198 10333 ts_tablet_manager.cc:1434] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.001s
I20260709 13:06:52.175182 10333 tablet_bootstrap.cc:492] T 375f151edb2245daa6d1b06187f21b79 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:52.177791 10423 raft_consensus.cc:493] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.180064 10423 raft_consensus.cc:515] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:52.186493 10430 consensus_queue.cc:1048] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.187870 10333 tablet_bootstrap.cc:654] T 375f151edb2245daa6d1b06187f21b79 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.192581 10423 leader_election.cc:290] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:52.194199 10278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9cb1393c4f549f0bc9b5e82121b543a" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:52.194916 10278 raft_consensus.cc:2468] T d9cb1393c4f549f0bc9b5e82121b543a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 0.
I20260709 13:06:52.197672 10325 tablet_bootstrap.cc:654] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.198693 10347 consensus_queue.cc:237] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.202106 10095 leader_election.cc:304] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:06:52.203797 10423 raft_consensus.cc:2804] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.204432 10423 raft_consensus.cc:493] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.203621  9954 catalog_manager.cc:5697] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 0 to 1, leader changed from <none> to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.204947 10423 raft_consensus.cc:3060] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.212324 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9cb1393c4f549f0bc9b5e82121b543a" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:52.213330 10203 raft_consensus.cc:2468] T d9cb1393c4f549f0bc9b5e82121b543a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 0.
I20260709 13:06:52.221213 10333 tablet_bootstrap.cc:492] T 375f151edb2245daa6d1b06187f21b79 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:52.221740 10333 ts_tablet_manager.cc:1403] T 375f151edb2245daa6d1b06187f21b79 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.016s
I20260709 13:06:52.222234 10324 tablet_bootstrap.cc:492] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:52.222910 10324 ts_tablet_manager.cc:1403] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.085s	user 0.016s	sys 0.008s
I20260709 13:06:52.228223 10328 tablet_bootstrap.cc:654] T b41d6a8939d7411e901d88f31d2bdf38 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.232745 10324 raft_consensus.cc:359] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.237236 10324 raft_consensus.cc:385] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.237703 10324 raft_consensus.cc:740] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.238015 10490 raft_consensus.cc:493] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.238869 10490 raft_consensus.cc:515] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:52.241832 10490 leader_election.cc:290] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:52.242764 10324 consensus_queue.cc:260] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.243546 10446 consensus_queue.cc:1048] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:52.245803 10324 ts_tablet_manager.cc:1434] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 13:06:52.249614 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c136ae5f6aa4055a44f8aec8883b02a" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:52.250290 10055 raft_consensus.cc:2468] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:52.251976 10246 leader_election.cc:304] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:52.252084 10423 raft_consensus.cc:515] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:52.252712 10325 tablet_bootstrap.cc:492] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:52.253266 10325 ts_tablet_manager.cc:1403] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.094s	user 0.020s	sys 0.016s
I20260709 13:06:52.256150 10325 raft_consensus.cc:359] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.256788 10325 raft_consensus.cc:385] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.257036 10325 raft_consensus.cc:740] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.246675 10457 raft_consensus.cc:493] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.257581 10333 raft_consensus.cc:359] T 375f151edb2245daa6d1b06187f21b79 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:52.258220 10490 raft_consensus.cc:2804] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.258692 10333 raft_consensus.cc:385] T 375f151edb2245daa6d1b06187f21b79 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.258415 10457 raft_consensus.cc:515] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.258805 10490 raft_consensus.cc:493] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.259488 10490 raft_consensus.cc:3060] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.260664 10457 leader_election.cc:290] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:52.261147 10423 leader_election.cc:290] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:52.259033 10333 raft_consensus.cc:740] T 375f151edb2245daa6d1b06187f21b79 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.257990 10325 consensus_queue.cc:260] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.263412 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41d6a8939d7411e901d88f31d2bdf38" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:52.264165 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41d6a8939d7411e901d88f31d2bdf38" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:52.265621 10056 raft_consensus.cc:3060] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.265796 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9cb1393c4f549f0bc9b5e82121b543a" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
W20260709 13:06:52.266496 10169 leader_election.cc:343] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.266511 10277 raft_consensus.cc:3060] T d9cb1393c4f549f0bc9b5e82121b543a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.272221 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9cb1393c4f549f0bc9b5e82121b543a" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:52.272923 10205 raft_consensus.cc:3060] T d9cb1393c4f549f0bc9b5e82121b543a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.264081 10055 raft_consensus.cc:2468] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:52.274134 10277 raft_consensus.cc:2468] T d9cb1393c4f549f0bc9b5e82121b543a P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
I20260709 13:06:52.275372 10172 leader_election.cc:304] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 50999113203c482b9b9300caa485f154
I20260709 13:06:52.272832 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c136ae5f6aa4055a44f8aec8883b02a" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:52.286950 10446 raft_consensus.cc:2804] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.283982 10333 consensus_queue.cc:260] T 375f151edb2245daa6d1b06187f21b79 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:52.287398 10446 raft_consensus.cc:493] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.287683 10328 tablet_bootstrap.cc:492] T b41d6a8939d7411e901d88f31d2bdf38 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:52.287837 10446 raft_consensus.cc:3060] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.288188 10328 ts_tablet_manager.cc:1403] T b41d6a8939d7411e901d88f31d2bdf38 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.130s	user 0.011s	sys 0.006s
I20260709 13:06:52.290385 10325 ts_tablet_manager.cc:1434] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.001s
I20260709 13:06:52.291178 10333 ts_tablet_manager.cc:1434] T 375f151edb2245daa6d1b06187f21b79 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.069s	user 0.002s	sys 0.004s
I20260709 13:06:52.290899 10490 raft_consensus.cc:515] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:52.291509 10325 tablet_bootstrap.cc:492] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:52.291920 10324 tablet_bootstrap.cc:492] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:52.292356 10095 leader_election.cc:304] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
W20260709 13:06:52.296113 10243 leader_election.cc:343] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.316077 10324 tablet_bootstrap.cc:654] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.295763 10333 tablet_bootstrap.cc:492] T 5c136ae5f6aa4055a44f8aec8883b02a P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:52.320214 10493 consensus_queue.cc:1048] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:52.323818 10423 raft_consensus.cc:2804] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.324311 10423 raft_consensus.cc:697] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Becoming Leader. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:06:52.325135 10423 consensus_queue.cc:237] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:52.302335 10446 raft_consensus.cc:515] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.329263 10325 tablet_bootstrap.cc:654] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.287007 10205 raft_consensus.cc:2468] T d9cb1393c4f549f0bc9b5e82121b543a P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
I20260709 13:06:52.329746 10457 consensus_queue.cc:1048] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:52.331955 10333 tablet_bootstrap.cc:654] T 5c136ae5f6aa4055a44f8aec8883b02a P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.338812 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c136ae5f6aa4055a44f8aec8883b02a" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:52.339172 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c136ae5f6aa4055a44f8aec8883b02a" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:52.339450 10055 raft_consensus.cc:3060] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.350128 10328 raft_consensus.cc:359] T b41d6a8939d7411e901d88f31d2bdf38 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.350708 10328 raft_consensus.cc:385] T b41d6a8939d7411e901d88f31d2bdf38 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.350945 10328 raft_consensus.cc:740] T b41d6a8939d7411e901d88f31d2bdf38 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.351572 10328 consensus_queue.cc:260] T b41d6a8939d7411e901d88f31d2bdf38 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.353390 10328 ts_tablet_manager.cc:1434] T b41d6a8939d7411e901d88f31d2bdf38 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.065s	user 0.005s	sys 0.000s
I20260709 13:06:52.354805 10328 tablet_bootstrap.cc:492] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:52.361171 10328 tablet_bootstrap.cc:654] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.357995  9954 catalog_manager.cc:5697] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 reported cstate change: term changed from 0 to 1, leader changed from <none> to 575e44d297f045a389903d65a471d548 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "575e44d297f045a389903d65a471d548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:52.376312 10243 leader_election.cc:343] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.384574 10429 raft_consensus.cc:493] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.385154 10429 raft_consensus.cc:515] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.385787 10347 raft_consensus.cc:493] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.386272 10347 raft_consensus.cc:515] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.387310 10429 leader_election.cc:290] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:52.379128 10446 leader_election.cc:290] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Requested vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:52.388315 10347 leader_election.cc:290] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:52.389554 10324 tablet_bootstrap.cc:492] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:52.390051 10324 ts_tablet_manager.cc:1403] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.107s	user 0.005s	sys 0.007s
I20260709 13:06:52.390585 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41d6a8939d7411e901d88f31d2bdf38" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:52.391235 10277 raft_consensus.cc:3060] T b41d6a8939d7411e901d88f31d2bdf38 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.390995  9953 catalog_manager.cc:5697] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:52.392720 10324 raft_consensus.cc:359] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.393374 10324 raft_consensus.cc:385] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.393376 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64c791e7fb4a79bd498140ef0b0943" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:52.395432 10052 raft_consensus.cc:3060] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.397240 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e66a35e846da4843bda7db1f5fca14b6" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
W20260709 13:06:52.412747 10172 leader_election.cc:343] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.398451 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f64c791e7fb4a79bd498140ef0b0943" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:52.413757 10129 raft_consensus.cc:2393] T 5f64c791e7fb4a79bd498140ef0b0943 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa338cd132c457896b11e08d7ea787d in current term 1: Already voted for candidate 8fe33410d51a469e81be6edc602d7d78 in this term.
I20260709 13:06:52.399780 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41d6a8939d7411e901d88f31d2bdf38" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:52.400460 10325 tablet_bootstrap.cc:492] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:52.414996 10053 raft_consensus.cc:3060] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.415184 10325 ts_tablet_manager.cc:1403] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.124s	user 0.019s	sys 0.021s
I20260709 13:06:52.418076 10325 raft_consensus.cc:359] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:52.404736 10278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e66a35e846da4843bda7db1f5fca14b6" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:52.418773 10325 raft_consensus.cc:385] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.419103 10325 raft_consensus.cc:740] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.405926 10055 raft_consensus.cc:2468] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:52.419919 10325 consensus_queue.cc:260] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:52.407929 10347 consensus_queue.cc:1048] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.408347 10493 consensus_queue.cc:1048] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.422161 10325 ts_tablet_manager.cc:1434] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:52.423305 10325 tablet_bootstrap.cc:492] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:52.424422 10053 raft_consensus.cc:2468] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
W20260709 13:06:52.419922 10169 leader_election.cc:343] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.425465 10169 leader_election.cc:304] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:52.418751 10324 raft_consensus.cc:740] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.418247 10020 leader_election.cc:304] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa338cd132c457896b11e08d7ea787d; no voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:52.426815 10324 consensus_queue.cc:260] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.407665 10277 raft_consensus.cc:2468] T b41d6a8939d7411e901d88f31d2bdf38 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:52.344451 10490 leader_election.cc:290] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:52.437186 10324 ts_tablet_manager.cc:1434] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.000s
I20260709 13:06:52.433276 10347 raft_consensus.cc:2749] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.434211 10446 raft_consensus.cc:493] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.451118 10328 tablet_bootstrap.cc:492] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:52.451697 10328 ts_tablet_manager.cc:1403] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.097s	user 0.012s	sys 0.008s
I20260709 13:06:52.454922 10324 tablet_bootstrap.cc:492] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:52.428294 10246 leader_election.cc:304] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 575e44d297f045a389903d65a471d548
I20260709 13:06:52.449771 10421 raft_consensus.cc:493] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.464854 10490 raft_consensus.cc:2804] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.465772 10490 raft_consensus.cc:697] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:06:52.459940 10433 raft_consensus.cc:2749] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.453958 10446 raft_consensus.cc:515] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:52.467095 10490 consensus_queue.cc:237] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:52.468778 10333 tablet_bootstrap.cc:492] T 5c136ae5f6aa4055a44f8aec8883b02a P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:52.468961 10446 leader_election.cc:290] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:52.469295 10333 ts_tablet_manager.cc:1403] T 5c136ae5f6aa4055a44f8aec8883b02a P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.177s	user 0.007s	sys 0.009s
I20260709 13:06:52.465427 10421 raft_consensus.cc:515] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:52.459009 10169 leader_election.cc:304] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:52.481245 10333 raft_consensus.cc:359] T 5c136ae5f6aa4055a44f8aec8883b02a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:52.482085 10333 raft_consensus.cc:385] T 5c136ae5f6aa4055a44f8aec8883b02a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.482336 10333 raft_consensus.cc:740] T 5c136ae5f6aa4055a44f8aec8883b02a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.483224 10333 consensus_queue.cc:260] T 5c136ae5f6aa4055a44f8aec8883b02a P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:52.487246 10446 raft_consensus.cc:2804] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.490455 10130 raft_consensus.cc:3060] T 375f151edb2245daa6d1b06187f21b79 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.494288 10421 leader_election.cc:290] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:52.494185 10457 consensus_queue.cc:1048] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.494418 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eeb5566a2f4fa4af640a137fe578df" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:52.495158 10205 raft_consensus.cc:2468] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 0.
I20260709 13:06:52.497032 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eeb5566a2f4fa4af640a137fe578df" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:52.490540 10446 raft_consensus.cc:697] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:06:52.497931 10328 raft_consensus.cc:359] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.498679 10328 raft_consensus.cc:385] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.499147 10328 raft_consensus.cc:740] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.499850 10328 consensus_queue.cc:260] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.500439 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eeb5566a2f4fa4af640a137fe578df" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
W20260709 13:06:52.498752 10095 leader_election.cc:343] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.501919 10328 ts_tablet_manager.cc:1434] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.000s
I20260709 13:06:52.502046 10095 leader_election.cc:304] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 50999113203c482b9b9300caa485f154
I20260709 13:06:52.502966 10328 tablet_bootstrap.cc:492] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:52.503710 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eeb5566a2f4fa4af640a137fe578df" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:52.504354 10131 raft_consensus.cc:2468] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:52.507284 10333 ts_tablet_manager.cc:1434] T 5c136ae5f6aa4055a44f8aec8883b02a P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.001s
W20260709 13:06:52.508482 10169 leader_election.cc:343] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.509143 10333 tablet_bootstrap.cc:492] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:52.509490 10328 tablet_bootstrap.cc:654] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.509974 10421 raft_consensus.cc:2804] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.498378 10446 consensus_queue.cc:237] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.509243 10169 leader_election.cc:304] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 50999113203c482b9b9300caa485f154
I20260709 13:06:52.519439 10457 raft_consensus.cc:2804] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.519846 10457 raft_consensus.cc:493] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.520236 10457 raft_consensus.cc:3060] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.510421 10421 raft_consensus.cc:493] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.528970 10421 raft_consensus.cc:3060] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.529752 10333 tablet_bootstrap.cc:654] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.533041 10324 tablet_bootstrap.cc:654] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.551860 10131 raft_consensus.cc:3060] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.573995 10333 tablet_bootstrap.cc:492] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:52.574596 10333 ts_tablet_manager.cc:1403] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.066s	user 0.019s	sys 0.004s
I20260709 13:06:52.581094 10333 raft_consensus.cc:359] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.581815 10333 raft_consensus.cc:385] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.582098 10333 raft_consensus.cc:740] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.582849 10333 consensus_queue.cc:260] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.583854 10325 tablet_bootstrap.cc:654] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.583734 10457 raft_consensus.cc:515] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:52.584815 10333 ts_tablet_manager.cc:1434] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:06:52.586151 10333 tablet_bootstrap.cc:492] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:52.596822 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eeb5566a2f4fa4af640a137fe578df" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:52.597805 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eeb5566a2f4fa4af640a137fe578df" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
W20260709 13:06:52.598526 10169 leader_election.cc:343] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.599129 10457 leader_election.cc:290] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:52.599515 10328 tablet_bootstrap.cc:492] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:52.600543 10490 raft_consensus.cc:493] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.601123 10423 raft_consensus.cc:493] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.601740 10423 raft_consensus.cc:515] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:52.602722 10433 consensus_queue.cc:1048] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:52.602873 10421 raft_consensus.cc:515] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:52.601078 10490 raft_consensus.cc:515] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.605700 10127 raft_consensus.cc:2393] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fe33410d51a469e81be6edc602d7d78 in current term 1: Already voted for candidate 575e44d297f045a389903d65a471d548 in this term.
I20260709 13:06:52.607458 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eeb5566a2f4fa4af640a137fe578df" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:52.607899 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e66a35e846da4843bda7db1f5fca14b6" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:52.608093 10421 leader_election.cc:290] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:52.608301 10205 raft_consensus.cc:2393] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 575e44d297f045a389903d65a471d548 in current term 1: Already voted for candidate 8fe33410d51a469e81be6edc602d7d78 in this term.
I20260709 13:06:52.608677 10203 raft_consensus.cc:2468] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:52.609983 10333 tablet_bootstrap.cc:654] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.611486 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e66a35e846da4843bda7db1f5fca14b6" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:52.603892 10328 ts_tablet_manager.cc:1403] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.101s	user 0.013s	sys 0.010s
I20260709 13:06:52.613377  9953 catalog_manager.cc:5697] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.624858 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eeb5566a2f4fa4af640a137fe578df" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:52.625602 10243 leader_election.cc:304] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:52.616586 10490 leader_election.cc:290] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:52.626533 10493 raft_consensus.cc:2804] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.627166 10493 raft_consensus.cc:493] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.627713 10493 raft_consensus.cc:3060] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.628847 10324 tablet_bootstrap.cc:492] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:52.629583 10324 ts_tablet_manager.cc:1403] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.175s	user 0.011s	sys 0.006s
I20260709 13:06:52.636163 10423 leader_election.cc:290] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:52.639765 10278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50256706bf3a4bf29b64ea4ea6583da3" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:52.640484 10278 raft_consensus.cc:2468] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
W20260709 13:06:52.630196 10246 leader_election.cc:343] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.643237 10095 leader_election.cc:304] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:06:52.627774 10169 leader_election.cc:304] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548
I20260709 13:06:52.644378 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50256706bf3a4bf29b64ea4ea6583da3" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:52.646157 10451 raft_consensus.cc:2749] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.649606 10423 raft_consensus.cc:2804] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:52.652848 10423 raft_consensus.cc:493] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:52.649952 10095 leader_election.cc:343] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.653482 10423 raft_consensus.cc:3060] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:52.653905 10095 leader_election.cc:304] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:52.649240 10328 raft_consensus.cc:359] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:52.655566 10328 raft_consensus.cc:385] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.655861 10328 raft_consensus.cc:740] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.651166 10324 raft_consensus.cc:359] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.659579 10328 consensus_queue.cc:260] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:52.659533 10324 raft_consensus.cc:385] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.661053 10324 raft_consensus.cc:740] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.667672 10493 raft_consensus.cc:515] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.670400 10535 raft_consensus.cc:2749] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.671216 10433 consensus_queue.cc:1048] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:52.673177  9954 catalog_manager.cc:5697] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.675271 10423 raft_consensus.cc:515] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:52.666688 10324 consensus_queue.cc:260] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.676496 10129 raft_consensus.cc:1240] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Rejecting Update request from peer efa338cd132c457896b11e08d7ea787d for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:06:52.679661 10325 tablet_bootstrap.cc:492] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:52.679811 10328 ts_tablet_manager.cc:1434] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.066s	user 0.003s	sys 0.004s
I20260709 13:06:52.680202 10325 ts_tablet_manager.cc:1403] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.257s	user 0.009s	sys 0.019s
I20260709 13:06:52.680917 10328 tablet_bootstrap.cc:492] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:52.686198 10433 consensus_queue.cc:1059] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.687142 10433 raft_consensus.cc:3055] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:52.687455 10433 raft_consensus.cc:740] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:06:52.688270 10433 consensus_queue.cc:260] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:52.689411 10433 raft_consensus.cc:3060] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:52.690300 10324 ts_tablet_manager.cc:1434] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.060s	user 0.006s	sys 0.000s
I20260709 13:06:52.692811 10325 raft_consensus.cc:359] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.693775 10325 raft_consensus.cc:385] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.694139 10325 raft_consensus.cc:740] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.688251 10328 tablet_bootstrap.cc:654] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.697046 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e66a35e846da4843bda7db1f5fca14b6" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:52.697732 10203 raft_consensus.cc:3060] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.697851 10493 leader_election.cc:290] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:52.698539 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e66a35e846da4843bda7db1f5fca14b6" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
W20260709 13:06:52.700551 10246 leader_election.cc:343] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.693297 10324 tablet_bootstrap.cc:492] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:52.698923 10325 consensus_queue.cc:260] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.702380 10457 consensus_queue.cc:1048] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:52.702869 10325 ts_tablet_manager.cc:1434] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 13:06:52.703843 10325 tablet_bootstrap.cc:492] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:52.709105 10203 raft_consensus.cc:2468] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:52.711879 10328 tablet_bootstrap.cc:492] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:52.730423 10328 ts_tablet_manager.cc:1403] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.050s	user 0.007s	sys 0.006s
I20260709 13:06:52.732506 10534 consensus_queue.cc:1048] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.733837 10243 leader_election.cc:304] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:06:52.712728 10433 raft_consensus.cc:493] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.716590 10325 tablet_bootstrap.cc:654] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.735072 10433 raft_consensus.cc:515] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.737341 10433 leader_election.cc:290] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:52.738306 10328 raft_consensus.cc:359] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.738657 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d98d0921054d6a932a308e4adb1937" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:52.739042 10328 raft_consensus.cc:385] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.739372 10328 raft_consensus.cc:740] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.739970 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d98d0921054d6a932a308e4adb1937" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:52.740661 10130 raft_consensus.cc:2468] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 0.
I20260709 13:06:52.740267 10328 consensus_queue.cc:260] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.720510 10423 leader_election.cc:290] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 election: Requested vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:52.724282 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50256706bf3a4bf29b64ea4ea6583da3" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:52.742280 10052 raft_consensus.cc:2410] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 575e44d297f045a389903d65a471d548 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:06:52.722404 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50256706bf3a4bf29b64ea4ea6583da3" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:52.742933 10328 ts_tablet_manager.cc:1434] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 13:06:52.743955 10328 tablet_bootstrap.cc:492] T 3666fc7cbb0d4f698fb337ad4c238c38 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:52.745685 10020 leader_election.cc:304] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:52.746739 10098 leader_election.cc:304] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:52.747917 10534 raft_consensus.cc:2749] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:52.726922 10324 tablet_bootstrap.cc:654] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.728384 10333 tablet_bootstrap.cc:492] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:52.749462 10333 ts_tablet_manager.cc:1403] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.164s	user 0.009s	sys 0.008s
I20260709 13:06:52.750304 10429 raft_consensus.cc:2804] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.750749 10429 raft_consensus.cc:493] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.751080 10429 raft_consensus.cc:3060] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.752249 10333 raft_consensus.cc:359] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.752966 10333 raft_consensus.cc:385] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.753238 10333 raft_consensus.cc:740] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.734606 10493 raft_consensus.cc:2804] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.753911 10493 raft_consensus.cc:697] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:06:52.754788 10493 consensus_queue.cc:237] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
W20260709 13:06:52.756151 10020 leader_election.cc:343] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.759213 10333 consensus_queue.cc:260] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.762185 10333 ts_tablet_manager.cc:1434] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:06:52.763396 10333 tablet_bootstrap.cc:492] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:52.775487 10325 tablet_bootstrap.cc:492] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:52.776010 10325 ts_tablet_manager.cc:1403] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.072s	user 0.011s	sys 0.000s
I20260709 13:06:52.778821 10325 raft_consensus.cc:359] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.779548 10325 raft_consensus.cc:385] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.779830 10325 raft_consensus.cc:740] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.780277 10534 consensus_queue.cc:1048] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:52.781037 10325 consensus_queue.cc:260] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.786981 10325 ts_tablet_manager.cc:1434] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:06:52.788040 10325 tablet_bootstrap.cc:492] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:52.789019 10324 tablet_bootstrap.cc:492] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:52.789623 10324 ts_tablet_manager.cc:1403] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.097s	user 0.009s	sys 0.000s
I20260709 13:06:52.791407 10347 consensus_queue.cc:1048] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:52.799400 10535 raft_consensus.cc:493] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.811605 10535 raft_consensus.cc:515] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.815446 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec007b8bb1e4c69a49a9ba8aeb56eee" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:52.816243 10052 raft_consensus.cc:2468] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 0.
I20260709 13:06:52.817510 10098 leader_election.cc:304] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:52.818749 10452 raft_consensus.cc:493] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.819366 10535 leader_election.cc:290] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:52.819309 10452 raft_consensus.cc:515] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.820654 10325 tablet_bootstrap.cc:654] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.818744 10423 raft_consensus.cc:2804] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.821165 10423 raft_consensus.cc:493] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.821499 10423 raft_consensus.cc:3060] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.821606 10452 leader_election.cc:290] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:52.822806 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d98d0921054d6a932a308e4adb1937" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:52.829298 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d98d0921054d6a932a308e4adb1937" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:52.830082 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec007b8bb1e4c69a49a9ba8aeb56eee" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:52.830152 10128 raft_consensus.cc:2468] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:52.831578 10169 leader_election.cc:304] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:52.832580 10347 raft_consensus.cc:2804] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.833045 10347 raft_consensus.cc:493] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.833444 10347 raft_consensus.cc:3060] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.835777 10429 raft_consensus.cc:515] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.837185 10056 raft_consensus.cc:2393] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fe33410d51a469e81be6edc602d7d78 in current term 1: Already voted for candidate efa338cd132c457896b11e08d7ea787d in this term.
W20260709 13:06:52.838155 10095 leader_election.cc:343] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.842394 10328 tablet_bootstrap.cc:654] T 3666fc7cbb0d4f698fb337ad4c238c38 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.792611 10324 raft_consensus.cc:359] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.847227 10324 raft_consensus.cc:385] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.847489 10324 raft_consensus.cc:740] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.848193 10324 consensus_queue.cc:260] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:52.849150 10333 tablet_bootstrap.cc:654] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.860378 10423 raft_consensus.cc:515] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.869822 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec007b8bb1e4c69a49a9ba8aeb56eee" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:52.870210 10535 raft_consensus.cc:493] T 5c136ae5f6aa4055a44f8aec8883b02a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.871040 10056 raft_consensus.cc:3060] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.870875 10535 raft_consensus.cc:515] T 5c136ae5f6aa4055a44f8aec8883b02a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:52.873420 10535 leader_election.cc:290] T 5c136ae5f6aa4055a44f8aec8883b02a P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:52.863492 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d98d0921054d6a932a308e4adb1937" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:52.874751 10128 raft_consensus.cc:3060] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.877295 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c136ae5f6aa4055a44f8aec8883b02a" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:52.880362 10429 leader_election.cc:290] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:52.883960 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d98d0921054d6a932a308e4adb1937" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:52.885924 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c136ae5f6aa4055a44f8aec8883b02a" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:52.886854 10053 raft_consensus.cc:2393] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 575e44d297f045a389903d65a471d548 in current term 1: Already voted for candidate 50999113203c482b9b9300caa485f154 in this term.
I20260709 13:06:52.887852 10098 leader_election.cc:304] T 5c136ae5f6aa4055a44f8aec8883b02a P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:52.889602 10535 raft_consensus.cc:3060] T 5c136ae5f6aa4055a44f8aec8883b02a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.891620 10128 raft_consensus.cc:2468] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:06:52.864152 10324 ts_tablet_manager.cc:1434] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.074s	user 0.006s	sys 0.000s
I20260709 13:06:52.894162 10324 tablet_bootstrap.cc:492] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:52.894447 10020 leader_election.cc:304] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:52.895407 10354 raft_consensus.cc:2804] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.895929 10354 raft_consensus.cc:697] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:06:52.896762 10354 consensus_queue.cc:237] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.866346  9954 catalog_manager.cc:5697] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.862373 10347 raft_consensus.cc:515] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:52.907791 10203 raft_consensus.cc:2393] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate efa338cd132c457896b11e08d7ea787d in current term 1: Already voted for candidate 8fe33410d51a469e81be6edc602d7d78 in this term.
I20260709 13:06:52.909767 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d98d0921054d6a932a308e4adb1937" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:52.912683 10325 tablet_bootstrap.cc:492] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:52.913432 10325 ts_tablet_manager.cc:1403] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.126s	user 0.010s	sys 0.001s
I20260709 13:06:52.919049 10347 leader_election.cc:290] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:52.918166 10423 leader_election.cc:290] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:52.934695 10333 tablet_bootstrap.cc:492] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:52.935315 10333 ts_tablet_manager.cc:1403] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.172s	user 0.013s	sys 0.000s
I20260709 13:06:52.922946 10325 raft_consensus.cc:359] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:52.946265 10325 raft_consensus.cc:385] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.946638 10325 raft_consensus.cc:740] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.947326 10325 consensus_queue.cc:260] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:53.053072 10333 raft_consensus.cc:359] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.053824 10333 raft_consensus.cc:385] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.054211 10333 raft_consensus.cc:740] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.055253 10333 consensus_queue.cc:260] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.057565 10333 ts_tablet_manager.cc:1434] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.122s	user 0.011s	sys 0.000s
I20260709 13:06:53.059077 10333 tablet_bootstrap.cc:492] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:53.065150 10333 tablet_bootstrap.cc:654] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.074478 10333 tablet_bootstrap.cc:492] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:53.074999 10333 ts_tablet_manager.cc:1403] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.004s
I20260709 13:06:53.078717 10333 raft_consensus.cc:359] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:53.081713 10333 raft_consensus.cc:385] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.082054 10333 raft_consensus.cc:740] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.082228 10493 raft_consensus.cc:493] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.082868 10493 raft_consensus.cc:515] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:53.083003 10333 consensus_queue.cc:260] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:53.084911 10493 leader_election.cc:290] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:53.085822 10333 ts_tablet_manager.cc:1434] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:06:53.086946 10333 tablet_bootstrap.cc:492] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:53.087568 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d98d0921054d6a932a308e4adb1937" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:53.088238 10130 raft_consensus.cc:2393] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fe33410d51a469e81be6edc602d7d78 in current term 1: Already voted for candidate efa338cd132c457896b11e08d7ea787d in this term.
I20260709 13:06:53.089625 10056 raft_consensus.cc:2468] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
I20260709 13:06:53.090971 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec007b8bb1e4c69a49a9ba8aeb56eee" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:53.091929 10098 leader_election.cc:304] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d; no voters: 
W20260709 13:06:53.092617 10095 leader_election.cc:343] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.092868 10423 raft_consensus.cc:2804] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.093441 10423 raft_consensus.cc:697] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Becoming Leader. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:06:53.094020 10534 raft_consensus.cc:493] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.094364 10423 consensus_queue.cc:237] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:53.094645 10534 raft_consensus.cc:515] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.095186 10333 tablet_bootstrap.cc:654] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.095011 10324 tablet_bootstrap.cc:700] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d: Time spent opening tablet: real 0.172s	user 0.002s	sys 0.002s
I20260709 13:06:53.096019 10324 tablet_bootstrap.cc:654] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.097772 10534 leader_election.cc:290] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:53.104319 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd93f389d32493b9dcc10e602f452da" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
W20260709 13:06:53.106762 10098 leader_election.cc:343] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.112082 10333 tablet_bootstrap.cc:492] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:53.112691 10333 ts_tablet_manager.cc:1403] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.000s
I20260709 13:06:53.114183 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eeb5566a2f4fa4af640a137fe578df" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:53.115456 10130 raft_consensus.cc:2393] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50999113203c482b9b9300caa485f154 in current term 1: Already voted for candidate 575e44d297f045a389903d65a471d548 in this term.
I20260709 13:06:53.115415 10333 raft_consensus.cc:359] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.116350 10333 raft_consensus.cc:385] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.116669 10333 raft_consensus.cc:740] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.117401 10333 consensus_queue.cc:260] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.118930 10493 raft_consensus.cc:493] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.119896 10493 raft_consensus.cc:515] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:53.120337 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eeb5566a2f4fa4af640a137fe578df" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:53.121266 10205 raft_consensus.cc:2393] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50999113203c482b9b9300caa485f154 in current term 1: Already voted for candidate 8fe33410d51a469e81be6edc602d7d78 in this term.
I20260709 13:06:53.122936 10493 leader_election.cc:290] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:53.123991 10333 ts_tablet_manager.cc:1434] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:06:53.125138 10333 tablet_bootstrap.cc:492] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:53.129911 10324 tablet_bootstrap.cc:492] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:53.129699 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd93f389d32493b9dcc10e602f452da" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:53.132226 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccaebf4041f14bada42fd7122f34419c" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:53.132252 10333 tablet_bootstrap.cc:654] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:53.133430 10095 leader_election.cc:343] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.133944 10095 leader_election.cc:304] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:53.133350 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccaebf4041f14bada42fd7122f34419c" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
W20260709 13:06:53.134604 10243 leader_election.cc:343] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.135188 10423 raft_consensus.cc:2749] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:53.135601 10246 leader_election.cc:343] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.135375 10243 leader_election.cc:304] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:53.136123 10246 leader_election.cc:304] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:53.130559 10324 ts_tablet_manager.cc:1403] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.237s	user 0.020s	sys 0.014s
I20260709 13:06:53.143651 10324 raft_consensus.cc:359] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:53.146171 10324 raft_consensus.cc:385] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.147290 10324 raft_consensus.cc:740] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.152877 10324 consensus_queue.cc:260] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:53.153573 10556 raft_consensus.cc:493] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.173699 10560 consensus_queue.cc:1048] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.175761 10560 raft_consensus.cc:3060] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.175951 10324 ts_tablet_manager.cc:1434] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.002s
I20260709 13:06:53.179131 10564 raft_consensus.cc:2749] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.183239 10324 tablet_bootstrap.cc:492] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
W20260709 13:06:53.183055 10243 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:40419, user_credentials={real_user=slave}} blocked reactor thread for 47759.4us
I20260709 13:06:53.184126 10169 leader_election.cc:304] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:53.161307 10556 raft_consensus.cc:515] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:53.191321 10560 raft_consensus.cc:2749] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.192255 10558 raft_consensus.cc:493] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.192757 10558 raft_consensus.cc:515] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:53.195022 10558 leader_election.cc:290] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
W20260709 13:06:53.185352 10246 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:42425, user_credentials={real_user=slave}} blocked reactor thread for 49792.7us
I20260709 13:06:53.196072 10452 raft_consensus.cc:2749] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.196727 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc1d8020dcd34e63b72a7de2db565ff6" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:53.198796 10556 leader_election.cc:290] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:53.199019 10055 raft_consensus.cc:3060] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.201028 10556 consensus_queue.cc:1048] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:53.202072 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc1d8020dcd34e63b72a7de2db565ff6" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:53.202761 10052 raft_consensus.cc:2468] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:53.204685 10333 tablet_bootstrap.cc:492] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:53.205188 10333 ts_tablet_manager.cc:1403] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.080s	user 0.035s	sys 0.008s
I20260709 13:06:53.208390 10333 raft_consensus.cc:359] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.209048 10333 raft_consensus.cc:385] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.209317 10333 raft_consensus.cc:740] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.209746 10324 tablet_bootstrap.cc:654] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.210906 10333 consensus_queue.cc:260] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.218269 10559 consensus_queue.cc:1048] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.220080 10172 leader_election.cc:304] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:53.218261 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3666fc7cbb0d4f698fb337ad4c238c38" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
W20260709 13:06:53.221067 10169 leader_election.cc:343] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.221285 10452 raft_consensus.cc:2804] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.221803 10452 raft_consensus.cc:493] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.222153 10452 raft_consensus.cc:3060] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.223063 10333 ts_tablet_manager.cc:1434] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.003s
I20260709 13:06:53.224195 10333 tablet_bootstrap.cc:492] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:53.246362 10325 ts_tablet_manager.cc:1434] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.332s	user 0.012s	sys 0.000s
I20260709 13:06:53.247530 10325 tablet_bootstrap.cc:492] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:53.221220 10127 raft_consensus.cc:2468] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:53.256287 10328 tablet_bootstrap.cc:492] T 3666fc7cbb0d4f698fb337ad4c238c38 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:53.256821 10328 ts_tablet_manager.cc:1403] T 3666fc7cbb0d4f698fb337ad4c238c38 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.513s	user 0.019s	sys 0.008s
I20260709 13:06:53.257248 10333 tablet_bootstrap.cc:654] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.264750 10328 raft_consensus.cc:359] T 3666fc7cbb0d4f698fb337ad4c238c38 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:53.265834 10328 raft_consensus.cc:385] T 3666fc7cbb0d4f698fb337ad4c238c38 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.266170 10328 raft_consensus.cc:740] T 3666fc7cbb0d4f698fb337ad4c238c38 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.266968 10328 consensus_queue.cc:260] T 3666fc7cbb0d4f698fb337ad4c238c38 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:53.270860 10452 raft_consensus.cc:515] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:53.273470 10328 ts_tablet_manager.cc:1434] T 3666fc7cbb0d4f698fb337ad4c238c38 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:06:53.274776 10328 tablet_bootstrap.cc:492] T 39e4808ea4be41f483e08e83162c59c8 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:53.275327 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc1d8020dcd34e63b72a7de2db565ff6" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:53.276234 10056 raft_consensus.cc:3060] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.278928 10325 tablet_bootstrap.cc:654] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.309494 10328 tablet_bootstrap.cc:654] T 39e4808ea4be41f483e08e83162c59c8 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:53.312199 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:53.316669 10324 tablet_bootstrap.cc:492] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:53.318209 10324 ts_tablet_manager.cc:1403] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.135s	user 0.013s	sys 0.005s
W20260709 13:06:53.311687 10243 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:33415, user_credentials={real_user=slave}} blocked reactor thread for 95382.6us
I20260709 13:06:53.324244 10535 raft_consensus.cc:2749] T 5c136ae5f6aa4055a44f8aec8883b02a P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.332533 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3666fc7cbb0d4f698fb337ad4c238c38" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:53.333312 10280 raft_consensus.cc:2468] T 3666fc7cbb0d4f698fb337ad4c238c38 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:53.335584 10490 consensus_queue.cc:1048] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:53.339273 10452 leader_election.cc:290] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Requested vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:53.359658 10433 consensus_queue.cc:1048] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:53.368921 10325 tablet_bootstrap.cc:492] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:53.369498 10325 ts_tablet_manager.cc:1403] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.122s	user 0.014s	sys 0.000s
I20260709 13:06:53.375155 10325 raft_consensus.cc:359] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:53.378394 10433 consensus_queue.cc:1048] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:53.343318 10324 raft_consensus.cc:359] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:53.398945 10324 raft_consensus.cc:385] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.399258 10324 raft_consensus.cc:740] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.399964 10324 consensus_queue.cc:260] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:53.402454 10324 ts_tablet_manager.cc:1434] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.084s	user 0.006s	sys 0.001s
I20260709 13:06:53.403473 10324 tablet_bootstrap.cc:492] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:53.405089 10328 tablet_bootstrap.cc:492] T 39e4808ea4be41f483e08e83162c59c8 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:53.405731 10325 raft_consensus.cc:385] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.405882 10328 ts_tablet_manager.cc:1403] T 39e4808ea4be41f483e08e83162c59c8 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.131s	user 0.043s	sys 0.033s
I20260709 13:06:53.406068 10325 raft_consensus.cc:740] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.381443 10056 raft_consensus.cc:2468] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
W20260709 13:06:53.381699 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:40841, user_credentials={real_user=slave}} blocked reactor thread for 155381us
I20260709 13:06:53.387414 10333 tablet_bootstrap.cc:492] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:53.392199 10563 consensus_queue.cc:1048] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.408203 10333 ts_tablet_manager.cc:1403] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.184s	user 0.049s	sys 0.044s
I20260709 13:06:53.410589 10328 raft_consensus.cc:359] T 39e4808ea4be41f483e08e83162c59c8 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.411367 10328 raft_consensus.cc:385] T 39e4808ea4be41f483e08e83162c59c8 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.411746 10328 raft_consensus.cc:740] T 39e4808ea4be41f483e08e83162c59c8 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.411047 10333 raft_consensus.cc:359] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:53.413316 10333 raft_consensus.cc:385] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.413714 10333 raft_consensus.cc:740] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.414654 10333 consensus_queue.cc:260] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:53.413327 10169 leader_election.cc:304] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:53.415946 10452 raft_consensus.cc:2804] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.416332 10452 raft_consensus.cc:493] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.416713 10452 raft_consensus.cc:3060] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.417701 10333 ts_tablet_manager.cc:1434] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:06:53.418888 10333 tablet_bootstrap.cc:492] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:53.422712  9952 catalog_manager.cc:5697] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 reported cstate change: term changed from 0 to 1, leader changed from <none> to 575e44d297f045a389903d65a471d548 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "575e44d297f045a389903d65a471d548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.412503 10328 consensus_queue.cc:260] T 39e4808ea4be41f483e08e83162c59c8 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.425411 10328 ts_tablet_manager.cc:1434] T 39e4808ea4be41f483e08e83162c59c8 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.001s
I20260709 13:06:53.425149 10452 raft_consensus.cc:515] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:53.426602 10328 tablet_bootstrap.cc:492] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:53.427446 10452 leader_election.cc:290] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:53.427801 10333 tablet_bootstrap.cc:654] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.431452 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3666fc7cbb0d4f698fb337ad4c238c38" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:53.432168 10132 raft_consensus.cc:3060] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.432018  9950 catalog_manager.cc:5697] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 0 to 1, leader changed from <none> to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.437263 10328 tablet_bootstrap.cc:654] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.451156 10328 tablet_bootstrap.cc:492] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:53.451884 10328 ts_tablet_manager.cc:1403] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.000s
I20260709 13:06:53.454834 10328 raft_consensus.cc:359] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:53.455579 10328 raft_consensus.cc:385] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.456136 10328 raft_consensus.cc:740] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.456949 10328 consensus_queue.cc:260] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:53.459116 10328 ts_tablet_manager.cc:1434] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:53.460181 10333 tablet_bootstrap.cc:492] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:53.460184 10328 tablet_bootstrap.cc:492] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:53.464859 10333 ts_tablet_manager.cc:1403] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.046s	user 0.015s	sys 0.013s
I20260709 13:06:53.469063 10333 raft_consensus.cc:359] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:53.469789 10333 raft_consensus.cc:385] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.470072 10333 raft_consensus.cc:740] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.470834 10328 tablet_bootstrap.cc:654] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.470714 10333 consensus_queue.cc:260] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:53.472589 10535 raft_consensus.cc:493] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.473158 10333 ts_tablet_manager.cc:1434] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:06:53.473098 10535 raft_consensus.cc:515] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.474215 10333 tablet_bootstrap.cc:492] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:53.476997 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39e4808ea4be41f483e08e83162c59c8" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:53.477825 10280 raft_consensus.cc:2468] T 39e4808ea4be41f483e08e83162c59c8 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 0.
I20260709 13:06:53.479423 10095 leader_election.cc:304] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:06:53.480396 10423 raft_consensus.cc:2804] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.480809 10333 tablet_bootstrap.cc:654] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.480794 10423 raft_consensus.cc:493] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.481293 10423 raft_consensus.cc:3060] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.485780 10535 leader_election.cc:290] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:53.487589 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39e4808ea4be41f483e08e83162c59c8" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
W20260709 13:06:53.489375 10098 leader_election.cc:343] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.492552 10333 tablet_bootstrap.cc:492] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:53.493094 10333 ts_tablet_manager.cc:1403] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.019s	user 0.001s	sys 0.015s
I20260709 13:06:53.495975 10333 raft_consensus.cc:359] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:53.496704 10333 raft_consensus.cc:385] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.496970 10333 raft_consensus.cc:740] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.497887 10333 consensus_queue.cc:260] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:53.500133 10333 ts_tablet_manager.cc:1434] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:06:53.500576 10423 raft_consensus.cc:515] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.501082 10333 tablet_bootstrap.cc:492] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:53.507524 10333 tablet_bootstrap.cc:654] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.508934 10172 leader_election.cc:304] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:53.510325 10451 raft_consensus.cc:2804] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.510957 10451 raft_consensus.cc:697] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:06:53.511973 10451 consensus_queue.cc:237] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:53.515780 10333 tablet_bootstrap.cc:492] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:53.516366 10333 ts_tablet_manager.cc:1403] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.015s	user 0.014s	sys 0.001s
I20260709 13:06:53.519304 10333 raft_consensus.cc:359] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:53.519994 10333 raft_consensus.cc:385] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.520363 10333 raft_consensus.cc:740] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.521140 10333 consensus_queue.cc:260] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:53.522213 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc1d8020dcd34e63b72a7de2db565ff6" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:53.523476 10333 ts_tablet_manager.cc:1434] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:53.524448 10333 tablet_bootstrap.cc:492] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:53.530706 10333 tablet_bootstrap.cc:654] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.536325 10333 tablet_bootstrap.cc:492] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:53.536800 10333 ts_tablet_manager.cc:1403] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.013s	user 0.007s	sys 0.003s
I20260709 13:06:53.539716 10333 raft_consensus.cc:359] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.540472 10333 raft_consensus.cc:385] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.540839 10333 raft_consensus.cc:740] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.541668 10333 consensus_queue.cc:260] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.543743 10333 ts_tablet_manager.cc:1434] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:53.544620 10333 tablet_bootstrap.cc:492] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:53.549080 10132 raft_consensus.cc:2468] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:53.550345 10333 tablet_bootstrap.cc:654] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.550561 10169 leader_election.cc:304] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:53.551504 10451 raft_consensus.cc:2804] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.551998 10451 raft_consensus.cc:697] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:06:53.552843 10451 consensus_queue.cc:237] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:53.559759 10325 consensus_queue.cc:260] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:53.566421 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39e4808ea4be41f483e08e83162c59c8" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:53.572316 10280 raft_consensus.cc:3060] T 39e4808ea4be41f483e08e83162c59c8 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.580611 10325 ts_tablet_manager.cc:1434] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.208s	user 0.006s	sys 0.000s
I20260709 13:06:53.586215 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39e4808ea4be41f483e08e83162c59c8" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:53.656312 10325 tablet_bootstrap.cc:492] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
W20260709 13:06:53.658309 10098 leader_election.cc:343] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.603765 10423 leader_election.cc:290] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Requested vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:53.560776  9952 catalog_manager.cc:5697] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.615003 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3666fc7cbb0d4f698fb337ad4c238c38" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
W20260709 13:06:53.628309 10095 consensus_peers.cc:597] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 -> Peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Couldn't send request to peer 8fe33410d51a469e81be6edc602d7d78. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:53.673557 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:40419, user_credentials={real_user=slave}} blocked reactor thread for 45375.8us
I20260709 13:06:53.673652 10277 raft_consensus.cc:3060] T 3666fc7cbb0d4f698fb337ad4c238c38 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.675204 10328 tablet_bootstrap.cc:492] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:53.675957 10280 raft_consensus.cc:2468] T 39e4808ea4be41f483e08e83162c59c8 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
I20260709 13:06:53.676196 10328 ts_tablet_manager.cc:1403] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.216s	user 0.082s	sys 0.106s
I20260709 13:06:53.677767 10324 tablet_bootstrap.cc:654] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.678989 10095 leader_election.cc:304] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:06:53.629449 10535 consensus_queue.cc:1048] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.680403 10562 raft_consensus.cc:493] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:53.653504 10169 leader_election.cc:343] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.680974 10562 raft_consensus.cc:515] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:53.680320 10545 raft_consensus.cc:2804] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.682525 10545 raft_consensus.cc:697] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Becoming Leader. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:06:53.683184 10562 leader_election.cc:290] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:53.683252 10558 consensus_queue.cc:1048] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.679813 10328 raft_consensus.cc:359] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:53.683504 10545 consensus_queue.cc:237] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.684245 10328 raft_consensus.cc:385] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.684772 10328 raft_consensus.cc:740] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.685797 10328 consensus_queue.cc:260] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:53.688383 10328 ts_tablet_manager.cc:1434] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:06:53.690452 10325 tablet_bootstrap.cc:654] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.693470 10328 tablet_bootstrap.cc:492] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:53.695996 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccaebf4041f14bada42fd7122f34419c" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:53.699712 10333 tablet_bootstrap.cc:492] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:53.700305 10333 ts_tablet_manager.cc:1403] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.156s	user 0.054s	sys 0.074s
I20260709 13:06:53.704656 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccaebf4041f14bada42fd7122f34419c" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:53.705483 10280 raft_consensus.cc:2468] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 0.
I20260709 13:06:53.706187 10328 tablet_bootstrap.cc:654] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.710002 10535 raft_consensus.cc:493] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.710609 10535 raft_consensus.cc:515] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.710911 10457 raft_consensus.cc:493] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.711584 10457 raft_consensus.cc:515] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:53.712893 10535 leader_election.cc:290] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:53.714888 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7364742faea3470694e1a23f151f1b57" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:53.716305 10457 leader_election.cc:290] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:53.717593 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0529bf107d07493eb0984c4cbb9ea01e" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:53.718400 10129 raft_consensus.cc:2468] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
W20260709 13:06:53.720099 10095 leader_election.cc:343] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.730245 10333 raft_consensus.cc:359] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:53.731129 10333 raft_consensus.cc:385] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.731446 10333 raft_consensus.cc:740] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.731848  9950 catalog_manager.cc:5697] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 reported cstate change: term changed from 0 to 1, leader changed from <none> to 575e44d297f045a389903d65a471d548 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "575e44d297f045a389903d65a471d548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.735961 10328 tablet_bootstrap.cc:492] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:53.735280 10333 consensus_queue.cc:260] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:53.736613 10328 ts_tablet_manager.cc:1403] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.043s	user 0.012s	sys 0.005s
I20260709 13:06:53.738834 10333 ts_tablet_manager.cc:1434] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 13:06:53.740684 10328 raft_consensus.cc:359] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.740875 10278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0529bf107d07493eb0984c4cbb9ea01e" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:53.741703 10278 raft_consensus.cc:2468] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:53.742480 10333 tablet_bootstrap.cc:492] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:53.742578 10328 raft_consensus.cc:385] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.743018 10328 raft_consensus.cc:740] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.743654 10328 consensus_queue.cc:260] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
W20260709 13:06:53.744962 10020 leader_election.cc:343] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.750620 10333 tablet_bootstrap.cc:654] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.741613 10169 leader_election.cc:304] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:53.747995 10328 ts_tablet_manager.cc:1434] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.002s
I20260709 13:06:53.751534 10457 raft_consensus.cc:2804] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.751967 10328 tablet_bootstrap.cc:492] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:53.752040 10457 raft_consensus.cc:493] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.752646 10457 raft_consensus.cc:3060] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.758728 10328 tablet_bootstrap.cc:654] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.763223 10423 raft_consensus.cc:493] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.775192 10545 raft_consensus.cc:493] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.776008 10545 raft_consensus.cc:515] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:53.779372 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21c65d190c541ab8e2d6ae0d4be85e7" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:53.780567 10324 tablet_bootstrap.cc:492] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:53.781092 10324 ts_tablet_manager.cc:1403] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.378s	user 0.021s	sys 0.030s
I20260709 13:06:53.783929 10324 raft_consensus.cc:359] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.784708 10324 raft_consensus.cc:385] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.786228 10324 raft_consensus.cc:740] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.787154 10324 consensus_queue.cc:260] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.764618 10020 leader_election.cc:304] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:53.790856 10429 raft_consensus.cc:2804] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.791208 10429 raft_consensus.cc:493] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.791529 10429 raft_consensus.cc:3060] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.792187 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21c65d190c541ab8e2d6ae0d4be85e7" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:53.793973 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7364742faea3470694e1a23f151f1b57" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:53.787907 10423 raft_consensus.cc:515] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
W20260709 13:06:53.788465 10095 leader_election.cc:343] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.796008 10423 leader_election.cc:290] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:53.797462 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc4e649e35ee49a2a224ecc9f962980a" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
W20260709 13:06:53.799101 10098 leader_election.cc:343] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.802832 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc4e649e35ee49a2a224ecc9f962980a" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:53.805353 10324 ts_tablet_manager.cc:1434] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
W20260709 13:06:53.808307 10095 leader_election.cc:343] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.789897 10277 raft_consensus.cc:2468] T 3666fc7cbb0d4f698fb337ad4c238c38 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
W20260709 13:06:53.810832 10095 leader_election.cc:343] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.812847 10095 leader_election.cc:304] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:53.810922 10545 leader_election.cc:290] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:53.814915 10545 raft_consensus.cc:2749] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.801272 10098 leader_election.cc:304] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:53.820326 10457 raft_consensus.cc:515] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:53.821863 10561 raft_consensus.cc:493] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.822350 10561 raft_consensus.cc:515] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.824800 10561 leader_election.cc:290] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:53.828976 10457 leader_election.cc:290] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:53.830241  9952 catalog_manager.cc:5697] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.836125 10423 raft_consensus.cc:2749] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.837940 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd93f389d32493b9dcc10e602f452da" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:53.838666 10132 raft_consensus.cc:2468] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:53.838944 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd93f389d32493b9dcc10e602f452da" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:53.839602 10055 raft_consensus.cc:2468] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:53.840626 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0529bf107d07493eb0984c4cbb9ea01e" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:53.841163 10277 raft_consensus.cc:3060] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.844000 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0529bf107d07493eb0984c4cbb9ea01e" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:53.844628 10132 raft_consensus.cc:3060] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.848448 10277 raft_consensus.cc:2468] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:53.849889 10169 leader_election.cc:304] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:53.850827 10457 raft_consensus.cc:2804] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:06:53.850749 10098 leader_election.cc:343] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.851616 10243 leader_election.cc:304] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:06:53.852313 10457 raft_consensus.cc:697] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:06:53.853264 10457 consensus_queue.cc:237] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:53.853746 10561 raft_consensus.cc:2804] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.854178 10561 raft_consensus.cc:493] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.854600 10561 raft_consensus.cc:3060] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.860944 10132 raft_consensus.cc:2468] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:53.808718 10324 tablet_bootstrap.cc:492] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:53.867036 10333 tablet_bootstrap.cc:492] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:53.876130 10333 ts_tablet_manager.cc:1403] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.134s	user 0.008s	sys 0.012s
I20260709 13:06:53.879092 10333 raft_consensus.cc:359] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.880416 10333 raft_consensus.cc:385] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.879992  9952 catalog_manager.cc:5697] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.880929 10333 raft_consensus.cc:740] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.883968 10564 raft_consensus.cc:493] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.883203 10333 consensus_queue.cc:260] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.884519 10564 raft_consensus.cc:515] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:53.889243 10333 ts_tablet_manager.cc:1434] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:06:53.888515 10429 raft_consensus.cc:515] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:53.894043 10429 leader_election.cc:290] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:53.899641 10328 tablet_bootstrap.cc:492] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:53.900363 10328 ts_tablet_manager.cc:1403] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.149s	user 0.012s	sys 0.008s
I20260709 13:06:53.851672 10098 leader_election.cc:304] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:53.902199 10333 tablet_bootstrap.cc:492] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:53.906177 10325 tablet_bootstrap.cc:492] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:53.906736 10325 ts_tablet_manager.cc:1403] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.251s	user 0.041s	sys 0.067s
I20260709 13:06:53.912420 10324 tablet_bootstrap.cc:654] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.905265 10545 raft_consensus.cc:2749] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.914722 10328 raft_consensus.cc:359] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:53.915779 10328 raft_consensus.cc:385] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.916182 10328 raft_consensus.cc:740] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.914743 10325 raft_consensus.cc:359] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:53.920054 10325 raft_consensus.cc:385] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.920401 10325 raft_consensus.cc:740] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.931851 10328 consensus_queue.cc:260] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:53.933347 10325 consensus_queue.cc:260] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:53.933964 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccaebf4041f14bada42fd7122f34419c" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
W20260709 13:06:53.935457 10020 leader_election.cc:343] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.938665 10325 ts_tablet_manager.cc:1434] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 13:06:53.945762 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc1d8020dcd34e63b72a7de2db565ff6" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:53.949079 10328 ts_tablet_manager.cc:1434] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.048s	user 0.010s	sys 0.002s
W20260709 13:06:53.938660 10098 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:42425, user_credentials={real_user=slave}} blocked reactor thread for 87937us
I20260709 13:06:53.953841 10564 leader_election.cc:290] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:53.956144 10325 tablet_bootstrap.cc:492] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:53.963881 10328 tablet_bootstrap.cc:492] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:53.964771 10333 tablet_bootstrap.cc:654] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.965308 10423 raft_consensus.cc:493] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.965844 10423 raft_consensus.cc:515] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.968219 10423 leader_election.cc:290] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:53.968606 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc1d8020dcd34e63b72a7de2db565ff6" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:53.969448 10056 raft_consensus.cc:2393] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50999113203c482b9b9300caa485f154 in current term 1: Already voted for candidate 8fe33410d51a469e81be6edc602d7d78 in this term.
I20260709 13:06:53.970199 10325 tablet_bootstrap.cc:654] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:53.960892  9986 debug-util.cc:398] Leaking SignalData structure 0x7b08002f1960 after lost signal to thread 434
W20260709 13:06:53.971873  9986 debug-util.cc:398] Leaking SignalData structure 0x7b0800158d80 after lost signal to thread 9989
I20260709 13:06:53.974016 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccaebf4041f14bada42fd7122f34419c" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:53.974774 10277 raft_consensus.cc:3060] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.978811 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75aced19d8c94fc78e5510b592bb01be" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:53.963626 10561 raft_consensus.cc:515] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:53.988371 10561 leader_election.cc:290] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
W20260709 13:06:53.973488  9986 debug-util.cc:398] Leaking SignalData structure 0x7b08000bf280 after lost signal to thread 10081
W20260709 13:06:54.002355  9986 debug-util.cc:398] Leaking SignalData structure 0x7b08001f7860 after lost signal to thread 10156
I20260709 13:06:54.002905 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75aced19d8c94fc78e5510b592bb01be" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
W20260709 13:06:54.003886  9986 debug-util.cc:398] Leaking SignalData structure 0x7b08000b37c0 after lost signal to thread 10230
I20260709 13:06:54.012008 10246 leader_election.cc:304] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
W20260709 13:06:53.999444 10098 leader_election.cc:343] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.013018 10561 raft_consensus.cc:3060] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.013777 10423 raft_consensus.cc:493] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.014317 10423 raft_consensus.cc:515] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:54.014957 10324 tablet_bootstrap.cc:492] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:53.997174 10277 raft_consensus.cc:2468] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:06:54.015481 10324 ts_tablet_manager.cc:1403] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.207s	user 0.025s	sys 0.013s
I20260709 13:06:54.016434 10423 leader_election.cc:290] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:54.017889 10020 leader_election.cc:304] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:54.020875 10561 raft_consensus.cc:2749] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.023494 10324 raft_consensus.cc:359] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:54.024242 10324 raft_consensus.cc:385] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.024518 10324 raft_consensus.cc:740] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
W20260709 13:06:54.026291  9986 debug-util.cc:398] Leaking SignalData structure 0x7b080024f8a0 after lost signal to thread 10304
I20260709 13:06:54.026834 10324 consensus_queue.cc:260] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:54.027963 10562 raft_consensus.cc:2804] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.027787 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd93f389d32493b9dcc10e602f452da" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:54.028610 10562 raft_consensus.cc:697] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:06:54.028762 10132 raft_consensus.cc:3060] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.029536 10562 consensus_queue.cc:237] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:54.031498 10333 tablet_bootstrap.cc:492] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
W20260709 13:06:54.031744 10095 leader_election.cc:343] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.032261 10095 leader_election.cc:304] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:54.033710 10535 raft_consensus.cc:2749] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.034947 10333 ts_tablet_manager.cc:1403] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.133s	user 0.019s	sys 0.006s
I20260709 13:06:54.036876 10324 ts_tablet_manager.cc:1434] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.003s
I20260709 13:06:54.038084 10324 tablet_bootstrap.cc:492] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:54.039865 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d139f1fbdd384830bc7ee062ab3d281c" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:54.040933 10333 raft_consensus.cc:359] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:54.041659 10333 raft_consensus.cc:385] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.041939 10333 raft_consensus.cc:740] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.042610 10333 consensus_queue.cc:260] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:54.044902 10333 ts_tablet_manager.cc:1434] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.000s
I20260709 13:06:54.048456 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d139f1fbdd384830bc7ee062ab3d281c" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:54.043692 10457 consensus_queue.cc:1048] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
W20260709 13:06:54.054994 10098 leader_election.cc:343] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.060266 10333 tablet_bootstrap.cc:492] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:54.068332 10328 tablet_bootstrap.cc:654] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.067091 10324 tablet_bootstrap.cc:654] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.073103 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd93f389d32493b9dcc10e602f452da" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:54.077956 10052 raft_consensus.cc:3060] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.081681 10561 raft_consensus.cc:493] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.082499 10561 raft_consensus.cc:515] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:54.083161 10333 tablet_bootstrap.cc:654] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.105305 10333 tablet_bootstrap.cc:492] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:54.106822 10325 tablet_bootstrap.cc:492] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:54.107326 10325 ts_tablet_manager.cc:1403] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.151s	user 0.035s	sys 0.033s
I20260709 13:06:54.108202 10457 consensus_queue.cc:1048] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:54.109542  9952 catalog_manager.cc:5697] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 0 to 1, leader changed from <none> to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:54.112161 10333 ts_tablet_manager.cc:1403] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.052s	user 0.015s	sys 0.009s
I20260709 13:06:54.086090 10561 leader_election.cc:290] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
W20260709 13:06:54.112664 10095 leader_election.cc:343] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.119031 10452 raft_consensus.cc:493] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.126513 10562 raft_consensus.cc:493] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.126711 10452 raft_consensus.cc:515] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:54.127408 10052 raft_consensus.cc:2468] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:54.127185 10562 raft_consensus.cc:515] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:54.129086 10452 leader_election.cc:290] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:54.129578 10562 leader_election.cc:290] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:54.127005 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0981226de91f4f9e96cf2e8b926aaf19" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:54.131265 10246 leader_election.cc:304] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:54.132259 10132 raft_consensus.cc:2468] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:54.132336 10561 raft_consensus.cc:2804] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.132674 10095 leader_election.cc:304] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:54.133975 10535 raft_consensus.cc:2749] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.134938 10561 raft_consensus.cc:697] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:06:54.134354 10452 consensus_queue.cc:1048] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:06:54.114105 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0981226de91f4f9e96cf2e8b926aaf19" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
W20260709 13:06:54.141500 10246 leader_election.cc:343] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.142504 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39e4808ea4be41f483e08e83162c59c8" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
W20260709 13:06:54.144345 10243 leader_election.cc:343] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.144881 10243 leader_election.cc:304] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:54.146248 10564 raft_consensus.cc:2749] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.134950 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec007b8bb1e4c69a49a9ba8aeb56eee" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:54.147787 10545 raft_consensus.cc:493] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.148327 10545 raft_consensus.cc:515] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:54.149751 10561 consensus_queue.cc:237] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:54.150857 10545 leader_election.cc:290] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:54.123684 10325 raft_consensus.cc:359] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:54.153867 10325 raft_consensus.cc:385] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.154299 10535 consensus_queue.cc:1048] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:54.155073 10452 consensus_queue.cc:1048] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:54.155272 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39e4808ea4be41f483e08e83162c59c8" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:54.157270 10325 raft_consensus.cc:740] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.157397 10277 raft_consensus.cc:2393] T 39e4808ea4be41f483e08e83162c59c8 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa338cd132c457896b11e08d7ea787d in current term 1: Already voted for candidate 575e44d297f045a389903d65a471d548 in this term.
I20260709 13:06:54.158468 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ec007b8bb1e4c69a49a9ba8aeb56eee" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:54.161427 10325 consensus_queue.cc:260] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:54.175225 10564 raft_consensus.cc:493] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:06:54.175869 10564 raft_consensus.cc:515] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:54.178252 10564 leader_election.cc:290] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:54.182055 10325 ts_tablet_manager.cc:1434] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.074s	user 0.005s	sys 0.003s
I20260709 13:06:54.185755 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e003c3e14c2444b937f547eef84b672" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:54.189800 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e003c3e14c2444b937f547eef84b672" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:54.191259 10325 tablet_bootstrap.cc:492] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:54.160197 10333 raft_consensus.cc:359] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:54.195044 10333 raft_consensus.cc:385] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.195401 10333 raft_consensus.cc:740] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.195773  9950 catalog_manager.cc:5697] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:54.207213 10333 consensus_queue.cc:260] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:54.203756 10056 raft_consensus.cc:3060] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.214536 10452 raft_consensus.cc:493] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.215600 10020 leader_election.cc:304] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa338cd132c457896b11e08d7ea787d; no voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548
I20260709 13:06:54.215340 10452 raft_consensus.cc:515] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:54.218125 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50256706bf3a4bf29b64ea4ea6583da3" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
W20260709 13:06:54.218919 10098 leader_election.cc:343] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.219148 10052 raft_consensus.cc:2468] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 2.
I20260709 13:06:54.220546 10246 leader_election.cc:304] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:54.224857 10561 raft_consensus.cc:2804] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:54.225426 10561 raft_consensus.cc:493] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:06:54.225874 10561 raft_consensus.cc:3060] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.228533 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50256706bf3a4bf29b64ea4ea6583da3" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:54.231761 10130 raft_consensus.cc:2393] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50999113203c482b9b9300caa485f154 in current term 2: Already voted for candidate 575e44d297f045a389903d65a471d548 in this term.
I20260709 13:06:54.232386 10452 leader_election.cc:290] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:54.235639 10333 ts_tablet_manager.cc:1434] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.123s	user 0.005s	sys 0.002s
I20260709 13:06:54.236768 10333 tablet_bootstrap.cc:492] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:54.238422 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eeb5566a2f4fa4af640a137fe578df" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:54.239148 10277 raft_consensus.cc:2468] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:54.240109 10169 leader_election.cc:304] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:54.245178 10457 raft_consensus.cc:2749] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.245720 10169 leader_election.cc:304] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:54.246534 10457 raft_consensus.cc:2804] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:54.246901 10457 raft_consensus.cc:493] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.255164 10325 tablet_bootstrap.cc:654] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.246862 10535 raft_consensus.cc:493] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.258464 10535 raft_consensus.cc:515] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:54.260576 10562 raft_consensus.cc:2749] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.261266 10535 leader_election.cc:290] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:54.247442 10457 raft_consensus.cc:3060] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.264534 10535 consensus_queue.cc:1048] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:54.266234 10423 raft_consensus.cc:493] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.266922 10423 raft_consensus.cc:515] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:54.268865 10279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f7da1cc13f64f56b02defc2e119c507" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:54.274953 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eeb5566a2f4fa4af640a137fe578df" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:54.275703 10130 raft_consensus.cc:2468] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
W20260709 13:06:54.276997 10095 leader_election.cc:343] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.277940 10328 tablet_bootstrap.cc:492] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:54.288028 10328 ts_tablet_manager.cc:1403] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.329s	user 0.025s	sys 0.026s
I20260709 13:06:54.286676 10452 raft_consensus.cc:493] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.296000 10452 raft_consensus.cc:515] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:54.299711 10452 leader_election.cc:290] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:54.301403 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccaebf4041f14bada42fd7122f34419c" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:54.279345 10095 leader_election.cc:304] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:54.266156 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f7da1cc13f64f56b02defc2e119c507" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:54.288254 10324 tablet_bootstrap.cc:492] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:54.305363 10324 ts_tablet_manager.cc:1403] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.267s	user 0.013s	sys 0.019s
I20260709 13:06:54.305766 10328 raft_consensus.cc:359] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:54.306617 10328 raft_consensus.cc:385] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.289608 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d642d68d234d668cb67cb81b80556e" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
W20260709 13:06:54.311498 10098 leader_election.cc:343] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.306888 10328 raft_consensus.cc:740] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.312863 10457 raft_consensus.cc:515] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:54.313740 10328 consensus_queue.cc:260] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:54.314990 10457 leader_election.cc:290] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:54.315923 10328 ts_tablet_manager.cc:1434] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:06:54.316740 10328 tablet_bootstrap.cc:492] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:54.308980 10324 raft_consensus.cc:359] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:54.318594 10324 raft_consensus.cc:385] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.319010 10324 raft_consensus.cc:740] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.319707 10324 consensus_queue.cc:260] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:54.321105 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eeb5566a2f4fa4af640a137fe578df" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:54.293710 10333 tablet_bootstrap.cc:654] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.301348 10279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccaebf4041f14bada42fd7122f34419c" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:54.324784 10280 raft_consensus.cc:3060] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.325169 10279 raft_consensus.cc:2393] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fe33410d51a469e81be6edc602d7d78 in current term 1: Already voted for candidate efa338cd132c457896b11e08d7ea787d in this term.
I20260709 13:06:54.324245 10535 raft_consensus.cc:2749] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.287155 10423 leader_election.cc:290] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:54.292631 10561 raft_consensus.cc:515] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
W20260709 13:06:54.325210 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:40419, user_credentials={real_user=slave}} blocked reactor thread for 48259.4us
I20260709 13:06:54.325624 10324 ts_tablet_manager.cc:1434] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:06:54.353807 10324 tablet_bootstrap.cc:492] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:54.348133 10280 raft_consensus.cc:2468] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 2.
I20260709 13:06:54.355119 10203 raft_consensus.cc:3060] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.324795 10328 tablet_bootstrap.cc:654] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.358457 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d642d68d234d668cb67cb81b80556e" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:54.350072 10325 tablet_bootstrap.cc:492] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:54.360587 10325 ts_tablet_manager.cc:1403] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.170s	user 0.014s	sys 0.006s
I20260709 13:06:54.361846 10561 leader_election.cc:290] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:54.362126 10333 tablet_bootstrap.cc:492] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:54.362068 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50256706bf3a4bf29b64ea4ea6583da3" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:54.362768 10333 ts_tablet_manager.cc:1403] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.126s	user 0.011s	sys 0.000s
I20260709 13:06:54.363698 10325 raft_consensus.cc:359] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:54.364168 10169 leader_election.cc:304] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:54.365387 10452 raft_consensus.cc:3060] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:54.365803 10095 leader_election.cc:343] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.366202 10333 raft_consensus.cc:359] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:54.366920 10333 raft_consensus.cc:385] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.367197 10333 raft_consensus.cc:740] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.364450 10325 raft_consensus.cc:385] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.370316 10052 raft_consensus.cc:2468] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 2.
I20260709 13:06:54.377061 10325 raft_consensus.cc:740] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.379734 10246 leader_election.cc:304] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:54.380471 10333 consensus_queue.cc:260] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:54.370198 10169 leader_election.cc:304] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:54.338768 10558 consensus_queue.cc:1048] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.044s
I20260709 13:06:54.382582 10561 raft_consensus.cc:2804] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:54.383111 10561 raft_consensus.cc:697] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:06:54.383924 10561 consensus_queue.cc:237] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:54.384994 10451 raft_consensus.cc:2804] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:54.387732 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50256706bf3a4bf29b64ea4ea6583da3" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:54.388593 10130 raft_consensus.cc:2393] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 50999113203c482b9b9300caa485f154 in current term 2: Already voted for candidate 575e44d297f045a389903d65a471d548 in this term.
I20260709 13:06:54.385535 10451 raft_consensus.cc:697] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 2 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:06:54.392321 10451 consensus_queue.cc:237] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:54.393921 10325 consensus_queue.cc:260] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:54.393261 10452 raft_consensus.cc:2749] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.398854 10324 tablet_bootstrap.cc:654] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:54.371230 10095 leader_election.cc:343] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.402391 10095 leader_election.cc:304] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:54.403298 10423 raft_consensus.cc:2749] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.411798 10333 ts_tablet_manager.cc:1434] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.049s	user 0.004s	sys 0.002s
I20260709 13:06:54.418545 10333 tablet_bootstrap.cc:492] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:54.401989 10325 ts_tablet_manager.cc:1434] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.004s
I20260709 13:06:54.426970 10325 tablet_bootstrap.cc:492] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:54.426461 10562 consensus_queue.cc:1048] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 13:06:54.453904 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:54.455825 10324 tablet_bootstrap.cc:492] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:54.456590 10324 ts_tablet_manager.cc:1403] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.103s	user 0.014s	sys 0.004s
I20260709 13:06:54.459877 10324 raft_consensus.cc:359] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:54.460646 10324 raft_consensus.cc:385] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.460947 10324 raft_consensus.cc:740] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.461742 10324 consensus_queue.cc:260] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:54.464823 10328 tablet_bootstrap.cc:492] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:54.465340 10328 ts_tablet_manager.cc:1403] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.149s	user 0.017s	sys 0.024s
I20260709 13:06:54.466784 10333 tablet_bootstrap.cc:654] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:54.467826 10095 leader_election.cc:343] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.468322 10095 leader_election.cc:304] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:54.469812 10535 raft_consensus.cc:2749] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.473925 10535 raft_consensus.cc:493] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.474414 10535 raft_consensus.cc:515] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:54.451460 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90eeb5566a2f4fa4af640a137fe578df" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:54.476553 10535 leader_election.cc:290] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:54.476747 10130 raft_consensus.cc:3060] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.477491 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f060317dcbd4f0db70f83fd18320215" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
W20260709 13:06:54.478968 10098 leader_election.cc:343] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.479734  9953 catalog_manager.cc:5697] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 0 to 2, leader changed from <none> to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:54.484349 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f060317dcbd4f0db70f83fd18320215" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:54.487145 10328 raft_consensus.cc:359] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:54.487869 10328 raft_consensus.cc:385] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.488159 10328 raft_consensus.cc:740] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.489167  9952 catalog_manager.cc:5697] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 1 to 2, leader changed from efa338cd132c457896b11e08d7ea787d (127.0.107.193) to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:54.491149 10457 raft_consensus.cc:493] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.491641 10457 raft_consensus.cc:515] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:54.502058 10325 tablet_bootstrap.cc:654] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.502842 10457 leader_election.cc:290] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:54.493300 10328 consensus_queue.cc:260] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:54.507102 10324 ts_tablet_manager.cc:1434] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.050s	user 0.006s	sys 0.000s
I20260709 13:06:54.508581 10324 tablet_bootstrap.cc:492] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:54.515372 10423 raft_consensus.cc:493] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.520061 10423 raft_consensus.cc:515] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:54.522542 10423 leader_election.cc:290] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:54.525432 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50256706bf3a4bf29b64ea4ea6583da3" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:54.525772 10324 tablet_bootstrap.cc:654] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.526649 10052 raft_consensus.cc:2410] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 575e44d297f045a389903d65a471d548 for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:06:54.529141 10457 consensus_queue.cc:1048] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:54.530191 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21c65d190c541ab8e2d6ae0d4be85e7" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:54.535041 10328 ts_tablet_manager.cc:1434] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.069s	user 0.007s	sys 0.000s
I20260709 13:06:54.540964 10325 tablet_bootstrap.cc:492] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:54.542783 10325 ts_tablet_manager.cc:1403] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.116s	user 0.013s	sys 0.006s
I20260709 13:06:54.543123 10056 raft_consensus.cc:2468] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:54.544337 10328 tablet_bootstrap.cc:492] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:54.546926 10172 leader_election.cc:304] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:54.547883 10452 raft_consensus.cc:2804] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.548267 10452 raft_consensus.cc:493] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.548632 10452 raft_consensus.cc:3060] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.548938 10325 raft_consensus.cc:359] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
W20260709 13:06:54.549698 10095 leader_election.cc:343] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.549955 10325 raft_consensus.cc:385] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.550442 10325 raft_consensus.cc:740] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.550287 10095 leader_election.cc:304] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:54.545781 10561 raft_consensus.cc:493] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.551501 10534 raft_consensus.cc:2749] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.551378 10561 raft_consensus.cc:515] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:54.555346 10325 consensus_queue.cc:260] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:54.559767 10325 ts_tablet_manager.cc:1434] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.003s
I20260709 13:06:54.556996 10130 raft_consensus.cc:2468] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 2.
I20260709 13:06:54.561590 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc4e649e35ee49a2a224ecc9f962980a" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:54.562204 10325 tablet_bootstrap.cc:492] T dc4e649e35ee49a2a224ecc9f962980a P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:54.562366 10132 raft_consensus.cc:2468] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:54.564459 10243 leader_election.cc:304] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:06:54.565797 10564 raft_consensus.cc:2804] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.566201 10564 raft_consensus.cc:493] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.566582 10564 raft_consensus.cc:3060] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.568405 10325 tablet_bootstrap.cc:654] T dc4e649e35ee49a2a224ecc9f962980a P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.572065 10598 raft_consensus.cc:493] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.572666 10598 raft_consensus.cc:515] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:54.574374 10452 raft_consensus.cc:515] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:54.582976 10564 raft_consensus.cc:515] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:54.594712 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50256706bf3a4bf29b64ea4ea6583da3" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:54.596529 10452 leader_election.cc:290] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:54.604857 10561 leader_election.cc:290] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:54.610289 10598 leader_election.cc:290] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:54.612038 10324 tablet_bootstrap.cc:492] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:54.618062 10564 leader_election.cc:290] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:54.606283 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21c65d190c541ab8e2d6ae0d4be85e7" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:54.633710 10324 ts_tablet_manager.cc:1403] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.126s	user 0.011s	sys 0.003s
I20260709 13:06:54.634063 10052 raft_consensus.cc:3060] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.622680 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc4e649e35ee49a2a224ecc9f962980a" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:54.623114 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7364742faea3470694e1a23f151f1b57" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:54.623114 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc4e649e35ee49a2a224ecc9f962980a" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:54.624231 10562 consensus_queue.cc:1048] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:06:54.637863 10246 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42425, user_credentials={real_user=slave}} blocked reactor thread for 66200.6us
I20260709 13:06:54.635715 10130 raft_consensus.cc:3060] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.636682 10132 raft_consensus.cc:2468] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:54.628886 10333 tablet_bootstrap.cc:492] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
W20260709 13:06:54.641350 10243 leader_election.cc:343] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.641595 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7364742faea3470694e1a23f151f1b57" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:54.641896 10333 ts_tablet_manager.cc:1403] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.229s	user 0.008s	sys 0.015s
I20260709 13:06:54.642247 10056 raft_consensus.cc:2468] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:54.642565 10243 leader_election.cc:304] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:06:54.645784 10333 raft_consensus.cc:359] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:54.646809 10333 raft_consensus.cc:385] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.647106 10333 raft_consensus.cc:740] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.648702 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21c65d190c541ab8e2d6ae0d4be85e7" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:54.646898 10628 raft_consensus.cc:2804] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.649361 10132 raft_consensus.cc:2468] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:54.649384 10628 raft_consensus.cc:493] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.650187 10628 raft_consensus.cc:3060] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.654284 10333 consensus_queue.cc:260] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:54.656639 10333 ts_tablet_manager.cc:1434] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:06:54.657318 10130 raft_consensus.cc:2468] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:54.658973 10243 leader_election.cc:304] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:06:54.660266 10598 raft_consensus.cc:2804] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.660919 10598 raft_consensus.cc:697] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:06:54.662037 10598 consensus_queue.cc:237] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:54.663385 10333 tablet_bootstrap.cc:492] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:54.668865 10328 tablet_bootstrap.cc:654] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.654620 10052 raft_consensus.cc:2468] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:54.670935 10172 leader_election.cc:304] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:54.674678 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc4e649e35ee49a2a224ecc9f962980a" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:54.675338 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21c65d190c541ab8e2d6ae0d4be85e7" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:54.676208 10129 raft_consensus.cc:3060] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.673437 10452 raft_consensus.cc:2804] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.677537 10452 raft_consensus.cc:697] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:06:54.678624 10452 consensus_queue.cc:237] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
W20260709 13:06:54.679356 10243 leader_election.cc:343] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.710894 10095 leader_election.cc:304] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:54.712646 10423 raft_consensus.cc:2749] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:54.713882 10328 tablet_bootstrap.cc:492] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:54.714903 10328 ts_tablet_manager.cc:1403] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.171s	user 0.019s	sys 0.010s
I20260709 13:06:54.715647 10324 raft_consensus.cc:359] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:54.721843 10561 consensus_queue.cc:1048] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:54.722894 10129 raft_consensus.cc:2468] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:54.726022 10324 raft_consensus.cc:385] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.730024 10324 raft_consensus.cc:740] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.730916 10324 consensus_queue.cc:260] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:54.741847 10325 tablet_bootstrap.cc:492] T dc4e649e35ee49a2a224ecc9f962980a P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:54.742420 10325 ts_tablet_manager.cc:1403] T dc4e649e35ee49a2a224ecc9f962980a P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.180s	user 0.051s	sys 0.061s
I20260709 13:06:54.746801 10328 raft_consensus.cc:359] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:54.748974 10328 raft_consensus.cc:385] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.749269 10328 raft_consensus.cc:740] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.748391 10325 raft_consensus.cc:359] T dc4e649e35ee49a2a224ecc9f962980a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:54.750245 10325 raft_consensus.cc:385] T dc4e649e35ee49a2a224ecc9f962980a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.750447 10324 ts_tablet_manager.cc:1434] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.116s	user 0.004s	sys 0.003s
I20260709 13:06:54.750505 10325 raft_consensus.cc:740] T dc4e649e35ee49a2a224ecc9f962980a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.751556 10324 tablet_bootstrap.cc:492] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:54.751413 10328 consensus_queue.cc:260] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:54.752132 10325 consensus_queue.cc:260] T dc4e649e35ee49a2a224ecc9f962980a P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:54.760658 10328 ts_tablet_manager.cc:1434] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
I20260709 13:06:54.761686 10328 tablet_bootstrap.cc:492] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:54.769455  9953 catalog_manager.cc:5697] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:54.774003 10325 ts_tablet_manager.cc:1434] T dc4e649e35ee49a2a224ecc9f962980a P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.002s
I20260709 13:06:54.774839 10325 tablet_bootstrap.cc:492] T d139f1fbdd384830bc7ee062ab3d281c P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:54.784070 10324 tablet_bootstrap.cc:654] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.792640 10628 raft_consensus.cc:515] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:54.793660 10325 tablet_bootstrap.cc:654] T d139f1fbdd384830bc7ee062ab3d281c P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.770869 10534 raft_consensus.cc:493] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.798610 10534 raft_consensus.cc:515] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:54.819113 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cded7cb005f64f9c894f568140883a77" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:54.823480 10328 tablet_bootstrap.cc:654] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:54.826992 10095 leader_election.cc:343] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.829471 10278 raft_consensus.cc:1275] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Refusing update from remote peer 8fe33410d51a469e81be6edc602d7d78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:06:54.811441 10534 leader_election.cc:290] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:54.831933 10628 leader_election.cc:290] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:54.835274 10130 raft_consensus.cc:1275] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Refusing update from remote peer 8fe33410d51a469e81be6edc602d7d78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:06:54.837809 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7364742faea3470694e1a23f151f1b57" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:54.838424 10055 raft_consensus.cc:3060] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.840690 10333 tablet_bootstrap.cc:654] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.842980 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7364742faea3470694e1a23f151f1b57" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:54.843657 10130 raft_consensus.cc:3060] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.845080 10558 consensus_queue.cc:1048] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:54.847425 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cded7cb005f64f9c894f568140883a77" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
W20260709 13:06:54.849718 10095 leader_election.cc:343] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.850178 10095 leader_election.cc:304] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:54.851193 10534 raft_consensus.cc:2749] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.852851 10628 consensus_queue.cc:1048] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:54.853937 10446 consensus_queue.cc:1048] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:54.887182 10429 raft_consensus.cc:493] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.887777 10429 raft_consensus.cc:515] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:54.904237 10328 tablet_bootstrap.cc:492] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:54.904830 10328 ts_tablet_manager.cc:1403] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.143s	user 0.021s	sys 0.012s
I20260709 13:06:54.900635  9950 catalog_manager.cc:5697] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:54.909778 10055 raft_consensus.cc:2468] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:54.921140 10333 tablet_bootstrap.cc:492] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:54.921718 10333 ts_tablet_manager.cc:1403] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.259s	user 0.016s	sys 0.001s
I20260709 13:06:54.922394 10246 leader_election.cc:304] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:54.923103 10324 tablet_bootstrap.cc:492] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:54.923581 10324 ts_tablet_manager.cc:1403] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.172s	user 0.012s	sys 0.012s
I20260709 13:06:54.925355 10628 raft_consensus.cc:2804] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.926062 10628 raft_consensus.cc:697] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:06:54.926854 10429 leader_election.cc:290] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:54.927397 10051 raft_consensus.cc:1275] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Refusing update from remote peer 50999113203c482b9b9300caa485f154: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.926972 10628 consensus_queue.cc:237] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:54.924425 10333 raft_consensus.cc:359] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:54.928815 10333 raft_consensus.cc:385] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.929184 10333 raft_consensus.cc:740] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.930044 10333 consensus_queue.cc:260] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:54.936611 10561 consensus_queue.cc:1048] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:54.939060 10279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0981226de91f4f9e96cf2e8b926aaf19" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:54.940027 10279 raft_consensus.cc:2468] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 0.
I20260709 13:06:54.939172 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0981226de91f4f9e96cf2e8b926aaf19" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:54.941243 10325 tablet_bootstrap.cc:492] T d139f1fbdd384830bc7ee062ab3d281c P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:54.941112 10205 raft_consensus.cc:2468] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 0.
I20260709 13:06:54.940418 10130 raft_consensus.cc:2468] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:54.947322 10325 ts_tablet_manager.cc:1403] T d139f1fbdd384830bc7ee062ab3d281c P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.173s	user 0.008s	sys 0.020s
I20260709 13:06:54.950307 10325 raft_consensus.cc:359] T d139f1fbdd384830bc7ee062ab3d281c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:54.951619 10333 ts_tablet_manager.cc:1434] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 13:06:54.952132 10328 raft_consensus.cc:359] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:54.953045 10328 raft_consensus.cc:385] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.950940 10020 leader_election.cc:304] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:54.953490 10328 raft_consensus.cc:740] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.956103 10324 raft_consensus.cc:359] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:54.956993 10324 raft_consensus.cc:385] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.957330 10324 raft_consensus.cc:740] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.958146 10324 consensus_queue.cc:260] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:54.955477 10328 consensus_queue.cc:260] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:54.960202 10324 ts_tablet_manager.cc:1434] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.000s
I20260709 13:06:54.961127 10324 tablet_bootstrap.cc:492] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:54.951014 10325 raft_consensus.cc:385] T d139f1fbdd384830bc7ee062ab3d281c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.966678 10325 raft_consensus.cc:740] T d139f1fbdd384830bc7ee062ab3d281c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.967602 10325 consensus_queue.cc:260] T d139f1fbdd384830bc7ee062ab3d281c P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:54.956521 10429 raft_consensus.cc:2804] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.968464 10429 raft_consensus.cc:493] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.968807 10429 raft_consensus.cc:3060] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.969894 10325 ts_tablet_manager.cc:1434] T d139f1fbdd384830bc7ee062ab3d281c P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:06:54.970883 10325 tablet_bootstrap.cc:492] T cded7cb005f64f9c894f568140883a77 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:54.978206 10328 ts_tablet_manager.cc:1434] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.073s	user 0.008s	sys 0.000s
I20260709 13:06:54.980294 10328 tablet_bootstrap.cc:492] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:54.982966 10130 raft_consensus.cc:1275] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Refusing update from remote peer 50999113203c482b9b9300caa485f154: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.977913 10423 raft_consensus.cc:493] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.984522 10423 raft_consensus.cc:515] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:54.985461 10629 consensus_queue.cc:1048] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:54.987322 10278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2258980c3a7540dea8ae11850194e183" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:54.987569 10423 leader_election.cc:290] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:54.989977 10333 tablet_bootstrap.cc:492] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548: Bootstrap starting.
W20260709 13:06:54.990885 10095 leader_election.cc:343] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.999191 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2258980c3a7540dea8ae11850194e183" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
W20260709 13:06:55.001514 10098 leader_election.cc:343] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.002262 10098 leader_election.cc:304] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:55.005852 10534 raft_consensus.cc:2749] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:55.006045 10562 raft_consensus.cc:493] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.006863 10562 raft_consensus.cc:515] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:55.022588 10629 raft_consensus.cc:493] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.023181 10629 raft_consensus.cc:515] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:55.024466 10324 tablet_bootstrap.cc:654] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.025478 10325 tablet_bootstrap.cc:654] T cded7cb005f64f9c894f568140883a77 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.027626 10629 leader_election.cc:290] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:55.028213 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d242deba350e42cb8e2b623027a26516" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:55.029083 10562 leader_election.cc:290] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
W20260709 13:06:55.029845 10243 leader_election.cc:343] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.030252 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d139f1fbdd384830bc7ee062ab3d281c" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:55.036017 10127 raft_consensus.cc:2468] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 0.
I20260709 13:06:55.037711 10020 leader_election.cc:304] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:55.038568 10562 raft_consensus.cc:2804] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.038957 10562 raft_consensus.cc:493] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.039283 10562 raft_consensus.cc:3060] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.030294 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d139f1fbdd384830bc7ee062ab3d281c" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:55.044783 10205 raft_consensus.cc:2468] T d139f1fbdd384830bc7ee062ab3d281c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 0.
I20260709 13:06:55.058897 10328 tablet_bootstrap.cc:654] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.061120 10457 raft_consensus.cc:493] T dc4e649e35ee49a2a224ecc9f962980a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.063889 10457 raft_consensus.cc:515] T dc4e649e35ee49a2a224ecc9f962980a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:55.064998 10429 raft_consensus.cc:515] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:55.068050 10429 leader_election.cc:290] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
W20260709 13:06:55.086370 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40841, user_credentials={real_user=slave}} blocked reactor thread for 81337us
I20260709 13:06:55.101090 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d242deba350e42cb8e2b623027a26516" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:55.104126 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0981226de91f4f9e96cf2e8b926aaf19" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:55.111459 10205 raft_consensus.cc:3060] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.098148 10278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0981226de91f4f9e96cf2e8b926aaf19" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:55.112349 10278 raft_consensus.cc:3060] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:55.089267 10098 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42425, user_credentials={real_user=slave}} blocked reactor thread for 60550.8us
I20260709 13:06:55.104255 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc4e649e35ee49a2a224ecc9f962980a" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:55.115666 10129 raft_consensus.cc:2393] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fe33410d51a469e81be6edc602d7d78 in current term 1: Already voted for candidate 50999113203c482b9b9300caa485f154 in this term.
I20260709 13:06:55.104112 10457 leader_election.cc:290] T dc4e649e35ee49a2a224ecc9f962980a P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:55.119325 10279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc4e649e35ee49a2a224ecc9f962980a" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:55.129033 10169 leader_election.cc:304] T dc4e649e35ee49a2a224ecc9f962980a P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548
W20260709 13:06:55.129868 10246 leader_election.cc:343] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.130467 10246 leader_election.cc:304] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:55.131733 10561 raft_consensus.cc:2749] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:55.119516 10333 tablet_bootstrap.cc:654] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.142334 10446 raft_consensus.cc:3060] T dc4e649e35ee49a2a224ecc9f962980a P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.110293 10562 raft_consensus.cc:515] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:55.175137 10562 leader_election.cc:290] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:55.175232 10205 raft_consensus.cc:2468] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:06:55.183801 10324 tablet_bootstrap.cc:492] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:55.184298 10324 ts_tablet_manager.cc:1403] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.223s	user 0.023s	sys 0.016s
I20260709 13:06:55.188577 10020 leader_election.cc:304] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:55.189707 10562 raft_consensus.cc:2804] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:55.190217 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d139f1fbdd384830bc7ee062ab3d281c" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:55.190685 10325 tablet_bootstrap.cc:492] T cded7cb005f64f9c894f568140883a77 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:55.190980 10127 raft_consensus.cc:3060] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.191327 10325 ts_tablet_manager.cc:1403] T cded7cb005f64f9c894f568140883a77 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.221s	user 0.017s	sys 0.024s
I20260709 13:06:55.194043 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d139f1fbdd384830bc7ee062ab3d281c" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:55.194136 10558 consensus_queue.cc:1048] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:55.194859 10203 raft_consensus.cc:3060] T d139f1fbdd384830bc7ee062ab3d281c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.190263 10562 raft_consensus.cc:697] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:06:55.200369 10562 consensus_queue.cc:237] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:55.203749 10325 raft_consensus.cc:359] T cded7cb005f64f9c894f568140883a77 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:55.204998 10325 raft_consensus.cc:385] T cded7cb005f64f9c894f568140883a77 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.205295 10325 raft_consensus.cc:740] T cded7cb005f64f9c894f568140883a77 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.205843 10325 consensus_queue.cc:260] T cded7cb005f64f9c894f568140883a77 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:55.179414 10278 raft_consensus.cc:2468] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:06:55.207885  9953 catalog_manager.cc:5697] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.221735 10325 ts_tablet_manager.cc:1434] T cded7cb005f64f9c894f568140883a77 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 13:06:55.223451 10325 tablet_bootstrap.cc:492] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:55.229983 10328 tablet_bootstrap.cc:492] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:55.230540 10328 ts_tablet_manager.cc:1403] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.251s	user 0.024s	sys 0.015s
I20260709 13:06:55.232826 10324 raft_consensus.cc:359] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:55.234174 10324 raft_consensus.cc:385] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.234563 10324 raft_consensus.cc:740] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.241118 10324 consensus_queue.cc:260] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:55.244824 10333 tablet_bootstrap.cc:492] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:55.245352 10333 ts_tablet_manager.cc:1403] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.256s	user 0.012s	sys 0.004s
I20260709 13:06:55.224530 10446 raft_consensus.cc:2749] T dc4e649e35ee49a2a224ecc9f962980a P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:55.251377 10328 raft_consensus.cc:359] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:55.252374 10328 raft_consensus.cc:385] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.252750 10328 raft_consensus.cc:740] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.253597 10328 consensus_queue.cc:260] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:55.255185 10324 ts_tablet_manager.cc:1434] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.071s	user 0.005s	sys 0.003s
I20260709 13:06:55.238050 10561 raft_consensus.cc:493] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.257258 10324 tablet_bootstrap.cc:492] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:55.261158 10328 ts_tablet_manager.cc:1434] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.004s
I20260709 13:06:55.263145 10328 tablet_bootstrap.cc:492] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:55.259030 10564 raft_consensus.cc:493] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.265962 10564 raft_consensus.cc:515] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:55.266556 10127 raft_consensus.cc:2468] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:06:55.268702 10564 leader_election.cc:290] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:55.258174 10628 consensus_queue.cc:1048] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:06:55.271821 10020 leader_election.cc:304] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:55.272660 10628 consensus_queue.cc:1048] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:55.273357 10206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cded7cb005f64f9c894f568140883a77" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:55.274058 10206 raft_consensus.cc:2468] T cded7cb005f64f9c894f568140883a77 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:55.276470 10243 leader_election.cc:304] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:55.277208 10652 raft_consensus.cc:2804] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:55.278023 10652 raft_consensus.cc:697] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:06:55.279111 10652 consensus_queue.cc:237] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:55.279742 10628 raft_consensus.cc:2804] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.280269 10628 raft_consensus.cc:493] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.280431 10333 raft_consensus.cc:359] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:55.280738 10628 raft_consensus.cc:3060] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.259459 10561 raft_consensus.cc:515] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:55.279774 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cded7cb005f64f9c894f568140883a77" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:55.281921 10129 raft_consensus.cc:2468] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:55.282894 10561 leader_election.cc:290] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:55.285391 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75aced19d8c94fc78e5510b592bb01be" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:55.286229 10129 raft_consensus.cc:2468] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:55.288066 10558 consensus_queue.cc:1048] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:55.288767 10333 raft_consensus.cc:385] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.289367 10333 raft_consensus.cc:740] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.288414 10243 leader_election.cc:304] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:06:55.292286 10628 raft_consensus.cc:515] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:55.292673 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75aced19d8c94fc78e5510b592bb01be" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:55.293629 10051 raft_consensus.cc:2468] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:55.295213 10333 consensus_queue.cc:260] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:55.277246 10325 tablet_bootstrap.cc:654] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.308238 10333 ts_tablet_manager.cc:1434] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.063s	user 0.005s	sys 0.003s
I20260709 13:06:55.320359 10333 tablet_bootstrap.cc:492] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:55.308193 10203 raft_consensus.cc:2468] T d139f1fbdd384830bc7ee062ab3d281c P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:06:55.322230 10645 raft_consensus.cc:493] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.326428 10645 raft_consensus.cc:515] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:55.328783 10645 leader_election.cc:290] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:55.330789 10278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0265654cefc40a5937d7e194f03dd5c" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
W20260709 13:06:55.332701 10095 leader_election.cc:343] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.307555 10564 raft_consensus.cc:2804] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.342832 10333 tablet_bootstrap.cc:654] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.344579 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cded7cb005f64f9c894f568140883a77" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:55.345465 10132 raft_consensus.cc:3060] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.334635 10628 leader_election.cc:290] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:55.347314 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cded7cb005f64f9c894f568140883a77" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:55.356173 10132 raft_consensus.cc:2468] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:55.339964 10564 raft_consensus.cc:493] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.360344 10328 tablet_bootstrap.cc:654] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.361555 10325 tablet_bootstrap.cc:492] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:55.362226 10325 ts_tablet_manager.cc:1403] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.139s	user 0.018s	sys 0.001s
I20260709 13:06:55.362576 10564 raft_consensus.cc:3060] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.367070  9953 catalog_manager.cc:5697] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 0 to 1, leader changed from <none> to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.368943  9953 catalog_manager.cc:5697] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 0 to 1, leader changed from <none> to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:55.369946 10325 raft_consensus.cc:359] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:55.370836 10325 raft_consensus.cc:385] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.371187 10325 raft_consensus.cc:740] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.372334 10324 tablet_bootstrap.cc:654] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.371987 10325 consensus_queue.cc:260] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:55.375139 10203 raft_consensus.cc:3060] T cded7cb005f64f9c894f568140883a77 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.381534 10243 leader_election.cc:304] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:06:55.385322 10325 ts_tablet_manager.cc:1434] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 13:06:55.379995 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0265654cefc40a5937d7e194f03dd5c" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:55.388729 10325 tablet_bootstrap.cc:492] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:55.385414 10628 raft_consensus.cc:2804] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:55.394518 10628 raft_consensus.cc:697] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:06:55.395483 10628 consensus_queue.cc:237] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
W20260709 13:06:55.406723 10095 leader_election.cc:343] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.407235 10095 leader_election.cc:304] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:55.386380 10564 raft_consensus.cc:515] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:55.417507 10564 leader_election.cc:290] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:55.411123 10645 raft_consensus.cc:2749] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:55.427448 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75aced19d8c94fc78e5510b592bb01be" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:55.428304 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75aced19d8c94fc78e5510b592bb01be" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:55.430213 10647 raft_consensus.cc:493] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.437879 10598 consensus_queue.cc:1048] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:55.449908 10056 raft_consensus.cc:3060] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.449738 10132 raft_consensus.cc:3060] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.449549 10647 raft_consensus.cc:515] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:55.468946 10056 raft_consensus.cc:2468] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:55.470520 10246 leader_election.cc:304] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:55.471060 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6c60f07656843aeac5d469a6533aaaa" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:55.471553 10561 raft_consensus.cc:2804] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:55.472174 10561 raft_consensus.cc:697] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
W20260709 13:06:55.472579 10095 leader_election.cc:343] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.478252 10333 tablet_bootstrap.cc:492] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:55.478153  9954 catalog_manager.cc:5697] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:55.478787 10333 ts_tablet_manager.cc:1403] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.161s	user 0.022s	sys 0.013s
I20260709 13:06:55.465368 10647 leader_election.cc:290] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:55.483664 10328 tablet_bootstrap.cc:492] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:55.486487 10328 ts_tablet_manager.cc:1403] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.224s	user 0.018s	sys 0.006s
I20260709 13:06:55.473145 10561 consensus_queue.cc:237] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:55.496910 10132 raft_consensus.cc:2468] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:55.489292 10328 raft_consensus.cc:359] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:55.500469 10328 raft_consensus.cc:385] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.500913 10328 raft_consensus.cc:740] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.501827 10328 consensus_queue.cc:260] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:55.505539 10325 tablet_bootstrap.cc:654] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.507866 10328 ts_tablet_manager.cc:1434] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:06:55.499792 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6c60f07656843aeac5d469a6533aaaa" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:55.509119 10328 tablet_bootstrap.cc:492] T 9f7da1cc13f64f56b02defc2e119c507 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
W20260709 13:06:55.510504 10098 leader_election.cc:343] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.511242 10098 leader_election.cc:304] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:55.511948 10203 raft_consensus.cc:2468] T cded7cb005f64f9c894f568140883a77 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:55.512543 10534 raft_consensus.cc:2749] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:55.513831 10333 raft_consensus.cc:359] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:55.514567 10333 raft_consensus.cc:385] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.514838 10333 raft_consensus.cc:740] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.515525 10333 consensus_queue.cc:260] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:55.518790 10324 tablet_bootstrap.cc:492] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:55.519345 10324 ts_tablet_manager.cc:1403] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.263s	user 0.012s	sys 0.005s
I20260709 13:06:55.522221 10324 raft_consensus.cc:359] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:55.522927 10324 raft_consensus.cc:385] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.523207 10324 raft_consensus.cc:740] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.524845 10328 tablet_bootstrap.cc:654] T 9f7da1cc13f64f56b02defc2e119c507 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.536747 10324 consensus_queue.cc:260] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:55.539369 10324 ts_tablet_manager.cc:1434] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:06:55.553735 10324 tablet_bootstrap.cc:492] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:55.553577 10658 consensus_queue.cc:1048] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:55.556663  9953 catalog_manager.cc:5697] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.568162 10333 ts_tablet_manager.cc:1434] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.089s	user 0.005s	sys 0.001s
I20260709 13:06:55.569341 10333 tablet_bootstrap.cc:492] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:55.576728 10324 tablet_bootstrap.cc:654] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.586853 10333 tablet_bootstrap.cc:654] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.597157 10324 tablet_bootstrap.cc:492] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:55.601023 10328 tablet_bootstrap.cc:492] T 9f7da1cc13f64f56b02defc2e119c507 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:55.601589 10328 ts_tablet_manager.cc:1403] T 9f7da1cc13f64f56b02defc2e119c507 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.093s	user 0.022s	sys 0.011s
I20260709 13:06:55.604333 10328 raft_consensus.cc:359] T 9f7da1cc13f64f56b02defc2e119c507 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:55.604794 10324 ts_tablet_manager.cc:1403] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.051s	user 0.012s	sys 0.000s
I20260709 13:06:55.605054 10328 raft_consensus.cc:385] T 9f7da1cc13f64f56b02defc2e119c507 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.605389 10328 raft_consensus.cc:740] T 9f7da1cc13f64f56b02defc2e119c507 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.608991 10325 tablet_bootstrap.cc:492] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:55.606690 10328 consensus_queue.cc:260] T 9f7da1cc13f64f56b02defc2e119c507 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:55.608660 10652 consensus_queue.cc:1048] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:55.607647 10324 raft_consensus.cc:359] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:55.609865 10325 ts_tablet_manager.cc:1403] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.223s	user 0.016s	sys 0.003s
I20260709 13:06:55.611408 10328 ts_tablet_manager.cc:1434] T 9f7da1cc13f64f56b02defc2e119c507 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 13:06:55.621021 10324 raft_consensus.cc:385] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.622496 10324 raft_consensus.cc:740] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.623183 10324 consensus_queue.cc:260] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:55.624039 10325 raft_consensus.cc:359] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:55.625048 10325 raft_consensus.cc:385] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.625308 10325 raft_consensus.cc:740] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.626385 10325 consensus_queue.cc:260] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:55.629071 10325 ts_tablet_manager.cc:1434] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:55.630162 10325 tablet_bootstrap.cc:492] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:55.632649 10328 tablet_bootstrap.cc:492] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154: Bootstrap starting.
W20260709 13:06:55.633086 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:55.637588 10324 ts_tablet_manager.cc:1434] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.001s
I20260709 13:06:55.639648 10324 tablet_bootstrap.cc:492] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:55.641183 10534 raft_consensus.cc:493] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.641734 10534 raft_consensus.cc:515] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:55.644100 10534 leader_election.cc:290] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:55.645181 10325 tablet_bootstrap.cc:654] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.645929 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f4dd5b2c17747058bab6588c63b9d72" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:55.648018 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f4dd5b2c17747058bab6588c63b9d72" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
W20260709 13:06:55.649609 10098 leader_election.cc:343] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:55.650579 10095 leader_election.cc:343] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.650974 10095 leader_election.cc:304] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:55.651985 10534 raft_consensus.cc:2749] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:55.655545 10328 tablet_bootstrap.cc:654] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.661320 10333 tablet_bootstrap.cc:492] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:55.661855 10333 ts_tablet_manager.cc:1403] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.093s	user 0.016s	sys 0.001s
I20260709 13:06:55.664470 10652 consensus_queue.cc:1048] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:55.666465 10328 tablet_bootstrap.cc:492] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:55.667004 10328 ts_tablet_manager.cc:1403] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.001s
I20260709 13:06:55.670270 10328 raft_consensus.cc:359] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:55.672390 10328 raft_consensus.cc:385] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.672708 10328 raft_consensus.cc:740] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.673475 10328 consensus_queue.cc:260] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:55.673686 10333 raft_consensus.cc:359] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:55.675230 10333 raft_consensus.cc:385] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.675508 10333 raft_consensus.cc:740] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.676491 10328 ts_tablet_manager.cc:1434] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:06:55.677235 10333 consensus_queue.cc:260] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:55.677914 10328 tablet_bootstrap.cc:492] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:55.684916 10324 tablet_bootstrap.cc:654] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.696555 10328 tablet_bootstrap.cc:654] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.679580 10333 ts_tablet_manager.cc:1434] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 13:06:55.684322 10652 consensus_queue.cc:1048] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:55.711035 10325 tablet_bootstrap.cc:492] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:55.711555 10325 ts_tablet_manager.cc:1403] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.003s
I20260709 13:06:55.714303 10325 raft_consensus.cc:359] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:55.715014 10325 raft_consensus.cc:385] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.715298 10325 raft_consensus.cc:740] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.717974 10324 tablet_bootstrap.cc:492] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:55.718493 10324 ts_tablet_manager.cc:1403] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.079s	user 0.011s	sys 0.000s
I20260709 13:06:55.721074 10324 raft_consensus.cc:359] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:55.721701 10324 raft_consensus.cc:385] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.721968 10324 raft_consensus.cc:740] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.721943 10325 consensus_queue.cc:260] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:55.722862 10324 consensus_queue.cc:260] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:55.723824 10650 consensus_queue.cc:1048] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:55.726719 10324 ts_tablet_manager.cc:1434] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:06:55.727746 10324 tablet_bootstrap.cc:492] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:55.729503 10325 ts_tablet_manager.cc:1434] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.002s
I20260709 13:06:55.730525 10325 tablet_bootstrap.cc:492] T 1f060317dcbd4f0db70f83fd18320215 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:55.734881 10324 tablet_bootstrap.cc:654] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.736539 10325 tablet_bootstrap.cc:654] T 1f060317dcbd4f0db70f83fd18320215 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.745848 10647 raft_consensus.cc:493] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.746471 10647 raft_consensus.cc:515] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:55.749776 10647 leader_election.cc:290] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:55.752398 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ea79b0bb13439c9384cba78627c40f" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
W20260709 13:06:55.753907 10095 leader_election.cc:343] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.755782 10558 raft_consensus.cc:493] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.756541 10558 raft_consensus.cc:515] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:55.758366 10333 tablet_bootstrap.cc:492] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:55.761390 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d242deba350e42cb8e2b623027a26516" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:55.762601 10280 raft_consensus.cc:2468] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:55.764413 10169 leader_election.cc:304] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:55.766510 10663 raft_consensus.cc:2804] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.766953 10663 raft_consensus.cc:493] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.767336 10663 raft_consensus.cc:3060] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.777081 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ea79b0bb13439c9384cba78627c40f" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
W20260709 13:06:55.780452 10098 leader_election.cc:343] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.781317 10098 leader_election.cc:304] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:55.785799 10647 raft_consensus.cc:2749] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:55.786935 10558 leader_election.cc:290] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:55.790975 10333 tablet_bootstrap.cc:654] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.791935 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d242deba350e42cb8e2b623027a26516" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:55.792650 10056 raft_consensus.cc:2468] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:55.794206 10663 raft_consensus.cc:515] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:55.795881 10324 tablet_bootstrap.cc:492] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:55.796494 10324 ts_tablet_manager.cc:1403] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.069s	user 0.017s	sys 0.000s
I20260709 13:06:55.797250 10663 leader_election.cc:290] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Requested vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:55.796953 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d242deba350e42cb8e2b623027a26516" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:55.798499 10280 raft_consensus.cc:3060] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.798332 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d242deba350e42cb8e2b623027a26516" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:55.799118 10056 raft_consensus.cc:3060] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.800298 10324 raft_consensus.cc:359] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:55.801072 10324 raft_consensus.cc:385] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.801265 10328 tablet_bootstrap.cc:492] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:55.801401 10324 raft_consensus.cc:740] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.801828 10328 ts_tablet_manager.cc:1403] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.124s	user 0.014s	sys 0.000s
I20260709 13:06:55.802393 10324 consensus_queue.cc:260] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:55.804543 10324 ts_tablet_manager.cc:1434] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:06:55.804466 10328 raft_consensus.cc:359] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:55.805290 10328 raft_consensus.cc:385] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.806248 10280 raft_consensus.cc:2468] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:55.807531 10169 leader_election.cc:304] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:55.808013 10324 tablet_bootstrap.cc:492] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:55.809948 10663 raft_consensus.cc:2804] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:55.810491 10663 raft_consensus.cc:697] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:06:55.805601 10328 raft_consensus.cc:740] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.813695 10324 tablet_bootstrap.cc:654] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.811383 10663 consensus_queue.cc:237] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:55.814283 10328 consensus_queue.cc:260] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:55.816743 10328 ts_tablet_manager.cc:1434] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:06:55.817708 10328 tablet_bootstrap.cc:492] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:55.819367 10056 raft_consensus.cc:2468] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:55.826886 10325 tablet_bootstrap.cc:492] T 1f060317dcbd4f0db70f83fd18320215 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:55.827425 10325 ts_tablet_manager.cc:1403] T 1f060317dcbd4f0db70f83fd18320215 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.097s	user 0.013s	sys 0.018s
I20260709 13:06:55.830653 10325 raft_consensus.cc:359] T 1f060317dcbd4f0db70f83fd18320215 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:55.831415 10325 raft_consensus.cc:385] T 1f060317dcbd4f0db70f83fd18320215 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.831756 10325 raft_consensus.cc:740] T 1f060317dcbd4f0db70f83fd18320215 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.832535 10325 consensus_queue.cc:260] T 1f060317dcbd4f0db70f83fd18320215 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:55.834823 10325 ts_tablet_manager.cc:1434] T 1f060317dcbd4f0db70f83fd18320215 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:55.835865 10325 tablet_bootstrap.cc:492] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:55.838433 10658 consensus_queue.cc:1048] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:55.839504 10328 tablet_bootstrap.cc:654] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.839859 10652 raft_consensus.cc:493] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.840354 10652 raft_consensus.cc:515] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:55.842545 10652 leader_election.cc:290] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:55.844533 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d642d68d234d668cb67cb81b80556e" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:55.845212 10132 raft_consensus.cc:2468] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 0.
I20260709 13:06:55.846606 10020 leader_election.cc:304] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:55.847442 10652 raft_consensus.cc:2804] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.847877 10652 raft_consensus.cc:493] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.844981 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d642d68d234d668cb67cb81b80556e" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:55.848285 10652 raft_consensus.cc:3060] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.848894 10280 raft_consensus.cc:2468] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 0.
I20260709 13:06:55.849612 10651 raft_consensus.cc:493] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.850198 10651 raft_consensus.cc:515] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:55.853284 10651 leader_election.cc:290] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:55.854342 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e003c3e14c2444b937f547eef84b672" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:55.854660 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e003c3e14c2444b937f547eef84b672" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:55.855178 10132 raft_consensus.cc:2468] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 0.
I20260709 13:06:55.855301 10203 raft_consensus.cc:2468] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 0.
I20260709 13:06:55.856449 10020 leader_election.cc:304] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:55.858404 10651 raft_consensus.cc:2804] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.858964 10651 raft_consensus.cc:493] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.859407 10651 raft_consensus.cc:3060] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.863346 10652 raft_consensus.cc:515] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:55.879015 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d642d68d234d668cb67cb81b80556e" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:55.879681 10129 raft_consensus.cc:3060] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.888213 10129 raft_consensus.cc:2468] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:06:55.890183 10020 leader_election.cc:304] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:55.891058 10650 raft_consensus.cc:2804] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:55.891605 10650 raft_consensus.cc:697] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:06:55.892503 10650 consensus_queue.cc:237] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:55.893849 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d642d68d234d668cb67cb81b80556e" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:55.894666 10280 raft_consensus.cc:3060] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.890362  9953 catalog_manager.cc:5697] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.909646 10561 raft_consensus.cc:493] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.910291 10561 raft_consensus.cc:515] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:55.924396 10598 consensus_queue.cc:1048] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:55.925420  9954 catalog_manager.cc:5697] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 0 to 1, leader changed from <none> to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.869014 10325 tablet_bootstrap.cc:654] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.932955 10280 raft_consensus.cc:2468] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:06:55.930716 10652 leader_election.cc:290] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Requested vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:55.956862 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78325f9138849749ada9825bfe45cd8" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
W20260709 13:06:55.960001 10243 leader_election.cc:343] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.968861 10324 tablet_bootstrap.cc:492] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:55.969344 10324 ts_tablet_manager.cc:1403] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.162s	user 0.014s	sys 0.015s
I20260709 13:06:55.935357 10651 raft_consensus.cc:515] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:55.970362 10328 tablet_bootstrap.cc:492] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:55.968554 10561 leader_election.cc:290] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:55.954543 10598 consensus_queue.cc:1048] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:06:55.972334 10328 ts_tablet_manager.cc:1403] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.155s	user 0.025s	sys 0.006s
I20260709 13:06:55.982048 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78325f9138849749ada9825bfe45cd8" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:55.975488 10652 raft_consensus.cc:493] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.998468 10328 raft_consensus.cc:359] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:55.999331 10328 raft_consensus.cc:385] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.999722 10328 raft_consensus.cc:740] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.013224 10333 tablet_bootstrap.cc:492] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:56.013872 10333 ts_tablet_manager.cc:1403] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.257s	user 0.023s	sys 0.009s
I20260709 13:06:56.016795 10333 raft_consensus.cc:359] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:56.003916 10328 consensus_queue.cc:260] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:56.022670 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e003c3e14c2444b937f547eef84b672" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:56.023360 10206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e003c3e14c2444b937f547eef84b672" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:56.024187 10206 raft_consensus.cc:3060] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.024605 10328 ts_tablet_manager.cc:1434] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.000s
I20260709 13:06:56.025720 10328 tablet_bootstrap.cc:492] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:56.018294 10324 raft_consensus.cc:359] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:56.029831 10324 raft_consensus.cc:385] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.023392 10127 raft_consensus.cc:3060] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.991740 10652 raft_consensus.cc:515] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:56.032737 10324 raft_consensus.cc:740] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.032585 10333 raft_consensus.cc:385] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.049832 10333 raft_consensus.cc:740] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.036140 10206 raft_consensus.cc:2468] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:06:56.039397 10328 tablet_bootstrap.cc:654] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.022083 10651 leader_election.cc:290] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:56.045029 10598 raft_consensus.cc:493] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.046087 10561 raft_consensus.cc:493] T 9f7da1cc13f64f56b02defc2e119c507 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.049484 10324 consensus_queue.cc:260] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:56.054095 10561 raft_consensus.cc:515] T 9f7da1cc13f64f56b02defc2e119c507 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:56.054219 10598 raft_consensus.cc:515] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:56.055676 10324 ts_tablet_manager.cc:1434] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.086s	user 0.008s	sys 0.000s
I20260709 13:06:56.055924 10020 leader_election.cc:304] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:56.056161 10561 leader_election.cc:290] T 9f7da1cc13f64f56b02defc2e119c507 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:56.056362 10598 leader_election.cc:290] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:56.050917 10333 consensus_queue.cc:260] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:56.052260 10056 raft_consensus.cc:2468] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:56.056951 10651 raft_consensus.cc:2804] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:56.057708 10651 raft_consensus.cc:697] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:06:56.058815 10333 ts_tablet_manager.cc:1434] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.045s	user 0.004s	sys 0.002s
I20260709 13:06:56.056702 10324 tablet_bootstrap.cc:492] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:56.058645 10651 consensus_queue.cc:237] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:56.061880 10333 tablet_bootstrap.cc:492] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:56.062140 10446 raft_consensus.cc:493] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.062667 10446 raft_consensus.cc:515] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:56.064670 10446 leader_election.cc:290] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:56.066771 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f7da1cc13f64f56b02defc2e119c507" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:56.067167 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f7da1cc13f64f56b02defc2e119c507" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:56.061926 10246 leader_election.cc:304] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:56.070312 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2258980c3a7540dea8ae11850194e183" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:56.070720 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f7da1cc13f64f56b02defc2e119c507" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:56.070990 10130 raft_consensus.cc:2468] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:56.071346 10280 raft_consensus.cc:2468] T 9f7da1cc13f64f56b02defc2e119c507 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:56.071962 10658 raft_consensus.cc:2804] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:56.070124 10132 raft_consensus.cc:2468] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:56.072365 10658 raft_consensus.cc:493] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.072757 10658 raft_consensus.cc:3060] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.073263 10243 leader_election.cc:304] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:06:56.075201 10629 raft_consensus.cc:2804] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:56.075241 10169 leader_election.cc:304] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:56.075786 10629 raft_consensus.cc:493] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.076282 10629 raft_consensus.cc:3060] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.076436 10558 raft_consensus.cc:2804] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:56.076900 10558 raft_consensus.cc:493] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.077232 10558 raft_consensus.cc:3060] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.080780 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2258980c3a7540dea8ae11850194e183" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
W20260709 13:06:56.088821 10246 leader_election.cc:343] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.090296 10206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f7da1cc13f64f56b02defc2e119c507" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:56.097774 10127 raft_consensus.cc:2468] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:06:56.124398 10694 consensus_queue.cc:1048] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:56.127290 10558 raft_consensus.cc:515] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:56.129184 10206 raft_consensus.cc:2393] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50999113203c482b9b9300caa485f154 in current term 1: Already voted for candidate 8fe33410d51a469e81be6edc602d7d78 in this term.
I20260709 13:06:56.134510 10333 tablet_bootstrap.cc:654] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.109925 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78325f9138849749ada9825bfe45cd8" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:56.120005 10534 raft_consensus.cc:493] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.136461 10658 raft_consensus.cc:515] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:56.138326 10534 raft_consensus.cc:515] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:56.134212 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f7da1cc13f64f56b02defc2e119c507" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:56.100203 10629 raft_consensus.cc:515] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:56.136842 10243 leader_election.cc:304] T 9f7da1cc13f64f56b02defc2e119c507 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78
W20260709 13:06:56.142222 10020 leader_election.cc:343] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.140472 10534 leader_election.cc:290] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:56.144425 10132 raft_consensus.cc:3060] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.155543 10324 tablet_bootstrap.cc:654] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.157403 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f060317dcbd4f0db70f83fd18320215" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:56.158172 10055 raft_consensus.cc:2468] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 0.
I20260709 13:06:56.150583 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f060317dcbd4f0db70f83fd18320215" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:56.158973 10278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78325f9138849749ada9825bfe45cd8" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:56.159449 10203 raft_consensus.cc:2468] T 1f060317dcbd4f0db70f83fd18320215 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 0.
I20260709 13:06:56.159768 10278 raft_consensus.cc:2393] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa338cd132c457896b11e08d7ea787d in current term 1: Already voted for candidate 50999113203c482b9b9300caa485f154 in this term.
I20260709 13:06:56.160970 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78325f9138849749ada9825bfe45cd8" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:56.154549 10558 leader_election.cc:290] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:56.162551 10095 leader_election.cc:304] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:56.164098 10647 raft_consensus.cc:2804] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:56.164523 10647 raft_consensus.cc:493] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.164929 10647 raft_consensus.cc:3060] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.169210 10629 leader_election.cc:290] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:56.169508 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f7da1cc13f64f56b02defc2e119c507" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
W20260709 13:06:56.177210 10243 leader_election.cc:343] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.177462 10020 leader_election.cc:304] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa338cd132c457896b11e08d7ea787d; no voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:56.178275 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2258980c3a7540dea8ae11850194e183" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:56.178630 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2258980c3a7540dea8ae11850194e183" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:56.179347 10129 raft_consensus.cc:3060] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.144059 10652 leader_election.cc:290] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:56.145742 10693 raft_consensus.cc:3060] T 9f7da1cc13f64f56b02defc2e119c507 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.196388 10132 raft_consensus.cc:2468] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:56.193097 10650 raft_consensus.cc:3060] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:56.229862 10246 leader_election.cc:343] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.242174 10558 consensus_queue.cc:1048] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:56.245193 10328 tablet_bootstrap.cc:492] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:56.245803 10328 ts_tablet_manager.cc:1403] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.220s	user 0.016s	sys 0.030s
I20260709 13:06:56.250012 10325 tablet_bootstrap.cc:492] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:56.250547 10325 ts_tablet_manager.cc:1403] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.415s	user 0.014s	sys 0.007s
I20260709 13:06:56.248807 10328 raft_consensus.cc:359] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:56.252607 10328 raft_consensus.cc:385] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.252976 10328 raft_consensus.cc:740] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.253407 10325 raft_consensus.cc:359] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:56.254438 10325 raft_consensus.cc:385] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.254735 10325 raft_consensus.cc:740] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.205061 10658 leader_election.cc:290] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:56.255501 10650 raft_consensus.cc:2749] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:56.255366 10325 consensus_queue.cc:260] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:56.257759 10129 raft_consensus.cc:2468] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:56.261770 10328 consensus_queue.cc:260] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:56.262297 10243 leader_election.cc:304] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:06:56.263708 10598 raft_consensus.cc:2804] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:56.264237 10325 ts_tablet_manager.cc:1434] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:06:56.264317 10598 raft_consensus.cc:697] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:06:56.266531 10328 ts_tablet_manager.cc:1434] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.002s
I20260709 13:06:56.266742 10598 consensus_queue.cc:237] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:56.267812 10328 tablet_bootstrap.cc:492] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:56.267313  9950 catalog_manager.cc:5697] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 0 to 1, leader changed from <none> to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:56.269330 10325 tablet_bootstrap.cc:492] T a0265654cefc40a5937d7e194f03dd5c P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
W20260709 13:06:56.270094 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40841, user_credentials={real_user=slave}} blocked reactor thread for 89307.7us
I20260709 13:06:56.308676 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78325f9138849749ada9825bfe45cd8" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:56.321452 10647 raft_consensus.cc:515] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:56.322753 10324 tablet_bootstrap.cc:492] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:56.323523 10324 ts_tablet_manager.cc:1403] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.267s	user 0.026s	sys 0.036s
I20260709 13:06:56.324826 10647 leader_election.cc:290] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:56.352655 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f060317dcbd4f0db70f83fd18320215" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:56.353348 10056 raft_consensus.cc:3060] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.355872 10324 raft_consensus.cc:359] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:56.356459 10324 raft_consensus.cc:385] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.348394 10658 raft_consensus.cc:493] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.356683 10324 raft_consensus.cc:740] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.358628 10324 consensus_queue.cc:260] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:56.360617 10324 ts_tablet_manager.cc:1434] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 13:06:56.361328 10324 tablet_bootstrap.cc:492] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:56.365940 10333 tablet_bootstrap.cc:492] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:56.367951 10333 ts_tablet_manager.cc:1403] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.306s	user 0.020s	sys 0.042s
I20260709 13:06:56.371119 10333 raft_consensus.cc:359] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:56.371898 10333 raft_consensus.cc:385] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.371685 10698 consensus_queue.cc:1048] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:56.372293 10333 raft_consensus.cc:740] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.373242 10333 consensus_queue.cc:260] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:56.374033 10658 raft_consensus.cc:515] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:56.375541 10333 ts_tablet_manager.cc:1434] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:56.376190 10658 leader_election.cc:290] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:56.376673 10333 tablet_bootstrap.cc:492] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:56.378031 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0265654cefc40a5937d7e194f03dd5c" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:56.378708 10127 raft_consensus.cc:2468] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:56.384532 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0265654cefc40a5937d7e194f03dd5c" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:56.387255 10243 leader_election.cc:304] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:06:56.388089 10333 tablet_bootstrap.cc:654] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.388592 10707 raft_consensus.cc:2804] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:06:56.389140 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40841, user_credentials={real_user=slave}} blocked reactor thread for 214285us
I20260709 13:06:56.389995 10169 leader_election.cc:304] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:56.390771 10702 raft_consensus.cc:2804] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:56.391268 10702 raft_consensus.cc:697] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:06:56.389029 10707 raft_consensus.cc:493] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.392036 10707 raft_consensus.cc:3060] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.392180 10702 consensus_queue.cc:237] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
W20260709 13:06:56.398594 10243 leader_election.cc:343] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.410595 10056 raft_consensus.cc:2468] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
I20260709 13:06:56.444492 10694 raft_consensus.cc:493] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.468931 10328 tablet_bootstrap.cc:654] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.476915 10703 raft_consensus.cc:493] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.477574 10703 raft_consensus.cc:515] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:56.479959 10703 leader_election.cc:290] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:56.445263 10694 raft_consensus.cc:515] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:56.466472 10098 leader_election.cc:304] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:56.486114 10693 raft_consensus.cc:2749] T 9f7da1cc13f64f56b02defc2e119c507 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:56.509931 10534 raft_consensus.cc:493] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.505118 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f060317dcbd4f0db70f83fd18320215" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
W20260709 13:06:56.530496 10172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42425, user_credentials={real_user=slave}} blocked reactor thread for 51437.1us
I20260709 13:06:56.540486 10333 tablet_bootstrap.cc:492] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:56.542582 10333 ts_tablet_manager.cc:1403] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.166s	user 0.038s	sys 0.039s
I20260709 13:06:56.541617 10325 tablet_bootstrap.cc:654] T a0265654cefc40a5937d7e194f03dd5c P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.530259 10534 raft_consensus.cc:515] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:56.557874 10534 leader_election.cc:290] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:56.538053 10205 raft_consensus.cc:3060] T 1f060317dcbd4f0db70f83fd18320215 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:56.566181 10710 log.cc:927] Time spent T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:06:56.580183 10055 raft_consensus.cc:2468] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:56.586998 10694 leader_election.cc:290] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:56.588495 10333 raft_consensus.cc:359] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:56.592061 10446 raft_consensus.cc:493] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.596398 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abca9c95a1334ebc97985eedf46fc01f" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:56.592890 10446 raft_consensus.cc:515] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:56.609119 10446 leader_election.cc:290] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:56.535590 10647 raft_consensus.cc:2804] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:56.612620 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78325f9138849749ada9825bfe45cd8" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:56.612847 10647 raft_consensus.cc:697] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Becoming Leader. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:06:56.613526 10051 raft_consensus.cc:2393] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fe33410d51a469e81be6edc602d7d78 in current term 1: Already voted for candidate 50999113203c482b9b9300caa485f154 in this term.
I20260709 13:06:56.614372 10647 consensus_queue.cc:237] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:56.589555 10333 raft_consensus.cc:385] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.616465 10333 raft_consensus.cc:740] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.617744 10333 consensus_queue.cc:260] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:56.620513 10333 ts_tablet_manager.cc:1434] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.078s	user 0.008s	sys 0.000s
I20260709 13:06:56.620679 10246 leader_election.cc:304] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:56.627434 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f660e208f31341f0bc602253c996258c" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:56.637583 10333 tablet_bootstrap.cc:492] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548: Bootstrap starting.
I20260709 13:06:56.639169 10651 consensus_queue.cc:1048] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:56.641393 10652 consensus_queue.cc:1048] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:56.646934 10205 raft_consensus.cc:2468] T 1f060317dcbd4f0db70f83fd18320215 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
I20260709 13:06:56.651199 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f660e208f31341f0bc602253c996258c" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
W20260709 13:06:56.654712 10098 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:42425, user_credentials={real_user=slave}} blocked reactor thread for 208047us
I20260709 13:06:56.657757 10324 tablet_bootstrap.cc:654] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.663990 10714 raft_consensus.cc:2804] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:06:56.664157 10246 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:42425, user_credentials={real_user=slave}} blocked reactor thread for 43634.6us
I20260709 13:06:56.664827 10714 raft_consensus.cc:697] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
W20260709 13:06:56.665483 10246 leader_election.cc:343] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.667788 10328 tablet_bootstrap.cc:492] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:56.668313 10328 ts_tablet_manager.cc:1403] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.401s	user 0.050s	sys 0.065s
I20260709 13:06:56.665858 10714 consensus_queue.cc:237] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:56.671077 10707 raft_consensus.cc:515] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:56.671864 10325 tablet_bootstrap.cc:492] T a0265654cefc40a5937d7e194f03dd5c P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:56.672422 10325 ts_tablet_manager.cc:1403] T a0265654cefc40a5937d7e194f03dd5c P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.403s	user 0.011s	sys 0.002s
I20260709 13:06:56.673715 10707 leader_election.cc:290] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:56.678705 10280 raft_consensus.cc:2468] T 9f7da1cc13f64f56b02defc2e119c507 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:56.685678 10652 raft_consensus.cc:493] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.686257 10652 raft_consensus.cc:515] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:56.688301 10652 leader_election.cc:290] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:56.689574 10325 raft_consensus.cc:359] T a0265654cefc40a5937d7e194f03dd5c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:56.694303 10325 raft_consensus.cc:385] T a0265654cefc40a5937d7e194f03dd5c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.694722 10325 raft_consensus.cc:740] T a0265654cefc40a5937d7e194f03dd5c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.695557 10325 consensus_queue.cc:260] T a0265654cefc40a5937d7e194f03dd5c P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:56.693944 10278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78325f9138849749ada9825bfe45cd8" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
W20260709 13:06:56.678108 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40841, user_credentials={real_user=slave}} blocked reactor thread for 182415us
W20260709 13:06:56.714643 10243 leader_election.cc:343] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.728442 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2258980c3a7540dea8ae11850194e183" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:56.729372 10132 raft_consensus.cc:2393] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa338cd132c457896b11e08d7ea787d in current term 1: Already voted for candidate 50999113203c482b9b9300caa485f154 in this term.
I20260709 13:06:56.729211 10652 consensus_queue.cc:1048] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:56.717070 10243 leader_election.cc:304] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:56.739670 10325 ts_tablet_manager.cc:1434] T a0265654cefc40a5937d7e194f03dd5c P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.067s	user 0.009s	sys 0.000s
I20260709 13:06:56.740664 10325 tablet_bootstrap.cc:492] T d6c60f07656843aeac5d469a6533aaaa P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:56.748745 10598 raft_consensus.cc:2749] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:56.749660 10206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1249fe82a5ce4bd7b416e6bb46be8f79" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:56.766700 10328 raft_consensus.cc:359] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:56.767489 10328 raft_consensus.cc:385] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.750636 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2258980c3a7540dea8ae11850194e183" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:56.735906 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0265654cefc40a5937d7e194f03dd5c" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:56.764114 10333 tablet_bootstrap.cc:654] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.767961 10328 raft_consensus.cc:740] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.775888 10020 leader_election.cc:304] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa338cd132c457896b11e08d7ea787d; no voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548
W20260709 13:06:56.775322 10095 leader_election.cc:343] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:56.800765 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:56.804342 10717 raft_consensus.cc:493] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.805131 10717 raft_consensus.cc:515] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:56.789091 10203 raft_consensus.cc:3060] T a0265654cefc40a5937d7e194f03dd5c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.774214 10651 consensus_queue.cc:1048] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260709 13:06:56.760412 10276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1249fe82a5ce4bd7b416e6bb46be8f79" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:56.817433 10328 consensus_queue.cc:260] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:56.817906 10324 tablet_bootstrap.cc:492] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:56.823930 10324 ts_tablet_manager.cc:1403] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.463s	user 0.011s	sys 0.038s
I20260709 13:06:56.813608 10325 tablet_bootstrap.cc:654] T d6c60f07656843aeac5d469a6533aaaa P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.796468 10429 raft_consensus.cc:3060] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:56.819526 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40841, user_credentials={real_user=slave}} blocked reactor thread for 69625.5us
I20260709 13:06:56.814020 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abca9c95a1334ebc97985eedf46fc01f" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:56.947185 10328 ts_tablet_manager.cc:1434] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.278s	user 0.011s	sys 0.000s
I20260709 13:06:56.948354 10328 tablet_bootstrap.cc:492] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:56.954003 10717 leader_election.cc:290] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
W20260709 13:06:56.965159 10098 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42425, user_credentials={real_user=slave}} blocked reactor thread for 148680us
W20260709 13:06:56.989533 10098 leader_election.cc:343] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.990157 10098 leader_election.cc:304] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:56.991588 10647 raft_consensus.cc:2749] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:56.993767 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0265654cefc40a5937d7e194f03dd5c" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:56.994407 10130 raft_consensus.cc:3060] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.994927  9953 catalog_manager.cc:5697] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:56.998834 10169 leader_election.cc:304] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:56.999415 10333 tablet_bootstrap.cc:492] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:56.999943 10333 ts_tablet_manager.cc:1403] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.378s	user 0.025s	sys 0.017s
I20260709 13:06:57.000083 10715 raft_consensus.cc:3060] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.010934 10715 raft_consensus.cc:2749] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:57.088946 10333 raft_consensus.cc:359] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:57.090072 10333 raft_consensus.cc:385] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.090421 10333 raft_consensus.cc:740] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.093088 10333 consensus_queue.cc:260] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:57.097303 10324 raft_consensus.cc:359] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:57.098943 10324 raft_consensus.cc:385] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.099238 10324 raft_consensus.cc:740] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.104202 10324 consensus_queue.cc:260] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
W20260709 13:06:57.108145 10095 leader_election.cc:343] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.108435 10324 ts_tablet_manager.cc:1434] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.284s	user 0.008s	sys 0.001s
I20260709 13:06:57.111012 10429 raft_consensus.cc:2749] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:57.111614 10130 raft_consensus.cc:2468] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:57.112490 10324 tablet_bootstrap.cc:492] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:57.107456 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "925f342702b542288fd34463d0585a5b" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
W20260709 13:06:57.116986 10732 log.cc:927] Time spent T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:06:57.120864 10203 raft_consensus.cc:2468] T a0265654cefc40a5937d7e194f03dd5c P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:57.123023 10328 tablet_bootstrap.cc:654] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.126717 10333 ts_tablet_manager.cc:1434] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.126s	user 0.010s	sys 0.000s
I20260709 13:06:57.128612 10333 tablet_bootstrap.cc:492] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548: Bootstrap starting.
W20260709 13:06:57.139014 10095 leader_election.cc:343] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.142325 10095 leader_election.cc:304] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78
W20260709 13:06:57.142740 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:33415, user_credentials={real_user=slave}} blocked reactor thread for 44453.7us
I20260709 13:06:57.149853 10714 consensus_queue.cc:1048] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:57.151868 10718 raft_consensus.cc:2749] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:57.159536 10095 leader_election.cc:343] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.159221  9950 catalog_manager.cc:5697] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:57.162025  9954 catalog_manager.cc:5697] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 reported cstate change: term changed from 0 to 1, leader changed from <none> to 575e44d297f045a389903d65a471d548 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "575e44d297f045a389903d65a471d548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:57.179672 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "925f342702b542288fd34463d0585a5b" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:57.183277 10325 tablet_bootstrap.cc:492] T d6c60f07656843aeac5d469a6533aaaa P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:57.184029 10325 ts_tablet_manager.cc:1403] T d6c60f07656843aeac5d469a6533aaaa P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.444s	user 0.022s	sys 0.002s
I20260709 13:06:57.166505 10718 consensus_queue.cc:1048] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:06:57.192494 10629 consensus_queue.cc:1048] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:57.195992 10703 raft_consensus.cc:493] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.196746 10703 raft_consensus.cc:515] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:57.198959 10243 leader_election.cc:304] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
W20260709 13:06:57.199204 10095 leader_election.cc:343] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.200033 10095 leader_election.cc:304] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:57.204247 10534 raft_consensus.cc:2749] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:57.205811 10702 consensus_queue.cc:1048] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:57.211040 10703 leader_election.cc:290] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:57.214700 10325 raft_consensus.cc:359] T d6c60f07656843aeac5d469a6533aaaa P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:57.220480 10325 raft_consensus.cc:385] T d6c60f07656843aeac5d469a6533aaaa P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.220813 10325 raft_consensus.cc:740] T d6c60f07656843aeac5d469a6533aaaa P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.222896 10714 raft_consensus.cc:2804] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:57.223603 10714 raft_consensus.cc:697] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:06:57.223780 10694 consensus_queue.cc:1048] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:57.224452 10714 consensus_queue.cc:237] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:57.227263 10694 consensus_queue.cc:1048] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:57.231098 10325 consensus_queue.cc:260] T d6c60f07656843aeac5d469a6533aaaa P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:57.241159 10333 tablet_bootstrap.cc:654] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.241344 10718 raft_consensus.cc:493] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.245096 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6c60f07656843aeac5d469a6533aaaa" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
W20260709 13:06:57.256171 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40841, user_credentials={real_user=slave}} blocked reactor thread for 54289.8us
I20260709 13:06:57.264504 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6c60f07656843aeac5d469a6533aaaa" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:57.265272 10199 raft_consensus.cc:2468] T d6c60f07656843aeac5d469a6533aaaa P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 0.
I20260709 13:06:57.260876 10718 raft_consensus.cc:515] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:57.281121 10707 raft_consensus.cc:493] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.281862 10707 raft_consensus.cc:515] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:57.282956 10095 leader_election.cc:304] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:57.289393 10325 ts_tablet_manager.cc:1434] T d6c60f07656843aeac5d469a6533aaaa P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.105s	user 0.007s	sys 0.003s
I20260709 13:06:57.298630 10325 tablet_bootstrap.cc:492] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:57.283946 10703 consensus_queue.cc:1048] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
W20260709 13:06:57.307001 10098 leader_election.cc:343] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.309190 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f4dd5b2c17747058bab6588c63b9d72" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:57.322029 10718 leader_election.cc:290] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:57.325357 10707 leader_election.cc:290] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:57.330370 10734 raft_consensus.cc:2804] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:57.333113 10279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af4e546f1084d8cb71469fbf3886792" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:57.349231 10702 consensus_queue.cc:1048] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:57.345973 10246 leader_election.cc:343] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:57.352867 10095 leader_election.cc:343] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.366382 10324 tablet_bootstrap.cc:654] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.352177 10734 raft_consensus.cc:493] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.387328 10734 raft_consensus.cc:3060] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.390141 10718 raft_consensus.cc:493] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.391060 10718 raft_consensus.cc:515] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:57.395129 10275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f4dd5b2c17747058bab6588c63b9d72" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:57.396081 10275 raft_consensus.cc:2468] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 0.
I20260709 13:06:57.398094 10095 leader_election.cc:304] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:06:57.400434 10703 raft_consensus.cc:2804] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:57.404193 10703 raft_consensus.cc:493] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.404537 10703 raft_consensus.cc:3060] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.407714 10738 raft_consensus.cc:493] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.408394 10738 raft_consensus.cc:515] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:57.413319 10738 leader_election.cc:290] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:57.413843 10275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ea79b0bb13439c9384cba78627c40f" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:57.443197 10703 raft_consensus.cc:515] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:57.415072 10718 leader_election.cc:290] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:57.454442 10328 tablet_bootstrap.cc:492] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:57.455137 10328 ts_tablet_manager.cc:1403] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.507s	user 0.015s	sys 0.015s
I20260709 13:06:57.445055 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af4e546f1084d8cb71469fbf3886792" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:57.475811 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f4dd5b2c17747058bab6588c63b9d72" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:57.480778 10703 leader_election.cc:290] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Requested vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
W20260709 13:06:57.481464 10098 leader_election.cc:343] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.484020 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f4dd5b2c17747058bab6588c63b9d72" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
W20260709 13:06:57.485654 10095 leader_election.cc:343] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50999113203c482b9b9300caa485f154 (127.0.107.196:40841): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:57.486829 10098 leader_election.cc:343] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.469372 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f4dd5b2c17747058bab6588c63b9d72" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
W20260709 13:06:57.494067 10098 leader_election.cc:343] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.494530 10098 leader_election.cc:304] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:57.490358 10650 raft_consensus.cc:493] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.501151 10275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f4dd5b2c17747058bab6588c63b9d72" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:57.502523 10650 raft_consensus.cc:515] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:57.503327 10275 raft_consensus.cc:3060] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.504768 10650 leader_election.cc:290] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:57.505419 10129 raft_consensus.cc:2393] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50999113203c482b9b9300caa485f154 in current term 1: Already voted for candidate 575e44d297f045a389903d65a471d548 in this term.
I20260709 13:06:57.506613 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ea79b0bb13439c9384cba78627c40f" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:57.508281 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f660e208f31341f0bc602253c996258c" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:57.509867 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f660e208f31341f0bc602253c996258c" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:57.510622 10280 raft_consensus.cc:2468] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 0.
W20260709 13:06:57.511974 10020 leader_election.cc:343] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.514714 10020 leader_election.cc:304] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 8fe33410d51a469e81be6edc602d7d78
W20260709 13:06:57.513502 10098 leader_election.cc:343] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.516942 10098 leader_election.cc:304] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:57.500969 10717 raft_consensus.cc:2749] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:57.520213 10738 raft_consensus.cc:2749] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:57.524502 10650 raft_consensus.cc:2804] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:57.525025 10650 raft_consensus.cc:493] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.525390 10650 raft_consensus.cc:3060] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.526512 10734 raft_consensus.cc:515] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:57.528728 10734 leader_election.cc:290] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:57.513465 10243 leader_election.cc:304] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:57.530726 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6c60f07656843aeac5d469a6533aaaa" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:57.539388 10333 tablet_bootstrap.cc:492] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548: No bootstrap required, opened a new log
I20260709 13:06:57.540117 10333 ts_tablet_manager.cc:1403] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548: Time spent bootstrapping tablet: real 0.413s	user 0.022s	sys 0.005s
W20260709 13:06:57.540421 10098 leader_election.cc:343] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.542973 10328 raft_consensus.cc:359] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:57.559370 10328 raft_consensus.cc:385] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.559767 10328 raft_consensus.cc:740] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.560618 10328 consensus_queue.cc:260] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:57.533543 10325 tablet_bootstrap.cc:654] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.548717 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6c60f07656843aeac5d469a6533aaaa" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:57.561962 10200 raft_consensus.cc:3060] T d6c60f07656843aeac5d469a6533aaaa P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.565025 10328 ts_tablet_manager.cc:1434] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.110s	user 0.009s	sys 0.000s
I20260709 13:06:57.566145 10328 tablet_bootstrap.cc:492] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:57.550140 10333 raft_consensus.cc:359] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:57.555891 10324 tablet_bootstrap.cc:492] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:57.573307 10324 ts_tablet_manager.cc:1403] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.461s	user 0.007s	sys 0.007s
I20260709 13:06:57.576619 10324 raft_consensus.cc:359] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:57.577481 10324 raft_consensus.cc:385] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.577813 10324 raft_consensus.cc:740] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.545658 10650 raft_consensus.cc:515] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:57.578550 10324 consensus_queue.cc:260] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:57.579598 10333 raft_consensus.cc:385] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.579948 10333 raft_consensus.cc:740] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.580148 10650 leader_election.cc:290] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:57.580950 10333 consensus_queue.cc:260] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:57.579099 10275 raft_consensus.cc:2468] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
I20260709 13:06:57.583336 10324 ts_tablet_manager.cc:1434] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:06:57.583256 10629 raft_consensus.cc:2749] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:57.584386 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f660e208f31341f0bc602253c996258c" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:57.585115 10277 raft_consensus.cc:3060] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.586443 10200 raft_consensus.cc:2468] T d6c60f07656843aeac5d469a6533aaaa P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
I20260709 13:06:57.586897 10095 leader_election.cc:304] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:06:57.587700 10734 raft_consensus.cc:2804] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:57.588125 10095 leader_election.cc:304] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:06:57.589287 10703 raft_consensus.cc:2804] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:57.589782 10703 raft_consensus.cc:697] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Becoming Leader. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:06:57.590616 10703 consensus_queue.cc:237] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:57.588220 10734 raft_consensus.cc:697] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Becoming Leader. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:06:57.595413 10734 consensus_queue.cc:237] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:57.584438 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f660e208f31341f0bc602253c996258c" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:57.604722 10328 tablet_bootstrap.cc:654] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.584406 10324 tablet_bootstrap.cc:492] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:57.607848 10333 ts_tablet_manager.cc:1434] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548: Time spent starting tablet: real 0.065s	user 0.007s	sys 0.000s
W20260709 13:06:57.613181 10020 leader_election.cc:343] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.674188 10717 raft_consensus.cc:493] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.675737 10717 raft_consensus.cc:515] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:57.699373 10717 leader_election.cc:290] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:57.701879 10277 raft_consensus.cc:2468] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:06:57.722004 10325 tablet_bootstrap.cc:492] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:57.722806 10325 ts_tablet_manager.cc:1403] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.432s	user 0.018s	sys 0.008s
I20260709 13:06:57.721724 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce414a4edf504d78b1b1e83498a5cfeb" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:57.726337 10764 raft_consensus.cc:493] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.726894 10764 raft_consensus.cc:515] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
W20260709 13:06:57.730638 10095 leader_election.cc:343] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:57.745229 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:40419, user_credentials={real_user=slave}} blocked reactor thread for 48429.1us
I20260709 13:06:57.740268 10325 raft_consensus.cc:359] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:57.746114 10325 raft_consensus.cc:385] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.746423 10325 raft_consensus.cc:740] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.748404 10325 consensus_queue.cc:260] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:57.728963 10764 leader_election.cc:290] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:57.751901 10325 ts_tablet_manager.cc:1434] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:06:57.753360 10324 tablet_bootstrap.cc:654] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.755264 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48dcd8683f004a61808243f33d859911" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:57.756567 10325 tablet_bootstrap.cc:492] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:57.758056 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce414a4edf504d78b1b1e83498a5cfeb" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:57.759554 10020 leader_election.cc:304] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:57.760432 10650 raft_consensus.cc:2804] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:57.760979 10650 raft_consensus.cc:697] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:06:57.762458 10650 consensus_queue.cc:237] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
W20260709 13:06:57.764674 10246 leader_election.cc:343] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:57.765727 10098 leader_election.cc:343] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.766239 10098 leader_election.cc:304] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:57.768177 10717 raft_consensus.cc:2749] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:57.770004 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48dcd8683f004a61808243f33d859911" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:57.786690 10328 tablet_bootstrap.cc:492] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:57.787405 10328 ts_tablet_manager.cc:1403] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.221s	user 0.009s	sys 0.005s
W20260709 13:06:57.789266 10243 leader_election.cc:343] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.789804 10243 leader_election.cc:304] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:57.793133 10328 raft_consensus.cc:359] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:57.794577 10328 raft_consensus.cc:385] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.794988 10328 raft_consensus.cc:740] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.795852 10328 consensus_queue.cc:260] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:57.803952 10764 raft_consensus.cc:2749] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:57.806355 10328 ts_tablet_manager.cc:1434] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:06:57.807327 10328 tablet_bootstrap.cc:492] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:57.810472 10325 tablet_bootstrap.cc:654] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.819247 10324 tablet_bootstrap.cc:492] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:57.819823 10324 ts_tablet_manager.cc:1403] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.236s	user 0.007s	sys 0.006s
I20260709 13:06:57.821369  9954 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51788 (request call id 83) took 1086 ms. Trace:
I20260709 13:06:57.822562 10324 raft_consensus.cc:359] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:57.823259 10324 raft_consensus.cc:385] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.823529 10324 raft_consensus.cc:740] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.824185 10324 consensus_queue.cc:260] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:57.821810  9954 rpcz_store.cc:276] 0709 13:06:56.737509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.737819 (+   310us) service_pool.cc:224] Handling call
0709 13:06:57.821286 (+1083467us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:57.536925 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:57.537495 (+   570us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:57.537535 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:06:57.537548 (+    13us) tablet.cc:662] Decoding operations
0709 13:06:57.542333 (+  4785us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:57.542387 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:57.542416 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:06:57.542625 (+   209us) tablet.cc:701] Row locks acquired
0709 13:06:57.542642 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:06:57.542782 (+   140us) write_op.cc:270] Start()
0709 13:06:57.542901 (+   119us) write_op.cc:276] Timestamp: P: 1783602417542733 usec, L: 0
0709 13:06:57.542935 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:06:57.543574 (+   639us) log.cc:844] Serialized 540 byte log entry
0709 13:06:57.804325 (+260751us) op_driver.cc:464] REPLICATION: finished
0709 13:06:57.808202 (+  3877us) write_op.cc:301] APPLY: starting
0709 13:06:57.808264 (+    62us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:57.808351 (+    87us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:57.808433 (+    82us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:57.808657 (+   224us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:57.808840 (+   183us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:06:57.808887 (+    47us) write_op.cc:312] APPLY: finished
0709 13:06:57.809049 (+   162us) log.cc:844] Serialized 22 byte log entry
0709 13:06:57.810866 (+  1817us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:57.810932 (+    66us) write_op.cc:454] Released schema lock
0709 13:06:57.810971 (+    39us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":8175,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":3714,"apply.run_cpu_time_us":1391,"apply.run_wall_time_us":3025,"num_ops":1,"prepare.queue_time_us":368587,"prepare.run_cpu_time_us":2490,"prepare.run_wall_time_us":6861,"replication_time_us":261245,"thread_start_us":8279,"threads_started":1}]]}
I20260709 13:06:57.830698 10764 consensus_queue.cc:1048] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:57.834211 10764 raft_consensus.cc:493] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.834754 10764 raft_consensus.cc:515] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:57.836714 10764 leader_election.cc:290] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:57.840250 10324 ts_tablet_manager.cc:1434] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.003s
I20260709 13:06:57.841336 10324 tablet_bootstrap.cc:492] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:57.841459 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "925f342702b542288fd34463d0585a5b" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:57.842346 10129 raft_consensus.cc:2468] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:57.842964  9953 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51804 (request call id 80) took 1022 ms. Trace:
I20260709 13:06:57.847751 10324 tablet_bootstrap.cc:654] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.848621 10328 tablet_bootstrap.cc:654] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.848762 10243 leader_election.cc:304] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:06:57.849524 10658 raft_consensus.cc:2804] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:57.849749 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "925f342702b542288fd34463d0585a5b" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:57.850001 10658 raft_consensus.cc:493] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.850437 10658 raft_consensus.cc:3060] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.847806 10745 raft_consensus.cc:493] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.851773 10745 raft_consensus.cc:515] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:57.855305 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f660e208f31341f0bc602253c996258c" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:57.857010 10745 leader_election.cc:290] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:57.857976 10658 raft_consensus.cc:515] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:57.862934 10741 raft_consensus.cc:493] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.863674 10741 raft_consensus.cc:515] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
W20260709 13:06:57.867389 10243 leader_election.cc:343] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.843350  9953 rpcz_store.cc:276] 0709 13:06:56.820912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.821039 (+   127us) service_pool.cc:224] Handling call
0709 13:06:57.842906 (+1021867us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:57.331884 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:57.334652 (+  2768us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:57.334685 (+    33us) write_op.cc:435] Acquired schema lock
0709 13:06:57.334702 (+    17us) tablet.cc:662] Decoding operations
0709 13:06:57.335031 (+   329us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:57.335112 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:57.335144 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:06:57.335320 (+   176us) tablet.cc:701] Row locks acquired
0709 13:06:57.335338 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:06:57.335476 (+   138us) write_op.cc:270] Start()
0709 13:06:57.335591 (+   115us) write_op.cc:276] Timestamp: P: 1783602417335431 usec, L: 0
0709 13:06:57.335624 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:06:57.336264 (+   640us) log.cc:844] Serialized 540 byte log entry
0709 13:06:57.708901 (+372637us) op_driver.cc:464] REPLICATION: finished
0709 13:06:57.799237 (+ 90336us) write_op.cc:301] APPLY: starting
0709 13:06:57.799337 (+   100us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:57.799444 (+   107us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:57.799594 (+   150us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:57.799825 (+   231us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:57.799997 (+   172us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:06:57.800052 (+    55us) write_op.cc:312] APPLY: finished
0709 13:06:57.800246 (+   194us) log.cc:844] Serialized 22 byte log entry
0709 13:06:57.811809 (+ 11563us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:57.811885 (+    76us) write_op.cc:454] Released schema lock
0709 13:06:57.811920 (+    35us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:57.812215 (+   295us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:57.812243 (+    28us) write_op.cc:454] Released schema lock
0709 13:06:57.812859 (+   616us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:57.812896 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":27768,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":89522,"apply.run_cpu_time_us":1626,"apply.run_wall_time_us":12984,"num_ops":1,"prepare.queue_time_us":320792,"prepare.run_cpu_time_us":61219,"prepare.run_wall_time_us":114530,"raft.queue_time_us":387516,"raft.run_cpu_time_us":27801,"raft.run_wall_time_us":127304,"replication_time_us":370259,"thread_start_us":446880,"threads_started":4,"wal-append.queue_time_us":140355}]]}
I20260709 13:06:57.870003 10275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f660e208f31341f0bc602253c996258c" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:57.870956 10275 raft_consensus.cc:2393] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fe33410d51a469e81be6edc602d7d78 in current term 1: Already voted for candidate efa338cd132c457896b11e08d7ea787d in this term.
I20260709 13:06:57.876559 10169 leader_election.cc:304] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:57.877817 10736 raft_consensus.cc:3060] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.879376 10328 tablet_bootstrap.cc:492] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:57.879927 10328 ts_tablet_manager.cc:1403] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.073s	user 0.008s	sys 0.002s
I20260709 13:06:57.880612 10741 leader_election.cc:290] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:57.889743 10328 raft_consensus.cc:359] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:57.890545 10328 raft_consensus.cc:385] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.890794 10328 raft_consensus.cc:740] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.891438 10328 consensus_queue.cc:260] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:57.896898 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f4dd5b2c17747058bab6588c63b9d72" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:57.915542 10736 raft_consensus.cc:2749] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:57.916829 10324 tablet_bootstrap.cc:492] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:57.917384 10324 ts_tablet_manager.cc:1403] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.076s	user 0.018s	sys 0.034s
I20260709 13:06:57.920360 10324 raft_consensus.cc:359] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:57.921181 10324 raft_consensus.cc:385] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.921537 10324 raft_consensus.cc:740] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.926712 10658 leader_election.cc:290] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:57.926510 10328 ts_tablet_manager.cc:1434] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.046s	user 0.004s	sys 0.003s
I20260709 13:06:57.928027 10328 tablet_bootstrap.cc:492] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:57.930603  9954 catalog_manager.cc:5697] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 reported cstate change: term changed from 0 to 1, leader changed from <none> to 575e44d297f045a389903d65a471d548 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "575e44d297f045a389903d65a471d548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:57.941794  9954 catalog_manager.cc:5697] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 reported cstate change: term changed from 0 to 1, leader changed from <none> to 575e44d297f045a389903d65a471d548 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "575e44d297f045a389903d65a471d548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:57.950551 10324 consensus_queue.cc:260] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:57.960528 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f4dd5b2c17747058bab6588c63b9d72" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:57.961885 10280 raft_consensus.cc:2393] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa338cd132c457896b11e08d7ea787d in current term 1: Already voted for candidate 575e44d297f045a389903d65a471d548 in this term.
W20260709 13:06:57.963719 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:57.964687 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "925f342702b542288fd34463d0585a5b" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:57.977028 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "925f342702b542288fd34463d0585a5b" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:57.978533 10128 raft_consensus.cc:3060] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.981961 10694 consensus_queue.cc:1048] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:06:57.984884 10243 leader_election.cc:343] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.984908  9955 catalog_manager.cc:5697] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:57.987897  9955 catalog_manager.cc:5697] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.009411 10325 tablet_bootstrap.cc:492] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:58.010011 10325 ts_tablet_manager.cc:1403] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.254s	user 0.024s	sys 0.018s
I20260709 13:06:58.012820 10325 raft_consensus.cc:359] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:58.019693 10020 leader_election.cc:304] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa338cd132c457896b11e08d7ea787d; no voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548
I20260709 13:06:58.022575 10741 raft_consensus.cc:3060] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.013577 10325 raft_consensus.cc:385] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.023449 10325 raft_consensus.cc:740] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.025063 10057 raft_consensus.cc:3060] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.025048 10325 consensus_queue.cc:260] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:58.028837 10325 ts_tablet_manager.cc:1434] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:06:58.030050 10325 tablet_bootstrap.cc:492] T 4457dec3dec94c1dbac542a298184d0f P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:58.045540 10738 consensus_queue.cc:1048] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.085115 10324 ts_tablet_manager.cc:1434] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.167s	user 0.007s	sys 0.000s
I20260709 13:06:58.088763 10741 raft_consensus.cc:2749] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:58.085157  9950 catalog_manager.cc:5697] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 0 to 1, leader changed from <none> to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.091818 10738 consensus_queue.cc:1048] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:58.096103 10324 tablet_bootstrap.cc:492] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:58.111548 10325 tablet_bootstrap.cc:654] T 4457dec3dec94c1dbac542a298184d0f P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.107900 10128 raft_consensus.cc:2468] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:58.131634 10243 leader_election.cc:304] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:58.152737 10328 tablet_bootstrap.cc:654] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.137483 10714 raft_consensus.cc:2804] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.158675 10714 raft_consensus.cc:697] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:06:58.159627 10714 consensus_queue.cc:237] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:58.205838 10738 consensus_queue.cc:1048] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:58.235615 10534 consensus_queue.cc:1048] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:58.256209 10785 log.cc:927] Time spent T d6c60f07656843aeac5d469a6533aaaa P 8fe33410d51a469e81be6edc602d7d78: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:06:58.256285 10741 consensus_queue.cc:1048] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.306249 10324 tablet_bootstrap.cc:654] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.295014 10651 consensus_queue.cc:1048] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.034s
W20260709 13:06:58.339895 10172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42425, user_credentials={real_user=slave}} blocked reactor thread for 72913.7us
I20260709 13:06:58.314657 10650 raft_consensus.cc:493] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.388150  9954 catalog_manager.cc:5697] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.395001 10325 tablet_bootstrap.cc:492] T 4457dec3dec94c1dbac542a298184d0f P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:58.395666 10325 ts_tablet_manager.cc:1403] T 4457dec3dec94c1dbac542a298184d0f P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.366s	user 0.032s	sys 0.013s
I20260709 13:06:58.353803 10650 raft_consensus.cc:515] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:58.380264 10745 raft_consensus.cc:493] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.432991 10714 raft_consensus.cc:493] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.456712 10328 tablet_bootstrap.cc:492] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:58.458190 10328 ts_tablet_manager.cc:1403] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.530s	user 0.017s	sys 0.019s
I20260709 13:06:58.458993 10650 leader_election.cc:290] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:58.430027 10745 raft_consensus.cc:515] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:58.476996 10328 raft_consensus.cc:359] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:58.477842 10328 raft_consensus.cc:385] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.478096 10328 raft_consensus.cc:740] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.478740 10328 consensus_queue.cc:260] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:58.480007 10328 ts_tablet_manager.cc:1434] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 13:06:58.480827 10328 tablet_bootstrap.cc:492] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:58.485867 10324 tablet_bootstrap.cc:492] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:58.486670 10324 ts_tablet_manager.cc:1403] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.392s	user 0.033s	sys 0.021s
I20260709 13:06:58.452600 10714 raft_consensus.cc:515] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:58.510408 10714 leader_election.cc:290] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:58.513120 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ea79b0bb13439c9384cba78627c40f" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:58.514029 10127 raft_consensus.cc:2468] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:58.533301 10745 leader_election.cc:290] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:58.526093 10717 raft_consensus.cc:493] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.534171 10717 raft_consensus.cc:515] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:58.537683 10658 raft_consensus.cc:493] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.540102 10658 raft_consensus.cc:515] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:58.543514 10325 raft_consensus.cc:359] T 4457dec3dec94c1dbac542a298184d0f P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:58.544675 10325 raft_consensus.cc:385] T 4457dec3dec94c1dbac542a298184d0f P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.545217 10325 raft_consensus.cc:740] T 4457dec3dec94c1dbac542a298184d0f P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.571657 10658 leader_election.cc:290] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:58.574470 10324 raft_consensus.cc:359] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:58.575264 10324 raft_consensus.cc:385] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.575541 10324 raft_consensus.cc:740] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.576201 10324 consensus_queue.cc:260] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:58.579643 10324 ts_tablet_manager.cc:1434] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.093s	user 0.008s	sys 0.000s
I20260709 13:06:58.580071 10325 consensus_queue.cc:260] T 4457dec3dec94c1dbac542a298184d0f P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:58.584890 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4457dec3dec94c1dbac542a298184d0f" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:58.585788 10203 raft_consensus.cc:2468] T 4457dec3dec94c1dbac542a298184d0f P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:58.586925 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48dcd8683f004a61808243f33d859911" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:58.587661 10052 raft_consensus.cc:2468] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:58.589816 10717 leader_election.cc:290] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:58.590277 10243 leader_election.cc:304] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:58.587798 10328 tablet_bootstrap.cc:654] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:58.582549 10793 log.cc:927] Time spent T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:06:58.595336 10694 raft_consensus.cc:2804] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:58.595770 10694 raft_consensus.cc:493] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.596132 10694 raft_consensus.cc:3060] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.608695 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48dcd8683f004a61808243f33d859911" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:58.609470 10277 raft_consensus.cc:2468] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 0.
I20260709 13:06:58.611634 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48dcd8683f004a61808243f33d859911" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:58.612320 10280 raft_consensus.cc:2468] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:58.618494 10020 leader_election.cc:304] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:58.621002 10325 ts_tablet_manager.cc:1434] T 4457dec3dec94c1dbac542a298184d0f P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.225s	user 0.008s	sys 0.001s
I20260709 13:06:58.623449 10169 leader_election.cc:304] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:58.624958 10324 tablet_bootstrap.cc:492] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:58.629976 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48dcd8683f004a61808243f33d859911" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:58.654588 10325 tablet_bootstrap.cc:492] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:58.653875 10801 raft_consensus.cc:2804] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:06:58.655845 10098 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42425, user_credentials={real_user=slave}} blocked reactor thread for 49911.8us
I20260709 13:06:58.658875 10243 leader_election.cc:304] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:06:58.659770 10629 raft_consensus.cc:2804] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:58.660159 10629 raft_consensus.cc:493] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.660488 10629 raft_consensus.cc:3060] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.655917 10801 raft_consensus.cc:493] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:58.664455 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:40841, user_credentials={real_user=slave}} blocked reactor thread for 46101.7us
W20260709 13:06:58.667562 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:40419, user_credentials={real_user=slave}} blocked reactor thread for 142424us
I20260709 13:06:58.663746 10801 raft_consensus.cc:3060] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.684983 10629 raft_consensus.cc:515] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:58.666874 10741 raft_consensus.cc:2804] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:58.694646 10741 raft_consensus.cc:493] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.694883 10629 leader_election.cc:290] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:58.695142 10741 raft_consensus.cc:3060] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.697151 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abca9c95a1334ebc97985eedf46fc01f" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:58.704236 10325 tablet_bootstrap.cc:654] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.711973 10801 raft_consensus.cc:515] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:58.721953 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abca9c95a1334ebc97985eedf46fc01f" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:58.724539 10203 raft_consensus.cc:2393] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa338cd132c457896b11e08d7ea787d in current term 1: Already voted for candidate 8fe33410d51a469e81be6edc602d7d78 in this term.
W20260709 13:06:58.725934 10095 leader_election.cc:343] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.738173 10801 leader_election.cc:290] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Requested vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:58.740731 10057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48dcd8683f004a61808243f33d859911" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:58.741117 10772 raft_consensus.cc:493] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.741696 10772 raft_consensus.cc:515] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:58.743796 10772 leader_election.cc:290] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:58.762558 10741 raft_consensus.cc:515] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:58.764160 10057 raft_consensus.cc:2393] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fe33410d51a469e81be6edc602d7d78 in current term 1: Already voted for candidate efa338cd132c457896b11e08d7ea787d in this term.
I20260709 13:06:58.771332 10324 tablet_bootstrap.cc:654] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.778981 10694 raft_consensus.cc:515] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:58.782610 10694 leader_election.cc:290] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:58.784195 10741 leader_election.cc:290] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
W20260709 13:06:58.788686 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:33415, user_credentials={real_user=slave}} blocked reactor thread for 76696.8us
W20260709 13:06:58.797165 10098 leader_election.cc:343] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.797680 10098 leader_election.cc:304] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:58.798947 10796 raft_consensus.cc:2749] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:58.811408 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ea79b0bb13439c9384cba78627c40f" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:58.812105 10132 raft_consensus.cc:3060] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.813274 10324 tablet_bootstrap.cc:492] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:58.813822 10324 ts_tablet_manager.cc:1403] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.189s	user 0.019s	sys 0.003s
I20260709 13:06:58.816475 10324 raft_consensus.cc:359] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:58.817206 10324 raft_consensus.cc:385] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.817474 10324 raft_consensus.cc:740] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.820150 10132 raft_consensus.cc:2468] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:58.821930 10324 consensus_queue.cc:260] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:58.824558 10324 ts_tablet_manager.cc:1434] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:06:58.830170 10328 tablet_bootstrap.cc:492] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:58.830767 10328 ts_tablet_manager.cc:1403] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.350s	user 0.021s	sys 0.015s
I20260709 13:06:58.835597 10324 tablet_bootstrap.cc:492] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:58.835916 10328 raft_consensus.cc:359] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:58.836702 10328 raft_consensus.cc:385] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.837016 10328 raft_consensus.cc:740] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.837865 10328 consensus_queue.cc:260] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:58.845201 10325 tablet_bootstrap.cc:492] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:58.845881 10325 ts_tablet_manager.cc:1403] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.192s	user 0.010s	sys 0.007s
I20260709 13:06:58.852800 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4457dec3dec94c1dbac542a298184d0f" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:58.853560 10201 raft_consensus.cc:3060] T 4457dec3dec94c1dbac542a298184d0f P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.856046 10328 ts_tablet_manager.cc:1434] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:06:58.856508 10324 tablet_bootstrap.cc:654] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.867205 10325 raft_consensus.cc:359] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:58.868034 10325 raft_consensus.cc:385] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.868100 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce414a4edf504d78b1b1e83498a5cfeb" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:58.868314 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce414a4edf504d78b1b1e83498a5cfeb" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:58.868896 10129 raft_consensus.cc:2468] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 0.
I20260709 13:06:58.871447 10328 tablet_bootstrap.cc:492] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:58.871399 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48dcd8683f004a61808243f33d859911" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:58.872402 10203 raft_consensus.cc:2393] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate efa338cd132c457896b11e08d7ea787d in current term 1: Already voted for candidate 8fe33410d51a469e81be6edc602d7d78 in this term.
W20260709 13:06:58.874209 10243 consensus_peers.cc:597] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 -> Peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Couldn't send request to peer 8fe33410d51a469e81be6edc602d7d78. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:58.879562 10328 tablet_bootstrap.cc:654] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.868428 10325 raft_consensus.cc:740] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.880121 10201 raft_consensus.cc:2468] T 4457dec3dec94c1dbac542a298184d0f P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:58.882930 10325 consensus_queue.cc:260] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:58.885198 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ea79b0bb13439c9384cba78627c40f" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:58.901432 10243 leader_election.cc:304] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:06:58.904397 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48dcd8683f004a61808243f33d859911" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:58.904553 10325 ts_tablet_manager.cc:1434] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.000s
I20260709 13:06:58.905236 10324 tablet_bootstrap.cc:492] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:58.905805 10324 ts_tablet_manager.cc:1403] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.070s	user 0.009s	sys 0.003s
I20260709 13:06:58.906126 10325 tablet_bootstrap.cc:492] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:58.905141 10277 raft_consensus.cc:3060] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.912750 10818 consensus_queue.cc:1048] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:58.912041 10629 raft_consensus.cc:2804] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.918226 10629 raft_consensus.cc:697] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:06:58.919306 10629 consensus_queue.cc:237] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:58.920785 10277 raft_consensus.cc:2468] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:06:58.909365 10324 raft_consensus.cc:359] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:58.923969 10328 tablet_bootstrap.cc:492] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:58.924515 10328 ts_tablet_manager.cc:1403] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.053s	user 0.013s	sys 0.020s
I20260709 13:06:58.927079 10324 raft_consensus.cc:385] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.927520 10324 raft_consensus.cc:740] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.927377 10328 raft_consensus.cc:359] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:58.928344 10328 raft_consensus.cc:385] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.928722 10328 raft_consensus.cc:740] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.930614 10328 consensus_queue.cc:260] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:58.932222 10328 ts_tablet_manager.cc:1434] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:06:58.932320 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48dcd8683f004a61808243f33d859911" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:58.933163 10280 raft_consensus.cc:2393] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fe33410d51a469e81be6edc602d7d78 in current term 1: Already voted for candidate efa338cd132c457896b11e08d7ea787d in this term.
I20260709 13:06:58.933183 10328 tablet_bootstrap.cc:492] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154: Bootstrap starting.
I20260709 13:06:58.935706 10324 consensus_queue.cc:260] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:58.938320 10324 ts_tablet_manager.cc:1434] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 13:06:58.940083 10328 tablet_bootstrap.cc:654] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.942623 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4457dec3dec94c1dbac542a298184d0f" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:58.943383 10053 raft_consensus.cc:2468] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:58.943508 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ea79b0bb13439c9384cba78627c40f" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:58.946893 10324 tablet_bootstrap.cc:492] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:58.948606 10169 leader_election.cc:304] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:58.952458 10811 raft_consensus.cc:2749] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:58.953258 10324 tablet_bootstrap.cc:654] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.955029 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4457dec3dec94c1dbac542a298184d0f" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:58.955839 10050 raft_consensus.cc:3060] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.948596 10243 leader_election.cc:304] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:58.973116 10818 raft_consensus.cc:2804] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.980141 10818 raft_consensus.cc:697] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:06:58.984207 10818 consensus_queue.cc:237] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:58.994305 10050 raft_consensus.cc:2468] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:58.995170 10328 tablet_bootstrap.cc:492] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154: No bootstrap required, opened a new log
I20260709 13:06:58.995699 10328 ts_tablet_manager.cc:1403] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154: Time spent bootstrapping tablet: real 0.063s	user 0.015s	sys 0.015s
W20260709 13:06:59.013229 10020 leader_election.cc:343] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.015429 10324 tablet_bootstrap.cc:492] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:59.017019 10324 ts_tablet_manager.cc:1403] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.070s	user 0.010s	sys 0.006s
W20260709 13:06:59.023411 10246 leader_election.cc:343] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.010856 10629 raft_consensus.cc:493] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.035912 10629 raft_consensus.cc:515] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:59.038071 10629 leader_election.cc:290] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:59.040446 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af4e546f1084d8cb71469fbf3886792" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:59.030287  9954 catalog_manager.cc:5697] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.041154 10132 raft_consensus.cc:2468] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 0.
I20260709 13:06:59.043711 10243 leader_election.cc:304] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
W20260709 13:06:59.029173 10246 leader_election.cc:343] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.046518 10629 raft_consensus.cc:2804] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:59.064910 10629 raft_consensus.cc:493] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.063127 10325 tablet_bootstrap.cc:654] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:59.060410 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:59.070245 10629 raft_consensus.cc:3060] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:59.070206 10324 raft_consensus.cc:359] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:59.071220 10324 raft_consensus.cc:385] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:59.071540 10324 raft_consensus.cc:740] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.072320 10324 consensus_queue.cc:260] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:59.079630 10328 raft_consensus.cc:359] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:59.089279 10328 raft_consensus.cc:385] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:59.089721 10328 raft_consensus.cc:740] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.090375 10324 ts_tablet_manager.cc:1434] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.073s	user 0.008s	sys 0.001s
I20260709 13:06:59.091735 10324 tablet_bootstrap.cc:492] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:59.090437 10328 consensus_queue.cc:260] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:59.098948 10020 leader_election.cc:304] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:59.103173 10328 ts_tablet_manager.cc:1434] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154: Time spent starting tablet: real 0.107s	user 0.010s	sys 0.000s
I20260709 13:06:59.105367 10651 raft_consensus.cc:2804] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:59.112201 10651 raft_consensus.cc:697] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:06:59.113334 10651 consensus_queue.cc:237] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:59.100435 10629 raft_consensus.cc:515] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:59.151767 10629 leader_election.cc:290] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:59.152697 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af4e546f1084d8cb71469fbf3886792" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:59.153421 10129 raft_consensus.cc:3060] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:59.158407 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40841, user_credentials={real_user=slave}} blocked reactor thread for 56095.1us
I20260709 13:06:59.160241 10020 leader_election.cc:304] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d; no voters: 8fe33410d51a469e81be6edc602d7d78
I20260709 13:06:59.160998 10797 raft_consensus.cc:2804] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:59.161404 10797 raft_consensus.cc:493] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.161762 10797 raft_consensus.cc:3060] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:59.170393 10797 raft_consensus.cc:515] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:59.172176 10797 leader_election.cc:290] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:06:59.170641 10325 tablet_bootstrap.cc:492] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:59.172947 10325 ts_tablet_manager.cc:1403] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.267s	user 0.014s	sys 0.013s
I20260709 13:06:59.176170 10325 raft_consensus.cc:359] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:59.176939 10325 raft_consensus.cc:385] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:59.177219 10325 raft_consensus.cc:740] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.177953 10325 consensus_queue.cc:260] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:59.161702 10129 raft_consensus.cc:2468] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:06:59.180285 10243 leader_election.cc:304] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:06:59.181520 10324 tablet_bootstrap.cc:654] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:59.182755 10629 raft_consensus.cc:2804] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:59.183983 10629 raft_consensus.cc:697] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:06:59.186432 10629 consensus_queue.cc:237] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:59.191718 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af4e546f1084d8cb71469fbf3886792" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:59.194886 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af4e546f1084d8cb71469fbf3886792" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:59.196445 10325 ts_tablet_manager.cc:1434] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.002s
I20260709 13:06:59.197885 10325 tablet_bootstrap.cc:492] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:59.198706 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce414a4edf504d78b1b1e83498a5cfeb" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:59.199358 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce414a4edf504d78b1b1e83498a5cfeb" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:59.199971 10127 raft_consensus.cc:3060] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:59.201210 10794 raft_consensus.cc:493] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.202122 10794 raft_consensus.cc:515] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:59.205600 10764 raft_consensus.cc:493] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.205837 10325 tablet_bootstrap.cc:654] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:59.206161 10764 raft_consensus.cc:515] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:59.205469 10794 leader_election.cc:290] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:59.207129 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abca9c95a1334ebc97985eedf46fc01f" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
W20260709 13:06:59.207573 10246 leader_election.cc:343] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.207721 10129 raft_consensus.cc:2468] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:59.208227 10764 leader_election.cc:290] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:06:59.209653 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abca9c95a1334ebc97985eedf46fc01f" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:59.210194 10169 leader_election.cc:304] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:59.210343 10052 raft_consensus.cc:2468] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 0.
I20260709 13:06:59.211609 10805 raft_consensus.cc:2804] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:59.211969 10805 raft_consensus.cc:493] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.212309 10805 raft_consensus.cc:3060] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:59.212340 10808 raft_consensus.cc:493] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.213066 10808 raft_consensus.cc:515] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:59.213737 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ea0c11178d43888a5596f49ffc7a7e" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
W20260709 13:06:59.215273 10243 leader_election.cc:343] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.215529 10808 leader_election.cc:290] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:59.217484 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1249fe82a5ce4bd7b416e6bb46be8f79" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:59.218190 10280 raft_consensus.cc:2468] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 0.
I20260709 13:06:59.224050 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1249fe82a5ce4bd7b416e6bb46be8f79" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:59.225699 10199 raft_consensus.cc:2468] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 0.
I20260709 13:06:59.227326 10095 leader_election.cc:304] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:59.229404 10127 raft_consensus.cc:2468] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:06:59.229576 10796 raft_consensus.cc:2804] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:59.230083 10796 raft_consensus.cc:493] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.230451 10796 raft_consensus.cc:3060] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:59.233280 10246 leader_election.cc:343] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.229468 10805 raft_consensus.cc:515] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:59.241444 10020 leader_election.cc:304] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:59.243949 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abca9c95a1334ebc97985eedf46fc01f" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:06:59.244941 10052 raft_consensus.cc:3060] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:59.243965  9953 catalog_manager.cc:5697] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 0 to 1, leader changed from <none> to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:59.249827 10796 raft_consensus.cc:515] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:59.254138 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1249fe82a5ce4bd7b416e6bb46be8f79" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:06:59.255028 10199 raft_consensus.cc:3060] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:59.256947 10805 leader_election.cc:290] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:59.257746 10741 raft_consensus.cc:2804] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:59.258306 10741 raft_consensus.cc:697] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:06:59.259379 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abca9c95a1334ebc97985eedf46fc01f" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:06:59.259182 10741 consensus_queue.cc:237] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:59.261160 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ea0c11178d43888a5596f49ffc7a7e" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
W20260709 13:06:59.261659 10020 leader_election.cc:343] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:59.265826 10246 leader_election.cc:343] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.266448 10246 leader_election.cc:304] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:59.267750 10764 raft_consensus.cc:2749] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:59.263973 10127 raft_consensus.cc:3060] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:59.275372 10199 raft_consensus.cc:2468] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
I20260709 13:06:59.212435 10324 tablet_bootstrap.cc:492] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:59.279529 10324 ts_tablet_manager.cc:1403] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.188s	user 0.011s	sys 0.001s
I20260709 13:06:59.278116 10796 leader_election.cc:290] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:59.282472 10324 raft_consensus.cc:359] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:59.283196 10324 raft_consensus.cc:385] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:59.283461 10324 raft_consensus.cc:740] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.284355 10324 consensus_queue.cc:260] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:59.283926 10325 tablet_bootstrap.cc:492] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:59.286130 10325 ts_tablet_manager.cc:1403] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.089s	user 0.014s	sys 0.004s
I20260709 13:06:59.286479 10324 ts_tablet_manager.cc:1434] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:59.287515 10324 tablet_bootstrap.cc:492] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:06:59.289309 10325 raft_consensus.cc:359] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:59.293795 10325 raft_consensus.cc:385] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:59.276955 10095 leader_election.cc:304] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:59.298442 10796 raft_consensus.cc:2804] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:59.299041 10796 raft_consensus.cc:697] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Becoming Leader. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:06:59.299846 10796 consensus_queue.cc:237] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:59.316285 10325 raft_consensus.cc:740] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.319818 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1249fe82a5ce4bd7b416e6bb46be8f79" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:06:59.317385  9953 catalog_manager.cc:5697] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.320802 10277 raft_consensus.cc:3060] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:59.321811  9953 catalog_manager.cc:5697] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.320053 10325 consensus_queue.cc:260] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:59.296388 10127 raft_consensus.cc:2468] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:59.336090 10325 ts_tablet_manager.cc:1434] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.050s	user 0.004s	sys 0.003s
I20260709 13:06:59.339408 10325 tablet_bootstrap.cc:492] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:59.339524 10169 leader_election.cc:304] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:06:59.275897 10052 raft_consensus.cc:2468] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:06:59.342988 10805 raft_consensus.cc:2804] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:59.350003 10805 raft_consensus.cc:697] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:06:59.351192 10805 consensus_queue.cc:237] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:59.352010 10324 tablet_bootstrap.cc:654] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:59.355038 10325 tablet_bootstrap.cc:654] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:59.355334  9955 catalog_manager.cc:5697] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 reported cstate change: term changed from 0 to 1, leader changed from <none> to 575e44d297f045a389903d65a471d548 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "575e44d297f045a389903d65a471d548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.361235 10797 raft_consensus.cc:493] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.361881 10797 raft_consensus.cc:515] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:59.358949  9950 catalog_manager.cc:5697] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 0 to 1, leader changed from <none> to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.364024 10797 leader_election.cc:290] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:59.366446 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ea79b0bb13439c9384cba78627c40f" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:59.370091 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ea79b0bb13439c9384cba78627c40f" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:59.370919 10128 raft_consensus.cc:2393] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa338cd132c457896b11e08d7ea787d in current term 1: Already voted for candidate 50999113203c482b9b9300caa485f154 in this term.
I20260709 13:06:59.379581 10020 leader_election.cc:304] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa338cd132c457896b11e08d7ea787d; no voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548
I20260709 13:06:59.383949 10797 raft_consensus.cc:3060] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:59.398126 10797 raft_consensus.cc:2749] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:59.401418 10324 tablet_bootstrap.cc:492] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d: No bootstrap required, opened a new log
I20260709 13:06:59.402359 10324 ts_tablet_manager.cc:1403] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.115s	user 0.015s	sys 0.006s
I20260709 13:06:59.403601 10277 raft_consensus.cc:2468] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
I20260709 13:06:59.405306 10324 raft_consensus.cc:359] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:59.406177 10324 raft_consensus.cc:385] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:59.406486 10324 raft_consensus.cc:740] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.408124 10324 consensus_queue.cc:260] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:06:59.409864 10325 tablet_bootstrap.cc:492] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:59.410440 10325 ts_tablet_manager.cc:1403] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.071s	user 0.010s	sys 0.008s
I20260709 13:06:59.410969 10324 ts_tablet_manager.cc:1434] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:06:59.413400 10325 raft_consensus.cc:359] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:59.414331 10325 raft_consensus.cc:385] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:59.414685 10325 raft_consensus.cc:740] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.415408 10325 consensus_queue.cc:260] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:59.416968 10325 ts_tablet_manager.cc:1434] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:06:59.417961 10325 tablet_bootstrap.cc:492] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78: Bootstrap starting.
I20260709 13:06:59.425029 10325 tablet_bootstrap.cc:654] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:59.446990  9955 catalog_manager.cc:5697] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:59.451036 10325 tablet_bootstrap.cc:492] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78: No bootstrap required, opened a new log
I20260709 13:06:59.451610 10325 ts_tablet_manager.cc:1403] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.012s
I20260709 13:06:59.457418 10325 raft_consensus.cc:359] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:59.458506 10325 raft_consensus.cc:385] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:59.458845 10325 raft_consensus.cc:740] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.460214 10325 consensus_queue.cc:260] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:59.463135 10325 ts_tablet_manager.cc:1434] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:06:59.476176 10816 consensus_queue.cc:1048] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:59.497082 10816 consensus_queue.cc:1048] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:59.503719 10629 consensus_queue.cc:1048] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:59.512166 10629 consensus_queue.cc:1048] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:06:59.559167 10805 raft_consensus.cc:493] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.559976 10805 raft_consensus.cc:515] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:06:59.563695 10805 leader_election.cc:290] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:06:59.566355 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "925f342702b542288fd34463d0585a5b" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:59.573148 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "925f342702b542288fd34463d0585a5b" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:59.575841 10169 leader_election.cc:304] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548
I20260709 13:06:59.577054 10805 raft_consensus.cc:2749] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:59.580451 10805 raft_consensus.cc:493] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.580974 10805 raft_consensus.cc:515] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:06:59.584228 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce414a4edf504d78b1b1e83498a5cfeb" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:06:59.584412 10805 leader_election.cc:290] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:06:59.588311 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce414a4edf504d78b1b1e83498a5cfeb" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:06:59.590526 10132 raft_consensus.cc:2393] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fe33410d51a469e81be6edc602d7d78 in current term 1: Already voted for candidate efa338cd132c457896b11e08d7ea787d in this term.
I20260709 13:06:59.592455 10169 leader_election.cc:304] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d
I20260709 13:06:59.593538 10803 raft_consensus.cc:3060] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:59.608114 10803 raft_consensus.cc:2749] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:59.654361 10054 raft_consensus.cc:3060] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:59.655030 10629 consensus_queue.cc:1048] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:59.663316 10741 consensus_queue.cc:1048] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:59.667922 10629 consensus_queue.cc:1048] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:59.685801 10741 consensus_queue.cc:1048] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:59.808319 10651 raft_consensus.cc:493] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.828740 10651 raft_consensus.cc:515] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:06:59.841178 10201 raft_consensus.cc:3060] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:59.884917 10651 leader_election.cc:290] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
W20260709 13:06:59.872661 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:40841, user_credentials={real_user=slave}} blocked reactor thread for 44987.5us
I20260709 13:06:59.936805 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ea0c11178d43888a5596f49ffc7a7e" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:06:59.938160 10198 raft_consensus.cc:2468] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 0.
I20260709 13:06:59.951506 10797 consensus_queue.cc:1048] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:59.951542 10020 leader_election.cc:304] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:06:59.954166 10797 raft_consensus.cc:2804] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:59.954663 10797 raft_consensus.cc:493] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.959020 10741 consensus_queue.cc:1048] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:59.955039 10797 raft_consensus.cc:3060] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:59.975956 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ea0c11178d43888a5596f49ffc7a7e" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:06:59.977164 10280 raft_consensus.cc:2468] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 0.
I20260709 13:07:00.016539 10858 consensus_queue.cc:1048] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:00.093091 10805 consensus_queue.cc:1048] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:00.131858 10797 raft_consensus.cc:515] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
W20260709 13:07:00.137787 10172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42425, user_credentials={real_user=slave}} blocked reactor thread for 62643us
I20260709 13:07:00.170568 10797 leader_election.cc:290] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:07:00.180110 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ea0c11178d43888a5596f49ffc7a7e" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:00.180939 10201 raft_consensus.cc:3060] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:00.215939 10201 raft_consensus.cc:2468] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
I20260709 13:07:00.221486 10796 consensus_queue.cc:1048] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:00.249948 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ea0c11178d43888a5596f49ffc7a7e" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:00.264428 10277 raft_consensus.cc:3060] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:00.266268 10020 leader_election.cc:304] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:07:00.267112 10797 raft_consensus.cc:2804] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:00.267684 10797 raft_consensus.cc:697] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:07:00.268800 10797 consensus_queue.cc:237] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:00.283319 10805 consensus_queue.cc:1048] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:00.308055  9954 catalog_manager.cc:5697] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 0 to 1, leader changed from <none> to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:00.314061 10277 raft_consensus.cc:2468] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 1.
W20260709 13:07:00.324276 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:00.744453   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 13:07:00.773367 10172 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20260709 13:07:00.818290 10172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42425, user_credentials={real_user=slave}} blocked reactor thread for 44953.3us
I20260709 13:07:00.820608 10651 consensus_queue.cc:1048] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
W20260709 13:07:00.829510 10098 consensus_peers.cc:597] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.836704 10797 consensus_queue.cc:1048] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:07:00.844332 10246 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42425, user_credentials={real_user=slave}} blocked reactor thread for 52603us
W20260709 13:07:00.846557 10172 consensus_peers.cc:597] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.854300 10246 consensus_peers.cc:597] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.871752 10246 consensus_peers.cc:597] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.872370 10246 consensus_peers.cc:597] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.872908 10246 consensus_peers.cc:597] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.881858 10246 consensus_peers.cc:597] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.900801 10098 consensus_peers.cc:597] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.902498 10172 consensus_peers.cc:597] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.903326 10172 consensus_peers.cc:597] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.905684 10246 consensus_peers.cc:597] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.920374 10246 consensus_peers.cc:597] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.957955 10246 consensus_peers.cc:597] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.975404 10172 consensus_peers.cc:597] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.989594   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:00.990736   431 tablet_replica.cc:333] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:00.991989   431 raft_consensus.cc:2243] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.992681   431 raft_consensus.cc:2272] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.996212   431 tablet_replica.cc:333] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:00.997227   431 raft_consensus.cc:2243] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:00.998270   431 raft_consensus.cc:2272] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.001991   431 tablet_replica.cc:333] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.003274   431 raft_consensus.cc:2243] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.003960   431 raft_consensus.cc:2272] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:01.004276 10246 consensus_peers.cc:597] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:01.008003 10246 consensus_peers.cc:597] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:01.008196   431 tablet_replica.cc:333] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.009272   431 raft_consensus.cc:2243] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.010017   431 raft_consensus.cc:2272] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:01.011206 10098 consensus_peers.cc:597] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:01.013304   431 tablet_replica.cc:333] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.014389   431 raft_consensus.cc:2243] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.015019   431 raft_consensus.cc:2272] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.018463   431 tablet_replica.cc:333] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.019622   431 raft_consensus.cc:2243] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:01.020923   431 raft_consensus.cc:2272] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.024169   431 tablet_replica.cc:333] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.024991   431 raft_consensus.cc:2243] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.025570   431 raft_consensus.cc:2272] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:01.027187 10098 consensus_peers.cc:597] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:01.028486   431 tablet_replica.cc:333] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.030305   431 raft_consensus.cc:2243] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.030998   431 raft_consensus.cc:2272] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.033200   431 tablet_replica.cc:333] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.033951   431 raft_consensus.cc:2243] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.034600   431 raft_consensus.cc:2272] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.039278   431 tablet_replica.cc:333] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
W20260709 13:07:01.040129 10098 consensus_peers.cc:597] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:01.040441   431 raft_consensus.cc:2243] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:01.041500   431 raft_consensus.cc:2272] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.044467   431 tablet_replica.cc:333] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.045395   431 raft_consensus.cc:2243] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.046032   431 raft_consensus.cc:2272] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.048842   431 tablet_replica.cc:333] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.049600   431 raft_consensus.cc:2243] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:01.051403   431 raft_consensus.cc:2272] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.055593   431 tablet_replica.cc:333] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.057087   431 raft_consensus.cc:2243] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.057789   431 raft_consensus.cc:2272] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:01.058842 10098 consensus_peers.cc:597] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:01.060936   431 tablet_replica.cc:333] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.063251   431 raft_consensus.cc:2243] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.063961   431 raft_consensus.cc:2272] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.069658   431 tablet_replica.cc:333] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.070562   431 raft_consensus.cc:2243] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:01.071794   431 raft_consensus.cc:2272] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.074990   431 tablet_replica.cc:333] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.075873   431 raft_consensus.cc:2243] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.076452   431 raft_consensus.cc:2272] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:01.077019 10246 consensus_peers.cc:597] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:01.079720   431 tablet_replica.cc:333] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.080713   431 raft_consensus.cc:2243] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:01.081800   431 raft_consensus.cc:2272] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.085326   431 tablet_replica.cc:333] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.086171   431 raft_consensus.cc:2243] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:01.088577 10246 consensus_peers.cc:597] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:01.089808   431 raft_consensus.cc:2272] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:01.090970 10172 consensus_peers.cc:597] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:01.092914   431 tablet_replica.cc:333] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.093850   431 raft_consensus.cc:2243] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.094333   431 raft_consensus.cc:2272] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.096855   431 tablet_replica.cc:333] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.097486   431 raft_consensus.cc:2243] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.098067   431 raft_consensus.cc:2272] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.101037   431 tablet_replica.cc:333] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.101969   431 raft_consensus.cc:2243] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:01.103098   431 raft_consensus.cc:2272] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.106350   431 tablet_replica.cc:333] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.106972   431 raft_consensus.cc:2243] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.107496   431 raft_consensus.cc:2272] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.110289   431 tablet_replica.cc:333] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.111128   431 raft_consensus.cc:2243] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.111716   431 raft_consensus.cc:2272] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.115183   431 tablet_replica.cc:333] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.116019   431 raft_consensus.cc:2243] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.116590   431 raft_consensus.cc:2272] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.120365   431 tablet_replica.cc:333] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.121198   431 raft_consensus.cc:2243] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:01.122529   431 raft_consensus.cc:2272] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.125793   431 tablet_replica.cc:333] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.126590   431 raft_consensus.cc:2243] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.127183   431 raft_consensus.cc:2272] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.129676   431 tablet_replica.cc:333] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.130564   431 raft_consensus.cc:2243] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.131197   431 raft_consensus.cc:2272] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.134243   431 tablet_replica.cc:333] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.135035   431 raft_consensus.cc:2243] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.135722   431 raft_consensus.cc:2272] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.138883   431 tablet_replica.cc:333] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.139644   431 raft_consensus.cc:2243] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.140269   431 raft_consensus.cc:2272] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.143489   431 tablet_replica.cc:333] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.144181   431 raft_consensus.cc:2243] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.144706   431 raft_consensus.cc:2272] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.147555   431 tablet_replica.cc:333] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.148923   431 raft_consensus.cc:2243] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:01.150175   431 raft_consensus.cc:2272] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:01.152374 10172 consensus_peers.cc:597] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:01.153419   431 tablet_replica.cc:333] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.154279   431 raft_consensus.cc:2243] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:01.154749 10098 consensus_peers.cc:597] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:01.154845   431 raft_consensus.cc:2272] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.158128   431 tablet_replica.cc:333] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.158732   431 raft_consensus.cc:2243] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.159313   431 raft_consensus.cc:2272] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.161876   431 tablet_replica.cc:333] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.162602   431 raft_consensus.cc:2243] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:01.163707   431 raft_consensus.cc:2272] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.166188   431 tablet_replica.cc:333] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.166965   431 raft_consensus.cc:2243] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:01.168076   431 raft_consensus.cc:2272] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.171211   431 tablet_replica.cc:333] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.172008   431 raft_consensus.cc:2243] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.172554   431 raft_consensus.cc:2272] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:01.174719 10172 consensus_peers.cc:597] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:01.175331   431 tablet_replica.cc:333] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.175997   431 raft_consensus.cc:2243] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.176565   431 raft_consensus.cc:2272] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.179823   431 tablet_replica.cc:333] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.180650   431 raft_consensus.cc:2243] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.181242   431 raft_consensus.cc:2272] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.184286   431 tablet_replica.cc:333] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.184805   431 raft_consensus.cc:2243] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.185380   431 raft_consensus.cc:2272] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.187899   431 tablet_replica.cc:333] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.188455   431 raft_consensus.cc:2243] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:01.189361   431 raft_consensus.cc:2272] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.191550   431 tablet_replica.cc:333] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.192134   431 raft_consensus.cc:2243] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:01.193066   431 raft_consensus.cc:2272] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.195842   431 tablet_replica.cc:333] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
W20260709 13:07:01.196549 10098 consensus_peers.cc:597] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:01.196549   431 raft_consensus.cc:2243] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:01.198307   431 raft_consensus.cc:2272] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.201123   431 tablet_replica.cc:333] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.201912   431 raft_consensus.cc:2243] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.202478   431 raft_consensus.cc:2272] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:01.203502 10172 consensus_peers.cc:597] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:01.205179   431 tablet_replica.cc:333] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.205852   431 raft_consensus.cc:2243] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.206319   431 raft_consensus.cc:2272] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.209376   431 tablet_replica.cc:333] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d: stopping tablet replica
I20260709 13:07:01.210045   431 raft_consensus.cc:2243] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.210536   431 raft_consensus.cc:2272] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:01.238740 10172 consensus_peers.cc:597] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:01.286412 10246 consensus_peers.cc:597] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:01.300778 10172 consensus_peers.cc:597] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:01.322584   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 13:07:01.541162   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:01.560340 10897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:01.565128 10898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:01.568343 10900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:01.569123   431 server_base.cc:1061] running on GCE node
I20260709 13:07:01.571712   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:01.572001   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:01.572238   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602421572219 us; error 0 us; skew 500 ppm
I20260709 13:07:01.572938   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:01.577678   431 webserver.cc:533] Webserver started at http://127.0.107.193:38361/ using document root <none> and password file <none>
I20260709 13:07:01.578404   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:01.578665   431 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:07:01.586341   431 fs_manager.cc:714] Time spent opening directory manager: real 0.006s	user 0.003s	sys 0.005s
I20260709 13:07:01.598986 10905 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:01.600155   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.005s	sys 0.000s
I20260709 13:07:01.600592   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-0-root
uuid: "efa338cd132c457896b11e08d7ea787d"
format_stamp: "Formatted at 2026-07-09 13:06:47 on dist-test-slave-mxd3"
I20260709 13:07:01.601219   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602054858684-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:01.625012   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:01.626863   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:01.629030   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:01.639564 10912 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 13:07:02.022522   431 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:07:02.023025   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.390s	user 0.007s	sys 0.003s
I20260709 13:07:02.023653   431 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:07:02.035998 10912 tablet_bootstrap.cc:492] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:02.059378 10860 raft_consensus.cc:493] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.060187 10860 raft_consensus.cc:515] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:02.063155 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f660e208f31341f0bc602253c996258c" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
W20260709 13:07:02.068037 10246 leader_election.cc:336] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.068574 10246 leader_election.cc:304] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.069428 10694 raft_consensus.cc:2749] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:02.069746 10912 tablet_bootstrap.cc:492] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:02.070842 10912 tablet_bootstrap.cc:492] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:02.071501 10912 ts_tablet_manager.cc:1403] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.016s
I20260709 13:07:02.072057 10860 leader_election.cc:290] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:02.075266 10912 raft_consensus.cc:359] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:02.075945 10912 raft_consensus.cc:740] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.076646 10912 consensus_queue.cc:260] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:02.078958 10912 ts_tablet_manager.cc:1434] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:07:02.079658 10912 tablet_bootstrap.cc:492] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:02.092924 10912 tablet_bootstrap.cc:492] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:02.094269 10912 tablet_bootstrap.cc:492] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:02.095090 10912 ts_tablet_manager.cc:1403] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
I20260709 13:07:02.098031 10912 raft_consensus.cc:359] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:02.098784 10912 raft_consensus.cc:740] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.099996 10912 consensus_queue.cc:260] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:02.112299 10912 ts_tablet_manager.cc:1434] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 13:07:02.115149 10912 tablet_bootstrap.cc:492] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:02.131932 10912 tablet_bootstrap.cc:492] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:02.132902 10912 tablet_bootstrap.cc:492] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:02.133586 10912 ts_tablet_manager.cc:1403] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.004s
I20260709 13:07:02.136121 10912 raft_consensus.cc:359] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:02.136807 10912 raft_consensus.cc:740] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.137629 10912 consensus_queue.cc:260] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:02.139845 10912 ts_tablet_manager.cc:1434] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:07:02.140632 10912 tablet_bootstrap.cc:492] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:02.169410 10912 tablet_bootstrap.cc:492] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:02.170470 10912 tablet_bootstrap.cc:492] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:02.171339 10912 ts_tablet_manager.cc:1403] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.031s	user 0.020s	sys 0.000s
I20260709 13:07:02.173861 10912 raft_consensus.cc:359] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:02.174764 10912 raft_consensus.cc:740] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.175360 10890 raft_consensus.cc:493] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.175519 10912 consensus_queue.cc:260] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:02.176354 10890 raft_consensus.cc:515] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.177970 10912 ts_tablet_manager.cc:1434] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.004s
I20260709 13:07:02.178783 10912 tablet_bootstrap.cc:492] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:02.187462 10890 leader_election.cc:290] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:07:02.187621 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce414a4edf504d78b1b1e83498a5cfeb" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
W20260709 13:07:02.191689 10098 leader_election.cc:336] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.192276 10098 leader_election.cc:304] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.193115 10890 raft_consensus.cc:2749] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:02.199165 10890 raft_consensus.cc:493] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.199710 10890 raft_consensus.cc:515] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.201872 10890 leader_election.cc:290] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:07:02.203747 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d98d0921054d6a932a308e4adb1937" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:07:02.207489 10890 raft_consensus.cc:493] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
W20260709 13:07:02.208418 10098 leader_election.cc:336] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.208921 10098 leader_election.cc:304] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.208066 10890 raft_consensus.cc:515] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:02.211071 10895 raft_consensus.cc:2749] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:02.212420 10276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d642d68d234d668cb67cb81b80556e" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:07:02.214838 10890 leader_election.cc:290] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
W20260709 13:07:02.220569 10098 leader_election.cc:336] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.221737 10098 leader_election.cc:304] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.222752 10890 raft_consensus.cc:2749] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:02.226452 10912 tablet_bootstrap.cc:492] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:02.227581 10912 tablet_bootstrap.cc:492] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:02.228358 10912 ts_tablet_manager.cc:1403] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.050s	user 0.017s	sys 0.004s
I20260709 13:07:02.231048 10912 raft_consensus.cc:359] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.231802 10912 raft_consensus.cc:740] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.232681 10912 consensus_queue.cc:260] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.235282 10912 ts_tablet_manager.cc:1434] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:07:02.236280 10912 tablet_bootstrap.cc:492] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:02.255517 10694 raft_consensus.cc:493] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.256158 10694 raft_consensus.cc:515] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:02.258437 10694 leader_election.cc:290] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
W20260709 13:07:02.266175 10246 leader_election.cc:336] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.270712 10805 raft_consensus.cc:493] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.271471 10805 raft_consensus.cc:515] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.272514 10694 raft_consensus.cc:493] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.273106 10694 raft_consensus.cc:515] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:02.275368 10694 leader_election.cc:290] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:02.276352 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d966d297f346da9e984aa2e181590a" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:07:02.282433 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d98d0921054d6a932a308e4adb1937" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:07:02.283092 10132 raft_consensus.cc:2468] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:07:02.284663 10169 leader_election.cc:304] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:07:02.285473 10891 raft_consensus.cc:2804] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.285864 10891 raft_consensus.cc:493] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.286197 10891 raft_consensus.cc:3060] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:02.288820 10246 leader_election.cc:336] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.289582 10246 leader_election.cc:304] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.290700 10694 raft_consensus.cc:2749] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:02.293272 10694 raft_consensus.cc:493] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.294085 10694 raft_consensus.cc:515] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:02.296434 10694 leader_election.cc:290] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:07:02.299310 10694 raft_consensus.cc:493] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.300055 10805 leader_election.cc:290] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:07:02.299844 10694 raft_consensus.cc:515] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:02.304931 10694 leader_election.cc:290] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:02.299481 10891 raft_consensus.cc:515] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.311035 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fa9341e7fbe49e1a832721bfa4abff3" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
W20260709 13:07:02.315182 10172 leader_election.cc:336] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.320194 10912 tablet_bootstrap.cc:492] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:02.321429 10912 tablet_bootstrap.cc:492] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:02.322445 10912 ts_tablet_manager.cc:1403] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.087s	user 0.052s	sys 0.028s
I20260709 13:07:02.325316 10912 raft_consensus.cc:359] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.326089 10912 raft_consensus.cc:740] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.326830 10912 consensus_queue.cc:260] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.327092 10923 raft_consensus.cc:493] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.328197 10923 raft_consensus.cc:515] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:02.329268 10912 ts_tablet_manager.cc:1434] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:07:02.330266 10912 tablet_bootstrap.cc:492] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:02.330433 10923 leader_election.cc:290] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:02.334795 10276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f660e208f31341f0bc602253c996258c" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:07:02.335618 10276 raft_consensus.cc:2468] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:07:02.343843 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d642d68d234d668cb67cb81b80556e" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:07:02.344473 10129 raft_consensus.cc:2468] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:07:02.347167 10169 leader_election.cc:304] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:07:02.348773 10891 leader_election.cc:290] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:07:02.353022 10923 raft_consensus.cc:2804] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.364400 10923 raft_consensus.cc:493] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.364909 10923 raft_consensus.cc:3060] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.367154 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d98d0921054d6a932a308e4adb1937" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:07:02.368036 10129 raft_consensus.cc:3060] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.368709 10890 raft_consensus.cc:493] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.369261 10890 raft_consensus.cc:515] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:02.371353 10890 leader_election.cc:290] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:02.373308 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fa9341e7fbe49e1a832721bfa4abff3" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:07:02.356098 10243 leader_election.cc:304] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.374287 10280 raft_consensus.cc:2468] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
I20260709 13:07:02.375363 10694 raft_consensus.cc:2749] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:07:02.356252 10172 leader_election.cc:336] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.360921 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccaebf4041f14bada42fd7122f34419c" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:07:02.376364 10201 raft_consensus.cc:2468] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:07:02.377410 10095 leader_election.cc:304] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:07:02.379163 10890 raft_consensus.cc:2804] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.379707 10890 raft_consensus.cc:493] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.380189 10890 raft_consensus.cc:3060] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:02.380447 10172 leader_election.cc:336] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.375741 10243 leader_election.cc:304] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:07:02.390093 10694 raft_consensus.cc:2804] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.389284 10921 raft_consensus.cc:493] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.390509 10694 raft_consensus.cc:493] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.391074 10694 raft_consensus.cc:3060] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:02.393504 10246 leader_election.cc:336] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.390789 10921 raft_consensus.cc:515] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:02.400447 10895 raft_consensus.cc:493] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.401181   431 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:07:02.403532   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.380s	user 0.294s	sys 0.018s
I20260709 13:07:02.403183 10895 raft_consensus.cc:515] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.407033 10895 leader_election.cc:290] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:07:02.408660 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a64aca789f4ea8a4925aa79dcb0235" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:07:02.409934 10243 leader_election.cc:304] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
W20260709 13:07:02.412726 10246 leader_election.cc:336] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.412395 10860 raft_consensus.cc:2804] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.414095 10860 raft_consensus.cc:493] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.414480 10860 raft_consensus.cc:3060] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.425967 10891 raft_consensus.cc:493] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.427019 10805 raft_consensus.cc:493] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.426934 10891 raft_consensus.cc:515] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.427842 10805 raft_consensus.cc:515] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:02.438287 10921 leader_election.cc:290] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:02.439150 10921 raft_consensus.cc:493] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.439859 10921 raft_consensus.cc:515] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:02.398058 10923 raft_consensus.cc:515] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
W20260709 13:07:02.442538 10172 leader_election.cc:336] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.443688 10129 raft_consensus.cc:2468] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 2.
I20260709 13:07:02.448628 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a64aca789f4ea8a4925aa79dcb0235" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:07:02.449362 10132 raft_consensus.cc:2468] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:07:02.450646 10169 leader_election.cc:304] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:07:02.446437 10805 leader_election.cc:290] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:02.452594 10805 raft_consensus.cc:2804] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.452962 10921 leader_election.cc:290] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:02.453217 10805 raft_consensus.cc:493] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.454641 10805 raft_consensus.cc:3060] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.455041 10891 leader_election.cc:290] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
W20260709 13:07:02.459514 10098 leader_election.cc:336] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.453366 10920 raft_consensus.cc:493] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.460870 10920 raft_consensus.cc:515] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.463069 10920 leader_election.cc:290] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:07:02.452626 10694 raft_consensus.cc:515] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
W20260709 13:07:02.464035 10098 leader_election.cc:336] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.463862 10890 raft_consensus.cc:515] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
W20260709 13:07:02.453562 10172 leader_election.cc:336] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.465220 10694 leader_election.cc:290] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:07:02.464604 10098 leader_election.cc:304] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.455205 10169 leader_election.cc:304] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.467998 10920 raft_consensus.cc:493] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.468549 10920 raft_consensus.cc:515] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:02.469014 10629 raft_consensus.cc:493] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.469544 10629 raft_consensus.cc:515] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:02.461184 10860 raft_consensus.cc:515] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
W20260709 13:07:02.470937 10246 leader_election.cc:336] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.471692 10891 raft_consensus.cc:2804] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.471899 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52ffbaf3a54145c2a791c4a67dbdf64a" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:07:02.472435 10891 raft_consensus.cc:697] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 2 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:07:02.473511 10629 leader_election.cc:290] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:02.473417 10891 consensus_queue.cc:237] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.486183 10926 raft_consensus.cc:2749] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:07:02.486627 10246 leader_election.cc:336] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.487637 10923 leader_election.cc:290] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: Requested vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:02.485459 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d642d68d234d668cb67cb81b80556e" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:07:02.492126 10127 raft_consensus.cc:3060] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:02.489861 10172 leader_election.cc:336] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
W20260709 13:07:02.492806 10172 leader_election.cc:336] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
W20260709 13:07:02.494473 10172 leader_election.cc:336] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.494552 10276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d966d297f346da9e984aa2e181590a" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:07:02.495314 10276 raft_consensus.cc:2468] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:07:02.495604 10920 leader_election.cc:290] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:02.497731 10860 leader_election.cc:290] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:02.498507 10912 tablet_bootstrap.cc:492] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:02.499797 10912 tablet_bootstrap.cc:492] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:02.500490 10912 ts_tablet_manager.cc:1403] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.170s	user 0.058s	sys 0.053s
W20260709 13:07:02.503557 10246 leader_election.cc:336] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.504199 10172 leader_election.cc:304] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.503492 10912 raft_consensus.cc:359] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.505795 10912 raft_consensus.cc:740] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.506412 10912 consensus_queue.cc:260] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.509341 10127 raft_consensus.cc:2468] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 2.
I20260709 13:07:02.510030 10890 leader_election.cc:290] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 election: Requested vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:02.511241 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0981226de91f4f9e96cf2e8b926aaf19" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:07:02.511476 10276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48dcd8683f004a61808243f33d859911" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:07:02.512267 10912 ts_tablet_manager.cc:1434] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 13:07:02.510237 10920 raft_consensus.cc:2749] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:02.513262 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f660e208f31341f0bc602253c996258c" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:02.513959 10280 raft_consensus.cc:3060] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.513170 10912 tablet_bootstrap.cc:492] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:02.514446 10276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ea0c11178d43888a5596f49ffc7a7e" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:07:02.515105 10276 raft_consensus.cc:2468] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:07:02.514997 10805 raft_consensus.cc:515] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.516207 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ea0c11178d43888a5596f49ffc7a7e" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:07:02.516857 10200 raft_consensus.cc:2468] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:07:02.517153 10805 leader_election.cc:290] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:02.525159 10276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fa9341e7fbe49e1a832721bfa4abff3" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:02.525916 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccaebf4041f14bada42fd7122f34419c" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
W20260709 13:07:02.526320 10098 leader_election.cc:336] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.526556 10200 raft_consensus.cc:3060] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.525229 10169 leader_election.cc:304] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.527798 10169 leader_election.cc:304] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.529052 10921 raft_consensus.cc:2804] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.529538 10921 raft_consensus.cc:493] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.529951 10921 raft_consensus.cc:3060] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.530515 10280 raft_consensus.cc:2468] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 2.
I20260709 13:07:02.532183 10923 raft_consensus.cc:493] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.529214 10169 leader_election.cc:304] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.525913 10276 raft_consensus.cc:3060] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.534427 10860 raft_consensus.cc:493] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.532445 10890 raft_consensus.cc:493] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.537133 10890 raft_consensus.cc:515] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.538074 10243 leader_election.cc:304] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.536089 10924 raft_consensus.cc:2749] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:02.539075 10928 raft_consensus.cc:2804] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.540503 10928 raft_consensus.cc:697] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:07:02.541388 10928 consensus_queue.cc:237] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:02.533324 10923 raft_consensus.cc:515] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.545971 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a64aca789f4ea8a4925aa79dcb0235" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:07:02.546816 10127 raft_consensus.cc:3060] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.548600 10169 leader_election.cc:304] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:07:02.549536 10169 leader_election.cc:304] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:07:02.529084 10920 raft_consensus.cc:2749] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:02.550510 10924 raft_consensus.cc:2804] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.551301 10805 raft_consensus.cc:2804] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.551653 10924 raft_consensus.cc:493] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.552067 10924 raft_consensus.cc:3060] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.552196 10805 raft_consensus.cc:697] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [term 2 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:07:02.553215 10805 consensus_queue.cc:237] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:02.536710 10860 raft_consensus.cc:515] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
W20260709 13:07:02.550992 10172 leader_election.cc:336] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.555316 10890 leader_election.cc:290] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
W20260709 13:07:02.556891 10172 leader_election.cc:336] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.557705 10694 raft_consensus.cc:493] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
W20260709 13:07:02.559296 10172 leader_election.cc:336] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.544471 10923 leader_election.cc:290] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:07:02.562050 10694 raft_consensus.cc:515] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:02.564630 10127 raft_consensus.cc:2468] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 2.
I20260709 13:07:02.567090 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e003c3e14c2444b937f547eef84b672" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:07:02.570246 10132 raft_consensus.cc:2468] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:07:02.575265 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e003c3e14c2444b937f547eef84b672" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:07:02.575994 10199 raft_consensus.cc:2468] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
I20260709 13:07:02.578069  9955 catalog_manager.cc:5697] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 1 to 2, leader changed from efa338cd132c457896b11e08d7ea787d (127.0.107.193) to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:02.582542 10169 leader_election.cc:304] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:07:02.584446 10169 leader_election.cc:304] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.583984 10095 leader_election.cc:304] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:07:02.567533 10694 leader_election.cc:290] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:02.595239 10890 raft_consensus.cc:2804] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.595713 10890 raft_consensus.cc:493] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.596084 10890 raft_consensus.cc:3060] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.604018 10890 raft_consensus.cc:515] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.565827 10860 leader_election.cc:290] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:02.618603 10922 raft_consensus.cc:2804] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.619400 10922 raft_consensus.cc:697] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [term 2 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:07:02.619762 10276 raft_consensus.cc:2468] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 2.
I20260709 13:07:02.620510 10922 consensus_queue.cc:237] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.622725 10095 leader_election.cc:304] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.626055 10924 raft_consensus.cc:515] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:02.628302 10895 raft_consensus.cc:2804] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.629024 10895 raft_consensus.cc:697] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [term 2 LEADER]: Becoming Leader. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:07:02.629973 10895 consensus_queue.cc:237] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:02.624289 10200 raft_consensus.cc:2468] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 2.
I20260709 13:07:02.631325 10924 leader_election.cc:290] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: Requested vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
W20260709 13:07:02.636029 10098 leader_election.cc:336] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.637565 10929 raft_consensus.cc:493] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.638188 10929 raft_consensus.cc:515] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.624065  9950 catalog_manager.cc:5697] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 1 to 2, leader changed from efa338cd132c457896b11e08d7ea787d (127.0.107.193) to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:02.633205 10246 leader_election.cc:336] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
W20260709 13:07:02.641623 10246 leader_election.cc:336] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.648301 10929 leader_election.cc:290] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:07:02.615471 10890 leader_election.cc:290] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:07:02.654776 10921 raft_consensus.cc:515] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
W20260709 13:07:02.658695 10098 leader_election.cc:336] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.657310 10921 leader_election.cc:290] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: Requested vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:02.592996 10891 raft_consensus.cc:2804] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:07:02.672621 10098 leader_election.cc:336] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.673431 10890 raft_consensus.cc:493] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.674393 10890 raft_consensus.cc:515] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
W20260709 13:07:02.670958 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:33415, user_credentials={real_user=slave}} blocked reactor thread for 86487us
I20260709 13:07:02.676617 10890 leader_election.cc:290] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:07:02.667328 10891 raft_consensus.cc:493] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.680680 10891 raft_consensus.cc:3060] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.682281 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e003c3e14c2444b937f547eef84b672" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:02.684379 10934 raft_consensus.cc:493] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.686362 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d139f1fbdd384830bc7ee062ab3d281c" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:07:02.687124 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52ffbaf3a54145c2a791c4a67dbdf64a" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
W20260709 13:07:02.688036 10098 leader_election.cc:336] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.690419 10201 raft_consensus.cc:2468] T d139f1fbdd384830bc7ee062ab3d281c P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
I20260709 13:07:02.689808 10934 raft_consensus.cc:515] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.691314 10199 raft_consensus.cc:2468] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
I20260709 13:07:02.693856 10934 leader_election.cc:290] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:07:02.692265 10243 leader_election.cc:304] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.695220 10860 raft_consensus.cc:2804] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.695608 10860 raft_consensus.cc:493] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.696007 10860 raft_consensus.cc:3060] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.706068 10860 raft_consensus.cc:515] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:02.708149 10243 leader_election.cc:304] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.711625 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce414a4edf504d78b1b1e83498a5cfeb" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:07:02.715860 10127 raft_consensus.cc:2468] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 1.
I20260709 13:07:02.719059 10935 raft_consensus.cc:2804] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.719658 10935 raft_consensus.cc:697] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:07:02.720680 10935 consensus_queue.cc:237] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:02.725762 10169 leader_election.cc:304] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:07:02.726366 10860 leader_election.cc:290] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:02.727653 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48dcd8683f004a61808243f33d859911" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:07:02.729400 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0981226de91f4f9e96cf2e8b926aaf19" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:07:02.729925 10199 raft_consensus.cc:2468] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
I20260709 13:07:02.730067 10201 raft_consensus.cc:2468] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 1.
W20260709 13:07:02.728083 10246 leader_election.cc:336] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
W20260709 13:07:02.725342 10172 leader_election.cc:336] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.735832 10934 raft_consensus.cc:2804] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.736393 10934 raft_consensus.cc:493] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.736850 10934 raft_consensus.cc:3060] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.742842 10891 raft_consensus.cc:515] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.744175 10200 raft_consensus.cc:2393] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 575e44d297f045a389903d65a471d548 in current term 2: Already voted for candidate 8fe33410d51a469e81be6edc602d7d78 in this term.
I20260709 13:07:02.752509 10891 leader_election.cc:290] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:07:02.753881 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e003c3e14c2444b937f547eef84b672" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:07:02.755115 10132 raft_consensus.cc:2393] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fe33410d51a469e81be6edc602d7d78 in current term 2: Already voted for candidate 575e44d297f045a389903d65a471d548 in this term.
I20260709 13:07:02.760247 10095 leader_election.cc:304] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.761576 10095 leader_election.cc:304] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.763105 10890 raft_consensus.cc:2804] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.763625 10890 raft_consensus.cc:493] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.764055 10890 raft_consensus.cc:3060] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.769591 10934 raft_consensus.cc:515] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.773818 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ea0c11178d43888a5596f49ffc7a7e" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:02.792227 10201 raft_consensus.cc:2393] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 50999113203c482b9b9300caa485f154 in current term 2: Already voted for candidate 8fe33410d51a469e81be6edc602d7d78 in this term.
I20260709 13:07:02.777396 10276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ea0c11178d43888a5596f49ffc7a7e" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:02.793773 10276 raft_consensus.cc:2393] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8fe33410d51a469e81be6edc602d7d78 in current term 2: Already voted for candidate 50999113203c482b9b9300caa485f154 in this term.
I20260709 13:07:02.777720 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d966d297f346da9e984aa2e181590a" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:02.795219 10280 raft_consensus.cc:3060] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.796728 10934 leader_election.cc:290] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:07:02.782027 10095 leader_election.cc:304] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 575e44d297f045a389903d65a471d548; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.799206 10243 leader_election.cc:304] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.799872 10929 raft_consensus.cc:2749] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:02.800081 10935 raft_consensus.cc:2804] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.782223 10895 raft_consensus.cc:2804] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.800556 10935 raft_consensus.cc:493] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.800751 10895 raft_consensus.cc:493] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.801048 10935 raft_consensus.cc:3060] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.801280 10895 raft_consensus.cc:3060] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.808068 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce414a4edf504d78b1b1e83498a5cfeb" candidate_uuid: "8fe33410d51a469e81be6edc602d7d78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:07:02.808848 10132 raft_consensus.cc:3060] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.810315 10935 raft_consensus.cc:515] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:02.812536 10243 leader_election.cc:304] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.816469 10860 raft_consensus.cc:2804] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.816926 10860 raft_consensus.cc:493] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:02.817330 10860 raft_consensus.cc:3060] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.819320 10243 leader_election.cc:304] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.783521 10890 raft_consensus.cc:515] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.780057 10912 tablet_bootstrap.cc:492] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:02.778728  9954 catalog_manager.cc:5697] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 reported cstate change: term changed from 1 to 2, leader changed from efa338cd132c457896b11e08d7ea787d (127.0.107.193) to 575e44d297f045a389903d65a471d548 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "575e44d297f045a389903d65a471d548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:02.821429 10912 tablet_bootstrap.cc:492] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:02.822033 10890 leader_election.cc:290] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:07:02.822434 10912 ts_tablet_manager.cc:1403] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.309s	user 0.034s	sys 0.026s
W20260709 13:07:02.824849 10246 leader_election.cc:336] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.825367 10912 raft_consensus.cc:359] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:02.807318 10280 raft_consensus.cc:2468] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 2.
I20260709 13:07:02.826709 10912 raft_consensus.cc:740] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.827622 10912 consensus_queue.cc:260] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
W20260709 13:07:02.828102 10098 leader_election.cc:336] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.828104 10928 raft_consensus.cc:2749] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:07:02.782330 10172 leader_election.cc:336] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.820101 10935 leader_election.cc:290] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
W20260709 13:07:02.830051 10172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:42425, user_credentials={real_user=slave}} blocked reactor thread for 47748.7us
W20260709 13:07:02.831107 10172 leader_election.cc:336] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.832615 10169 leader_election.cc:304] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.835230 10132 raft_consensus.cc:2468] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe33410d51a469e81be6edc602d7d78 in term 2.
I20260709 13:07:02.837983 10934 raft_consensus.cc:2749] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:02.856654 10169 leader_election.cc:304] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.830044 10912 ts_tablet_manager.cc:1434] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:02.859469 10922 raft_consensus.cc:2804] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.860085 10912 tablet_bootstrap.cc:492] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:02.861254 10922 raft_consensus.cc:697] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 2 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:07:02.862346 10922 consensus_queue.cc:237] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
W20260709 13:07:02.858031 10172 leader_election.cc:336] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.845655 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48dcd8683f004a61808243f33d859911" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:02.840270 10860 raft_consensus.cc:515] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:02.866047 10201 raft_consensus.cc:3060] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.867329 10860 leader_election.cc:290] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:02.846380 10895 raft_consensus.cc:515] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
W20260709 13:07:02.872855 10246 leader_election.cc:336] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.874523 10169 leader_election.cc:304] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
W20260709 13:07:02.873006 10172 leader_election.cc:336] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.875581 10172 leader_election.cc:304] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78; no voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.875746 10895 leader_election.cc:290] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:07:02.876356 10891 raft_consensus.cc:2804] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.876930 10891 raft_consensus.cc:697] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 2 LEADER]: Becoming Leader. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:07:02.877830 10891 consensus_queue.cc:237] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:02.881539 10934 raft_consensus.cc:2749] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:02.884665 10201 raft_consensus.cc:2468] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 2.
W20260709 13:07:02.888411 10098 leader_election.cc:336] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.892347 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0981226de91f4f9e96cf2e8b926aaf19" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:02.893085 10200 raft_consensus.cc:3060] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.901293 10243 leader_election.cc:304] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.902204 10860 raft_consensus.cc:2804] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.902670 10860 raft_consensus.cc:697] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:07:02.903493 10860 consensus_queue.cc:237] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:02.905830 10200 raft_consensus.cc:2468] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 2.
I20260709 13:07:02.920186 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d139f1fbdd384830bc7ee062ab3d281c" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:02.921672 10200 raft_consensus.cc:3060] T d139f1fbdd384830bc7ee062ab3d281c P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.926474 10243 leader_election.cc:304] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: efa338cd132c457896b11e08d7ea787d
W20260709 13:07:02.927881 10172 leader_election.cc:336] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:02.928988 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52ffbaf3a54145c2a791c4a67dbdf64a" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:02.935284 10199 raft_consensus.cc:3060] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.934609 10935 raft_consensus.cc:2804] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.964417 10199 raft_consensus.cc:2468] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 2.
I20260709 13:07:02.964699  9950 catalog_manager.cc:5697] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 1 to 2, leader changed from efa338cd132c457896b11e08d7ea787d (127.0.107.193) to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:02.978259 10095 leader_election.cc:304] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.982331 10280 raft_consensus.cc:1275] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Refusing update from remote peer 8fe33410d51a469e81be6edc602d7d78: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:02.983400 10200 raft_consensus.cc:2468] T d139f1fbdd384830bc7ee062ab3d281c P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 2.
I20260709 13:07:02.957101 10935 raft_consensus.cc:697] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:07:02.987831 10935 consensus_queue.cc:237] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:02.987578  9955 catalog_manager.cc:5697] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 1 to 2, leader changed from efa338cd132c457896b11e08d7ea787d (127.0.107.193) to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:02.989526 10895 raft_consensus.cc:2804] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.990195 10895 raft_consensus.cc:697] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [term 2 LEADER]: Becoming Leader. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
W20260709 13:07:02.991235 10246 consensus_peers.cc:597] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:02.993830 10095 leader_election.cc:304] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:07:02.996958  9955 catalog_manager.cc:5697] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 1 to 2, leader changed from efa338cd132c457896b11e08d7ea787d (127.0.107.193) to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:02.999436  9955 catalog_manager.cc:5697] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 1 to 2, leader changed from efa338cd132c457896b11e08d7ea787d (127.0.107.193) to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:03.001050  9955 catalog_manager.cc:5697] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 reported cstate change: term changed from 1 to 2, leader changed from efa338cd132c457896b11e08d7ea787d (127.0.107.193) to 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "8fe33410d51a469e81be6edc602d7d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:02.997972 10929 raft_consensus.cc:2804] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.995448 10922 consensus_queue.cc:1048] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:07:03.018890 10127 raft_consensus.cc:1275] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Refusing update from remote peer 50999113203c482b9b9300caa485f154: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:07:03.019315 10172 consensus_peers.cc:597] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:03.028465 10629 consensus_queue.cc:1048] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:03.012495 10895 consensus_queue.cc:237] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:03.041580 10132 raft_consensus.cc:1275] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Refusing update from remote peer 8fe33410d51a469e81be6edc602d7d78: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:03.052542 10924 consensus_queue.cc:1048] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:03.027916 10929 raft_consensus.cc:697] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [term 2 LEADER]: Becoming Leader. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:07:03.062193 10280 raft_consensus.cc:1275] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Refusing update from remote peer 575e44d297f045a389903d65a471d548: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:03.063362 10929 consensus_queue.cc:237] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:03.066001 10890 consensus_queue.cc:1048] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:03.076215 10912 tablet_bootstrap.cc:492] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:03.082638 10912 tablet_bootstrap.cc:492] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:03.089524 10912 ts_tablet_manager.cc:1403] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.230s	user 0.023s	sys 0.023s
I20260709 13:07:03.096045  9955 catalog_manager.cc:5697] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 1 to 2, leader changed from efa338cd132c457896b11e08d7ea787d (127.0.107.193) to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:03.117540  9955 catalog_manager.cc:5697] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 1 to 2, leader changed from efa338cd132c457896b11e08d7ea787d (127.0.107.193) to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:03.144011 10912 raft_consensus.cc:359] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:03.145152 10912 raft_consensus.cc:740] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:03.146924 10912 consensus_queue.cc:260] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:03.150691 10912 ts_tablet_manager.cc:1434] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.000s
I20260709 13:07:03.152215 10912 tablet_bootstrap.cc:492] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
W20260709 13:07:03.199816 10098 consensus_peers.cc:597] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.204624 10246 consensus_peers.cc:597] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.221875 10172 consensus_peers.cc:597] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:03.226326 10200 raft_consensus.cc:1275] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Refusing update from remote peer 50999113203c482b9b9300caa485f154: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:07:03.231586 10098 consensus_peers.cc:597] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:03.240676 10935 consensus_queue.cc:1048] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:03.257875 10132 raft_consensus.cc:1275] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Refusing update from remote peer 8fe33410d51a469e81be6edc602d7d78: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:03.271345 10924 consensus_queue.cc:1048] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:03.299953  9955 catalog_manager.cc:5697] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 reported cstate change: term changed from 1 to 2, leader changed from efa338cd132c457896b11e08d7ea787d (127.0.107.193) to 575e44d297f045a389903d65a471d548 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "575e44d297f045a389903d65a471d548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:03.303673 10127 raft_consensus.cc:1275] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Refusing update from remote peer 8fe33410d51a469e81be6edc602d7d78: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:03.305205  9955 catalog_manager.cc:5697] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 reported cstate change: term changed from 1 to 2, leader changed from efa338cd132c457896b11e08d7ea787d (127.0.107.193) to 575e44d297f045a389903d65a471d548 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "575e44d297f045a389903d65a471d548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:03.317139 10246 consensus_peers.cc:597] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:03.353559 10924 consensus_queue.cc:1048] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:07:03.357133 10246 consensus_peers.cc:597] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.359591 10172 consensus_peers.cc:597] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:03.365513 10172 consensus_peers.cc:597] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.367807 10172 consensus_peers.cc:597] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:03.379105 10172 consensus_peers.cc:597] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.382030 10172 consensus_peers.cc:597] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.390005 10098 consensus_peers.cc:597] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.390537 10098 consensus_peers.cc:597] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.396220 10246 consensus_peers.cc:597] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.398386 10172 consensus_peers.cc:597] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.405580 10246 consensus_peers.cc:597] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.406376 10246 consensus_peers.cc:597] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:03.411967 10172 consensus_peers.cc:597] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:03.436286 10912 tablet_bootstrap.cc:492] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:03.443487 10912 tablet_bootstrap.cc:492] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
W20260709 13:07:03.453682 10098 consensus_peers.cc:597] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:03.456470 10246 consensus_peers.cc:597] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.459532 10098 consensus_peers.cc:597] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:03.467988 10200 raft_consensus.cc:1275] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Refusing update from remote peer 50999113203c482b9b9300caa485f154: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:07:03.468338 10246 consensus_peers.cc:597] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:03.469977 10199 raft_consensus.cc:1275] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Refusing update from remote peer 50999113203c482b9b9300caa485f154: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:03.470814 10860 consensus_queue.cc:1048] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:03.445722 10912 ts_tablet_manager.cc:1403] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.294s	user 0.022s	sys 0.005s
I20260709 13:07:03.476934 10912 raft_consensus.cc:359] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:03.478403 10949 consensus_queue.cc:1048] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:03.480252 10199 raft_consensus.cc:1275] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Refusing update from remote peer 575e44d297f045a389903d65a471d548: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:03.486084 10275 raft_consensus.cc:1275] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Refusing update from remote peer 8fe33410d51a469e81be6edc602d7d78: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:03.489346 10912 raft_consensus.cc:740] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:03.490363 10912 consensus_queue.cc:260] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:03.492672 10912 ts_tablet_manager.cc:1434] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
W20260709 13:07:03.495035 10246 consensus_peers.cc:597] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:03.494481 10912 tablet_bootstrap.cc:492] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
W20260709 13:07:03.495525 10246 consensus_peers.cc:597] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:03.495409 10938 consensus_queue.cc:1048] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:07:03.503405 10981 consensus_queue.cc:1048] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:03.505556 10914 raft_consensus.cc:493] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.506196 10914 raft_consensus.cc:515] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:03.520267 10914 leader_election.cc:290] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
W20260709 13:07:03.523461 10246 consensus_peers.cc:597] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:03.534742 10914 raft_consensus.cc:493] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.541610 10914 raft_consensus.cc:515] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:03.544497 10914 leader_election.cc:290] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:07:03.550377 10198 raft_consensus.cc:1275] T d139f1fbdd384830bc7ee062ab3d281c P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Refusing update from remote peer 575e44d297f045a389903d65a471d548: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:03.561707 10914 raft_consensus.cc:493] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.566655 10914 raft_consensus.cc:515] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:03.553323 10929 consensus_queue.cc:1048] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
W20260709 13:07:03.599783 10246 consensus_peers.cc:597] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.600368 10246 consensus_peers.cc:597] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.613345 10098 consensus_peers.cc:597] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:03.623217 10914 leader_election.cc:290] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
W20260709 13:07:03.623587 10246 consensus_peers.cc:597] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.628731 10172 consensus_peers.cc:597] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.636068 10098 consensus_peers.cc:597] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.646539 10098 consensus_peers.cc:597] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.679045 10246 consensus_peers.cc:597] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:03.696990 10912 tablet_bootstrap.cc:492] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:07:03.704478 10172 consensus_peers.cc:597] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.712208 10098 consensus_peers.cc:597] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:03.715164 10914 raft_consensus.cc:493] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.715757 10914 raft_consensus.cc:515] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:03.719345 10912 tablet_bootstrap.cc:492] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:03.723899 10912 ts_tablet_manager.cc:1403] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.230s	user 0.014s	sys 0.009s
W20260709 13:07:03.720927 10172 consensus_peers.cc:597] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:03.728669 10914 leader_election.cc:290] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:07:03.731083 10275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f4dd5b2c17747058bab6588c63b9d72" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:07:03.731878 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd93f389d32493b9dcc10e602f452da" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:07:03.744426 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f0446529cd446d0ba97ee860c6ae221" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
W20260709 13:07:03.744465 10246 consensus_peers.cc:597] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:03.745098 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52ffbaf3a54145c2a791c4a67dbdf64a" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:07:03.748363 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd93f389d32493b9dcc10e602f452da" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:07:03.748849 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f4dd5b2c17747058bab6588c63b9d72" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:07:03.749569 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52ffbaf3a54145c2a791c4a67dbdf64a" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:07:03.750157 10906 leader_election.cc:304] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa338cd132c457896b11e08d7ea787d; no voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548
I20260709 13:07:03.750885 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f0446529cd446d0ba97ee860c6ae221" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:07:03.751022 10914 raft_consensus.cc:2749] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:03.753185 10912 raft_consensus.cc:359] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:03.753963 10912 raft_consensus.cc:740] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:03.753814 10906 leader_election.cc:304] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa338cd132c457896b11e08d7ea787d; no voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548
I20260709 13:07:03.754755 10912 consensus_queue.cc:260] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:03.756919 10912 ts_tablet_manager.cc:1434] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.032s	user 0.003s	sys 0.000s
I20260709 13:07:03.761514 10914 raft_consensus.cc:2749] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:03.763001 10906 leader_election.cc:304] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa338cd132c457896b11e08d7ea787d; no voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78
I20260709 13:07:03.763803 10914 raft_consensus.cc:2749] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:03.765219 10906 leader_election.cc:304] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa338cd132c457896b11e08d7ea787d; no voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78
I20260709 13:07:03.766201 10914 raft_consensus.cc:2749] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:03.768934 10912 tablet_bootstrap.cc:492] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:03.795262 10914 raft_consensus.cc:493] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.795845 10914 raft_consensus.cc:515] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:03.800069 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af4e546f1084d8cb71469fbf3886792" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:07:03.801600 10914 leader_election.cc:290] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
W20260709 13:07:03.801529 10098 consensus_peers.cc:597] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:03.807096 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af4e546f1084d8cb71469fbf3886792" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:07:03.823441 10912 tablet_bootstrap.cc:492] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:03.825599 10906 leader_election.cc:304] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa338cd132c457896b11e08d7ea787d; no voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548
I20260709 13:07:03.829835 10914 raft_consensus.cc:2749] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:03.832274 10912 tablet_bootstrap.cc:492] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:03.833258 10912 ts_tablet_manager.cc:1403] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.076s	user 0.016s	sys 0.008s
I20260709 13:07:03.837577 10912 raft_consensus.cc:359] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:03.838574 10912 raft_consensus.cc:740] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:03.839457 10912 consensus_queue.cc:260] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:03.842576 10912 ts_tablet_manager.cc:1434] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.003s
I20260709 13:07:03.843549 10912 tablet_bootstrap.cc:492] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
W20260709 13:07:03.875891 10172 consensus_peers.cc:597] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.880429 10172 consensus_peers.cc:597] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:03.890035 10912 tablet_bootstrap.cc:492] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:03.891283 10912 tablet_bootstrap.cc:492] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:03.891971 10912 ts_tablet_manager.cc:1403] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.049s	user 0.023s	sys 0.013s
I20260709 13:07:03.895476 10912 raft_consensus.cc:359] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:03.896440 10912 raft_consensus.cc:740] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:03.897375 10912 consensus_queue.cc:260] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:03.900352 10912 ts_tablet_manager.cc:1434] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:07:03.901573 10912 tablet_bootstrap.cc:492] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:03.932606 10912 tablet_bootstrap.cc:492] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:03.935020 10912 tablet_bootstrap.cc:492] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:03.935928 10912 ts_tablet_manager.cc:1403] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.013s
I20260709 13:07:03.940395 10912 raft_consensus.cc:359] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:03.941360 10912 raft_consensus.cc:740] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:03.942440 10912 consensus_queue.cc:260] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:03.946206 10912 ts_tablet_manager.cc:1434] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 13:07:03.947007 10912 tablet_bootstrap.cc:492] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:03.974242 10912 tablet_bootstrap.cc:492] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:03.975529 10912 tablet_bootstrap.cc:492] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:03.981369 10912 ts_tablet_manager.cc:1403] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.035s	user 0.026s	sys 0.000s
I20260709 13:07:03.984911 10912 raft_consensus.cc:359] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:03.986734 10912 raft_consensus.cc:740] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:03.988390 10912 consensus_queue.cc:260] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:03.991971 10912 ts_tablet_manager.cc:1434] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:07:03.993072 10912 tablet_bootstrap.cc:492] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
W20260709 13:07:04.014161 10172 consensus_peers.cc:597] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:04.015038 10914 raft_consensus.cc:493] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.015576 10914 raft_consensus.cc:515] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:04.019222 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e66a35e846da4843bda7db1f5fca14b6" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:07:04.023535 10914 leader_election.cc:290] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:07:04.025549 10275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e66a35e846da4843bda7db1f5fca14b6" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:07:04.027631 10906 leader_election.cc:304] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa338cd132c457896b11e08d7ea787d; no voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78
I20260709 13:07:04.029253 10914 raft_consensus.cc:2749] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:04.037799 10912 tablet_bootstrap.cc:492] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.039126 10912 tablet_bootstrap.cc:492] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:04.039996 10912 ts_tablet_manager.cc:1403] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.047s	user 0.022s	sys 0.005s
I20260709 13:07:04.041982 11044 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:42425 every 8 connection(s)
I20260709 13:07:04.042065   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:42425
I20260709 13:07:04.043857 10912 raft_consensus.cc:359] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:04.044687 10912 raft_consensus.cc:740] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.045428 10912 consensus_queue.cc:260] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:04.047778 10912 ts_tablet_manager.cc:1434] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:07:04.048599 10912 tablet_bootstrap.cc:492] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:04.138813 10912 tablet_bootstrap.cc:492] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.140161 10912 tablet_bootstrap.cc:492] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:04.141077 10912 ts_tablet_manager.cc:1403] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.093s	user 0.030s	sys 0.048s
I20260709 13:07:04.144511 10912 raft_consensus.cc:359] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:04.145430 10912 raft_consensus.cc:740] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.147469 10912 consensus_queue.cc:260] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:04.150321 10912 ts_tablet_manager.cc:1434] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:07:04.151681 10912 tablet_bootstrap.cc:492] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:04.170037 11045 heartbeater.cc:344] Connected to a master server at 127.0.107.254:43521
I20260709 13:07:04.170542 11045 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:04.171418 11045 heartbeater.cc:507] Master 127.0.107.254:43521 requested a full tablet report, sending...
I20260709 13:07:04.180513 10912 tablet_bootstrap.cc:492] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.182993 10912 tablet_bootstrap.cc:492] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:04.183781 10912 ts_tablet_manager.cc:1403] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.032s	user 0.024s	sys 0.000s
I20260709 13:07:04.186508 10912 raft_consensus.cc:359] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:04.187255 10912 raft_consensus.cc:740] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.188012 10912 consensus_queue.cc:260] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:04.190505 10912 ts_tablet_manager.cc:1434] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:07:04.192027 10912 tablet_bootstrap.cc:492] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:04.206892  9950 ts_manager.cc:194] Re-registered known tserver with Master: efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:04.215947 10912 tablet_bootstrap.cc:492] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.217805 10912 tablet_bootstrap.cc:492] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:04.218591 10912 ts_tablet_manager.cc:1403] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.027s	user 0.020s	sys 0.000s
I20260709 13:07:04.221465 10912 raft_consensus.cc:359] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:04.222290 10912 raft_consensus.cc:740] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.223028 10912 consensus_queue.cc:260] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:04.226591 10912 ts_tablet_manager.cc:1434] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:04.227437 10912 tablet_bootstrap.cc:492] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:04.262633 10912 tablet_bootstrap.cc:492] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.263978 10912 tablet_bootstrap.cc:492] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:04.264816 10912 ts_tablet_manager.cc:1403] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.038s	user 0.020s	sys 0.011s
I20260709 13:07:04.267477 10912 raft_consensus.cc:359] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:04.268213 10912 raft_consensus.cc:740] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.268911 10912 consensus_queue.cc:260] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:04.271461 10912 ts_tablet_manager.cc:1434] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:07:04.272603 10912 tablet_bootstrap.cc:492] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:04.275297  9950 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47516
I20260709 13:07:04.281419 11045 heartbeater.cc:499] Master 127.0.107.254:43521 was elected leader, sending a full tablet report...
I20260709 13:07:04.312533 11012 raft_consensus.cc:3060] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:04.314570 10912 tablet_bootstrap.cc:492] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.317536 10912 tablet_bootstrap.cc:492] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:04.318320 10912 ts_tablet_manager.cc:1403] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.046s	user 0.025s	sys 0.005s
I20260709 13:07:04.321403 10912 raft_consensus.cc:359] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:04.322199 10912 raft_consensus.cc:740] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.323235 10912 consensus_queue.cc:260] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:04.325754 10912 ts_tablet_manager.cc:1434] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:04.326655 10912 tablet_bootstrap.cc:492] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:04.356714 11015 raft_consensus.cc:3060] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:04.363025 10912 tablet_bootstrap.cc:492] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.364436 10912 tablet_bootstrap.cc:492] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:04.365276 10912 ts_tablet_manager.cc:1403] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.039s	user 0.014s	sys 0.017s
I20260709 13:07:04.368731 10912 raft_consensus.cc:359] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:04.371510 10912 raft_consensus.cc:740] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.372704 10912 consensus_queue.cc:260] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:04.374282 10912 ts_tablet_manager.cc:1434] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.003s
I20260709 13:07:04.375126 10912 tablet_bootstrap.cc:492] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:04.400940 10912 tablet_bootstrap.cc:492] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.402052 10912 tablet_bootstrap.cc:492] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:04.402971 10912 ts_tablet_manager.cc:1403] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.028s	user 0.023s	sys 0.000s
I20260709 13:07:04.405928 10912 raft_consensus.cc:359] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:04.406731 10912 raft_consensus.cc:740] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.407496 10912 consensus_queue.cc:260] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:04.410142 10912 ts_tablet_manager.cc:1434] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:07:04.411682 10912 tablet_bootstrap.cc:492] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:04.430584 10912 tablet_bootstrap.cc:492] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.431627 10912 tablet_bootstrap.cc:492] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:04.432391 10912 ts_tablet_manager.cc:1403] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.000s
I20260709 13:07:04.435506 10912 raft_consensus.cc:359] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:04.436589 10912 raft_consensus.cc:740] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.437403 10912 consensus_queue.cc:260] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:04.440177 10912 ts_tablet_manager.cc:1434] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260709 13:07:04.444096 10928 raft_consensus.cc:493] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.444653 10928 raft_consensus.cc:515] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:04.446431 10912 tablet_bootstrap.cc:492] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:04.446792 10928 leader_election.cc:290] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:04.448354 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ea0c11178d43888a5596f49ffc7a7e" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:07:04.449071 10199 raft_consensus.cc:2468] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 2.
I20260709 13:07:04.453353 10243 leader_election.cc:304] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:07:04.454461 11015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ea0c11178d43888a5596f49ffc7a7e" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:07:04.457613 10860 raft_consensus.cc:2804] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:04.459996 10860 raft_consensus.cc:493] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.460423 10860 raft_consensus.cc:3060] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:04.463079 10914 raft_consensus.cc:493] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.463660 10914 raft_consensus.cc:515] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:04.465909 10914 leader_election.cc:290] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
W20260709 13:07:04.470747 10246 leader_election.cc:343] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:04.475870 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ed1b358949418a9d300990f8f81652" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "575e44d297f045a389903d65a471d548" is_pre_election: true
I20260709 13:07:04.473091 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ed1b358949418a9d300990f8f81652" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50999113203c482b9b9300caa485f154" is_pre_election: true
I20260709 13:07:04.488435 10906 leader_election.cc:304] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa338cd132c457896b11e08d7ea787d; no voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548
I20260709 13:07:04.498769 10914 raft_consensus.cc:2749] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:04.499058 11012 raft_consensus.cc:3060] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:04.533671 10860 raft_consensus.cc:515] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:04.565680 10860 leader_election.cc:290] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 3 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
W20260709 13:07:04.577416 10243 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:33415, user_credentials={real_user=slave}} blocked reactor thread for 62536.4us
I20260709 13:07:04.583777 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ea0c11178d43888a5596f49ffc7a7e" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:04.584808 10199 raft_consensus.cc:3060] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:04.608438 10246 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42425, user_credentials={real_user=slave}} blocked reactor thread for 118608us
I20260709 13:07:04.613248 11016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ea0c11178d43888a5596f49ffc7a7e" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
W20260709 13:07:04.618686 10246 leader_election.cc:343] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:04.624526 10912 tablet_bootstrap.cc:492] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.626147 10912 tablet_bootstrap.cc:492] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:04.627280 10912 ts_tablet_manager.cc:1403] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.181s	user 0.059s	sys 0.088s
I20260709 13:07:04.700233 10199 raft_consensus.cc:2468] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 3.
I20260709 13:07:04.712601 10912 raft_consensus.cc:359] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:04.713501 10912 raft_consensus.cc:740] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.721194 10912 consensus_queue.cc:260] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:04.723907 10912 ts_tablet_manager.cc:1434] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.096s	user 0.007s	sys 0.001s
I20260709 13:07:04.724804 10912 tablet_bootstrap.cc:492] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:04.730063 10243 leader_election.cc:304] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: efa338cd132c457896b11e08d7ea787d
I20260709 13:07:04.751585 10912 tablet_bootstrap.cc:492] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.752666 10912 tablet_bootstrap.cc:492] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:04.753329 10912 ts_tablet_manager.cc:1403] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.000s
I20260709 13:07:04.755895 11067 raft_consensus.cc:2804] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:04.756716 11067 raft_consensus.cc:697] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 3 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:07:04.756817 10912 raft_consensus.cc:359] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:04.757788 11067 consensus_queue.cc:237] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:04.757763 10912 raft_consensus.cc:740] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.760859 10912 consensus_queue.cc:260] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:04.764891 10912 ts_tablet_manager.cc:1434] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 13:07:04.766115 10912 tablet_bootstrap.cc:492] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:04.810881 10912 tablet_bootstrap.cc:492] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.812366 10912 tablet_bootstrap.cc:492] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:04.814685 10912 ts_tablet_manager.cc:1403] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.049s	user 0.029s	sys 0.002s
I20260709 13:07:04.818221 10912 raft_consensus.cc:359] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:04.820482 10912 raft_consensus.cc:740] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.821362 10912 consensus_queue.cc:260] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:04.830149 10912 ts_tablet_manager.cc:1434] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:07:04.831270 10912 tablet_bootstrap.cc:492] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:04.843767 11012 raft_consensus.cc:3060] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:04.856606 10912 tablet_bootstrap.cc:492] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.857661 10912 tablet_bootstrap.cc:492] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:04.858438 10912 ts_tablet_manager.cc:1403] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.027s	user 0.022s	sys 0.003s
I20260709 13:07:04.861570 10912 raft_consensus.cc:359] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:04.862418 10912 raft_consensus.cc:740] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.863556 10912 consensus_queue.cc:260] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:04.869577 10912 ts_tablet_manager.cc:1434] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:07:04.870534 10912 tablet_bootstrap.cc:492] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:04.878142 11074 raft_consensus.cc:493] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.878765 11074 raft_consensus.cc:515] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:04.880563 11016 raft_consensus.cc:3060] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:04.881735 11074 leader_election.cc:290] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
W20260709 13:07:04.882373 10243 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:33415, user_credentials={real_user=slave}} blocked reactor thread for 56425.5us
I20260709 13:07:04.883366 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e003c3e14c2444b937f547eef84b672" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78" is_pre_election: true
I20260709 13:07:04.884171 10199 raft_consensus.cc:2468] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 2.
I20260709 13:07:04.883366 11011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e003c3e14c2444b937f547eef84b672" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d" is_pre_election: true
I20260709 13:07:04.886046 11011 raft_consensus.cc:2468] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 1.
I20260709 13:07:04.887719 10098 leader_election.cc:304] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:07:04.888674 11074 raft_consensus.cc:2804] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:04.889210 11074 raft_consensus.cc:493] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.890652 11074 raft_consensus.cc:3060] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:04.946982  9950 catalog_manager.cc:5697] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 1 to 3, leader changed from efa338cd132c457896b11e08d7ea787d (127.0.107.193) to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 3 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:04.957082 11074 raft_consensus.cc:515] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:04.961135 11074 leader_election.cc:290] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 3 election: Requested vote from peers efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:07:04.966197 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e003c3e14c2444b937f547eef84b672" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:04.967186 10201 raft_consensus.cc:3060] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:05.004202 11012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e003c3e14c2444b937f547eef84b672" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa338cd132c457896b11e08d7ea787d"
I20260709 13:07:05.010289 11012 raft_consensus.cc:3060] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:07:05.031023 11012 raft_consensus.cc:2468] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 3.
I20260709 13:07:05.032306 10098 leader_election.cc:304] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:07:05.036700 11074 raft_consensus.cc:2804] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:05.039108 11074 raft_consensus.cc:697] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 3 LEADER]: Becoming Leader. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:07:05.040843 11074 consensus_queue.cc:237] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:05.038379 10201 raft_consensus.cc:2468] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 3.
I20260709 13:07:05.051405 10912 tablet_bootstrap.cc:492] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.055825 10912 tablet_bootstrap.cc:492] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:05.056639 10912 ts_tablet_manager.cc:1403] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.186s	user 0.028s	sys 0.007s
I20260709 13:07:05.059310 10912 raft_consensus.cc:359] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:05.060560 10912 raft_consensus.cc:740] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.061551 10912 consensus_queue.cc:260] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:05.064777 10912 ts_tablet_manager.cc:1434] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:05.065724 10912 tablet_bootstrap.cc:492] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:05.081755  9950 catalog_manager.cc:5697] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 reported cstate change: term changed from 1 to 3, leader changed from efa338cd132c457896b11e08d7ea787d (127.0.107.193) to 575e44d297f045a389903d65a471d548 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "575e44d297f045a389903d65a471d548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:05.093446 11014 raft_consensus.cc:3060] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:05.103964 10912 tablet_bootstrap.cc:492] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.105353 10912 tablet_bootstrap.cc:492] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:05.106151 10912 ts_tablet_manager.cc:1403] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.041s	user 0.017s	sys 0.006s
I20260709 13:07:05.108769 10912 raft_consensus.cc:359] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:05.110533 10912 raft_consensus.cc:740] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.111677 10912 consensus_queue.cc:260] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:05.114010 10912 ts_tablet_manager.cc:1434] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:07:05.114951 10912 tablet_bootstrap.cc:492] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:05.140383 10912 tablet_bootstrap.cc:492] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.141669 10912 tablet_bootstrap.cc:492] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:05.142427 10912 ts_tablet_manager.cc:1403] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.028s	user 0.023s	sys 0.000s
I20260709 13:07:05.145125 10912 raft_consensus.cc:359] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:05.146014 10912 raft_consensus.cc:740] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.146939 10912 consensus_queue.cc:260] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:05.149425 10912 ts_tablet_manager.cc:1434] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:07:05.151578 10912 tablet_bootstrap.cc:492] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:05.179440 10912 tablet_bootstrap.cc:492] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.180637 10912 tablet_bootstrap.cc:492] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:05.181393 10912 ts_tablet_manager.cc:1403] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.006s
I20260709 13:07:05.184759 10912 raft_consensus.cc:359] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:05.185600 10912 raft_consensus.cc:740] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.186537 10912 consensus_queue.cc:260] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:05.189417 10912 ts_tablet_manager.cc:1434] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.004s
I20260709 13:07:05.195008 10912 tablet_bootstrap.cc:492] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:05.276827 11014 raft_consensus.cc:3060] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:05.286113 10199 raft_consensus.cc:1275] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Refusing update from remote peer 50999113203c482b9b9300caa485f154: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:07:05.293665 10246 consensus_peers.cc:597] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:05.311780 11068 consensus_queue.cc:1048] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:05.315080 11012 raft_consensus.cc:3060] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:05.330713 10912 tablet_bootstrap.cc:492] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.331789 10912 tablet_bootstrap.cc:492] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:05.347549 10912 ts_tablet_manager.cc:1403] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.153s	user 0.051s	sys 0.056s
I20260709 13:07:05.357883 10912 raft_consensus.cc:359] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:05.358745 10912 raft_consensus.cc:740] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.359515 10912 consensus_queue.cc:260] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:05.370705 10912 ts_tablet_manager.cc:1434] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:07:05.371917 10912 tablet_bootstrap.cc:492] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:05.528112 10912 tablet_bootstrap.cc:492] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.529110 10912 tablet_bootstrap.cc:492] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:05.529784 10912 ts_tablet_manager.cc:1403] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.158s	user 0.037s	sys 0.031s
I20260709 13:07:05.532159 10912 raft_consensus.cc:359] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:05.532673 10912 raft_consensus.cc:740] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.533331 10912 consensus_queue.cc:260] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:05.534931 10912 ts_tablet_manager.cc:1434] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:07:05.535755 10912 tablet_bootstrap.cc:492] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:05.572177 10912 tablet_bootstrap.cc:492] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.573302 10912 tablet_bootstrap.cc:492] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:05.574096 10912 ts_tablet_manager.cc:1403] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.039s	user 0.016s	sys 0.007s
I20260709 13:07:05.576821 10200 raft_consensus.cc:1275] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Refusing update from remote peer 575e44d297f045a389903d65a471d548: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:05.578722 10912 raft_consensus.cc:359] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:05.579514 10912 raft_consensus.cc:740] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.580444 10912 consensus_queue.cc:260] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:05.582301 10912 ts_tablet_manager.cc:1434] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:07:05.582208 10929 consensus_queue.cc:1048] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:05.583585 10912 tablet_bootstrap.cc:492] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:05.603103 11014 raft_consensus.cc:3060] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:05.620196 10172 consensus_peers.cc:597] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:05.629212 11011 raft_consensus.cc:1275] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Refusing update from remote peer 575e44d297f045a389903d65a471d548: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:05.645951 11104 consensus_queue.cc:1048] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:05.664768 10912 tablet_bootstrap.cc:492] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.665802 10912 tablet_bootstrap.cc:492] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:05.666512 10912 ts_tablet_manager.cc:1403] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.083s	user 0.013s	sys 0.004s
I20260709 13:07:05.669090 10912 raft_consensus.cc:359] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:05.669811 10912 raft_consensus.cc:740] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.670508 10912 consensus_queue.cc:260] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:05.672610 10912 ts_tablet_manager.cc:1434] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:07:05.673422 10912 tablet_bootstrap.cc:492] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
W20260709 13:07:05.680485 10246 consensus_peers.cc:597] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:05.705545 10912 tablet_bootstrap.cc:492] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.706779 10912 tablet_bootstrap.cc:492] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:05.707899 10912 ts_tablet_manager.cc:1403] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.035s	user 0.028s	sys 0.003s
I20260709 13:07:05.712025 10912 raft_consensus.cc:359] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:05.712795 10912 raft_consensus.cc:740] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.713755 10912 consensus_queue.cc:260] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:05.716051 10912 ts_tablet_manager.cc:1434] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:07:05.716930 10912 tablet_bootstrap.cc:492] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:05.738605 10912 tablet_bootstrap.cc:492] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.739634 10912 tablet_bootstrap.cc:492] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:05.740548 10912 ts_tablet_manager.cc:1403] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.003s
I20260709 13:07:05.743456 10912 raft_consensus.cc:359] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:05.744349 10912 raft_consensus.cc:740] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.745230 10912 consensus_queue.cc:260] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:05.747586 10912 ts_tablet_manager.cc:1434] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:07:05.753345 10912 tablet_bootstrap.cc:492] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:05.792503 10912 tablet_bootstrap.cc:492] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.793819 10912 tablet_bootstrap.cc:492] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:05.794529 10912 ts_tablet_manager.cc:1403] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.041s	user 0.016s	sys 0.016s
I20260709 13:07:05.797894 10912 raft_consensus.cc:359] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:05.798633 10912 raft_consensus.cc:740] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.799321 10912 consensus_queue.cc:260] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:05.800942 10912 ts_tablet_manager.cc:1434] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:07:05.802591 10912 tablet_bootstrap.cc:492] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:05.870684 10912 tablet_bootstrap.cc:492] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.871683 10912 tablet_bootstrap.cc:492] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:05.872378 10912 ts_tablet_manager.cc:1403] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.070s	user 0.025s	sys 0.031s
I20260709 13:07:05.875445 10912 raft_consensus.cc:359] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:05.876188 10912 raft_consensus.cc:740] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.876884 10912 consensus_queue.cc:260] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:05.879081 10912 ts_tablet_manager.cc:1434] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:07:05.884382 10912 tablet_bootstrap.cc:492] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:05.890301 11011 raft_consensus.cc:3060] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:07:05.893393 10246 consensus_peers.cc:597] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:05.965181 10912 tablet_bootstrap.cc:492] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:07:05.966220 10098 consensus_peers.cc:597] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:05.966429 10912 tablet_bootstrap.cc:492] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:05.971421 10912 ts_tablet_manager.cc:1403] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.087s	user 0.015s	sys 0.004s
I20260709 13:07:05.974121 10912 raft_consensus.cc:359] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:05.974852 10912 raft_consensus.cc:740] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.975634 10912 consensus_queue.cc:260] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
W20260709 13:07:05.976871 10098 consensus_peers.cc:597] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:05.985888 10912 ts_tablet_manager.cc:1434] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:07:05.991665 10912 tablet_bootstrap.cc:492] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
W20260709 13:07:05.995280 10172 consensus_peers.cc:597] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:06.007248 11011 raft_consensus.cc:3060] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:06.028873 10912 tablet_bootstrap.cc:492] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:06.029968 10912 tablet_bootstrap.cc:492] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:06.030658 10912 ts_tablet_manager.cc:1403] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.039s	user 0.015s	sys 0.007s
I20260709 13:07:06.032666 10912 raft_consensus.cc:359] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:06.033190 10912 raft_consensus.cc:740] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:06.033883 10912 consensus_queue.cc:260] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:06.036271 10912 ts_tablet_manager.cc:1434] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.003s
I20260709 13:07:06.036983 10912 tablet_bootstrap.cc:492] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:06.056475 10912 tablet_bootstrap.cc:492] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:06.057246 10912 tablet_bootstrap.cc:492] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:06.057849 10912 ts_tablet_manager.cc:1403] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.007s
I20260709 13:07:06.060710 10912 raft_consensus.cc:359] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:06.062098 10912 raft_consensus.cc:740] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:06.062793 10912 consensus_queue.cc:260] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:06.065578 10912 ts_tablet_manager.cc:1434] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 13:07:06.066344 11011 raft_consensus.cc:3060] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:06.066802 10912 tablet_bootstrap.cc:492] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:06.088219 11012 raft_consensus.cc:3060] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:06.095842 10912 tablet_bootstrap.cc:492] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:06.096931 10912 tablet_bootstrap.cc:492] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:06.097572 10912 ts_tablet_manager.cc:1403] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.012s
I20260709 13:07:06.100455 10912 raft_consensus.cc:359] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:06.101151 10912 raft_consensus.cc:740] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:06.101997 10912 consensus_queue.cc:260] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:06.104212 10912 ts_tablet_manager.cc:1434] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 13:07:06.106112 10912 tablet_bootstrap.cc:492] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d: Bootstrap starting.
I20260709 13:07:06.274302 10912 tablet_bootstrap.cc:492] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:06.275323 10912 tablet_bootstrap.cc:492] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d: Bootstrap complete.
I20260709 13:07:06.279006 10912 ts_tablet_manager.cc:1403] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d: Time spent bootstrapping tablet: real 0.173s	user 0.026s	sys 0.043s
I20260709 13:07:06.283396 10912 raft_consensus.cc:359] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:06.284461 10912 raft_consensus.cc:740] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Initialized, Role: FOLLOWER
I20260709 13:07:06.285386 10912 consensus_queue.cc:260] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:06.289116 10912 ts_tablet_manager.cc:1434] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:07:14.083302   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:14.146395 10200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3666fc7cbb0d4f698fb337ad4c238c38"
dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
mode: GRACEFUL
new_leader_uuid: "575e44d297f045a389903d65a471d548"
 from {username='slave'} at 127.0.0.1:58228
I20260709 13:07:14.147389 10200 raft_consensus.cc:606] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Received request to transfer leadership to TS 575e44d297f045a389903d65a471d548
I20260709 13:07:14.159723 10277 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48dcd8683f004a61808243f33d859911"
dest_uuid: "50999113203c482b9b9300caa485f154"
mode: GRACEFUL
new_leader_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:58788
I20260709 13:07:14.160192 10277 raft_consensus.cc:606] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Received request to transfer leadership to TS efa338cd132c457896b11e08d7ea787d
I20260709 13:07:14.162799 10277 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4af4e546f1084d8cb71469fbf3886792"
dest_uuid: "50999113203c482b9b9300caa485f154"
mode: GRACEFUL
new_leader_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:58788
I20260709 13:07:14.163252 10277 raft_consensus.cc:606] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Received request to transfer leadership to TS efa338cd132c457896b11e08d7ea787d
I20260709 13:07:14.165890 10277 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75ed1b358949418a9d300990f8f81652"
dest_uuid: "50999113203c482b9b9300caa485f154"
mode: GRACEFUL
new_leader_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:58788
I20260709 13:07:14.166416 10277 raft_consensus.cc:606] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Received request to transfer leadership to TS efa338cd132c457896b11e08d7ea787d
I20260709 13:07:14.169104 10277 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "925f342702b542288fd34463d0585a5b"
dest_uuid: "50999113203c482b9b9300caa485f154"
mode: GRACEFUL
new_leader_uuid: "575e44d297f045a389903d65a471d548"
 from {username='slave'} at 127.0.0.1:58788
I20260709 13:07:14.169581 10277 raft_consensus.cc:606] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Received request to transfer leadership to TS 575e44d297f045a389903d65a471d548
I20260709 13:07:14.172202 10277 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ccaebf4041f14bada42fd7122f34419c"
dest_uuid: "50999113203c482b9b9300caa485f154"
mode: GRACEFUL
new_leader_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:58788
I20260709 13:07:14.172785 10277 raft_consensus.cc:606] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Received request to transfer leadership to TS efa338cd132c457896b11e08d7ea787d
I20260709 13:07:14.174021   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:07:14.175207   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:14.196862 11129 raft_consensus.cc:993] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154: : Instructing follower efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:14.197403 11129 raft_consensus.cc:1081] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Signalling peer efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:14.199064 11011 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "75ed1b358949418a9d300990f8f81652"
dest_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:35172
I20260709 13:07:14.199568 11011 raft_consensus.cc:493] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:14.199957 11011 raft_consensus.cc:3060] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.207226 11011 raft_consensus.cc:515] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:14.209282 11011 leader_election.cc:290] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 election: Requested vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:07:14.210145 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ed1b358949418a9d300990f8f81652" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:14.210742 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ed1b358949418a9d300990f8f81652" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:07:14.210973 10277 raft_consensus.cc:3055] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:14.211339 10130 raft_consensus.cc:3060] T 75ed1b358949418a9d300990f8f81652 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.211335 10277 raft_consensus.cc:740] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:07:14.213193 10277 consensus_queue.cc:260] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:14.214372 10277 raft_consensus.cc:3060] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.217080 10130 raft_consensus.cc:2468] T 75ed1b358949418a9d300990f8f81652 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 2.
I20260709 13:07:14.218547 10906 leader_election.cc:304] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:07:14.220022 11140 raft_consensus.cc:993] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154: : Instructing follower 575e44d297f045a389903d65a471d548 to start an election
I20260709 13:07:14.220737 11140 raft_consensus.cc:1081] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Signalling peer 575e44d297f045a389903d65a471d548 to start an election
I20260709 13:07:14.222172 11144 raft_consensus.cc:2804] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:14.223537 10128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "925f342702b542288fd34463d0585a5b"
dest_uuid: "575e44d297f045a389903d65a471d548"
 from {username='slave'} at 127.0.0.1:52882
I20260709 13:07:14.224107 10128 raft_consensus.cc:493] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:14.224442 10128 raft_consensus.cc:3060] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.224920 11144 raft_consensus.cc:697] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [term 2 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:07:14.226344 10277 raft_consensus.cc:2468] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 2.
I20260709 13:07:14.226096 11144 consensus_queue.cc:237] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:14.231490 10128 raft_consensus.cc:515] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:14.233716 10128 leader_election.cc:290] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:07:14.234720 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "925f342702b542288fd34463d0585a5b" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:14.234907 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "925f342702b542288fd34463d0585a5b" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:14.235431 10200 raft_consensus.cc:3060] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.235666 10277 raft_consensus.cc:3055] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:14.236099 10277 raft_consensus.cc:740] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:07:14.237008 10277 consensus_queue.cc:260] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:14.240778 10277 raft_consensus.cc:3060] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.244702 10200 raft_consensus.cc:2468] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 2.
I20260709 13:07:14.246273 10095 leader_election.cc:304] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:07:14.245748  9954 catalog_manager.cc:5697] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 1 to 2, leader changed from 50999113203c482b9b9300caa485f154 (127.0.107.196) to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 2 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:14.247184 11138 raft_consensus.cc:2804] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:14.247659 11138 raft_consensus.cc:697] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [term 2 LEADER]: Becoming Leader. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:07:14.248708 11138 consensus_queue.cc:237] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:14.250481 10277 raft_consensus.cc:2468] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 2.
I20260709 13:07:14.257153 11139 raft_consensus.cc:993] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78: : Instructing follower 575e44d297f045a389903d65a471d548 to start an election
I20260709 13:07:14.258704 11139 raft_consensus.cc:1081] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Signalling peer 575e44d297f045a389903d65a471d548 to start an election
I20260709 13:07:14.273895 10130 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3666fc7cbb0d4f698fb337ad4c238c38"
dest_uuid: "575e44d297f045a389903d65a471d548"
 from {username='slave'} at 127.0.0.1:52892
I20260709 13:07:14.274582 10130 raft_consensus.cc:493] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:14.275161 10130 raft_consensus.cc:3060] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.290114 10130 raft_consensus.cc:515] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:14.290297  9955 catalog_manager.cc:5697] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 reported cstate change: term changed from 1 to 2, leader changed from 50999113203c482b9b9300caa485f154 (127.0.107.196) to 575e44d297f045a389903d65a471d548 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "575e44d297f045a389903d65a471d548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:14.295361 10130 leader_election.cc:290] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:07:14.296490 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3666fc7cbb0d4f698fb337ad4c238c38" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:14.295922 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3666fc7cbb0d4f698fb337ad4c238c38" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:14.297663 10277 raft_consensus.cc:3060] T 3666fc7cbb0d4f698fb337ad4c238c38 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.297557 10200 raft_consensus.cc:3055] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:14.309144 10200 raft_consensus.cc:740] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:07:14.310223 10200 consensus_queue.cc:260] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:14.311973 10200 raft_consensus.cc:3060] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.318662 10277 raft_consensus.cc:2468] T 3666fc7cbb0d4f698fb337ad4c238c38 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 2.
I20260709 13:07:14.319931 10095 leader_election.cc:304] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:07:14.320791 11150 raft_consensus.cc:2804] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:14.321264 11150 raft_consensus.cc:697] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548 [term 2 LEADER]: Becoming Leader. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:07:14.321962 10200 raft_consensus.cc:2468] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 2.
I20260709 13:07:14.322448 11150 consensus_queue.cc:237] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:14.340898  9955 catalog_manager.cc:5697] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548 reported cstate change: term changed from 1 to 2, leader changed from 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195) to 575e44d297f045a389903d65a471d548 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "575e44d297f045a389903d65a471d548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:14.375396 11140 raft_consensus.cc:993] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154: : Instructing follower efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:14.376068 11135 raft_consensus.cc:1081] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Signalling peer efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:14.377490 11014 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ccaebf4041f14bada42fd7122f34419c"
dest_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:35172
I20260709 13:07:14.378026 11014 raft_consensus.cc:493] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:14.378316 11014 raft_consensus.cc:3060] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.383041 11014 raft_consensus.cc:515] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:14.384912 11014 leader_election.cc:290] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 3 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:07:14.385579 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccaebf4041f14bada42fd7122f34419c" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:14.386261 10200 raft_consensus.cc:3060] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.386067 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccaebf4041f14bada42fd7122f34419c" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:14.387025 10277 raft_consensus.cc:3055] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:14.387316 10277 raft_consensus.cc:740] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:07:14.388182 10277 consensus_queue.cc:260] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:14.389531 10277 raft_consensus.cc:3060] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.394670 10200 raft_consensus.cc:2468] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 3.
I20260709 13:07:14.395852 10906 leader_election.cc:304] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:07:14.396783 11144 raft_consensus.cc:2804] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:14.397317 11144 raft_consensus.cc:697] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 3 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:07:14.398368 11144 consensus_queue.cc:237] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:14.401914 10277 raft_consensus.cc:2468] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 3.
I20260709 13:07:14.405329 11129 raft_consensus.cc:993] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154: : Instructing follower efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:14.407236 11135 raft_consensus.cc:1081] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Signalling peer efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:14.409092 11014 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4af4e546f1084d8cb71469fbf3886792"
dest_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:35172
I20260709 13:07:14.409654 11014 raft_consensus.cc:493] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:14.410024 11014 raft_consensus.cc:3060] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.421411  9955 catalog_manager.cc:5697] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 2 to 3, leader changed from 50999113203c482b9b9300caa485f154 (127.0.107.196) to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 3 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:14.423050 11014 raft_consensus.cc:515] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:14.425381 11014 leader_election.cc:290] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 election: Requested vote from peers 50999113203c482b9b9300caa485f154 (127.0.107.196:40841), 575e44d297f045a389903d65a471d548 (127.0.107.194:33415)
I20260709 13:07:14.426307 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af4e546f1084d8cb71469fbf3886792" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:07:14.426611 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af4e546f1084d8cb71469fbf3886792" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:14.426964 10130 raft_consensus.cc:3060] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.427241 10277 raft_consensus.cc:3055] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:14.427589 10277 raft_consensus.cc:740] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:07:14.429086 10277 consensus_queue.cc:260] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:14.430296 10277 raft_consensus.cc:3060] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.434978 10130 raft_consensus.cc:2468] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 2.
I20260709 13:07:14.436854 10906 leader_election.cc:304] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:07:14.437203 10277 raft_consensus.cc:2468] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 2.
I20260709 13:07:14.437815 11144 raft_consensus.cc:2804] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:14.438344 11144 raft_consensus.cc:697] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [term 2 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:07:14.439250 11144 consensus_queue.cc:237] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } }
I20260709 13:07:14.449627  9955 catalog_manager.cc:5697] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 1 to 2, leader changed from 50999113203c482b9b9300caa485f154 (127.0.107.196) to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 2 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:14.565214 11135 raft_consensus.cc:993] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154: : Instructing follower efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:14.566171 11159 raft_consensus.cc:1081] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Signalling peer efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:14.568010 11012 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48dcd8683f004a61808243f33d859911"
dest_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:35172
I20260709 13:07:14.568495 11012 raft_consensus.cc:493] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:14.568748 11012 raft_consensus.cc:3060] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.574059 11012 raft_consensus.cc:515] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:14.575730 11012 leader_election.cc:290] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 3 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:07:14.576686 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48dcd8683f004a61808243f33d859911" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:14.576684 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48dcd8683f004a61808243f33d859911" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:14.577497 10200 raft_consensus.cc:3060] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.577875 10277 raft_consensus.cc:3055] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:14.578209 10277 raft_consensus.cc:740] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:07:14.578907 10277 consensus_queue.cc:260] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:14.579838 10277 raft_consensus.cc:3060] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.583825 10200 raft_consensus.cc:2468] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 3.
I20260709 13:07:14.585304 10906 leader_election.cc:304] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:07:14.586112 11144 raft_consensus.cc:2804] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:14.586213 10277 raft_consensus.cc:2468] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 3.
I20260709 13:07:14.586588 11144 raft_consensus.cc:697] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 3 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:07:14.588529 11144 consensus_queue.cc:237] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:14.607743  9955 catalog_manager.cc:5697] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 2 to 3, leader changed from 50999113203c482b9b9300caa485f154 (127.0.107.196) to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 3 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:14.645735 10130 raft_consensus.cc:1275] T 75ed1b358949418a9d300990f8f81652 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.647336 11144 consensus_queue.cc:1048] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.676458 10280 raft_consensus.cc:1275] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.682061 11145 consensus_queue.cc:1048] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:14.774428 10280 raft_consensus.cc:1275] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Refusing update from remote peer 575e44d297f045a389903d65a471d548: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.775794 11150 consensus_queue.cc:1048] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.790961 10199 raft_consensus.cc:1275] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Refusing update from remote peer 575e44d297f045a389903d65a471d548: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.792527 11150 consensus_queue.cc:1048] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.816519 10277 raft_consensus.cc:1275] T 3666fc7cbb0d4f698fb337ad4c238c38 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Refusing update from remote peer 575e44d297f045a389903d65a471d548: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.818007 11150 consensus_queue.cc:1048] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.830222 10199 raft_consensus.cc:1275] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Refusing update from remote peer 575e44d297f045a389903d65a471d548: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.831799 11138 consensus_queue.cc:1048] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.935214 10277 raft_consensus.cc:1275] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.936610 11145 consensus_queue.cc:1048] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.953722 10277 raft_consensus.cc:1275] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:14.953814 10130 raft_consensus.cc:1275] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.958213 11145 consensus_queue.cc:1048] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.959137 11175 consensus_queue.cc:1048] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:14.971501 10199 raft_consensus.cc:1275] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:14.972885 11175 consensus_queue.cc:1048] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:15.055290 10200 raft_consensus.cc:1275] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:15.056582 11176 consensus_queue.cc:1048] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:15.066182 10277 raft_consensus.cc:1275] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:15.069121 11174 consensus_queue.cc:1048] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:15.176216   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:15.216962   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:15.259016 10200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0529bf107d07493eb0984c4cbb9ea01e"
dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
mode: GRACEFUL
new_leader_uuid: "50999113203c482b9b9300caa485f154"
 from {username='slave'} at 127.0.0.1:58228
I20260709 13:07:15.259573 10200 raft_consensus.cc:606] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Received request to transfer leadership to TS 50999113203c482b9b9300caa485f154
I20260709 13:07:15.280368 10130 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1249fe82a5ce4bd7b416e6bb46be8f79"
dest_uuid: "575e44d297f045a389903d65a471d548"
mode: GRACEFUL
new_leader_uuid: "50999113203c482b9b9300caa485f154"
 from {username='slave'} at 127.0.0.1:55924
I20260709 13:07:15.280972 10130 raft_consensus.cc:606] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Received request to transfer leadership to TS 50999113203c482b9b9300caa485f154
I20260709 13:07:15.283571 10280 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4457dec3dec94c1dbac542a298184d0f"
dest_uuid: "50999113203c482b9b9300caa485f154"
mode: GRACEFUL
new_leader_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:58788
I20260709 13:07:15.284047 10280 raft_consensus.cc:606] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Received request to transfer leadership to TS efa338cd132c457896b11e08d7ea787d
I20260709 13:07:15.286432 10280 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0265654cefc40a5937d7e194f03dd5c"
dest_uuid: "50999113203c482b9b9300caa485f154"
mode: GRACEFUL
new_leader_uuid: "575e44d297f045a389903d65a471d548"
 from {username='slave'} at 127.0.0.1:58788
I20260709 13:07:15.286880 10280 raft_consensus.cc:606] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Received request to transfer leadership to TS 575e44d297f045a389903d65a471d548
I20260709 13:07:15.289496 10280 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c78325f9138849749ada9825bfe45cd8"
dest_uuid: "50999113203c482b9b9300caa485f154"
mode: GRACEFUL
new_leader_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:58788
I20260709 13:07:15.290009 10280 raft_consensus.cc:606] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Received request to transfer leadership to TS efa338cd132c457896b11e08d7ea787d
I20260709 13:07:15.293129 10277 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e66a35e846da4843bda7db1f5fca14b6"
dest_uuid: "50999113203c482b9b9300caa485f154"
mode: GRACEFUL
new_leader_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:58788
I20260709 13:07:15.293593 10277 raft_consensus.cc:606] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Received request to transfer leadership to TS efa338cd132c457896b11e08d7ea787d
I20260709 13:07:15.294812   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:07:15.295900   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:15.357882 11135 raft_consensus.cc:993] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154: : Instructing follower efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:15.358438 11159 raft_consensus.cc:1081] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Signalling peer efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:15.359879 11012 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4457dec3dec94c1dbac542a298184d0f"
dest_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:35172
I20260709 13:07:15.360356 11012 raft_consensus.cc:493] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:15.360630 11012 raft_consensus.cc:3060] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.365295 11012 raft_consensus.cc:515] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:15.366838 11012 leader_election.cc:290] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:07:15.367558 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4457dec3dec94c1dbac542a298184d0f" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:15.367811 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4457dec3dec94c1dbac542a298184d0f" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:15.368203 10200 raft_consensus.cc:3060] T 4457dec3dec94c1dbac542a298184d0f P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.368343 10280 raft_consensus.cc:3055] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:15.368721 10280 raft_consensus.cc:740] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:07:15.369489 10280 consensus_queue.cc:260] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:15.370733 10280 raft_consensus.cc:3060] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.374948 10200 raft_consensus.cc:2468] T 4457dec3dec94c1dbac542a298184d0f P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 2.
I20260709 13:07:15.376149 10906 leader_election.cc:304] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:07:15.376940 11174 raft_consensus.cc:2804] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:15.377454 11174 raft_consensus.cc:697] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [term 2 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:07:15.377673 10280 raft_consensus.cc:2468] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 2.
I20260709 13:07:15.378436 11174 consensus_queue.cc:237] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:15.387990  9950 catalog_manager.cc:5697] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 1 to 2, leader changed from 50999113203c482b9b9300caa485f154 (127.0.107.196) to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 2 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:15.401325 11157 raft_consensus.cc:993] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154: : Instructing follower efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:15.401865 11157 raft_consensus.cc:1081] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Signalling peer efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:15.403381 11135 raft_consensus.cc:993] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154: : Instructing follower 575e44d297f045a389903d65a471d548 to start an election
I20260709 13:07:15.403863 11135 raft_consensus.cc:1081] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Signalling peer 575e44d297f045a389903d65a471d548 to start an election
I20260709 13:07:15.405368 11012 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c78325f9138849749ada9825bfe45cd8"
dest_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:35172
I20260709 13:07:15.406019 11012 raft_consensus.cc:493] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:15.405943 10130 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a0265654cefc40a5937d7e194f03dd5c"
dest_uuid: "575e44d297f045a389903d65a471d548"
 from {username='slave'} at 127.0.0.1:52882
I20260709 13:07:15.406881 11012 raft_consensus.cc:3060] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.407181 10130 raft_consensus.cc:493] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:15.407583 10130 raft_consensus.cc:3060] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.415696 10130 raft_consensus.cc:515] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:15.419173 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0265654cefc40a5937d7e194f03dd5c" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:15.419155 11012 raft_consensus.cc:515] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:15.420116 10200 raft_consensus.cc:3060] T a0265654cefc40a5937d7e194f03dd5c P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.421890 10130 leader_election.cc:290] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:07:15.421978 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78325f9138849749ada9825bfe45cd8" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:15.422765 10199 raft_consensus.cc:3060] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.423553 11012 leader_election.cc:290] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:07:15.424571 10280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0265654cefc40a5937d7e194f03dd5c" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:15.425211 10280 raft_consensus.cc:3055] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:15.425530 10280 raft_consensus.cc:740] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:07:15.426362 10280 consensus_queue.cc:260] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:15.427522 10280 raft_consensus.cc:3060] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.430009 10199 raft_consensus.cc:2468] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 2.
I20260709 13:07:15.430778 10275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78325f9138849749ada9825bfe45cd8" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:15.431394 10275 raft_consensus.cc:3055] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:15.431360 10906 leader_election.cc:304] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:07:15.431933 10275 raft_consensus.cc:740] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:07:15.432719 10275 consensus_queue.cc:260] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:15.433329 11174 raft_consensus.cc:2804] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:15.433367 10200 raft_consensus.cc:2468] T a0265654cefc40a5937d7e194f03dd5c P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 2.
I20260709 13:07:15.433930 11174 raft_consensus.cc:697] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 2 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:07:15.434239 10275 raft_consensus.cc:3060] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.434413 10280 raft_consensus.cc:2468] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 2.
I20260709 13:07:15.434993 10095 leader_election.cc:304] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:07:15.434880 11174 consensus_queue.cc:237] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:15.436525 11150 raft_consensus.cc:2804] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:15.437063 11150 raft_consensus.cc:697] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [term 2 LEADER]: Becoming Leader. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:07:15.438299 11150 consensus_queue.cc:237] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:15.442178 10275 raft_consensus.cc:2468] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 2.
I20260709 13:07:15.449878  9950 catalog_manager.cc:5697] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 1 to 2, leader changed from 50999113203c482b9b9300caa485f154 (127.0.107.196) to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 2 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:15.451026  9954 catalog_manager.cc:5697] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 reported cstate change: term changed from 1 to 2, leader changed from 50999113203c482b9b9300caa485f154 (127.0.107.196) to 575e44d297f045a389903d65a471d548 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "575e44d297f045a389903d65a471d548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:15.499189 11159 raft_consensus.cc:993] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154: : Instructing follower efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:15.499706 11157 raft_consensus.cc:1081] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Signalling peer efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:15.501127 11011 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e66a35e846da4843bda7db1f5fca14b6"
dest_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:35172
I20260709 13:07:15.501510 11011 raft_consensus.cc:493] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:15.501775 11011 raft_consensus.cc:3060] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.506898 11011 raft_consensus.cc:515] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:15.508523 11011 leader_election.cc:290] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:07:15.509097 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e66a35e846da4843bda7db1f5fca14b6" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:15.509464 10275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e66a35e846da4843bda7db1f5fca14b6" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:15.509774 10200 raft_consensus.cc:3060] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.510092 10275 raft_consensus.cc:3055] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:15.510372 10275 raft_consensus.cc:740] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:07:15.511085 10275 consensus_queue.cc:260] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:15.512181 10275 raft_consensus.cc:3060] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.517261 10200 raft_consensus.cc:2468] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 2.
I20260709 13:07:15.518174 10275 raft_consensus.cc:2468] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 2.
I20260709 13:07:15.518591 10906 leader_election.cc:304] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:07:15.519465 11174 raft_consensus.cc:2804] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:15.519945 11174 raft_consensus.cc:697] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [term 2 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:07:15.520900 11174 consensus_queue.cc:237] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:15.530084  9954 catalog_manager.cc:5697] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 1 to 2, leader changed from 50999113203c482b9b9300caa485f154 (127.0.107.196) to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 2 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:15.716871 11128 raft_consensus.cc:993] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78: : Instructing follower 50999113203c482b9b9300caa485f154 to start an election
I20260709 13:07:15.717310 11128 raft_consensus.cc:1081] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Signalling peer 50999113203c482b9b9300caa485f154 to start an election
I20260709 13:07:15.718642 10280 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0529bf107d07493eb0984c4cbb9ea01e"
dest_uuid: "50999113203c482b9b9300caa485f154"
 from {username='slave'} at 127.0.0.1:34002
I20260709 13:07:15.719156 10280 raft_consensus.cc:493] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:15.719473 10280 raft_consensus.cc:3060] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.725724 10280 raft_consensus.cc:515] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:15.730412 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0529bf107d07493eb0984c4cbb9ea01e" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:07:15.731091 10128 raft_consensus.cc:3060] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.731262 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0529bf107d07493eb0984c4cbb9ea01e" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:15.731805 10200 raft_consensus.cc:3055] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:15.732065 10200 raft_consensus.cc:740] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:07:15.732776 10200 consensus_queue.cc:260] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:15.733820 10200 raft_consensus.cc:3060] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.729928 10280 leader_election.cc:290] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:07:15.738154 10128 raft_consensus.cc:2468] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 2.
I20260709 13:07:15.740549 10243 leader_election.cc:304] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 575e44d297f045a389903d65a471d548; no voters: 
I20260709 13:07:15.741456 11159 raft_consensus.cc:2804] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:15.742033 11159 raft_consensus.cc:697] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:07:15.743160 10200 raft_consensus.cc:2468] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 2.
I20260709 13:07:15.742976 11159 consensus_queue.cc:237] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:15.754395  9954 catalog_manager.cc:5697] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 1 to 2, leader changed from 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195) to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:15.812630 11138 raft_consensus.cc:993] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548: : Instructing follower 50999113203c482b9b9300caa485f154 to start an election
I20260709 13:07:15.813299 11125 raft_consensus.cc:1081] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Signalling peer 50999113203c482b9b9300caa485f154 to start an election
I20260709 13:07:15.815150 10280 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1249fe82a5ce4bd7b416e6bb46be8f79"
dest_uuid: "50999113203c482b9b9300caa485f154"
 from {username='slave'} at 127.0.0.1:34026
I20260709 13:07:15.815696 10280 raft_consensus.cc:493] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:15.816037 10280 raft_consensus.cc:3060] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.822868 10280 raft_consensus.cc:515] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:15.824743 10280 leader_election.cc:290] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:07:15.825434 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1249fe82a5ce4bd7b416e6bb46be8f79" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:07:15.826025 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1249fe82a5ce4bd7b416e6bb46be8f79" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:15.826699 10200 raft_consensus.cc:3060] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.826067 10128 raft_consensus.cc:3055] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:15.827122 10128 raft_consensus.cc:740] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:07:15.827899 10128 consensus_queue.cc:260] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:15.829566 10128 raft_consensus.cc:3060] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.833668 10200 raft_consensus.cc:2468] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 2.
I20260709 13:07:15.834971 10243 leader_election.cc:304] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:07:15.835733 11159 raft_consensus.cc:2804] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:15.836208 11159 raft_consensus.cc:697] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:07:15.836990 11159 consensus_queue.cc:237] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:15.838431 10128 raft_consensus.cc:2468] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 2.
I20260709 13:07:15.839912 10280 raft_consensus.cc:1275] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:15.842720 11174 consensus_queue.cc:1048] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:15.848202 10275 raft_consensus.cc:1275] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:15.849612 11174 consensus_queue.cc:1048] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:15.850145  9954 catalog_manager.cc:5697] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 1 to 2, leader changed from 575e44d297f045a389903d65a471d548 (127.0.107.194) to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:15.870404 10200 raft_consensus.cc:1275] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:15.897575 11174 consensus_queue.cc:1048] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:15.901278 10200 raft_consensus.cc:1275] T 4457dec3dec94c1dbac542a298184d0f P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:15.903582 11174 consensus_queue.cc:1048] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:15.916525 10201 raft_consensus.cc:1275] T a0265654cefc40a5937d7e194f03dd5c P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Refusing update from remote peer 575e44d297f045a389903d65a471d548: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:15.918308 11125 consensus_queue.cc:1048] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:15.929267 10277 raft_consensus.cc:1275] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Refusing update from remote peer 575e44d297f045a389903d65a471d548: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:15.930611 11125 consensus_queue.cc:1048] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:16.049731 10277 raft_consensus.cc:1275] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:16.051213 11174 consensus_queue.cc:1048] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:16.061794 10201 raft_consensus.cc:1275] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:16.065532 11145 consensus_queue.cc:1048] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:16.207919 10200 raft_consensus.cc:1275] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Refusing update from remote peer 50999113203c482b9b9300caa485f154: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:16.209226 11159 consensus_queue.cc:1048] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:16.219661 10128 raft_consensus.cc:1275] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Refusing update from remote peer 50999113203c482b9b9300caa485f154: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:16.221886 11135 consensus_queue.cc:1048] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:16.296782   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:16.334044   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:16.335520 10201 raft_consensus.cc:1275] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Refusing update from remote peer 50999113203c482b9b9300caa485f154: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:16.343683 11135 consensus_queue.cc:1048] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:16.383375 10128 raft_consensus.cc:1275] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Refusing update from remote peer 50999113203c482b9b9300caa485f154: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:16.389253 11157 consensus_queue.cc:1048] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:07:16.393002 10277 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0529bf107d07493eb0984c4cbb9ea01e"
dest_uuid: "50999113203c482b9b9300caa485f154"
mode: GRACEFUL
new_leader_uuid: "575e44d297f045a389903d65a471d548"
 from {username='slave'} at 127.0.0.1:58788
I20260709 13:07:16.393720 10277 raft_consensus.cc:606] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Received request to transfer leadership to TS 575e44d297f045a389903d65a471d548
I20260709 13:07:16.401088 10277 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0981226de91f4f9e96cf2e8b926aaf19"
dest_uuid: "50999113203c482b9b9300caa485f154"
mode: GRACEFUL
new_leader_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:58788
I20260709 13:07:16.401835 10277 raft_consensus.cc:606] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Received request to transfer leadership to TS efa338cd132c457896b11e08d7ea787d
I20260709 13:07:16.404809 10201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc1d8020dcd34e63b72a7de2db565ff6"
dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
mode: GRACEFUL
new_leader_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:58228
I20260709 13:07:16.405448 10201 raft_consensus.cc:606] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Received request to transfer leadership to TS efa338cd132c457896b11e08d7ea787d
I20260709 13:07:16.408509 10201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7d966d297f346da9e984aa2e181590a"
dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
mode: GRACEFUL
new_leader_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:58228
I20260709 13:07:16.409406 10201 raft_consensus.cc:606] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 2 LEADER]: Received request to transfer leadership to TS efa338cd132c457896b11e08d7ea787d
I20260709 13:07:16.414098 10130 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6c60f07656843aeac5d469a6533aaaa"
dest_uuid: "575e44d297f045a389903d65a471d548"
mode: GRACEFUL
new_leader_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:55924
I20260709 13:07:16.414757 10130 raft_consensus.cc:606] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Received request to transfer leadership to TS efa338cd132c457896b11e08d7ea787d
I20260709 13:07:16.416118   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:07:16.417452   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:16.522496 11125 raft_consensus.cc:993] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548: : Instructing follower efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:16.523145 11138 raft_consensus.cc:1081] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Signalling peer efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:16.524856 11011 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d6c60f07656843aeac5d469a6533aaaa"
dest_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:35174
I20260709 13:07:16.525455 11011 raft_consensus.cc:493] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:16.525868 11011 raft_consensus.cc:3060] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:16.531585 11011 raft_consensus.cc:515] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:16.533613 11011 leader_election.cc:290] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:07:16.534637 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6c60f07656843aeac5d469a6533aaaa" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:07:16.534744 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6c60f07656843aeac5d469a6533aaaa" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:16.536012 10201 raft_consensus.cc:3060] T d6c60f07656843aeac5d469a6533aaaa P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:16.535468 10130 raft_consensus.cc:3055] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:16.537442 10130 raft_consensus.cc:740] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:07:16.538339 10130 consensus_queue.cc:260] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:16.539415 10130 raft_consensus.cc:3060] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:16.539770 11128 raft_consensus.cc:993] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78: : Instructing follower efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:16.540251 11128 raft_consensus.cc:1081] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 2 LEADER]: Signalling peer efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:16.543066 10201 raft_consensus.cc:2468] T d6c60f07656843aeac5d469a6533aaaa P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 2.
I20260709 13:07:16.546764 10130 raft_consensus.cc:2468] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 2.
I20260709 13:07:16.548383 10906 leader_election.cc:304] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:07:16.549578 11176 raft_consensus.cc:2804] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:16.550104 11176 raft_consensus.cc:697] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [term 2 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:07:16.550870 11176 consensus_queue.cc:237] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:16.553947 11012 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c7d966d297f346da9e984aa2e181590a"
dest_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:35176
I20260709 13:07:16.554613 11012 raft_consensus.cc:493] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:16.555008 11012 raft_consensus.cc:3060] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:16.561165 11012 raft_consensus.cc:515] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:16.563140 11012 leader_election.cc:290] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 3 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:07:16.562353  9954 catalog_manager.cc:5697] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 1 to 2, leader changed from 575e44d297f045a389903d65a471d548 (127.0.107.194) to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 2 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:16.566833 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d966d297f346da9e984aa2e181590a" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:16.567440 10201 raft_consensus.cc:3055] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:16.567664 10201 raft_consensus.cc:740] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:07:16.568331 10201 consensus_queue.cc:260] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:16.569180 10201 raft_consensus.cc:3060] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:16.572252 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d966d297f346da9e984aa2e181590a" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:16.573117 10277 raft_consensus.cc:3060] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:16.575891 10201 raft_consensus.cc:2468] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 3.
I20260709 13:07:16.577158 10906 leader_election.cc:304] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:07:16.577960 11176 raft_consensus.cc:2804] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:16.578433 11176 raft_consensus.cc:697] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 3 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:07:16.579317 11176 consensus_queue.cc:237] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:16.580564 10277 raft_consensus.cc:2468] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 3.
I20260709 13:07:16.600160  9954 catalog_manager.cc:5697] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 2 to 3, leader changed from 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195) to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 3 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:16.635679 11128 raft_consensus.cc:993] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78: : Instructing follower efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:16.636443 11128 raft_consensus.cc:1081] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Signalling peer efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:16.638258 11011 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bc1d8020dcd34e63b72a7de2db565ff6"
dest_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:35176
I20260709 13:07:16.638813 11011 raft_consensus.cc:493] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:16.639163 11011 raft_consensus.cc:3060] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:16.644060 11011 raft_consensus.cc:515] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:16.645782 11011 leader_election.cc:290] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:07:16.646529 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc1d8020dcd34e63b72a7de2db565ff6" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:16.646644 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc1d8020dcd34e63b72a7de2db565ff6" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:16.647363 10201 raft_consensus.cc:3055] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:16.647441 10277 raft_consensus.cc:3060] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:16.647751 10201 raft_consensus.cc:740] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:07:16.648715 10201 consensus_queue.cc:260] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:16.650020 10201 raft_consensus.cc:3060] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:16.654168 10277 raft_consensus.cc:2468] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 2.
I20260709 13:07:16.655215 10906 leader_election.cc:304] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:07:16.656025 11176 raft_consensus.cc:2804] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:16.656510 11176 raft_consensus.cc:697] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [term 2 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:07:16.656957 10201 raft_consensus.cc:2468] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 2.
I20260709 13:07:16.657510 11176 consensus_queue.cc:237] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:16.669998  9954 catalog_manager.cc:5697] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 1 to 2, leader changed from 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195) to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 2 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:16.749220 11212 raft_consensus.cc:993] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154: : Instructing follower 575e44d297f045a389903d65a471d548 to start an election
I20260709 13:07:16.749910 11157 raft_consensus.cc:1081] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Signalling peer 575e44d297f045a389903d65a471d548 to start an election
I20260709 13:07:16.751827 10130 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0529bf107d07493eb0984c4cbb9ea01e"
dest_uuid: "575e44d297f045a389903d65a471d548"
 from {username='slave'} at 127.0.0.1:52882
I20260709 13:07:16.753306 10130 raft_consensus.cc:493] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:16.753739 10130 raft_consensus.cc:3060] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:16.761061 10130 raft_consensus.cc:515] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:16.763427 10130 leader_election.cc:290] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 3 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:07:16.764024 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0529bf107d07493eb0984c4cbb9ea01e" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:16.764746 10201 raft_consensus.cc:3060] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:16.765398 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0529bf107d07493eb0984c4cbb9ea01e" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:16.766062 10277 raft_consensus.cc:3055] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:16.766360 10277 raft_consensus.cc:740] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:07:16.767004 10277 consensus_queue.cc:260] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:16.767901 10277 raft_consensus.cc:3060] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:16.772533 10201 raft_consensus.cc:2468] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 3.
I20260709 13:07:16.773814 10095 leader_election.cc:304] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:07:16.774842 11125 raft_consensus.cc:2804] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:16.775480 11125 raft_consensus.cc:697] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 3 LEADER]: Becoming Leader. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:07:16.776471 10277 raft_consensus.cc:2468] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 3.
I20260709 13:07:16.776322 11125 consensus_queue.cc:237] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:16.787683  9954 catalog_manager.cc:5697] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 reported cstate change: term changed from 2 to 3, leader changed from 50999113203c482b9b9300caa485f154 (127.0.107.196) to 575e44d297f045a389903d65a471d548 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "575e44d297f045a389903d65a471d548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:16.798348 11157 raft_consensus.cc:993] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154: : Instructing follower efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:16.798905 11212 raft_consensus.cc:1081] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Signalling peer efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:16.800760 11012 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0981226de91f4f9e96cf2e8b926aaf19"
dest_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:35172
I20260709 13:07:16.801211 11012 raft_consensus.cc:493] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:16.801458 11012 raft_consensus.cc:3060] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:16.806258 11012 raft_consensus.cc:515] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:16.807880 11012 leader_election.cc:290] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 3 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:07:16.808833 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0981226de91f4f9e96cf2e8b926aaf19" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:16.809193 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0981226de91f4f9e96cf2e8b926aaf19" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:16.809470 10201 raft_consensus.cc:3060] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:16.809839 10277 raft_consensus.cc:3055] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:16.810195 10277 raft_consensus.cc:740] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:07:16.810989 10277 consensus_queue.cc:260] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:16.812114 10277 raft_consensus.cc:3060] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:16.816732 10201 raft_consensus.cc:2468] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 3.
I20260709 13:07:16.818838 10906 leader_election.cc:304] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:07:16.819629 11176 raft_consensus.cc:2804] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:16.820130 11176 raft_consensus.cc:697] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 3 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:07:16.820972 11176 consensus_queue.cc:237] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:16.823503 10277 raft_consensus.cc:2468] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 3.
I20260709 13:07:16.833600  9954 catalog_manager.cc:5697] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 2 to 3, leader changed from 50999113203c482b9b9300caa485f154 (127.0.107.196) to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 3 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:16.933812 10201 raft_consensus.cc:1275] T d6c60f07656843aeac5d469a6533aaaa P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:16.935139 11176 consensus_queue.cc:1048] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:16.949384 10128 raft_consensus.cc:1275] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:16.951650 11145 consensus_queue.cc:1048] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:17.148730 10275 raft_consensus.cc:1275] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:17.150027 11145 consensus_queue.cc:1048] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:17.160104 10201 raft_consensus.cc:1275] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:17.162215 11145 consensus_queue.cc:1048] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:17.184522 10277 raft_consensus.cc:1275] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:17.185844 11145 consensus_queue.cc:1048] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:17.186286 10201 raft_consensus.cc:1275] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:17.187875 11145 consensus_queue.cc:1048] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:17.238772 10200 raft_consensus.cc:1275] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Refusing update from remote peer 575e44d297f045a389903d65a471d548: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:17.239995 11138 consensus_queue.cc:1048] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:17.240903 10200 raft_consensus.cc:1275] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:17.242949 11175 consensus_queue.cc:1048] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:17.252585 10275 raft_consensus.cc:1275] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:17.254891 11175 consensus_queue.cc:1048] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:17.273049 10277 raft_consensus.cc:1275] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Refusing update from remote peer 575e44d297f045a389903d65a471d548: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:17.274801 11125 consensus_queue.cc:1048] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:17.418579   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:17.467497   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:17.506152 10128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0529bf107d07493eb0984c4cbb9ea01e"
dest_uuid: "575e44d297f045a389903d65a471d548"
mode: GRACEFUL
new_leader_uuid: "50999113203c482b9b9300caa485f154"
 from {username='slave'} at 127.0.0.1:55924
I20260709 13:07:17.506706 10128 raft_consensus.cc:606] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 3 LEADER]: Received request to transfer leadership to TS 50999113203c482b9b9300caa485f154
I20260709 13:07:17.509244 10201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5461abd09b074193b3d03df659fa2ced"
dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
mode: GRACEFUL
new_leader_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:58228
I20260709 13:07:17.509788 10201 raft_consensus.cc:606] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Received request to transfer leadership to TS efa338cd132c457896b11e08d7ea787d
I20260709 13:07:17.512502 10200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce414a4edf504d78b1b1e83498a5cfeb"
dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
mode: GRACEFUL
new_leader_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:58228
I20260709 13:07:17.513036 10200 raft_consensus.cc:606] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 2 LEADER]: Received request to transfer leadership to TS efa338cd132c457896b11e08d7ea787d
I20260709 13:07:17.514389   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:07:17.515693   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:17.615577 11128 raft_consensus.cc:993] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78: : Instructing follower efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:17.616060 11198 raft_consensus.cc:1081] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Signalling peer efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:17.617623 11011 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5461abd09b074193b3d03df659fa2ced"
dest_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:35176
I20260709 13:07:17.618269 11011 raft_consensus.cc:493] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:17.618570 11011 raft_consensus.cc:3060] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:17.623688 11011 raft_consensus.cc:515] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:17.625190 11011 leader_election.cc:290] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:07:17.625849 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461abd09b074193b3d03df659fa2ced" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:17.626392 10200 raft_consensus.cc:3055] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:17.626555 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461abd09b074193b3d03df659fa2ced" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:17.626708 10200 raft_consensus.cc:740] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:07:17.627233 10277 raft_consensus.cc:3060] T 5461abd09b074193b3d03df659fa2ced P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:17.627689 10200 consensus_queue.cc:260] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:17.628917 10200 raft_consensus.cc:3060] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:17.632750 10277 raft_consensus.cc:2468] T 5461abd09b074193b3d03df659fa2ced P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 2.
I20260709 13:07:17.634123 10906 leader_election.cc:304] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:07:17.634585 10200 raft_consensus.cc:2468] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 2.
I20260709 13:07:17.634917 11236 raft_consensus.cc:2804] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:17.635676 11236 raft_consensus.cc:697] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [term 2 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:07:17.636514 11236 consensus_queue.cc:237] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:17.647070  9954 catalog_manager.cc:5697] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 1 to 2, leader changed from 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195) to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 2 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:17.648231 11128 raft_consensus.cc:993] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78: : Instructing follower efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:17.648769 11198 raft_consensus.cc:1081] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 2 LEADER]: Signalling peer efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:17.650328 11011 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ce414a4edf504d78b1b1e83498a5cfeb"
dest_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:35176
I20260709 13:07:17.650799 11011 raft_consensus.cc:493] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:17.651064 11011 raft_consensus.cc:3060] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:17.658138 11011 raft_consensus.cc:515] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:17.659745 11011 leader_election.cc:290] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 3 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:07:17.660385 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce414a4edf504d78b1b1e83498a5cfeb" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:07:17.660671 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce414a4edf504d78b1b1e83498a5cfeb" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:17.661401 10128 raft_consensus.cc:3060] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:17.661453 10200 raft_consensus.cc:3055] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:17.661914 10200 raft_consensus.cc:740] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8fe33410d51a469e81be6edc602d7d78, State: Running, Role: LEADER
I20260709 13:07:17.662636 10200 consensus_queue.cc:260] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:17.663619 10200 raft_consensus.cc:3060] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:17.669661 10128 raft_consensus.cc:2468] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 3.
I20260709 13:07:17.669981 10200 raft_consensus.cc:2468] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 3.
I20260709 13:07:17.670943 10906 leader_election.cc:304] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:07:17.671578 11236 raft_consensus.cc:2804] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:17.672061 11236 raft_consensus.cc:697] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 3 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:07:17.672895 11236 consensus_queue.cc:237] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } }
I20260709 13:07:17.681047  9954 catalog_manager.cc:5697] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 2 to 3, leader changed from 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195) to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 3 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:17.833122 11138 raft_consensus.cc:993] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548: : Instructing follower 50999113203c482b9b9300caa485f154 to start an election
I20260709 13:07:17.833604 11242 raft_consensus.cc:1081] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 3 LEADER]: Signalling peer 50999113203c482b9b9300caa485f154 to start an election
I20260709 13:07:17.835247 10277 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0529bf107d07493eb0984c4cbb9ea01e"
dest_uuid: "50999113203c482b9b9300caa485f154"
 from {username='slave'} at 127.0.0.1:34026
I20260709 13:07:17.835783 10277 raft_consensus.cc:493] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:17.836097 10277 raft_consensus.cc:3060] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:17.842749 10277 raft_consensus.cc:515] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:17.844305 10277 leader_election.cc:290] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 4 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419)
I20260709 13:07:17.845121 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0529bf107d07493eb0984c4cbb9ea01e" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:07:17.845372 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0529bf107d07493eb0984c4cbb9ea01e" candidate_uuid: "50999113203c482b9b9300caa485f154" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:17.846124 10200 raft_consensus.cc:3060] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:17.847018 10128 raft_consensus.cc:3055] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:17.847681 10128 raft_consensus.cc:740] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:07:17.848821 10128 consensus_queue.cc:260] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:17.850157 10128 raft_consensus.cc:3060] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:17.854146 10200 raft_consensus.cc:2468] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 4.
I20260709 13:07:17.855108 10128 raft_consensus.cc:2468] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50999113203c482b9b9300caa485f154 in term 4.
I20260709 13:07:17.855348 10243 leader_election.cc:304] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:07:17.856371 11157 raft_consensus.cc:2804] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:17.857722 11157 raft_consensus.cc:697] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 4 LEADER]: Becoming Leader. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:07:17.858839 11157 consensus_queue.cc:237] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:17.868130  9954 catalog_manager.cc:5697] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 reported cstate change: term changed from 3 to 4, leader changed from 575e44d297f045a389903d65a471d548 (127.0.107.194) to 50999113203c482b9b9300caa485f154 (127.0.107.196). New cstate: current_term: 4 leader_uuid: "50999113203c482b9b9300caa485f154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:18.106488 10130 raft_consensus.cc:1275] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 3 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:18.107820 11176 consensus_queue.cc:1048] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:18.110188 10201 raft_consensus.cc:1275] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:18.113387 11145 consensus_queue.cc:1048] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:18.121749 10277 raft_consensus.cc:1275] T 5461abd09b074193b3d03df659fa2ced P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:18.131691 11176 consensus_queue.cc:1048] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:18.160188 10201 raft_consensus.cc:1275] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:18.187377 11145 consensus_queue.cc:1048] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:18.446641 10199 raft_consensus.cc:1275] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 4 FOLLOWER]: Refusing update from remote peer 50999113203c482b9b9300caa485f154: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:18.447947 11157 consensus_queue.cc:1048] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:18.456897 10130 raft_consensus.cc:1275] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 4 FOLLOWER]: Refusing update from remote peer 50999113203c482b9b9300caa485f154: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:18.458774 11255 consensus_queue.cc:1048] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:18.516660   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:18.568785   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:18.611374 10275 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0529bf107d07493eb0984c4cbb9ea01e"
dest_uuid: "50999113203c482b9b9300caa485f154"
mode: GRACEFUL
new_leader_uuid: "575e44d297f045a389903d65a471d548"
 from {username='slave'} at 127.0.0.1:58788
I20260709 13:07:18.611970 10275 raft_consensus.cc:606] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 4 LEADER]: Received request to transfer leadership to TS 575e44d297f045a389903d65a471d548
I20260709 13:07:18.613294   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:07:18.614890   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:19.055177 11255 raft_consensus.cc:993] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154: : Instructing follower 575e44d297f045a389903d65a471d548 to start an election
I20260709 13:07:19.055754 11244 raft_consensus.cc:1081] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 4 LEADER]: Signalling peer 575e44d297f045a389903d65a471d548 to start an election
I20260709 13:07:19.057340 10130 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0529bf107d07493eb0984c4cbb9ea01e"
dest_uuid: "575e44d297f045a389903d65a471d548"
 from {username='slave'} at 127.0.0.1:52882
I20260709 13:07:19.058000 10130 raft_consensus.cc:493] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:19.058516 10130 raft_consensus.cc:3060] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:19.066112 10130 raft_consensus.cc:515] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:19.069180 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0529bf107d07493eb0984c4cbb9ea01e" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "8fe33410d51a469e81be6edc602d7d78"
I20260709 13:07:19.068612 10130 leader_election.cc:290] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 5 election: Requested vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:07:19.069847 10275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0529bf107d07493eb0984c4cbb9ea01e" candidate_uuid: "575e44d297f045a389903d65a471d548" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:19.070482 10275 raft_consensus.cc:3055] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:19.070829 10275 raft_consensus.cc:740] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 50999113203c482b9b9300caa485f154, State: Running, Role: LEADER
I20260709 13:07:19.071241 10199 raft_consensus.cc:3060] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:19.071547 10275 consensus_queue.cc:260] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:19.072834 10275 raft_consensus.cc:3060] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:19.078656 10199 raft_consensus.cc:2468] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 5.
I20260709 13:07:19.079674 10275 raft_consensus.cc:2468] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575e44d297f045a389903d65a471d548 in term 5.
I20260709 13:07:19.080176 10095 leader_election.cc:304] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575e44d297f045a389903d65a471d548, 8fe33410d51a469e81be6edc602d7d78; no voters: 
I20260709 13:07:19.080940 11138 raft_consensus.cc:2804] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:19.081377 11138 raft_consensus.cc:697] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 5 LEADER]: Becoming Leader. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:07:19.082036 11138 consensus_queue.cc:237] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } }
I20260709 13:07:19.091444  9953 catalog_manager.cc:5697] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 reported cstate change: term changed from 4 to 5, leader changed from 50999113203c482b9b9300caa485f154 (127.0.107.196) to 575e44d297f045a389903d65a471d548 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "575e44d297f045a389903d65a471d548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:19.542409 10275 raft_consensus.cc:1275] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 5 FOLLOWER]: Refusing update from remote peer 575e44d297f045a389903d65a471d548: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:07:19.543723 11138 consensus_queue.cc:1048] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:19.553870 10201 raft_consensus.cc:1275] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 5 FOLLOWER]: Refusing update from remote peer 575e44d297f045a389903d65a471d548: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:07:19.561074 11138 consensus_queue.cc:1048] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:19.615722   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:19.668792   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:19.702945 10128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "80b0c1b6201b46d5b1f46ccad59503b7"
dest_uuid: "575e44d297f045a389903d65a471d548"
mode: GRACEFUL
new_leader_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:55924
I20260709 13:07:19.703608 10128 raft_consensus.cc:606] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Received request to transfer leadership to TS efa338cd132c457896b11e08d7ea787d
I20260709 13:07:19.705098   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:07:19.706393   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:20.038021 11268 raft_consensus.cc:993] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548: : Instructing follower efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:20.039309 11266 raft_consensus.cc:1081] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Signalling peer efa338cd132c457896b11e08d7ea787d to start an election
I20260709 13:07:20.043691 11012 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "80b0c1b6201b46d5b1f46ccad59503b7"
dest_uuid: "efa338cd132c457896b11e08d7ea787d"
 from {username='slave'} at 127.0.0.1:35174
I20260709 13:07:20.044940 11012 raft_consensus.cc:493] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:20.045576 11012 raft_consensus.cc:3060] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:20.068756 11012 raft_consensus.cc:515] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:20.073302 11012 leader_election.cc:290] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 election: Requested vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), 50999113203c482b9b9300caa485f154 (127.0.107.196:40841)
I20260709 13:07:20.073735 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80b0c1b6201b46d5b1f46ccad59503b7" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "575e44d297f045a389903d65a471d548"
I20260709 13:07:20.074731 10277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80b0c1b6201b46d5b1f46ccad59503b7" candidate_uuid: "efa338cd132c457896b11e08d7ea787d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50999113203c482b9b9300caa485f154"
I20260709 13:07:20.074992 10130 raft_consensus.cc:3055] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:20.075492 10130 raft_consensus.cc:740] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 575e44d297f045a389903d65a471d548, State: Running, Role: LEADER
I20260709 13:07:20.075619 10277 raft_consensus.cc:3060] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:20.076707 10130 consensus_queue.cc:260] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:20.078742 10130 raft_consensus.cc:3060] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:20.086850 10277 raft_consensus.cc:2468] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 2.
I20260709 13:07:20.088414 10906 leader_election.cc:304] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50999113203c482b9b9300caa485f154, efa338cd132c457896b11e08d7ea787d; no voters: 
I20260709 13:07:20.088430 10130 raft_consensus.cc:2468] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa338cd132c457896b11e08d7ea787d in term 2.
I20260709 13:07:20.089406 11274 raft_consensus.cc:2804] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:20.090042 11274 raft_consensus.cc:697] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [term 2 LEADER]: Becoming Leader. State: Replica: efa338cd132c457896b11e08d7ea787d, State: Running, Role: LEADER
I20260709 13:07:20.091027 11274 consensus_queue.cc:237] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:20.127028  9954 catalog_manager.cc:5697] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d reported cstate change: term changed from 1 to 2, leader changed from 575e44d297f045a389903d65a471d548 (127.0.107.194) to efa338cd132c457896b11e08d7ea787d (127.0.107.193). New cstate: current_term: 2 leader_uuid: "efa338cd132c457896b11e08d7ea787d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:20.518358 10277 raft_consensus.cc:1275] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:20.519634 11145 consensus_queue.cc:1048] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:20.536720 10128 raft_consensus.cc:1275] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Refusing update from remote peer efa338cd132c457896b11e08d7ea787d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:20.541188 11145 consensus_queue.cc:1048] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [LEADER]: Connected to new peer: Peer: permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:20.707916   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:20.746698   431 tablet_server.cc:179] TabletServer@127.0.107.193:42425 shutting down...
W20260709 13:07:20.762320 10172 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 265 similar messages]
W20260709 13:07:20.767992 10172 consensus_peers.cc:597] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:20.781250 10246 consensus_peers.cc:597] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:20.784741 10246 consensus_peers.cc:597] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:20.794457 10098 consensus_peers.cc:597] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:20.798811   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:20.799917   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.801231   431 raft_consensus.cc:2243] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.801846   431 raft_consensus.cc:2272] T 1f060317dcbd4f0db70f83fd18320215 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.804317   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.804900   431 raft_consensus.cc:2243] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.805277   431 raft_consensus.cc:2272] T d242deba350e42cb8e2b623027a26516 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.807430   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.808223   431 raft_consensus.cc:2243] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.808928   431 raft_consensus.cc:2272] T a0ea0c11178d43888a5596f49ffc7a7e P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.812161   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:07:20.812536 10172 consensus_peers.cc:597] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:20.812950   431 raft_consensus.cc:2243] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.813489   431 raft_consensus.cc:2272] T aa03c270e7364d769700b877d171588e P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.816572   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.817472   431 raft_consensus.cc:2243] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.818142   431 raft_consensus.cc:2272] T d8d642d68d234d668cb67cb81b80556e P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.820922   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.821516   431 raft_consensus.cc:2243] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.821930   431 raft_consensus.cc:2272] T 39e4808ea4be41f483e08e83162c59c8 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.824049   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.825001   431 raft_consensus.cc:2243] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.825477   431 raft_consensus.cc:2272] T 6d2f27feb83847298256dd63dafcd35c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.828379   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.828994   431 raft_consensus.cc:2243] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:20.830073   431 raft_consensus.cc:2272] T 0981226de91f4f9e96cf2e8b926aaf19 P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.833416   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.834213   431 raft_consensus.cc:2243] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.834779   431 raft_consensus.cc:2272] T b626433d4aec4288976ffc448a79bb68 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.837783   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.838379   431 raft_consensus.cc:2243] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.838742   431 raft_consensus.cc:2272] T abca9c95a1334ebc97985eedf46fc01f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.841202   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.842010   431 raft_consensus.cc:2243] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.842653   431 raft_consensus.cc:2272] T 0e003c3e14c2444b937f547eef84b672 P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.846161   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.846961   431 raft_consensus.cc:2243] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.847486   431 raft_consensus.cc:2272] T 7364742faea3470694e1a23f151f1b57 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:20.848905 10098 consensus_peers.cc:597] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:20.850351   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.851083   431 raft_consensus.cc:2243] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.851495   431 raft_consensus.cc:2272] T 50256706bf3a4bf29b64ea4ea6583da3 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.853830   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.854591   431 raft_consensus.cc:2243] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.855245   431 raft_consensus.cc:2272] T f660e208f31341f0bc602253c996258c P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.858057   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.858584   431 raft_consensus.cc:2243] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:20.859328   431 raft_consensus.cc:2272] T 4457dec3dec94c1dbac542a298184d0f P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.861518   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.862063   431 raft_consensus.cc:2243] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.862422   431 raft_consensus.cc:2272] T 5c136ae5f6aa4055a44f8aec8883b02a P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.864585   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.865123   431 raft_consensus.cc:2243] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.865684   431 raft_consensus.cc:2272] T 83a64aca789f4ea8a4925aa79dcb0235 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.867884   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.868500   431 raft_consensus.cc:2243] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.869061   431 raft_consensus.cc:2272] T c21c65d190c541ab8e2d6ae0d4be85e7 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.871965   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.872807   431 raft_consensus.cc:2243] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.873283   431 raft_consensus.cc:2272] T 04676809490049a5923174641a3c1137 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.876569   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.877429   431 raft_consensus.cc:2243] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.878032   431 raft_consensus.cc:2272] T 2258980c3a7540dea8ae11850194e183 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.881258   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.881959   431 raft_consensus.cc:2243] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:20.882858   431 raft_consensus.cc:2272] T ccaebf4041f14bada42fd7122f34419c P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.885083   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.885700   431 raft_consensus.cc:2243] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.886209   431 raft_consensus.cc:2272] T 5f64c791e7fb4a79bd498140ef0b0943 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.889009   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.889719   431 raft_consensus.cc:2243] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.890228   431 raft_consensus.cc:2272] T 19d98d0921054d6a932a308e4adb1937 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.892745   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:07:20.892895 10246 consensus_peers.cc:597] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:20.893522   431 raft_consensus.cc:2243] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:20.894495   431 raft_consensus.cc:2272] T bc1d8020dcd34e63b72a7de2db565ff6 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:20.896633 10246 consensus_peers.cc:597] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:20.897078   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.898031   431 raft_consensus.cc:2243] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:20.899158   431 raft_consensus.cc:2272] T 48dcd8683f004a61808243f33d859911 P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.901770   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.902415   431 raft_consensus.cc:2243] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:20.903628   431 raft_consensus.cc:2272] T 5461abd09b074193b3d03df659fa2ced P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.906745   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.907624   431 raft_consensus.cc:2243] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.908332   431 raft_consensus.cc:2272] T d139f1fbdd384830bc7ee062ab3d281c P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:20.909713 10172 consensus_peers.cc:597] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:20.911484   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.912252   431 raft_consensus.cc:2243] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.912676   431 raft_consensus.cc:2272] T b41d6a8939d7411e901d88f31d2bdf38 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.915169   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.916114   431 raft_consensus.cc:2243] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:20.917150   431 raft_consensus.cc:2272] T 75ed1b358949418a9d300990f8f81652 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:20.919463 10246 consensus_peers.cc:597] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:20.920053   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.920650   431 raft_consensus.cc:2243] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.921092   431 raft_consensus.cc:2272] T 52ffbaf3a54145c2a791c4a67dbdf64a P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.923401   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.923938   431 raft_consensus.cc:2243] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.924310   431 raft_consensus.cc:2272] T 75aced19d8c94fc78e5510b592bb01be P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.926354   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.926941   431 raft_consensus.cc:2243] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:20.927816   431 raft_consensus.cc:2272] T d6c60f07656843aeac5d469a6533aaaa P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.930248   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.930794   431 raft_consensus.cc:2243] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:20.931507   431 raft_consensus.cc:2272] T 80b0c1b6201b46d5b1f46ccad59503b7 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.934095   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.934643   431 raft_consensus.cc:2243] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.935091   431 raft_consensus.cc:2272] T 64ea79b0bb13439c9384cba78627c40f P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.937134   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.937815   431 raft_consensus.cc:2243] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:20.938741   431 raft_consensus.cc:2272] T c78325f9138849749ada9825bfe45cd8 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.940909   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.941399   431 raft_consensus.cc:2243] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.941810   431 raft_consensus.cc:2272] T 1f3c90bb2ac14652a45cdcccba25f97c P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.943926   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.944393   431 raft_consensus.cc:2243] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:20.945137   431 raft_consensus.cc:2272] T ce414a4edf504d78b1b1e83498a5cfeb P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.947371   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.947961   431 raft_consensus.cc:2243] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:20.948822   431 raft_consensus.cc:2272] T e66a35e846da4843bda7db1f5fca14b6 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.950935   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.951392   431 raft_consensus.cc:2243] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.951735   431 raft_consensus.cc:2272] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.953953   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.954487   431 raft_consensus.cc:2243] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:20.955435   431 raft_consensus.cc:2272] T c7d966d297f346da9e984aa2e181590a P efa338cd132c457896b11e08d7ea787d [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:20.957875 10246 consensus_peers.cc:597] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:20.958418   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.959252   431 raft_consensus.cc:2243] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.959822   431 raft_consensus.cc:2272] T 7f0446529cd446d0ba97ee860c6ae221 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.962680   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.963428   431 raft_consensus.cc:2243] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.964066   431 raft_consensus.cc:2272] T 3fa9341e7fbe49e1a832721bfa4abff3 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.967098   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.968019   431 raft_consensus.cc:2243] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.968604   431 raft_consensus.cc:2272] T 7dd93f389d32493b9dcc10e602f452da P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.971731   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:07:20.971936 10172 consensus_peers.cc:597] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:20.972402   431 raft_consensus.cc:2243] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:20.973029   431 raft_consensus.cc:2272] T 1f4dd5b2c17747058bab6588c63b9d72 P efa338cd132c457896b11e08d7ea787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:20.975970   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:20.976727   431 raft_consensus.cc:2243] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:20.978001   431 raft_consensus.cc:2272] T 4af4e546f1084d8cb71469fbf3886792 P efa338cd132c457896b11e08d7ea787d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:20.984151 10246 consensus_peers.cc:597] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:20.992184 10098 consensus_peers.cc:597] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.008015 10172 consensus_peers.cc:597] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.021450 10098 consensus_peers.cc:597] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.038012 10172 consensus_peers.cc:597] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.041836 10172 consensus_peers.cc:597] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.042552 10172 consensus_peers.cc:597] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.070212 10172 consensus_peers.cc:597] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.073891   431 tablet_server.cc:196] TabletServer@127.0.107.193:42425 shutdown complete.
W20260709 13:07:21.078763 10098 consensus_peers.cc:597] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.081266 10098 consensus_peers.cc:597] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.163687   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:07:21.176434 10098 consensus_peers.cc:597] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.177358 10172 consensus_peers.cc:597] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.190956 10098 consensus_peers.cc:597] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.197780 10246 consensus_peers.cc:597] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.222255 10169 consensus_peers.cc:597] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.222506   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:07:21.226809 10243 consensus_peers.cc:597] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.227939 10246 consensus_peers.cc:597] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.233911   431 tablet_replica.cc:333] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548: stopping tablet replica
W20260709 13:07:21.234794 10172 consensus_peers.cc:597] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.235054   431 raft_consensus.cc:2243] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.235736   431 raft_consensus.cc:2272] T ce414a4edf504d78b1b1e83498a5cfeb P 575e44d297f045a389903d65a471d548 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.238567   431 tablet_replica.cc:333] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.239470   431 raft_consensus.cc:2243] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.240108   431 raft_consensus.cc:2272] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.243062   431 tablet_replica.cc:333] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.243811   431 raft_consensus.cc:2243] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.244438   431 raft_consensus.cc:2272] T 2258980c3a7540dea8ae11850194e183 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.248178   431 tablet_replica.cc:333] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.249033   431 raft_consensus.cc:2243] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.250406   431 raft_consensus.cc:2272] T 1f060317dcbd4f0db70f83fd18320215 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.253445   431 tablet_replica.cc:333] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548: stopping tablet replica
W20260709 13:07:21.254154 10243 consensus_peers.cc:597] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.254336   431 raft_consensus.cc:2243] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.254911   431 raft_consensus.cc:2272] T 4af4e546f1084d8cb71469fbf3886792 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.257352   431 tablet_replica.cc:333] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.258057   431 raft_consensus.cc:2243] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.258551   431 raft_consensus.cc:2272] T cded7cb005f64f9c894f568140883a77 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.260728   431 tablet_replica.cc:333] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.261389   431 raft_consensus.cc:2243] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.261904   431 raft_consensus.cc:2272] T 9f7da1cc13f64f56b02defc2e119c507 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.264189   431 tablet_replica.cc:333] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.265065   431 raft_consensus.cc:2243] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.266285   431 raft_consensus.cc:2272] T 1f4dd5b2c17747058bab6588c63b9d72 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:21.267546 10169 consensus_peers.cc:597] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.268746   431 tablet_replica.cc:333] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.269486   431 raft_consensus.cc:2243] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.270303   431 raft_consensus.cc:2272] T 0529bf107d07493eb0984c4cbb9ea01e P 575e44d297f045a389903d65a471d548 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.272889   431 tablet_replica.cc:333] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.273869   431 raft_consensus.cc:2243] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.275072   431 raft_consensus.cc:2272] T 925f342702b542288fd34463d0585a5b P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.278290   431 tablet_replica.cc:333] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.279359   431 raft_consensus.cc:2243] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.280006   431 raft_consensus.cc:2272] T 7364742faea3470694e1a23f151f1b57 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:21.281193 10169 consensus_peers.cc:597] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.282613   431 tablet_replica.cc:333] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.283401   431 raft_consensus.cc:2243] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.283921   431 raft_consensus.cc:2272] T d8d642d68d234d668cb67cb81b80556e P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.286221   431 tablet_replica.cc:333] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.287218   431 raft_consensus.cc:2243] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.287935   431 raft_consensus.cc:2272] T dc4e649e35ee49a2a224ecc9f962980a P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.291093   431 tablet_replica.cc:333] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.291805   431 raft_consensus.cc:2243] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.293028   431 raft_consensus.cc:2272] T 39e4808ea4be41f483e08e83162c59c8 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:21.293568 10098 consensus_peers.cc:597] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.296592   431 tablet_replica.cc:333] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.297610   431 raft_consensus.cc:2243] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.298388   431 raft_consensus.cc:2272] T 19d98d0921054d6a932a308e4adb1937 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.301761   431 tablet_replica.cc:333] T 5c136ae5f6aa4055a44f8aec8883b02a P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.302707   431 raft_consensus.cc:2243] T 5c136ae5f6aa4055a44f8aec8883b02a P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.303352   431 raft_consensus.cc:2272] T 5c136ae5f6aa4055a44f8aec8883b02a P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.306595   431 tablet_replica.cc:333] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548: stopping tablet replica
W20260709 13:07:21.307085 10246 consensus_peers.cc:597] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 -> Peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Couldn't send request to peer efa338cd132c457896b11e08d7ea787d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.307344   431 raft_consensus.cc:2243] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.307999   431 raft_consensus.cc:2272] T 50256706bf3a4bf29b64ea4ea6583da3 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.310734   431 tablet_replica.cc:333] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.311543   431 raft_consensus.cc:2243] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.312044   431 raft_consensus.cc:2272] T 90eeb5566a2f4fa4af640a137fe578df P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.314401   431 tablet_replica.cc:333] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.314976   431 raft_consensus.cc:2243] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.315824   431 raft_consensus.cc:2272] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.317893   431 tablet_replica.cc:333] T 7f0446529cd446d0ba97ee860c6ae221 P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.318542   431 raft_consensus.cc:2243] T 7f0446529cd446d0ba97ee860c6ae221 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.319103   431 raft_consensus.cc:2272] T 7f0446529cd446d0ba97ee860c6ae221 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.321996   431 tablet_replica.cc:333] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.322846   431 raft_consensus.cc:2243] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.323388   431 raft_consensus.cc:2272] T d6c60f07656843aeac5d469a6533aaaa P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.326171   431 tablet_replica.cc:333] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.326864   431 raft_consensus.cc:2243] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:07:21.327623 10169 consensus_peers.cc:597] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.327777   431 raft_consensus.cc:2272] T d9cb1393c4f549f0bc9b5e82121b543a P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.330312   431 tablet_replica.cc:333] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.330920   431 raft_consensus.cc:2243] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.331741   431 raft_consensus.cc:2272] T 3666fc7cbb0d4f698fb337ad4c238c38 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.333904   431 tablet_replica.cc:333] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.334609   431 raft_consensus.cc:2243] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.335548   431 raft_consensus.cc:2272] T d139f1fbdd384830bc7ee062ab3d281c P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.338238   431 tablet_replica.cc:333] T 56df6e7bddfd4832b31ccc4bb8b3521b P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.338852   431 raft_consensus.cc:2243] T 56df6e7bddfd4832b31ccc4bb8b3521b P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.339270   431 raft_consensus.cc:2272] T 56df6e7bddfd4832b31ccc4bb8b3521b P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.341463   431 tablet_replica.cc:333] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.342074   431 raft_consensus.cc:2243] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.342748   431 raft_consensus.cc:2272] T 7dd93f389d32493b9dcc10e602f452da P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.345764   431 tablet_replica.cc:333] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.346524   431 raft_consensus.cc:2243] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.347751   431 raft_consensus.cc:2272] T 3fa9341e7fbe49e1a832721bfa4abff3 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.350405   431 tablet_replica.cc:333] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.351207   431 raft_consensus.cc:2243] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.351801   431 raft_consensus.cc:2272] T c21c65d190c541ab8e2d6ae0d4be85e7 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.354910   431 tablet_replica.cc:333] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.355803   431 raft_consensus.cc:2243] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.356662   431 raft_consensus.cc:2272] T 83a64aca789f4ea8a4925aa79dcb0235 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:21.359752 10169 consensus_peers.cc:597] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.360442   431 tablet_replica.cc:333] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.361806   431 raft_consensus.cc:2243] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.362610   431 raft_consensus.cc:2272] T 75aced19d8c94fc78e5510b592bb01be P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.366966   431 tablet_replica.cc:333] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.368005   431 raft_consensus.cc:2243] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.369200   431 raft_consensus.cc:2272] T 52ffbaf3a54145c2a791c4a67dbdf64a P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.372324   431 tablet_replica.cc:333] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.373138   431 raft_consensus.cc:2243] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.374236   431 raft_consensus.cc:2272] T 1f3c90bb2ac14652a45cdcccba25f97c P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.377141   431 tablet_replica.cc:333] T 6d2f27feb83847298256dd63dafcd35c P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.377903   431 raft_consensus.cc:2243] T 6d2f27feb83847298256dd63dafcd35c P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.378484   431 raft_consensus.cc:2272] T 6d2f27feb83847298256dd63dafcd35c P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.381268   431 tablet_replica.cc:333] T 5f64c791e7fb4a79bd498140ef0b0943 P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.381943   431 raft_consensus.cc:2243] T 5f64c791e7fb4a79bd498140ef0b0943 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.382421   431 raft_consensus.cc:2272] T 5f64c791e7fb4a79bd498140ef0b0943 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.384857   431 tablet_replica.cc:333] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.385785   431 raft_consensus.cc:2243] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.387058   431 raft_consensus.cc:2272] T aa03c270e7364d769700b877d171588e P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.390103   431 tablet_replica.cc:333] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.390897   431 raft_consensus.cc:2243] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.391466   431 raft_consensus.cc:2272] T c7dfde2e205846058c75fc205443d330 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.394603   431 tablet_replica.cc:333] T 75ed1b358949418a9d300990f8f81652 P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.395435   431 raft_consensus.cc:2243] T 75ed1b358949418a9d300990f8f81652 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.396070   431 raft_consensus.cc:2272] T 75ed1b358949418a9d300990f8f81652 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.399050   431 tablet_replica.cc:333] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.399634   431 raft_consensus.cc:2243] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.400107   431 raft_consensus.cc:2272] T abca9c95a1334ebc97985eedf46fc01f P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.402174   431 tablet_replica.cc:333] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.402730   431 raft_consensus.cc:2243] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.403540   431 raft_consensus.cc:2272] T 0e003c3e14c2444b937f547eef84b672 P 575e44d297f045a389903d65a471d548 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.405845   431 tablet_replica.cc:333] T 375f151edb2245daa6d1b06187f21b79 P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.406380   431 raft_consensus.cc:2243] T 375f151edb2245daa6d1b06187f21b79 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.406916   431 raft_consensus.cc:2272] T 375f151edb2245daa6d1b06187f21b79 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.409032   431 tablet_replica.cc:333] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.409579   431 raft_consensus.cc:2243] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.410120   431 raft_consensus.cc:2272] T 64ea79b0bb13439c9384cba78627c40f P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.413251   431 tablet_replica.cc:333] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.414199   431 raft_consensus.cc:2243] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:07:21.414690 10243 consensus_peers.cc:597] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.415392   431 raft_consensus.cc:2272] T a0265654cefc40a5937d7e194f03dd5c P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.418190   431 tablet_replica.cc:333] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548: stopping tablet replica
W20260709 13:07:21.418516 10243 consensus_peers.cc:597] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.418866   431 raft_consensus.cc:2243] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.419420   431 raft_consensus.cc:2272] T 80b0c1b6201b46d5b1f46ccad59503b7 P 575e44d297f045a389903d65a471d548 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.421875   431 tablet_replica.cc:333] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548: stopping tablet replica
I20260709 13:07:21.422397   431 raft_consensus.cc:2243] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.422974   431 raft_consensus.cc:2272] T e959733d8ef94ea19aad472fcf384371 P 575e44d297f045a389903d65a471d548 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:21.429437 10169 consensus_peers.cc:597] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.447688 10169 consensus_peers.cc:597] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.453274 10243 consensus_peers.cc:597] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.475082 10169 consensus_peers.cc:597] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.479746 10243 consensus_peers.cc:597] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.485802 10243 consensus_peers.cc:597] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.513449   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:07:21.551892 10243 consensus_peers.cc:597] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.572098 10243 consensus_peers.cc:597] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.583276 10243 consensus_peers.cc:597] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.585798 10243 consensus_peers.cc:597] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.597405   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:07:21.615722 10169 consensus_peers.cc:597] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.620431 10243 consensus_peers.cc:597] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.644208 10169 consensus_peers.cc:597] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.669629 10169 consensus_peers.cc:597] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.670888   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:21.671865   431 tablet_replica.cc:333] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.672559   431 raft_consensus.cc:2243] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.673049   431 raft_consensus.cc:2272] T a0ea0c11178d43888a5596f49ffc7a7e P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.675450   431 tablet_replica.cc:333] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.676026   431 raft_consensus.cc:2243] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.676508   431 raft_consensus.cc:2272] T 925f342702b542288fd34463d0585a5b P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.679193   431 tablet_replica.cc:333] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.680123   431 raft_consensus.cc:2243] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.680758   431 raft_consensus.cc:2272] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.684317   431 tablet_replica.cc:333] T 4457dec3dec94c1dbac542a298184d0f P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.686147   431 raft_consensus.cc:2243] T 4457dec3dec94c1dbac542a298184d0f P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:21.686647 10243 consensus_peers.cc:597] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 -> Peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Couldn't send request to peer 8fe33410d51a469e81be6edc602d7d78. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.686817   431 raft_consensus.cc:2272] T 4457dec3dec94c1dbac542a298184d0f P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.690132   431 tablet_replica.cc:333] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.690752   431 raft_consensus.cc:2243] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.691572   431 raft_consensus.cc:2272] T f660e208f31341f0bc602253c996258c P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.694206   431 tablet_replica.cc:333] T a0265654cefc40a5937d7e194f03dd5c P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.694955   431 raft_consensus.cc:2243] T a0265654cefc40a5937d7e194f03dd5c P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.695441   431 raft_consensus.cc:2272] T a0265654cefc40a5937d7e194f03dd5c P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.697531   431 tablet_replica.cc:333] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.698149   431 raft_consensus.cc:2243] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.698647   431 raft_consensus.cc:2272] T c78325f9138849749ada9825bfe45cd8 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.700852   431 tablet_replica.cc:333] T 1f060317dcbd4f0db70f83fd18320215 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.701421   431 raft_consensus.cc:2243] T 1f060317dcbd4f0db70f83fd18320215 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.701845   431 raft_consensus.cc:2272] T 1f060317dcbd4f0db70f83fd18320215 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.703639   431 tablet_replica.cc:333] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.704124   431 raft_consensus.cc:2243] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.704841   431 raft_consensus.cc:2272] T 9f7da1cc13f64f56b02defc2e119c507 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.707183   431 tablet_replica.cc:333] T cded7cb005f64f9c894f568140883a77 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.707890   431 raft_consensus.cc:2243] T cded7cb005f64f9c894f568140883a77 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.708305   431 raft_consensus.cc:2272] T cded7cb005f64f9c894f568140883a77 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.710860   431 tablet_replica.cc:333] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.711534   431 raft_consensus.cc:2243] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.712201   431 raft_consensus.cc:2272] T d242deba350e42cb8e2b623027a26516 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.714146   431 tablet_replica.cc:333] T dc4e649e35ee49a2a224ecc9f962980a P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.714634   431 raft_consensus.cc:2243] T dc4e649e35ee49a2a224ecc9f962980a P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.714982   431 raft_consensus.cc:2272] T dc4e649e35ee49a2a224ecc9f962980a P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.716718   431 tablet_replica.cc:333] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.717476   431 raft_consensus.cc:2243] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.718114   431 raft_consensus.cc:2272] T 0981226de91f4f9e96cf2e8b926aaf19 P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.720080   431 tablet_replica.cc:333] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.720618   431 raft_consensus.cc:2243] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.721426   431 raft_consensus.cc:2272] T 19d98d0921054d6a932a308e4adb1937 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:21.722158 10243 consensus_peers.cc:597] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 -> Peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Couldn't send request to peer 575e44d297f045a389903d65a471d548. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.723559   431 tablet_replica.cc:333] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.724172   431 raft_consensus.cc:2243] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.724602   431 raft_consensus.cc:2272] T bc1d8020dcd34e63b72a7de2db565ff6 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.726531   431 tablet_replica.cc:333] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.727049   431 raft_consensus.cc:2243] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.727455   431 raft_consensus.cc:2272] T 1ec007b8bb1e4c69a49a9ba8aeb56eee P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.730244   431 tablet_replica.cc:333] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.731087   431 raft_consensus.cc:2243] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.732116   431 raft_consensus.cc:2272] T 90eeb5566a2f4fa4af640a137fe578df P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.734620   431 tablet_replica.cc:333] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.735435   431 raft_consensus.cc:2243] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.735963   431 raft_consensus.cc:2272] T 0529bf107d07493eb0984c4cbb9ea01e P 8fe33410d51a469e81be6edc602d7d78 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.738971   431 tablet_replica.cc:333] T d6c60f07656843aeac5d469a6533aaaa P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.739729   431 raft_consensus.cc:2243] T d6c60f07656843aeac5d469a6533aaaa P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.740381   431 raft_consensus.cc:2272] T d6c60f07656843aeac5d469a6533aaaa P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.743562   431 tablet_replica.cc:333] T d9cb1393c4f549f0bc9b5e82121b543a P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.744585   431 raft_consensus.cc:2243] T d9cb1393c4f549f0bc9b5e82121b543a P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.745237   431 raft_consensus.cc:2272] T d9cb1393c4f549f0bc9b5e82121b543a P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.748163   431 tablet_replica.cc:333] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.748853   431 raft_consensus.cc:2243] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.749331   431 raft_consensus.cc:2272] T c7d966d297f346da9e984aa2e181590a P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.751729   431 tablet_replica.cc:333] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.752362   431 raft_consensus.cc:2243] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.753199   431 raft_consensus.cc:2272] T 7f0446529cd446d0ba97ee860c6ae221 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.755704   431 tablet_replica.cc:333] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.756367   431 raft_consensus.cc:2243] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.757249   431 raft_consensus.cc:2272] T abca9c95a1334ebc97985eedf46fc01f P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.759666   431 tablet_replica.cc:333] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.760288   431 raft_consensus.cc:2243] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.760803   431 raft_consensus.cc:2272] T 0e003c3e14c2444b937f547eef84b672 P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.763077   431 tablet_replica.cc:333] T 375f151edb2245daa6d1b06187f21b79 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.763693   431 raft_consensus.cc:2243] T 375f151edb2245daa6d1b06187f21b79 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.764200   431 raft_consensus.cc:2272] T 375f151edb2245daa6d1b06187f21b79 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.766789   431 tablet_replica.cc:333] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.767403   431 raft_consensus.cc:2243] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.767901   431 raft_consensus.cc:2272] T b626433d4aec4288976ffc448a79bb68 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.770291   431 tablet_replica.cc:333] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.770910   431 raft_consensus.cc:2243] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.771770   431 raft_consensus.cc:2272] T c7dfde2e205846058c75fc205443d330 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.773917   431 tablet_replica.cc:333] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.774487   431 raft_consensus.cc:2243] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.775309   431 raft_consensus.cc:2272] T c21c65d190c541ab8e2d6ae0d4be85e7 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.777318   431 tablet_replica.cc:333] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.777994   431 raft_consensus.cc:2243] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.778831   431 raft_consensus.cc:2272] T 04676809490049a5923174641a3c1137 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.781301   431 tablet_replica.cc:333] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.781920   431 raft_consensus.cc:2243] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.782722   431 raft_consensus.cc:2272] T 83a64aca789f4ea8a4925aa79dcb0235 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.784616   431 tablet_replica.cc:333] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.785190   431 raft_consensus.cc:2243] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.786064   431 raft_consensus.cc:2272] T 6d2f27feb83847298256dd63dafcd35c P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.787958   431 tablet_replica.cc:333] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.788542   431 raft_consensus.cc:2243] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.789106   431 raft_consensus.cc:2272] T ccaebf4041f14bada42fd7122f34419c P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.791508   431 tablet_replica.cc:333] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.792193   431 raft_consensus.cc:2243] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.792991   431 raft_consensus.cc:2272] T 5f64c791e7fb4a79bd498140ef0b0943 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.795387   431 tablet_replica.cc:333] T aa03c270e7364d769700b877d171588e P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.795928   431 raft_consensus.cc:2243] T aa03c270e7364d769700b877d171588e P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.796351   431 raft_consensus.cc:2272] T aa03c270e7364d769700b877d171588e P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.798897   431 tablet_replica.cc:333] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.799432   431 raft_consensus.cc:2243] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.800204   431 raft_consensus.cc:2272] T e959733d8ef94ea19aad472fcf384371 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.802220   431 tablet_replica.cc:333] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.803169   431 raft_consensus.cc:2243] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.803817   431 raft_consensus.cc:2272] T 48dcd8683f004a61808243f33d859911 P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.806639   431 tablet_replica.cc:333] T d139f1fbdd384830bc7ee062ab3d281c P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.807464   431 raft_consensus.cc:2243] T d139f1fbdd384830bc7ee062ab3d281c P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.808120   431 raft_consensus.cc:2272] T d139f1fbdd384830bc7ee062ab3d281c P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:21.809180 10243 consensus_peers.cc:597] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 -> Peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Couldn't send request to peer 8fe33410d51a469e81be6edc602d7d78. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.811122   431 tablet_replica.cc:333] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.812090   431 raft_consensus.cc:2243] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.812763   431 raft_consensus.cc:2272] T 56df6e7bddfd4832b31ccc4bb8b3521b P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.816390   431 tablet_replica.cc:333] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.817152   431 raft_consensus.cc:2243] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.817602   431 raft_consensus.cc:2272] T 5461abd09b074193b3d03df659fa2ced P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.819796   431 tablet_replica.cc:333] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.820348   431 raft_consensus.cc:2243] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.820823   431 raft_consensus.cc:2272] T ce414a4edf504d78b1b1e83498a5cfeb P 8fe33410d51a469e81be6edc602d7d78 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.822606   431 tablet_replica.cc:333] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.823063   431 raft_consensus.cc:2243] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.823470   431 raft_consensus.cc:2272] T e66a35e846da4843bda7db1f5fca14b6 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.825152   431 tablet_replica.cc:333] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.825613   431 raft_consensus.cc:2243] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.826093   431 raft_consensus.cc:2272] T 52ffbaf3a54145c2a791c4a67dbdf64a P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.827880   431 tablet_replica.cc:333] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.828337   431 raft_consensus.cc:2243] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.828758   431 raft_consensus.cc:2272] T 3666fc7cbb0d4f698fb337ad4c238c38 P 8fe33410d51a469e81be6edc602d7d78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.830660   431 tablet_replica.cc:333] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78: stopping tablet replica
I20260709 13:07:21.831096   431 raft_consensus.cc:2243] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.831838   431 raft_consensus.cc:2272] T b41d6a8939d7411e901d88f31d2bdf38 P 8fe33410d51a469e81be6edc602d7d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.899289   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:07:21.906857 11271 raft_consensus.cc:493] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:21.907466 11271 raft_consensus.cc:515] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:21.909439 11271 leader_election.cc:290] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
W20260709 13:07:21.914147 10243 leader_election.cc:336] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Network error: Client connection negotiation failed: client connection to 127.0.107.195:40419: connect: Connection refused (error 111)
W20260709 13:07:21.915153 10246 leader_election.cc:336] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:21.915534 10246 leader_election.cc:304] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:07:21.916201 11271 raft_consensus.cc:2749] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:07:21.926265 10243 consensus_peers.cc:597] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 -> Peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Couldn't send request to peer 8fe33410d51a469e81be6edc602d7d78. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.974234 10243 consensus_peers.cc:597] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 -> Peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Couldn't send request to peer 8fe33410d51a469e81be6edc602d7d78. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.978658   431 tablet_server.cc:179] TabletServer@127.0.107.196:0 shutting down...
I20260709 13:07:21.993624 11271 raft_consensus.cc:493] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:21.994489 11271 raft_consensus.cc:515] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:21.997404 11271 leader_election.cc:290] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
W20260709 13:07:22.005394 10243 leader_election.cc:336] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Network error: Client connection negotiation failed: client connection to 127.0.107.195:40419: connect: Connection refused (error 111)
W20260709 13:07:22.006376 10246 leader_election.cc:336] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:22.006740 10246 leader_election.cc:304] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:07:22.007357 11271 raft_consensus.cc:2749] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:22.022471   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:22.023522   431 tablet_replica.cc:333] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.024519   431 raft_consensus.cc:2243] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:22.025370   431 raft_consensus.cc:2272] T a0ea0c11178d43888a5596f49ffc7a7e P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.028213   431 tablet_replica.cc:333] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.029023   431 raft_consensus.cc:2243] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:22.030010   431 raft_consensus.cc:2272] T 1249fe82a5ce4bd7b416e6bb46be8f79 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.030857 11271 raft_consensus.cc:493] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:22.031409 11271 raft_consensus.cc:515] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575e44d297f045a389903d65a471d548" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33415 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
I20260709 13:07:22.032883   431 tablet_replica.cc:333] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.033572 11271 leader_election.cc:290] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 575e44d297f045a389903d65a471d548 (127.0.107.194:33415), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:22.033883   431 raft_consensus.cc:2243] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.034520   431 raft_consensus.cc:2272] T 4457dec3dec94c1dbac542a298184d0f P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.037266   431 tablet_replica.cc:333] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.038125   431 raft_consensus.cc:2243] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:22.038437 10243 leader_election.cc:336] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 575e44d297f045a389903d65a471d548 (127.0.107.194:33415): Network error: Client connection negotiation failed: client connection to 127.0.107.194:33415: connect: Connection refused (error 111)
I20260709 13:07:22.038654   431 raft_consensus.cc:2272] T f660e208f31341f0bc602253c996258c P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:22.039281 10246 leader_election.cc:336] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:22.039697 10246 leader_election.cc:304] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 575e44d297f045a389903d65a471d548, efa338cd132c457896b11e08d7ea787d
I20260709 13:07:22.040352 11271 raft_consensus.cc:2749] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:22.041392   431 tablet_replica.cc:333] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.042052   431 raft_consensus.cc:2243] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:22.042784   431 raft_consensus.cc:2272] T 2258980c3a7540dea8ae11850194e183 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.044724   431 tablet_replica.cc:333] T 9f7da1cc13f64f56b02defc2e119c507 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.045241   431 raft_consensus.cc:2243] T 9f7da1cc13f64f56b02defc2e119c507 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.045586   431 raft_consensus.cc:2272] T 9f7da1cc13f64f56b02defc2e119c507 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.047529   431 tablet_replica.cc:333] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.048035   431 raft_consensus.cc:2243] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.048409   431 raft_consensus.cc:2272] T 4af4e546f1084d8cb71469fbf3886792 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.050298   431 tablet_replica.cc:333] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.050930   431 raft_consensus.cc:2243] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:22.051843   431 raft_consensus.cc:2272] T cded7cb005f64f9c894f568140883a77 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.054601   431 tablet_replica.cc:333] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.055334   431 raft_consensus.cc:2243] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.055886   431 raft_consensus.cc:2272] T d242deba350e42cb8e2b623027a26516 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:22.055969 10243 consensus_peers.cc:597] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 -> Peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Couldn't send request to peer 8fe33410d51a469e81be6edc602d7d78. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:22.058509   431 tablet_replica.cc:333] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.059224   431 raft_consensus.cc:2243] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:22.060096   431 raft_consensus.cc:2272] T 75aced19d8c94fc78e5510b592bb01be P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.062750   431 tablet_replica.cc:333] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.063309   431 raft_consensus.cc:2243] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.063804   431 raft_consensus.cc:2272] T 925f342702b542288fd34463d0585a5b P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.064949 11271 raft_consensus.cc:493] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader efa338cd132c457896b11e08d7ea787d)
I20260709 13:07:22.065490 11271 raft_consensus.cc:515] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe33410d51a469e81be6edc602d7d78" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40419 } } peers { permanent_uuid: "50999113203c482b9b9300caa485f154" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 40841 } } peers { permanent_uuid: "efa338cd132c457896b11e08d7ea787d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42425 } }
W20260709 13:07:22.066386 10243 consensus_peers.cc:597] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 -> Peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Couldn't send request to peer 8fe33410d51a469e81be6edc602d7d78. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:22.066684   431 tablet_replica.cc:333] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.067397   431 raft_consensus.cc:2243] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:22.067642 11271 leader_election.cc:290] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419), efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425)
I20260709 13:07:22.068239   431 raft_consensus.cc:2272] T 7364742faea3470694e1a23f151f1b57 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.070924   431 tablet_replica.cc:333] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.071812   431 raft_consensus.cc:2243] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:22.072022 10246 leader_election.cc:336] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer efa338cd132c457896b11e08d7ea787d (127.0.107.193:42425): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42425: connect: Connection refused (error 111)
I20260709 13:07:22.072365   431 raft_consensus.cc:2272] T 0981226de91f4f9e96cf2e8b926aaf19 P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:22.073535 10243 leader_election.cc:336] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8fe33410d51a469e81be6edc602d7d78 (127.0.107.195:40419): Network error: Client connection negotiation failed: client connection to 127.0.107.195:40419: connect: Connection refused (error 111)
I20260709 13:07:22.074097 10243 leader_election.cc:304] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50999113203c482b9b9300caa485f154; no voters: 8fe33410d51a469e81be6edc602d7d78, efa338cd132c457896b11e08d7ea787d
I20260709 13:07:22.074709 11271 raft_consensus.cc:2749] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:22.074796   431 tablet_replica.cc:333] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.075620   431 raft_consensus.cc:2243] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:22.076308   431 raft_consensus.cc:2272] T 7dd93f389d32493b9dcc10e602f452da P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.078327   431 tablet_replica.cc:333] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.078876   431 raft_consensus.cc:2243] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:22.079562   431 raft_consensus.cc:2272] T d8d642d68d234d668cb67cb81b80556e P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.081388   431 tablet_replica.cc:333] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.081983   431 raft_consensus.cc:2243] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:22.082762   431 raft_consensus.cc:2272] T dc4e649e35ee49a2a224ecc9f962980a P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.084623   431 tablet_replica.cc:333] T 39e4808ea4be41f483e08e83162c59c8 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.085224   431 raft_consensus.cc:2243] T 39e4808ea4be41f483e08e83162c59c8 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.085691   431 raft_consensus.cc:2272] T 39e4808ea4be41f483e08e83162c59c8 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.087493   431 tablet_replica.cc:333] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.088090   431 raft_consensus.cc:2243] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.088562   431 raft_consensus.cc:2272] T bc1d8020dcd34e63b72a7de2db565ff6 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.090468   431 tablet_replica.cc:333] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.091008   431 raft_consensus.cc:2243] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.091423   431 raft_consensus.cc:2272] T ccaebf4041f14bada42fd7122f34419c P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.093288   431 tablet_replica.cc:333] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.093837   431 raft_consensus.cc:2243] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.094255   431 raft_consensus.cc:2272] T 90eeb5566a2f4fa4af640a137fe578df P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.095966   431 tablet_replica.cc:333] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.096472   431 raft_consensus.cc:2243] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:22.097193   431 raft_consensus.cc:2272] T 5c136ae5f6aa4055a44f8aec8883b02a P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.099195   431 tablet_replica.cc:333] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.099740   431 raft_consensus.cc:2243] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.100149   431 raft_consensus.cc:2272] T 1f4dd5b2c17747058bab6588c63b9d72 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.101966   431 tablet_replica.cc:333] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.102468   431 raft_consensus.cc:2243] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.102901   431 raft_consensus.cc:2272] T 0529bf107d07493eb0984c4cbb9ea01e P 50999113203c482b9b9300caa485f154 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.104657   431 tablet_replica.cc:333] T d9cb1393c4f549f0bc9b5e82121b543a P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.105194   431 raft_consensus.cc:2243] T d9cb1393c4f549f0bc9b5e82121b543a P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.105656   431 raft_consensus.cc:2272] T d9cb1393c4f549f0bc9b5e82121b543a P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.107435   431 tablet_replica.cc:333] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.107970   431 raft_consensus.cc:2243] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:22.108726   431 raft_consensus.cc:2272] T 50256706bf3a4bf29b64ea4ea6583da3 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.110558   431 tablet_replica.cc:333] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.111081   431 raft_consensus.cc:2243] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:22.111833   431 raft_consensus.cc:2272] T 64ea79b0bb13439c9384cba78627c40f P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.113694   431 tablet_replica.cc:333] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.114310   431 raft_consensus.cc:2243] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.114743   431 raft_consensus.cc:2272] T a0265654cefc40a5937d7e194f03dd5c P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.116532   431 tablet_replica.cc:333] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.117067   431 raft_consensus.cc:2243] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.117509   431 raft_consensus.cc:2272] T 80b0c1b6201b46d5b1f46ccad59503b7 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.119340   431 tablet_replica.cc:333] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.119855   431 raft_consensus.cc:2243] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:22.120605   431 raft_consensus.cc:2272] T b626433d4aec4288976ffc448a79bb68 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.122606   431 tablet_replica.cc:333] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.123127   431 raft_consensus.cc:2243] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:22.123864   431 raft_consensus.cc:2272] T 375f151edb2245daa6d1b06187f21b79 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.125789   431 tablet_replica.cc:333] T e959733d8ef94ea19aad472fcf384371 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.126325   431 raft_consensus.cc:2243] T e959733d8ef94ea19aad472fcf384371 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.126745   431 raft_consensus.cc:2272] T e959733d8ef94ea19aad472fcf384371 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.128703   431 tablet_replica.cc:333] T c7dfde2e205846058c75fc205443d330 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.129252   431 raft_consensus.cc:2243] T c7dfde2e205846058c75fc205443d330 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.129680   431 raft_consensus.cc:2272] T c7dfde2e205846058c75fc205443d330 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.131775   431 tablet_replica.cc:333] T 3666fc7cbb0d4f698fb337ad4c238c38 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.132326   431 raft_consensus.cc:2243] T 3666fc7cbb0d4f698fb337ad4c238c38 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.132782   431 raft_consensus.cc:2272] T 3666fc7cbb0d4f698fb337ad4c238c38 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.134641   431 tablet_replica.cc:333] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.135180   431 raft_consensus.cc:2243] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.135593   431 raft_consensus.cc:2272] T 75ed1b358949418a9d300990f8f81652 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.137363   431 tablet_replica.cc:333] T b41d6a8939d7411e901d88f31d2bdf38 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.137889   431 raft_consensus.cc:2243] T b41d6a8939d7411e901d88f31d2bdf38 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.138324   431 raft_consensus.cc:2272] T b41d6a8939d7411e901d88f31d2bdf38 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.139991   431 tablet_replica.cc:333] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.140484   431 raft_consensus.cc:2243] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.140887   431 raft_consensus.cc:2272] T 48dcd8683f004a61808243f33d859911 P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.142709   431 tablet_replica.cc:333] T 5461abd09b074193b3d03df659fa2ced P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.143182   431 raft_consensus.cc:2243] T 5461abd09b074193b3d03df659fa2ced P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.143606   431 raft_consensus.cc:2272] T 5461abd09b074193b3d03df659fa2ced P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.145364   431 tablet_replica.cc:333] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.145867   431 raft_consensus.cc:2243] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:22.146663   431 raft_consensus.cc:2272] T 56df6e7bddfd4832b31ccc4bb8b3521b P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.148394   431 tablet_replica.cc:333] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.148896   431 raft_consensus.cc:2243] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.149289   431 raft_consensus.cc:2272] T c7d966d297f346da9e984aa2e181590a P 50999113203c482b9b9300caa485f154 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.151000   431 tablet_replica.cc:333] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.151472   431 raft_consensus.cc:2243] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.151885   431 raft_consensus.cc:2272] T 3fa9341e7fbe49e1a832721bfa4abff3 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.153669   431 tablet_replica.cc:333] T 04676809490049a5923174641a3c1137 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.154129   431 raft_consensus.cc:2243] T 04676809490049a5923174641a3c1137 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.154524   431 raft_consensus.cc:2272] T 04676809490049a5923174641a3c1137 P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.156302   431 tablet_replica.cc:333] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.156755   431 raft_consensus.cc:2243] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.157174   431 raft_consensus.cc:2272] T e66a35e846da4843bda7db1f5fca14b6 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.158975   431 tablet_replica.cc:333] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.159433   431 raft_consensus.cc:2243] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.159854   431 raft_consensus.cc:2272] T c78325f9138849749ada9825bfe45cd8 P 50999113203c482b9b9300caa485f154 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.161768   431 tablet_replica.cc:333] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154: stopping tablet replica
I20260709 13:07:22.162212   431 raft_consensus.cc:2243] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:22.162600   431 raft_consensus.cc:2272] T 1f3c90bb2ac14652a45cdcccba25f97c P 50999113203c482b9b9300caa485f154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.229295   431 tablet_server.cc:196] TabletServer@127.0.107.196:0 shutdown complete.
I20260709 13:07:22.302292   431 master.cc:562] Master@127.0.107.254:43521 shutting down...
I20260709 13:07:22.333072   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:22.334007   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.334486   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0b93df6119c0475a83c010dafc5ff2bd: stopping tablet replica
I20260709 13:07:22.388785   431 master.cc:584] Master@127.0.107.254:43521 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (35151 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:07:22.477496   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:37289
I20260709 13:07:22.479053   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:22.484706 11303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:22.484990 11304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:22.486040 11306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:22.486516   431 server_base.cc:1061] running on GCE node
I20260709 13:07:22.487612   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:22.487794   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:22.487906   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602442487896 us; error 0 us; skew 500 ppm
I20260709 13:07:22.488370   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:22.491089   431 webserver.cc:533] Webserver started at http://127.0.107.254:46687/ using document root <none> and password file <none>
I20260709 13:07:22.491533   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:22.491703   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:22.491919   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:22.493007   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/master-0-root/instance:
uuid: "f1fd10dba22f4381b09f218ff21d8ea8"
format_stamp: "Formatted at 2026-07-09 13:07:22 on dist-test-slave-mxd3"
I20260709 13:07:22.497383   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:07:22.501354 11311 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:22.502333   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:07:22.502696   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/master-0-root
uuid: "f1fd10dba22f4381b09f218ff21d8ea8"
format_stamp: "Formatted at 2026-07-09 13:07:22 on dist-test-slave-mxd3"
I20260709 13:07:22.502962   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:22.514834   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:22.516062   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:22.556190   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:37289
I20260709 13:07:22.556310 11362 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:37289 every 8 connection(s)
I20260709 13:07:22.560410 11363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.571982 11363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8: Bootstrap starting.
I20260709 13:07:22.577061 11363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:22.581378 11363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8: No bootstrap required, opened a new log
I20260709 13:07:22.583730 11363 raft_consensus.cc:359] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1fd10dba22f4381b09f218ff21d8ea8" member_type: VOTER }
I20260709 13:07:22.584112 11363 raft_consensus.cc:385] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:22.584407 11363 raft_consensus.cc:740] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1fd10dba22f4381b09f218ff21d8ea8, State: Initialized, Role: FOLLOWER
I20260709 13:07:22.585006 11363 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1fd10dba22f4381b09f218ff21d8ea8" member_type: VOTER }
I20260709 13:07:22.585455 11363 raft_consensus.cc:399] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:07:22.585695 11363 raft_consensus.cc:493] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:07:22.585964 11363 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:22.590576 11363 raft_consensus.cc:515] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1fd10dba22f4381b09f218ff21d8ea8" member_type: VOTER }
I20260709 13:07:22.591127 11363 leader_election.cc:304] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1fd10dba22f4381b09f218ff21d8ea8; no voters: 
I20260709 13:07:22.592267 11363 leader_election.cc:290] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:07:22.592661 11366 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:22.594251 11366 raft_consensus.cc:697] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8 [term 1 LEADER]: Becoming Leader. State: Replica: f1fd10dba22f4381b09f218ff21d8ea8, State: Running, Role: LEADER
I20260709 13:07:22.594921 11366 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1fd10dba22f4381b09f218ff21d8ea8" member_type: VOTER }
I20260709 13:07:22.595497 11363 sys_catalog.cc:565] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:07:22.599587 11367 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1fd10dba22f4381b09f218ff21d8ea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1fd10dba22f4381b09f218ff21d8ea8" member_type: VOTER } }
I20260709 13:07:22.600320 11367 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:22.600092 11368 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1fd10dba22f4381b09f218ff21d8ea8. Latest consensus state: current_term: 1 leader_uuid: "f1fd10dba22f4381b09f218ff21d8ea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1fd10dba22f4381b09f218ff21d8ea8" member_type: VOTER } }
I20260709 13:07:22.601738 11368 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:22.603070 11372 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:07:22.611439 11372 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:07:22.615478   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:07:22.622233 11372 catalog_manager.cc:1383] Generated new cluster ID: bef07046318d4faaacd670ab68d7bcfa
I20260709 13:07:22.622535 11372 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:07:22.651168 11372 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:07:22.652493 11372 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:07:22.675865 11372 catalog_manager.cc:6070] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8: Generated new TSK 0
I20260709 13:07:22.676522 11372 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:07:22.681910   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:22.687777 11384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:22.689096 11385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:22.690573   431 server_base.cc:1061] running on GCE node
W20260709 13:07:22.690598 11387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:22.691892   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:22.692075   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:22.692193   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602442692181 us; error 0 us; skew 500 ppm
I20260709 13:07:22.692649   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:22.694981   431 webserver.cc:533] Webserver started at http://127.0.107.193:43437/ using document root <none> and password file <none>
I20260709 13:07:22.695418   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:22.695575   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:22.695784   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:22.696743   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/ts-0-root/instance:
uuid: "7a94a4038cc649babfe75a46223a8bf9"
format_stamp: "Formatted at 2026-07-09 13:07:22 on dist-test-slave-mxd3"
I20260709 13:07:22.701195   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.006s
I20260709 13:07:22.704377 11392 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:22.705191   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:07:22.705540   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/ts-0-root
uuid: "7a94a4038cc649babfe75a46223a8bf9"
format_stamp: "Formatted at 2026-07-09 13:07:22 on dist-test-slave-mxd3"
I20260709 13:07:22.705847   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:22.716048   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:22.717267   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:22.718932   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:22.721499   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:22.721791   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.001s
I20260709 13:07:22.722108   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:22.722296   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:22.763111   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:40225
I20260709 13:07:22.763244 11454 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:40225 every 8 connection(s)
I20260709 13:07:22.768443   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:22.775875 11459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:22.780825 11460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:22.784584   431 server_base.cc:1061] running on GCE node
I20260709 13:07:22.785166 11455 heartbeater.cc:344] Connected to a master server at 127.0.107.254:37289
W20260709 13:07:22.785399 11462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:22.785673 11455 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:22.786500 11455 heartbeater.cc:507] Master 127.0.107.254:37289 requested a full tablet report, sending...
I20260709 13:07:22.786520   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:22.786926   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:22.787120   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602442787103 us; error 0 us; skew 500 ppm
I20260709 13:07:22.787813   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:22.788968 11328 ts_manager.cc:194] Registered new tserver with Master: 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225)
I20260709 13:07:22.790648   431 webserver.cc:533] Webserver started at http://127.0.107.194:38245/ using document root <none> and password file <none>
I20260709 13:07:22.791167   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:22.791141 11328 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60550
I20260709 13:07:22.791545   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:22.791803   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:22.793079   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/ts-1-root/instance:
uuid: "41c59a1982d3480d87bc6a0baa625970"
format_stamp: "Formatted at 2026-07-09 13:07:22 on dist-test-slave-mxd3"
I20260709 13:07:22.797915   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:07:22.801307 11467 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:22.802279   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:07:22.802613   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/ts-1-root
uuid: "41c59a1982d3480d87bc6a0baa625970"
format_stamp: "Formatted at 2026-07-09 13:07:22 on dist-test-slave-mxd3"
I20260709 13:07:22.802946   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:22.816604   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:22.817914   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:22.819412   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:22.821653   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:22.821884   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:22.822125   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:22.822304   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:22.864439   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:43293
I20260709 13:07:22.864593 11529 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:43293 every 8 connection(s)
I20260709 13:07:22.869485   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:22.876287 11533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:22.878166 11534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:22.880425 11536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:22.881629   431 server_base.cc:1061] running on GCE node
I20260709 13:07:22.882831   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:07:22.882902 11530 heartbeater.cc:344] Connected to a master server at 127.0.107.254:37289
W20260709 13:07:22.883111   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:22.883419   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602442883398 us; error 0 us; skew 500 ppm
I20260709 13:07:22.883443 11530 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:22.884132   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:22.884454 11530 heartbeater.cc:507] Master 127.0.107.254:37289 requested a full tablet report, sending...
I20260709 13:07:22.887032 11328 ts_manager.cc:194] Registered new tserver with Master: 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:22.887040   431 webserver.cc:533] Webserver started at http://127.0.107.195:40551/ using document root <none> and password file <none>
I20260709 13:07:22.887980   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:22.888245   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:22.888599   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:22.889088 11328 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60556
I20260709 13:07:22.890470   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/ts-2-root/instance:
uuid: "1115d86484ef4339933c2ba1911812f8"
format_stamp: "Formatted at 2026-07-09 13:07:22 on dist-test-slave-mxd3"
I20260709 13:07:22.895112   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:07:22.898594 11541 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:22.899397   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:07:22.899824   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/ts-2-root
uuid: "1115d86484ef4339933c2ba1911812f8"
format_stamp: "Formatted at 2026-07-09 13:07:22 on dist-test-slave-mxd3"
I20260709 13:07:22.900101   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:22.922159   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:22.923388   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:22.924954   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:22.927351   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:22.927587   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:22.927850   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:22.928041   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:22.969549   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:42537
I20260709 13:07:22.969710 11603 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:42537 every 8 connection(s)
I20260709 13:07:22.984025 11604 heartbeater.cc:344] Connected to a master server at 127.0.107.254:37289
I20260709 13:07:22.984475 11604 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:22.985248 11604 heartbeater.cc:507] Master 127.0.107.254:37289 requested a full tablet report, sending...
I20260709 13:07:22.987507 11328 ts_manager.cc:194] Registered new tserver with Master: 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:22.987608   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015141321s
I20260709 13:07:22.989187 11328 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60570
I20260709 13:07:23.016414 11328 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60580:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:23.019359 11328 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:23.100993 11569 tablet_service.cc:1511] Processing CreateTablet for tablet 3c510e78737545ebb0aa0366f008c5bd (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:07:23.103209 11569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c510e78737545ebb0aa0366f008c5bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.107360 11420 tablet_service.cc:1511] Processing CreateTablet for tablet 3c510e78737545ebb0aa0366f008c5bd (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:07:23.107493 11568 tablet_service.cc:1511] Processing CreateTablet for tablet f668526011204a8e8370e268ae661324 (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:07:23.109009 11420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c510e78737545ebb0aa0366f008c5bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.109292 11568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f668526011204a8e8370e268ae661324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.110786 11419 tablet_service.cc:1511] Processing CreateTablet for tablet f668526011204a8e8370e268ae661324 (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:07:23.112337 11419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f668526011204a8e8370e268ae661324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.118269 11566 tablet_service.cc:1511] Processing CreateTablet for tablet c8dad3e10536436f9dd13720971d29b7 (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:07:23.119974 11566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8dad3e10536436f9dd13720971d29b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.119855 11418 tablet_service.cc:1511] Processing CreateTablet for tablet 34fd2daee75140e8ad3c8617bc3b5592 (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:07:23.123238 11417 tablet_service.cc:1511] Processing CreateTablet for tablet c8dad3e10536436f9dd13720971d29b7 (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:07:23.123257 11567 tablet_service.cc:1511] Processing CreateTablet for tablet 34fd2daee75140e8ad3c8617bc3b5592 (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:07:23.125375 11416 tablet_service.cc:1511] Processing CreateTablet for tablet 4ddc5f5fecb0403386dd61f18d051b97 (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:07:23.127157 11417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8dad3e10536436f9dd13720971d29b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.128337 11565 tablet_service.cc:1511] Processing CreateTablet for tablet 4ddc5f5fecb0403386dd61f18d051b97 (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:07:23.129983 11565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ddc5f5fecb0403386dd61f18d051b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.131685 11567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34fd2daee75140e8ad3c8617bc3b5592. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.131335 11415 tablet_service.cc:1511] Processing CreateTablet for tablet c5f7a919c42640ea984e8ae25b9b2338 (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:07:23.132777 11415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5f7a919c42640ea984e8ae25b9b2338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.136626 11492 tablet_service.cc:1511] Processing CreateTablet for tablet c8dad3e10536436f9dd13720971d29b7 (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:07:23.137391 11495 tablet_service.cc:1511] Processing CreateTablet for tablet 3c510e78737545ebb0aa0366f008c5bd (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:07:23.138561 11414 tablet_service.cc:1511] Processing CreateTablet for tablet 4a4c406d34924a419654e30483424cbf (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:07:23.138960 11564 tablet_service.cc:1511] Processing CreateTablet for tablet c5f7a919c42640ea984e8ae25b9b2338 (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:07:23.139671 11494 tablet_service.cc:1511] Processing CreateTablet for tablet f668526011204a8e8370e268ae661324 (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:07:23.140386 11416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ddc5f5fecb0403386dd61f18d051b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.140655 11564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5f7a919c42640ea984e8ae25b9b2338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.142059 11414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a4c406d34924a419654e30483424cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.142829 11491 tablet_service.cc:1511] Processing CreateTablet for tablet 4ddc5f5fecb0403386dd61f18d051b97 (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:07:23.144658 11489 tablet_service.cc:1511] Processing CreateTablet for tablet 4a4c406d34924a419654e30483424cbf (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:07:23.144989 11490 tablet_service.cc:1511] Processing CreateTablet for tablet c5f7a919c42640ea984e8ae25b9b2338 (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:07:23.142096 11493 tablet_service.cc:1511] Processing CreateTablet for tablet 34fd2daee75140e8ad3c8617bc3b5592 (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:07:23.148154 11563 tablet_service.cc:1511] Processing CreateTablet for tablet 4a4c406d34924a419654e30483424cbf (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:07:23.148293 11413 tablet_service.cc:1511] Processing CreateTablet for tablet 01c031d7def6441ea735da3573686b3e (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:07:23.150400 11413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01c031d7def6441ea735da3573686b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.150974 11488 tablet_service.cc:1511] Processing CreateTablet for tablet 01c031d7def6441ea735da3573686b3e (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:07:23.151561 11412 tablet_service.cc:1511] Processing CreateTablet for tablet 72a07b3addad47cdad5e02a033cdd7cf (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:07:23.153035 11412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72a07b3addad47cdad5e02a033cdd7cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.153234 11562 tablet_service.cc:1511] Processing CreateTablet for tablet 01c031d7def6441ea735da3573686b3e (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:07:23.154167 11418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34fd2daee75140e8ad3c8617bc3b5592. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.155224 11492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8dad3e10536436f9dd13720971d29b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.159296 11493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34fd2daee75140e8ad3c8617bc3b5592. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.160501 11489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a4c406d34924a419654e30483424cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.163100 11563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a4c406d34924a419654e30483424cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.166188 11562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01c031d7def6441ea735da3573686b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.169803 11487 tablet_service.cc:1511] Processing CreateTablet for tablet 72a07b3addad47cdad5e02a033cdd7cf (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:07:23.178339 11486 tablet_service.cc:1511] Processing CreateTablet for tablet 1e0204688f55492b8bf3ec0ed3df4cf3 (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:07:23.179299 11411 tablet_service.cc:1511] Processing CreateTablet for tablet 1e0204688f55492b8bf3ec0ed3df4cf3 (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:07:23.180794 11411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e0204688f55492b8bf3ec0ed3df4cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.181373 11494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f668526011204a8e8370e268ae661324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.186329 11495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c510e78737545ebb0aa0366f008c5bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.187187 11487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72a07b3addad47cdad5e02a033cdd7cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.189126 11561 tablet_service.cc:1511] Processing CreateTablet for tablet 72a07b3addad47cdad5e02a033cdd7cf (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:07:23.189971 11491 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ddc5f5fecb0403386dd61f18d051b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.190685 11561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72a07b3addad47cdad5e02a033cdd7cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.194429 11486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e0204688f55492b8bf3ec0ed3df4cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.195436 11560 tablet_service.cc:1511] Processing CreateTablet for tablet 1e0204688f55492b8bf3ec0ed3df4cf3 (DEFAULT_TABLE table=test-workload [id=f4f70bf547c944eda360a85f0d9577a4]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:07:23.196982 11560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e0204688f55492b8bf3ec0ed3df4cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.197321 11488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01c031d7def6441ea735da3573686b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.202397 11490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5f7a919c42640ea984e8ae25b9b2338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.328032 11620 tablet_bootstrap.cc:492] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap starting.
I20260709 13:07:23.352414 11620 tablet_bootstrap.cc:654] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.381877 11621 tablet_bootstrap.cc:492] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970: Bootstrap starting.
I20260709 13:07:23.388274 11621 tablet_bootstrap.cc:654] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.408746 11619 tablet_bootstrap.cc:492] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8: Bootstrap starting.
I20260709 13:07:23.432844 11620 tablet_bootstrap.cc:492] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9: No bootstrap required, opened a new log
I20260709 13:07:23.433355 11620 ts_tablet_manager.cc:1403] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9: Time spent bootstrapping tablet: real 0.106s	user 0.024s	sys 0.025s
I20260709 13:07:23.433689 11621 tablet_bootstrap.cc:492] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970: No bootstrap required, opened a new log
I20260709 13:07:23.434172 11621 ts_tablet_manager.cc:1403] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970: Time spent bootstrapping tablet: real 0.053s	user 0.013s	sys 0.020s
I20260709 13:07:23.435771 11620 raft_consensus.cc:359] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.436388 11620 raft_consensus.cc:385] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.436268 11621 raft_consensus.cc:359] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.436856 11620 raft_consensus.cc:740] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.437071 11621 raft_consensus.cc:385] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.437413 11621 raft_consensus.cc:740] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.437759 11619 tablet_bootstrap.cc:654] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.437757 11620 consensus_queue.cc:260] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.438177 11621 consensus_queue.cc:260] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.440430 11455 heartbeater.cc:499] Master 127.0.107.254:37289 was elected leader, sending a full tablet report...
I20260709 13:07:23.440805 11620 ts_tablet_manager.cc:1434] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:23.442235 11620 tablet_bootstrap.cc:492] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap starting.
I20260709 13:07:23.447383 11621 ts_tablet_manager.cc:1434] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970: Time spent starting tablet: real 0.013s	user 0.011s	sys 0.001s
I20260709 13:07:23.447554 11530 heartbeater.cc:499] Master 127.0.107.254:37289 was elected leader, sending a full tablet report...
I20260709 13:07:23.449710 11620 tablet_bootstrap.cc:654] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.457921 11619 tablet_bootstrap.cc:492] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8: No bootstrap required, opened a new log
I20260709 13:07:23.458729 11619 ts_tablet_manager.cc:1403] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8: Time spent bootstrapping tablet: real 0.050s	user 0.009s	sys 0.013s
I20260709 13:07:23.462852 11619 raft_consensus.cc:359] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.463932 11619 raft_consensus.cc:385] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.464344 11619 raft_consensus.cc:740] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.465615 11619 consensus_queue.cc:260] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.474830 11621 tablet_bootstrap.cc:492] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970: Bootstrap starting.
I20260709 13:07:23.476837 11620 tablet_bootstrap.cc:492] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9: No bootstrap required, opened a new log
I20260709 13:07:23.477756 11620 ts_tablet_manager.cc:1403] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.019s
I20260709 13:07:23.482362 11621 tablet_bootstrap.cc:654] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.482252 11620 raft_consensus.cc:359] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } } peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } }
I20260709 13:07:23.483417 11620 raft_consensus.cc:385] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.483906 11620 raft_consensus.cc:740] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.485036 11620 consensus_queue.cc:260] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } } peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } }
I20260709 13:07:23.486434 11604 heartbeater.cc:499] Master 127.0.107.254:37289 was elected leader, sending a full tablet report...
I20260709 13:07:23.487046 11619 ts_tablet_manager.cc:1434] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8: Time spent starting tablet: real 0.028s	user 0.010s	sys 0.015s
I20260709 13:07:23.487854 11619 tablet_bootstrap.cc:492] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8: Bootstrap starting.
I20260709 13:07:23.489473 11628 raft_consensus.cc:493] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.489997 11628 raft_consensus.cc:515] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.493224 11620 ts_tablet_manager.cc:1434] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:07:23.500739 11620 tablet_bootstrap.cc:492] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap starting.
I20260709 13:07:23.497939 11628 leader_election.cc:290] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:23.510663 11620 tablet_bootstrap.cc:654] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.511361 11621 tablet_bootstrap.cc:492] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970: No bootstrap required, opened a new log
I20260709 13:07:23.511951 11621 ts_tablet_manager.cc:1403] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.006s
I20260709 13:07:23.515152 11619 tablet_bootstrap.cc:654] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:23.517943 11456 tablet.cc:2406] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:07:23.525797 11620 tablet_bootstrap.cc:492] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9: No bootstrap required, opened a new log
I20260709 13:07:23.526448 11620 ts_tablet_manager.cc:1403] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9: Time spent bootstrapping tablet: real 0.026s	user 0.002s	sys 0.015s
I20260709 13:07:23.526341 11621 raft_consensus.cc:359] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.527281 11621 raft_consensus.cc:385] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.527637 11621 raft_consensus.cc:740] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.529879 11620 raft_consensus.cc:359] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.530789 11620 raft_consensus.cc:385] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.531147 11620 raft_consensus.cc:740] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.532008 11620 consensus_queue.cc:260] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.534407 11620 ts_tablet_manager.cc:1434] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:07:23.535499 11620 tablet_bootstrap.cc:492] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap starting.
I20260709 13:07:23.537745 11619 tablet_bootstrap.cc:492] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8: No bootstrap required, opened a new log
I20260709 13:07:23.537501 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f668526011204a8e8370e268ae661324" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970" is_pre_election: true
I20260709 13:07:23.538393 11619 ts_tablet_manager.cc:1403] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8: Time spent bootstrapping tablet: real 0.051s	user 0.010s	sys 0.006s
W20260709 13:07:23.539299 11544 leader_election.cc:343] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.541990 11619 raft_consensus.cc:359] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.542654 11620 tablet_bootstrap.cc:654] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.542872 11619 raft_consensus.cc:385] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.543241 11619 raft_consensus.cc:740] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.544128 11619 consensus_queue.cc:260] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.547446 11621 consensus_queue.cc:260] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.550067 11621 ts_tablet_manager.cc:1434] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970: Time spent starting tablet: real 0.038s	user 0.010s	sys 0.000s
I20260709 13:07:23.550541 11430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f668526011204a8e8370e268ae661324" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a94a4038cc649babfe75a46223a8bf9" is_pre_election: true
I20260709 13:07:23.551076 11621 tablet_bootstrap.cc:492] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970: Bootstrap starting.
W20260709 13:07:23.553049 11542 leader_election.cc:343] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.553843 11542 leader_election.cc:304] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8; no voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9
I20260709 13:07:23.555542 11628 raft_consensus.cc:2749] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:23.558131 11619 ts_tablet_manager.cc:1434] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:07:23.558292 11621 tablet_bootstrap.cc:654] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.559891 11619 tablet_bootstrap.cc:492] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8: Bootstrap starting.
I20260709 13:07:23.568238 11619 tablet_bootstrap.cc:654] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.568825 11620 tablet_bootstrap.cc:492] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9: No bootstrap required, opened a new log
I20260709 13:07:23.569509 11620 ts_tablet_manager.cc:1403] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9: Time spent bootstrapping tablet: real 0.034s	user 0.020s	sys 0.008s
I20260709 13:07:23.573390 11620 raft_consensus.cc:359] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.574524 11620 raft_consensus.cc:385] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.575014 11620 raft_consensus.cc:740] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.576221 11620 consensus_queue.cc:260] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.581992 11621 tablet_bootstrap.cc:492] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970: No bootstrap required, opened a new log
I20260709 13:07:23.582756 11621 ts_tablet_manager.cc:1403] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.012s
I20260709 13:07:23.582772 11620 ts_tablet_manager.cc:1434] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.001s
I20260709 13:07:23.585083 11620 tablet_bootstrap.cc:492] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap starting.
I20260709 13:07:23.585418 11619 tablet_bootstrap.cc:492] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8: No bootstrap required, opened a new log
I20260709 13:07:23.585985 11619 ts_tablet_manager.cc:1403] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.014s
I20260709 13:07:23.586478 11621 raft_consensus.cc:359] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.587414 11621 raft_consensus.cc:385] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.587834 11621 raft_consensus.cc:740] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.588866 11621 consensus_queue.cc:260] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.589082 11619 raft_consensus.cc:359] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } } peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } }
I20260709 13:07:23.590023 11619 raft_consensus.cc:385] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.590345 11619 raft_consensus.cc:740] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.591173 11619 consensus_queue.cc:260] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } } peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } }
I20260709 13:07:23.592312 11620 tablet_bootstrap.cc:654] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.595816 11621 ts_tablet_manager.cc:1434] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970: Time spent starting tablet: real 0.013s	user 0.002s	sys 0.005s
I20260709 13:07:23.596602 11619 ts_tablet_manager.cc:1434] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 13:07:23.597308 11621 tablet_bootstrap.cc:492] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970: Bootstrap starting.
I20260709 13:07:23.598105 11619 tablet_bootstrap.cc:492] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8: Bootstrap starting.
I20260709 13:07:23.604394 11620 tablet_bootstrap.cc:492] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9: No bootstrap required, opened a new log
I20260709 13:07:23.604991 11620 ts_tablet_manager.cc:1403] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 13:07:23.605609 11619 tablet_bootstrap.cc:654] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.606133 11621 tablet_bootstrap.cc:654] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.608441 11620 raft_consensus.cc:359] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.609359 11620 raft_consensus.cc:385] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.609721 11620 raft_consensus.cc:740] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Initialized, Role: FOLLOWER
W20260709 13:07:23.610105 11376 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:23.610628 11620 consensus_queue.cc:260] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.617904 11620 ts_tablet_manager.cc:1434] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.004s
I20260709 13:07:23.619392 11620 tablet_bootstrap.cc:492] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap starting.
I20260709 13:07:23.629962 11620 tablet_bootstrap.cc:654] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.636240 11619 tablet_bootstrap.cc:492] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8: No bootstrap required, opened a new log
I20260709 13:07:23.636770 11619 ts_tablet_manager.cc:1403] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8: Time spent bootstrapping tablet: real 0.039s	user 0.017s	sys 0.016s
I20260709 13:07:23.637318 11620 tablet_bootstrap.cc:492] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9: No bootstrap required, opened a new log
I20260709 13:07:23.637976 11620 ts_tablet_manager.cc:1403] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.004s
I20260709 13:07:23.638732 11621 tablet_bootstrap.cc:492] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970: No bootstrap required, opened a new log
I20260709 13:07:23.639205 11621 ts_tablet_manager.cc:1403] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970: Time spent bootstrapping tablet: real 0.042s	user 0.014s	sys 0.024s
I20260709 13:07:23.639837 11619 raft_consensus.cc:359] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.640607 11619 raft_consensus.cc:385] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.640944 11619 raft_consensus.cc:740] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.640676 11620 raft_consensus.cc:359] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.641443 11620 raft_consensus.cc:385] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.641607 11621 raft_consensus.cc:359] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.641804 11620 raft_consensus.cc:740] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.642345 11621 raft_consensus.cc:385] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.641776 11619 consensus_queue.cc:260] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.642676 11621 raft_consensus.cc:740] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.642910 11620 consensus_queue.cc:260] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.643533 11621 consensus_queue.cc:260] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.644738 11619 ts_tablet_manager.cc:1434] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:07:23.645843 11619 tablet_bootstrap.cc:492] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 1115d86484ef4339933c2ba1911812f8: Bootstrap starting.
I20260709 13:07:23.648190 11621 ts_tablet_manager.cc:1434] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 13:07:23.648628 11620 ts_tablet_manager.cc:1434] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:07:23.649717 11621 tablet_bootstrap.cc:492] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970: Bootstrap starting.
I20260709 13:07:23.649840 11620 tablet_bootstrap.cc:492] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap starting.
I20260709 13:07:23.652204 11619 tablet_bootstrap.cc:654] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 1115d86484ef4339933c2ba1911812f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.656880 11621 tablet_bootstrap.cc:654] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.657389 11620 tablet_bootstrap.cc:654] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.664453 11619 tablet_bootstrap.cc:492] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 1115d86484ef4339933c2ba1911812f8: No bootstrap required, opened a new log
I20260709 13:07:23.665149 11619 ts_tablet_manager.cc:1403] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 1115d86484ef4339933c2ba1911812f8: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 13:07:23.669412 11619 raft_consensus.cc:359] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.670644 11619 raft_consensus.cc:385] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.671060 11619 raft_consensus.cc:740] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.671387 11621 tablet_bootstrap.cc:492] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970: No bootstrap required, opened a new log
I20260709 13:07:23.672087 11621 ts_tablet_manager.cc:1403] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.000s
I20260709 13:07:23.673345 11619 consensus_queue.cc:260] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 1115d86484ef4339933c2ba1911812f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.674779 11624 raft_consensus.cc:493] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.675025 11621 raft_consensus.cc:359] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.675328 11624 raft_consensus.cc:515] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.675846 11621 raft_consensus.cc:385] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.676083 11619 ts_tablet_manager.cc:1434] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 1115d86484ef4339933c2ba1911812f8: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.003s
I20260709 13:07:23.676221 11621 raft_consensus.cc:740] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.677491 11619 tablet_bootstrap.cc:492] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8: Bootstrap starting.
I20260709 13:07:23.677459 11621 consensus_queue.cc:260] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.680318 11621 ts_tablet_manager.cc:1434] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:07:23.681442 11621 tablet_bootstrap.cc:492] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970: Bootstrap starting.
I20260709 13:07:23.681806 11625 raft_consensus.cc:493] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.686622 11620 tablet_bootstrap.cc:492] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9: No bootstrap required, opened a new log
I20260709 13:07:23.685587 11624 leader_election.cc:290] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:23.687875 11620 ts_tablet_manager.cc:1403] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9: Time spent bootstrapping tablet: real 0.038s	user 0.021s	sys 0.004s
I20260709 13:07:23.687422 11625 raft_consensus.cc:515] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.691108 11628 raft_consensus.cc:493] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.694911 11628 raft_consensus.cc:515] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.701872 11628 leader_election.cc:290] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:23.702773 11430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a4c406d34924a419654e30483424cbf" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a94a4038cc649babfe75a46223a8bf9" is_pre_election: true
W20260709 13:07:23.704368 11542 leader_election.cc:343] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.702803 11625 leader_election.cc:290] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:23.707417 11625 raft_consensus.cc:493] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.708012 11625 raft_consensus.cc:515] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.710882 11625 leader_election.cc:290] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:23.728272 11620 raft_consensus.cc:359] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.728991 11620 raft_consensus.cc:385] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.729401 11620 raft_consensus.cc:740] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.730556 11620 consensus_queue.cc:260] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.732239 11619 tablet_bootstrap.cc:654] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.734144 11620 ts_tablet_manager.cc:1434] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9: Time spent starting tablet: real 0.046s	user 0.003s	sys 0.004s
I20260709 13:07:23.741220 11621 tablet_bootstrap.cc:654] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.742187 11620 tablet_bootstrap.cc:492] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap starting.
I20260709 13:07:23.744952 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a4c406d34924a419654e30483424cbf" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970" is_pre_election: true
I20260709 13:07:23.745332 11649 raft_consensus.cc:493] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.746328 11649 raft_consensus.cc:515] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.748669 11649 leader_election.cc:290] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
W20260709 13:07:23.751340 11544 leader_election.cc:343] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.751940 11544 leader_election.cc:304] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8; no voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9
I20260709 13:07:23.753569 11628 raft_consensus.cc:2749] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:23.753732 11620 tablet_bootstrap.cc:654] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.754763 11579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72a07b3addad47cdad5e02a033cdd7cf" candidate_uuid: "7a94a4038cc649babfe75a46223a8bf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8" is_pre_election: true
I20260709 13:07:23.757941 11649 raft_consensus.cc:493] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.758461 11649 raft_consensus.cc:515] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.760478 11649 leader_election.cc:290] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
W20260709 13:07:23.760322 11393 leader_election.cc:343] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.761507 11619 tablet_bootstrap.cc:492] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8: No bootstrap required, opened a new log
I20260709 13:07:23.762265 11619 ts_tablet_manager.cc:1403] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8: Time spent bootstrapping tablet: real 0.085s	user 0.009s	sys 0.016s
I20260709 13:07:23.762260 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72a07b3addad47cdad5e02a033cdd7cf" candidate_uuid: "7a94a4038cc649babfe75a46223a8bf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970" is_pre_election: true
I20260709 13:07:23.763418 11505 raft_consensus.cc:2468] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a94a4038cc649babfe75a46223a8bf9 in term 0.
I20260709 13:07:23.765507 11395 leader_election.cc:304] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9; no voters: 1115d86484ef4339933c2ba1911812f8
I20260709 13:07:23.767078 11624 raft_consensus.cc:2804] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:23.767488 11624 raft_consensus.cc:493] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.767844 11624 raft_consensus.cc:3060] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.769345 11619 raft_consensus.cc:359] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.770210 11619 raft_consensus.cc:385] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.770547 11619 raft_consensus.cc:740] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.771375 11619 consensus_queue.cc:260] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.772773 11430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c031d7def6441ea735da3573686b3e" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a94a4038cc649babfe75a46223a8bf9" is_pre_election: true
I20260709 13:07:23.773494 11430 raft_consensus.cc:2468] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 0.
I20260709 13:07:23.773883 11619 ts_tablet_manager.cc:1434] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:07:23.774720 11429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72a07b3addad47cdad5e02a033cdd7cf" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a94a4038cc649babfe75a46223a8bf9" is_pre_election: true
I20260709 13:07:23.774999 11619 tablet_bootstrap.cc:492] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8: Bootstrap starting.
I20260709 13:07:23.776055 11427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e0204688f55492b8bf3ec0ed3df4cf3" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a94a4038cc649babfe75a46223a8bf9" is_pre_election: true
I20260709 13:07:23.776778 11427 raft_consensus.cc:2468] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 0.
I20260709 13:07:23.777854 11428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34fd2daee75140e8ad3c8617bc3b5592" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a94a4038cc649babfe75a46223a8bf9" is_pre_election: true
I20260709 13:07:23.779122 11468 leader_election.cc:304] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:23.780218 11649 raft_consensus.cc:2804] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:23.780594 11649 raft_consensus.cc:493] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.780954 11649 raft_consensus.cc:3060] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.782709 11619 tablet_bootstrap.cc:654] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.789953 11468 leader_election.cc:304] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:23.790931 11625 raft_consensus.cc:2804] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:07:23.791045 11468 leader_election.cc:343] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.791358 11625 raft_consensus.cc:493] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.791889 11625 raft_consensus.cc:3060] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.789945 11649 raft_consensus.cc:515] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.794812 11428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c031d7def6441ea735da3573686b3e" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a94a4038cc649babfe75a46223a8bf9"
I20260709 13:07:23.795452 11428 raft_consensus.cc:3060] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.795606 11624 raft_consensus.cc:515] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.797962 11429 raft_consensus.cc:2393] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41c59a1982d3480d87bc6a0baa625970 in current term 1: Already voted for candidate 7a94a4038cc649babfe75a46223a8bf9 in this term.
I20260709 13:07:23.801851 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72a07b3addad47cdad5e02a033cdd7cf" candidate_uuid: "7a94a4038cc649babfe75a46223a8bf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
I20260709 13:07:23.802587 11505 raft_consensus.cc:3060] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.804355 11621 tablet_bootstrap.cc:492] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970: No bootstrap required, opened a new log
I20260709 13:07:23.804116 11625 raft_consensus.cc:515] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.807235 11625 leader_election.cc:290] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:23.808710 11621 ts_tablet_manager.cc:1403] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970: Time spent bootstrapping tablet: real 0.127s	user 0.006s	sys 0.029s
I20260709 13:07:23.812086 11621 raft_consensus.cc:359] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.812858 11621 raft_consensus.cc:385] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.812353 11505 raft_consensus.cc:2468] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a94a4038cc649babfe75a46223a8bf9 in term 1.
I20260709 13:07:23.813108 11429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e0204688f55492b8bf3ec0ed3df4cf3" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a94a4038cc649babfe75a46223a8bf9"
I20260709 13:07:23.814646 11429 raft_consensus.cc:3060] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.813172 11621 raft_consensus.cc:740] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.818257 11649 leader_election.cc:290] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:23.817342 11621 consensus_queue.cc:260] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.819867 11395 leader_election.cc:304] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:23.823242 11579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72a07b3addad47cdad5e02a033cdd7cf" candidate_uuid: "7a94a4038cc649babfe75a46223a8bf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8"
I20260709 13:07:23.826318 11578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c031d7def6441ea735da3573686b3e" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8" is_pre_election: true
I20260709 13:07:23.827611 11579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e0204688f55492b8bf3ec0ed3df4cf3" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8" is_pre_election: true
I20260709 13:07:23.828299 11579 raft_consensus.cc:2468] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 0.
I20260709 13:07:23.828253 11578 raft_consensus.cc:2468] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 0.
I20260709 13:07:23.828909 11621 ts_tablet_manager.cc:1434] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:07:23.830022 11621 tablet_bootstrap.cc:492] T f668526011204a8e8370e268ae661324 P 41c59a1982d3480d87bc6a0baa625970: Bootstrap starting.
I20260709 13:07:23.830021 11577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72a07b3addad47cdad5e02a033cdd7cf" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8" is_pre_election: true
I20260709 13:07:23.831907 11576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34fd2daee75140e8ad3c8617bc3b5592" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8" is_pre_election: true
I20260709 13:07:23.832898 11576 raft_consensus.cc:2468] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 0.
I20260709 13:07:23.835938 11628 raft_consensus.cc:493] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.836467 11628 raft_consensus.cc:515] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } } peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } }
I20260709 13:07:23.836793 11575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e0204688f55492b8bf3ec0ed3df4cf3" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8"
W20260709 13:07:23.837546 11468 leader_election.cc:343] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.837720 11575 raft_consensus.cc:3060] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.838575 11628 leader_election.cc:290] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:23.840333 11427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c510e78737545ebb0aa0366f008c5bd" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a94a4038cc649babfe75a46223a8bf9" is_pre_election: true
I20260709 13:07:23.840591 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c510e78737545ebb0aa0366f008c5bd" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970" is_pre_election: true
I20260709 13:07:23.841008 11427 raft_consensus.cc:2468] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 0.
W20260709 13:07:23.842121 11544 leader_election.cc:343] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.842319 11574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c031d7def6441ea735da3573686b3e" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8"
I20260709 13:07:23.843146 11542 leader_election.cc:304] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 7a94a4038cc649babfe75a46223a8bf9; no voters: 41c59a1982d3480d87bc6a0baa625970
I20260709 13:07:23.843294 11574 raft_consensus.cc:3060] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.844446 11621 tablet_bootstrap.cc:654] T f668526011204a8e8370e268ae661324 P 41c59a1982d3480d87bc6a0baa625970: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.845127 11628 raft_consensus.cc:2804] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:23.845592 11628 raft_consensus.cc:493] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:23.843698 11393 leader_election.cc:343] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.846290 11628 raft_consensus.cc:3060] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.821878 11624 leader_election.cc:290] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 election: Requested vote from peers 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:23.849596 11649 raft_consensus.cc:493] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.850091 11649 raft_consensus.cc:515] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.850916 11429 raft_consensus.cc:2468] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 1.
I20260709 13:07:23.852381 11649 leader_election.cc:290] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:23.852418 11624 raft_consensus.cc:2804] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:23.853528 11624 raft_consensus.cc:697] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 1 LEADER]: Becoming Leader. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Running, Role: LEADER
I20260709 13:07:23.854640 11620 tablet_bootstrap.cc:492] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9: No bootstrap required, opened a new log
I20260709 13:07:23.855229 11620 ts_tablet_manager.cc:1403] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9: Time spent bootstrapping tablet: real 0.113s	user 0.029s	sys 0.023s
I20260709 13:07:23.838065 11468 leader_election.cc:304] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970; no voters: 1115d86484ef4339933c2ba1911812f8, 7a94a4038cc649babfe75a46223a8bf9
I20260709 13:07:23.857057 11649 raft_consensus.cc:2749] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:23.858290 11468 leader_election.cc:304] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 41c59a1982d3480d87bc6a0baa625970; no voters: 7a94a4038cc649babfe75a46223a8bf9
I20260709 13:07:23.858987 11649 raft_consensus.cc:2804] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:23.859401 11619 tablet_bootstrap.cc:492] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8: No bootstrap required, opened a new log
I20260709 13:07:23.859508 11620 raft_consensus.cc:359] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.860127 11619 ts_tablet_manager.cc:1403] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8: Time spent bootstrapping tablet: real 0.085s	user 0.022s	sys 0.024s
I20260709 13:07:23.860460 11620 raft_consensus.cc:385] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.861001 11577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ddc5f5fecb0403386dd61f18d051b97" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8" is_pre_election: true
I20260709 13:07:23.860586 11468 leader_election.cc:304] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:23.862174 11625 raft_consensus.cc:2804] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:23.859385 11649 raft_consensus.cc:493] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.847954 11428 raft_consensus.cc:2468] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 1.
I20260709 13:07:23.861023 11620 raft_consensus.cc:740] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.864003 11620 consensus_queue.cc:260] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.862664 11649 raft_consensus.cc:3060] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.860447 11429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ddc5f5fecb0403386dd61f18d051b97" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a94a4038cc649babfe75a46223a8bf9" is_pre_election: true
I20260709 13:07:23.865846 11429 raft_consensus.cc:2468] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 0.
I20260709 13:07:23.865933 11619 raft_consensus.cc:359] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.866906 11619 raft_consensus.cc:385] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.867290 11619 raft_consensus.cc:740] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.868238 11619 consensus_queue.cc:260] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.854547 11624 consensus_queue.cc:237] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
W20260709 13:07:23.864952 11468 leader_election.cc:343] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.854636 11575 raft_consensus.cc:2468] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 1.
I20260709 13:07:23.871417 11468 leader_election.cc:304] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:23.876426 11628 raft_consensus.cc:515] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } } peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } }
I20260709 13:07:23.880656 11625 raft_consensus.cc:697] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [term 1 LEADER]: Becoming Leader. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Running, Role: LEADER
I20260709 13:07:23.881553 11620 ts_tablet_manager.cc:1434] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.003s
I20260709 13:07:23.881909 11657 raft_consensus.cc:2804] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:23.881740 11625 consensus_queue.cc:237] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.882611 11620 tablet_bootstrap.cc:492] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap starting.
I20260709 13:07:23.882491 11657 raft_consensus.cc:697] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [term 1 LEADER]: Becoming Leader. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Running, Role: LEADER
I20260709 13:07:23.872025 11619 ts_tablet_manager.cc:1434] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 13:07:23.879245 11574 raft_consensus.cc:2468] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 1.
I20260709 13:07:23.884678 11468 leader_election.cc:304] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9; no voters: 1115d86484ef4339933c2ba1911812f8
I20260709 13:07:23.885675 11628 leader_election.cc:290] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:23.887547 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c510e78737545ebb0aa0366f008c5bd" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
I20260709 13:07:23.888690 11429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c510e78737545ebb0aa0366f008c5bd" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a94a4038cc649babfe75a46223a8bf9"
I20260709 13:07:23.889881 11429 raft_consensus.cc:3060] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:23.893847 11544 leader_election.cc:343] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.894348 11621 tablet_bootstrap.cc:492] T f668526011204a8e8370e268ae661324 P 41c59a1982d3480d87bc6a0baa625970: No bootstrap required, opened a new log
I20260709 13:07:23.894923 11621 ts_tablet_manager.cc:1403] T f668526011204a8e8370e268ae661324 P 41c59a1982d3480d87bc6a0baa625970: Time spent bootstrapping tablet: real 0.065s	user 0.013s	sys 0.001s
I20260709 13:07:23.898427 11621 raft_consensus.cc:359] T f668526011204a8e8370e268ae661324 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.899232 11621 raft_consensus.cc:385] T f668526011204a8e8370e268ae661324 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.899302 11649 raft_consensus.cc:515] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.899919 11621 raft_consensus.cc:740] T f668526011204a8e8370e268ae661324 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.900269 11429 raft_consensus.cc:2468] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 1.
I20260709 13:07:23.898753 11619 tablet_bootstrap.cc:492] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8: Bootstrap starting.
I20260709 13:07:23.901229 11659 raft_consensus.cc:2804] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:23.901019 11621 consensus_queue.cc:260] T f668526011204a8e8370e268ae661324 P 41c59a1982d3480d87bc6a0baa625970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.903234 11542 leader_election.cc:304] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 7a94a4038cc649babfe75a46223a8bf9; no voters: 41c59a1982d3480d87bc6a0baa625970
I20260709 13:07:23.903815 11649 leader_election.cc:290] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:23.904493 11628 raft_consensus.cc:2804] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:23.905108 11621 ts_tablet_manager.cc:1434] T f668526011204a8e8370e268ae661324 P 41c59a1982d3480d87bc6a0baa625970: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 13:07:23.904719 11657 consensus_queue.cc:237] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.906314 11621 tablet_bootstrap.cc:492] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970: Bootstrap starting.
I20260709 13:07:23.905025 11628 raft_consensus.cc:697] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [term 1 LEADER]: Becoming Leader. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Running, Role: LEADER
I20260709 13:07:23.907330 11628 consensus_queue.cc:237] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } } peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } }
I20260709 13:07:23.909027 11658 raft_consensus.cc:493] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.909551 11658 raft_consensus.cc:515] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.910389 11619 tablet_bootstrap.cc:654] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.913779 11658 leader_election.cc:290] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:23.902884 11659 raft_consensus.cc:493] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.915400 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8dad3e10536436f9dd13720971d29b7" candidate_uuid: "7a94a4038cc649babfe75a46223a8bf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970" is_pre_election: true
I20260709 13:07:23.918433 11621 tablet_bootstrap.cc:654] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:23.918351 11395 leader_election.cc:343] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.919620 11574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8dad3e10536436f9dd13720971d29b7" candidate_uuid: "7a94a4038cc649babfe75a46223a8bf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8" is_pre_election: true
W20260709 13:07:23.921097 11393 leader_election.cc:343] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.927031 11393 leader_election.cc:304] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a94a4038cc649babfe75a46223a8bf9; no voters: 1115d86484ef4339933c2ba1911812f8, 41c59a1982d3480d87bc6a0baa625970
I20260709 13:07:23.926632 11620 tablet_bootstrap.cc:654] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.915545 11659 raft_consensus.cc:3060] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.928478 11429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34fd2daee75140e8ad3c8617bc3b5592" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a94a4038cc649babfe75a46223a8bf9"
I20260709 13:07:23.935118 11574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34fd2daee75140e8ad3c8617bc3b5592" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8"
I20260709 13:07:23.935711 11574 raft_consensus.cc:3060] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.933606 11624 raft_consensus.cc:2749] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:23.933874 11628 raft_consensus.cc:493] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.938019 11628 raft_consensus.cc:515] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.938920 11328 catalog_manager.cc:5697] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "41c59a1982d3480d87bc6a0baa625970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:23.940183 11628 leader_election.cc:290] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:23.940724 11428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5f7a919c42640ea984e8ae25b9b2338" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a94a4038cc649babfe75a46223a8bf9" is_pre_election: true
W20260709 13:07:23.941102 11468 leader_election.cc:343] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.930990 11326 catalog_manager.cc:5697] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1115d86484ef4339933c2ba1911812f8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "1115d86484ef4339933c2ba1911812f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:23.942113 11542 leader_election.cc:343] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.945859 11327 catalog_manager.cc:5697] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "7a94a4038cc649babfe75a46223a8bf9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:23.946664 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5f7a919c42640ea984e8ae25b9b2338" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970" is_pre_election: true
W20260709 13:07:23.948202 11544 leader_election.cc:343] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.948685 11544 leader_election.cc:304] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8; no voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9
I20260709 13:07:23.949433 11628 raft_consensus.cc:2749] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:23.950733 11574 raft_consensus.cc:2468] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 1.
I20260709 13:07:23.952248 11468 leader_election.cc:304] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 41c59a1982d3480d87bc6a0baa625970; no voters: 7a94a4038cc649babfe75a46223a8bf9
I20260709 13:07:23.955109 11657 raft_consensus.cc:2804] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:23.955325 11659 raft_consensus.cc:515] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.958557 11428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ddc5f5fecb0403386dd61f18d051b97" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a94a4038cc649babfe75a46223a8bf9"
I20260709 13:07:23.959196 11428 raft_consensus.cc:3060] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.963865 11621 tablet_bootstrap.cc:492] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970: No bootstrap required, opened a new log
I20260709 13:07:23.964252 11619 tablet_bootstrap.cc:492] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8: No bootstrap required, opened a new log
I20260709 13:07:23.966495 11428 raft_consensus.cc:2468] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 1.
I20260709 13:07:23.966655 11574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ddc5f5fecb0403386dd61f18d051b97" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8"
I20260709 13:07:23.967870 11468 leader_election.cc:304] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:23.966694 11619 ts_tablet_manager.cc:1403] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8: Time spent bootstrapping tablet: real 0.068s	user 0.011s	sys 0.002s
W20260709 13:07:23.969023 11468 leader_election.cc:343] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.969969 11625 raft_consensus.cc:2804] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:23.970485 11625 raft_consensus.cc:697] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 1 LEADER]: Becoming Leader. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Running, Role: LEADER
I20260709 13:07:23.970953 11619 raft_consensus.cc:359] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.971534 11625 consensus_queue.cc:237] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.973150 11620 tablet_bootstrap.cc:492] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9: No bootstrap required, opened a new log
I20260709 13:07:23.973628 11620 ts_tablet_manager.cc:1403] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9: Time spent bootstrapping tablet: real 0.091s	user 0.004s	sys 0.006s
I20260709 13:07:23.976279 11620 raft_consensus.cc:359] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.976953 11620 raft_consensus.cc:385] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.977217 11620 raft_consensus.cc:740] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.964450 11621 ts_tablet_manager.cc:1403] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970: Time spent bootstrapping tablet: real 0.058s	user 0.013s	sys 0.000s
I20260709 13:07:23.955821 11657 raft_consensus.cc:697] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 1 LEADER]: Becoming Leader. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Running, Role: LEADER
I20260709 13:07:23.978307 11657 consensus_queue.cc:237] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.966027 11659 leader_election.cc:290] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:23.971621 11619 raft_consensus.cc:385] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.982811 11619 raft_consensus.cc:740] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.983032 11621 raft_consensus.cc:359] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } } peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } }
I20260709 13:07:23.983834 11621 raft_consensus.cc:385] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.983573 11619 consensus_queue.cc:260] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.984143 11621 raft_consensus.cc:740] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.985000 11621 consensus_queue.cc:260] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } } peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } }
I20260709 13:07:23.985622 11619 ts_tablet_manager.cc:1434] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.000s
I20260709 13:07:23.986596 11619 tablet_bootstrap.cc:492] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8: Bootstrap starting.
I20260709 13:07:23.987411 11621 ts_tablet_manager.cc:1434] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:23.988415 11621 tablet_bootstrap.cc:492] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970: Bootstrap starting.
I20260709 13:07:23.988487 11624 raft_consensus.cc:493] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.989029 11624 raft_consensus.cc:515] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.989104 11620 consensus_queue.cc:260] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:23.991182 11624 leader_election.cc:290] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:23.992897 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f668526011204a8e8370e268ae661324" candidate_uuid: "7a94a4038cc649babfe75a46223a8bf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970" is_pre_election: true
I20260709 13:07:23.993718 11619 tablet_bootstrap.cc:654] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.995059 11621 tablet_bootstrap.cc:654] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.997314 11620 ts_tablet_manager.cc:1434] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 13:07:23.993590 11505 raft_consensus.cc:2468] T f668526011204a8e8370e268ae661324 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a94a4038cc649babfe75a46223a8bf9 in term 0.
I20260709 13:07:23.998543 11620 tablet_bootstrap.cc:492] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap starting.
I20260709 13:07:23.998929 11395 leader_election.cc:304] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:24.000135 11574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f668526011204a8e8370e268ae661324" candidate_uuid: "7a94a4038cc649babfe75a46223a8bf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8" is_pre_election: true
I20260709 13:07:24.000837 11574 raft_consensus.cc:2468] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a94a4038cc649babfe75a46223a8bf9 in term 0.
I20260709 13:07:24.006861 11620 tablet_bootstrap.cc:654] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:24.009876 11624 raft_consensus.cc:2804] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:24.010716 11624 raft_consensus.cc:493] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.014711 11624 raft_consensus.cc:3060] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:24.022640 11621 tablet_bootstrap.cc:492] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970: No bootstrap required, opened a new log
I20260709 13:07:24.023226 11621 ts_tablet_manager.cc:1403] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970: Time spent bootstrapping tablet: real 0.035s	user 0.007s	sys 0.015s
I20260709 13:07:24.028450 11326 catalog_manager.cc:5697] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "41c59a1982d3480d87bc6a0baa625970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:24.029256 11621 raft_consensus.cc:359] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.029974 11621 raft_consensus.cc:385] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:24.030274 11621 raft_consensus.cc:740] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Initialized, Role: FOLLOWER
I20260709 13:07:24.030967 11621 consensus_queue.cc:260] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.030679 11326 catalog_manager.cc:5697] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "41c59a1982d3480d87bc6a0baa625970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:24.031893 11619 tablet_bootstrap.cc:492] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8: No bootstrap required, opened a new log
I20260709 13:07:24.032407 11619 ts_tablet_manager.cc:1403] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8: Time spent bootstrapping tablet: real 0.046s	user 0.009s	sys 0.010s
I20260709 13:07:24.032090 11624 raft_consensus.cc:515] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.032989 11326 catalog_manager.cc:5697] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "41c59a1982d3480d87bc6a0baa625970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:24.035379 11619 raft_consensus.cc:359] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.036096 11619 raft_consensus.cc:385] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:24.036237 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f668526011204a8e8370e268ae661324" candidate_uuid: "7a94a4038cc649babfe75a46223a8bf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
I20260709 13:07:24.037237 11505 raft_consensus.cc:3060] T f668526011204a8e8370e268ae661324 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:24.038223 11621 ts_tablet_manager.cc:1434] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:07:24.039216 11621 tablet_bootstrap.cc:492] T c8dad3e10536436f9dd13720971d29b7 P 41c59a1982d3480d87bc6a0baa625970: Bootstrap starting.
I20260709 13:07:24.036398 11619 raft_consensus.cc:740] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:24.042565 11619 consensus_queue.cc:260] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.044886 11621 tablet_bootstrap.cc:654] T c8dad3e10536436f9dd13720971d29b7 P 41c59a1982d3480d87bc6a0baa625970: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:24.045238 11619 ts_tablet_manager.cc:1434] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:07:24.051050 11505 raft_consensus.cc:2468] T f668526011204a8e8370e268ae661324 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a94a4038cc649babfe75a46223a8bf9 in term 1.
I20260709 13:07:24.057447 11395 leader_election.cc:304] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:24.052424 11619 tablet_bootstrap.cc:492] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8: Bootstrap starting.
I20260709 13:07:24.054543 11574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f668526011204a8e8370e268ae661324" candidate_uuid: "7a94a4038cc649babfe75a46223a8bf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8"
I20260709 13:07:24.058909 11574 raft_consensus.cc:3060] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:24.061913 11658 raft_consensus.cc:2804] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:24.062706 11658 raft_consensus.cc:697] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 LEADER]: Becoming Leader. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Running, Role: LEADER
I20260709 13:07:24.063608 11658 consensus_queue.cc:237] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.064896 11619 tablet_bootstrap.cc:654] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:24.055678 11624 leader_election.cc:290] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 election: Requested vote from peers 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:24.057928 11620 tablet_bootstrap.cc:492] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9: No bootstrap required, opened a new log
I20260709 13:07:24.072022 11620 ts_tablet_manager.cc:1403] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9: Time spent bootstrapping tablet: real 0.074s	user 0.029s	sys 0.026s
I20260709 13:07:24.076035 11620 raft_consensus.cc:359] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.077180 11620 raft_consensus.cc:385] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:24.077661 11620 raft_consensus.cc:740] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Initialized, Role: FOLLOWER
I20260709 13:07:24.078984 11620 consensus_queue.cc:260] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.080379 11619 tablet_bootstrap.cc:492] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8: No bootstrap required, opened a new log
I20260709 13:07:24.080804 11621 tablet_bootstrap.cc:492] T c8dad3e10536436f9dd13720971d29b7 P 41c59a1982d3480d87bc6a0baa625970: No bootstrap required, opened a new log
I20260709 13:07:24.080986 11619 ts_tablet_manager.cc:1403] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.005s
I20260709 13:07:24.081257 11621 ts_tablet_manager.cc:1403] T c8dad3e10536436f9dd13720971d29b7 P 41c59a1982d3480d87bc6a0baa625970: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.015s
I20260709 13:07:24.082345 11620 ts_tablet_manager.cc:1434] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:07:24.077358 11574 raft_consensus.cc:2468] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a94a4038cc649babfe75a46223a8bf9 in term 1.
I20260709 13:07:24.084348 11619 raft_consensus.cc:359] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.085047 11619 raft_consensus.cc:385] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:24.085290 11619 raft_consensus.cc:740] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:24.086082 11619 consensus_queue.cc:260] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.086609 11621 raft_consensus.cc:359] T c8dad3e10536436f9dd13720971d29b7 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.087363 11621 raft_consensus.cc:385] T c8dad3e10536436f9dd13720971d29b7 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:24.087651 11621 raft_consensus.cc:740] T c8dad3e10536436f9dd13720971d29b7 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Initialized, Role: FOLLOWER
I20260709 13:07:24.088411 11619 ts_tablet_manager.cc:1434] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:24.088383 11621 consensus_queue.cc:260] T c8dad3e10536436f9dd13720971d29b7 P 41c59a1982d3480d87bc6a0baa625970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.090574 11621 ts_tablet_manager.cc:1434] T c8dad3e10536436f9dd13720971d29b7 P 41c59a1982d3480d87bc6a0baa625970: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:07:24.090444 11328 catalog_manager.cc:5697] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "7a94a4038cc649babfe75a46223a8bf9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:24.103451 11658 raft_consensus.cc:493] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.104048 11658 raft_consensus.cc:515] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.106312 11658 leader_election.cc:290] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:24.107064 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a4c406d34924a419654e30483424cbf" candidate_uuid: "7a94a4038cc649babfe75a46223a8bf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970" is_pre_election: true
I20260709 13:07:24.107937 11505 raft_consensus.cc:2468] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a94a4038cc649babfe75a46223a8bf9 in term 0.
I20260709 13:07:24.109046 11395 leader_election.cc:304] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:24.109910 11658 raft_consensus.cc:2804] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:24.109951 11574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a4c406d34924a419654e30483424cbf" candidate_uuid: "7a94a4038cc649babfe75a46223a8bf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8" is_pre_election: true
I20260709 13:07:24.110415 11658 raft_consensus.cc:493] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.110675 11574 raft_consensus.cc:2468] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a94a4038cc649babfe75a46223a8bf9 in term 0.
I20260709 13:07:24.110795 11658 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:24.115996 11658 raft_consensus.cc:515] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.117568 11658 leader_election.cc:290] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 election: Requested vote from peers 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:24.118495 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a4c406d34924a419654e30483424cbf" candidate_uuid: "7a94a4038cc649babfe75a46223a8bf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
I20260709 13:07:24.118623 11574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a4c406d34924a419654e30483424cbf" candidate_uuid: "7a94a4038cc649babfe75a46223a8bf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8"
I20260709 13:07:24.119115 11505 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:24.119199 11574 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:24.124172 11574 raft_consensus.cc:2468] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a94a4038cc649babfe75a46223a8bf9 in term 1.
I20260709 13:07:24.124186 11505 raft_consensus.cc:2468] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a94a4038cc649babfe75a46223a8bf9 in term 1.
I20260709 13:07:24.125300 11393 leader_election.cc:304] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:24.126088 11658 raft_consensus.cc:2804] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:24.126561 11658 raft_consensus.cc:697] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 1 LEADER]: Becoming Leader. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Running, Role: LEADER
I20260709 13:07:24.127205 11658 consensus_queue.cc:237] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.130630 11628 raft_consensus.cc:493] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.131243 11628 raft_consensus.cc:515] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.132351 11630 raft_consensus.cc:493] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.132920 11630 raft_consensus.cc:515] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.133394 11628 leader_election.cc:290] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:24.134408 11428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ddc5f5fecb0403386dd61f18d051b97" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a94a4038cc649babfe75a46223a8bf9" is_pre_election: true
I20260709 13:07:24.134948 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ddc5f5fecb0403386dd61f18d051b97" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970" is_pre_election: true
I20260709 13:07:24.135082 11630 leader_election.cc:290] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:24.135231 11428 raft_consensus.cc:2393] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1115d86484ef4339933c2ba1911812f8 in current term 1: Already voted for candidate 41c59a1982d3480d87bc6a0baa625970 in this term.
I20260709 13:07:24.135885 11326 catalog_manager.cc:5697] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "7a94a4038cc649babfe75a46223a8bf9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:24.137022 11428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72a07b3addad47cdad5e02a033cdd7cf" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a94a4038cc649babfe75a46223a8bf9" is_pre_election: true
I20260709 13:07:24.137367 11542 leader_election.cc:304] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8; no voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9
I20260709 13:07:24.138191 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72a07b3addad47cdad5e02a033cdd7cf" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970" is_pre_election: true
I20260709 13:07:24.138569 11630 raft_consensus.cc:3060] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:24.138962 11505 raft_consensus.cc:2393] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1115d86484ef4339933c2ba1911812f8 in current term 1: Already voted for candidate 7a94a4038cc649babfe75a46223a8bf9 in this term.
I20260709 13:07:24.140231 11544 leader_election.cc:304] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8; no voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9
I20260709 13:07:24.141364 11628 raft_consensus.cc:3060] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:24.146245 11630 raft_consensus.cc:2749] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:24.148144 11628 raft_consensus.cc:2749] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:24.174862 11658 raft_consensus.cc:493] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.175462 11658 raft_consensus.cc:515] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.177265 11658 leader_election.cc:290] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:24.178181 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34fd2daee75140e8ad3c8617bc3b5592" candidate_uuid: "7a94a4038cc649babfe75a46223a8bf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970" is_pre_election: true
I20260709 13:07:24.178452 11574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34fd2daee75140e8ad3c8617bc3b5592" candidate_uuid: "7a94a4038cc649babfe75a46223a8bf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8" is_pre_election: true
I20260709 13:07:24.179306 11574 raft_consensus.cc:2393] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a94a4038cc649babfe75a46223a8bf9 in current term 1: Already voted for candidate 41c59a1982d3480d87bc6a0baa625970 in this term.
I20260709 13:07:24.180250 11393 leader_election.cc:304] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a94a4038cc649babfe75a46223a8bf9; no voters: 1115d86484ef4339933c2ba1911812f8, 41c59a1982d3480d87bc6a0baa625970
I20260709 13:07:24.181016 11658 raft_consensus.cc:3060] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:24.186252 11658 raft_consensus.cc:2749] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:24.198117 11658 raft_consensus.cc:493] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.198525 11658 raft_consensus.cc:515] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.200253 11658 leader_election.cc:290] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:24.201020 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5f7a919c42640ea984e8ae25b9b2338" candidate_uuid: "7a94a4038cc649babfe75a46223a8bf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970" is_pre_election: true
I20260709 13:07:24.201335 11574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5f7a919c42640ea984e8ae25b9b2338" candidate_uuid: "7a94a4038cc649babfe75a46223a8bf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8" is_pre_election: true
I20260709 13:07:24.201777 11505 raft_consensus.cc:2468] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a94a4038cc649babfe75a46223a8bf9 in term 0.
I20260709 13:07:24.202327 11574 raft_consensus.cc:2468] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a94a4038cc649babfe75a46223a8bf9 in term 0.
I20260709 13:07:24.202893 11395 leader_election.cc:304] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:24.203670 11658 raft_consensus.cc:2804] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:24.204020 11658 raft_consensus.cc:493] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.204406 11658 raft_consensus.cc:3060] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:24.209697 11658 raft_consensus.cc:515] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.211320 11658 leader_election.cc:290] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 election: Requested vote from peers 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:24.212067 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5f7a919c42640ea984e8ae25b9b2338" candidate_uuid: "7a94a4038cc649babfe75a46223a8bf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
I20260709 13:07:24.212342 11574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5f7a919c42640ea984e8ae25b9b2338" candidate_uuid: "7a94a4038cc649babfe75a46223a8bf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8"
I20260709 13:07:24.212702 11505 raft_consensus.cc:3060] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:24.212872 11574 raft_consensus.cc:3060] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:24.219741 11505 raft_consensus.cc:2468] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a94a4038cc649babfe75a46223a8bf9 in term 1.
I20260709 13:07:24.219741 11574 raft_consensus.cc:2468] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a94a4038cc649babfe75a46223a8bf9 in term 1.
I20260709 13:07:24.221202 11393 leader_election.cc:304] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:24.221956 11658 raft_consensus.cc:2804] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:24.222442 11658 raft_consensus.cc:697] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 LEADER]: Becoming Leader. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Running, Role: LEADER
I20260709 13:07:24.223265 11658 consensus_queue.cc:237] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.231978 11326 catalog_manager.cc:5697] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "7a94a4038cc649babfe75a46223a8bf9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:24.281041 11657 raft_consensus.cc:493] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.281351 11658 consensus_queue.cc:1048] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:24.281757 11657 raft_consensus.cc:515] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } } peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } }
I20260709 13:07:24.283854 11657 leader_election.cc:290] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537), 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225)
I20260709 13:07:24.284636 11574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c510e78737545ebb0aa0366f008c5bd" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8" is_pre_election: true
I20260709 13:07:24.284904 11428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c510e78737545ebb0aa0366f008c5bd" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a94a4038cc649babfe75a46223a8bf9" is_pre_election: true
I20260709 13:07:24.285718 11428 raft_consensus.cc:2393] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41c59a1982d3480d87bc6a0baa625970 in current term 1: Already voted for candidate 1115d86484ef4339933c2ba1911812f8 in this term.
I20260709 13:07:24.286813 11468 leader_election.cc:304] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970; no voters: 1115d86484ef4339933c2ba1911812f8, 7a94a4038cc649babfe75a46223a8bf9
I20260709 13:07:24.287680 11657 raft_consensus.cc:3060] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:24.297585 11657 raft_consensus.cc:2749] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:24.303941 11658 consensus_queue.cc:1048] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:24.306087 11657 consensus_queue.cc:1048] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:24.310866 11628 raft_consensus.cc:493] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.311542 11628 raft_consensus.cc:515] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.313620 11628 leader_election.cc:290] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:24.316215 11428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8dad3e10536436f9dd13720971d29b7" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a94a4038cc649babfe75a46223a8bf9" is_pre_election: true
I20260709 13:07:24.319164 11659 consensus_queue.cc:1048] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:24.319885 11428 raft_consensus.cc:2468] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 0.
I20260709 13:07:24.321069 11542 leader_election.cc:304] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:24.322141 11628 raft_consensus.cc:2804] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:24.322508 11628 raft_consensus.cc:493] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.322938 11628 raft_consensus.cc:3060] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:24.330314 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8dad3e10536436f9dd13720971d29b7" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970" is_pre_election: true
I20260709 13:07:24.331266 11505 raft_consensus.cc:2468] T c8dad3e10536436f9dd13720971d29b7 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 0.
I20260709 13:07:24.336511 11628 raft_consensus.cc:515] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.340648 11428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8dad3e10536436f9dd13720971d29b7" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a94a4038cc649babfe75a46223a8bf9"
I20260709 13:07:24.341248 11428 raft_consensus.cc:3060] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:24.341843 11625 consensus_queue.cc:1048] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:24.344772 11628 leader_election.cc:290] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:24.346815 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8dad3e10536436f9dd13720971d29b7" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
I20260709 13:07:24.347451 11505 raft_consensus.cc:3060] T c8dad3e10536436f9dd13720971d29b7 P 41c59a1982d3480d87bc6a0baa625970 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:24.351495 11428 raft_consensus.cc:2468] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 1.
I20260709 13:07:24.353214 11542 leader_election.cc:304] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:24.354209 11628 raft_consensus.cc:2804] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:24.354760 11628 raft_consensus.cc:697] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 [term 1 LEADER]: Becoming Leader. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Running, Role: LEADER
I20260709 13:07:24.355250 11505 raft_consensus.cc:2468] T c8dad3e10536436f9dd13720971d29b7 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 1.
I20260709 13:07:24.355639 11628 consensus_queue.cc:237] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:24.366488 11659 consensus_queue.cc:1048] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:24.370898 11326 catalog_manager.cc:5697] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1115d86484ef4339933c2ba1911812f8 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "1115d86484ef4339933c2ba1911812f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:24.379379 11659 consensus_queue.cc:1048] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:24.390611 11625 consensus_queue.cc:1048] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:24.399029 11628 consensus_queue.cc:1048] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:24.416850 11628 consensus_queue.cc:1048] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:24.476894   431 ts_manager.cc:295] Set tserver state for 7a94a4038cc649babfe75a46223a8bf9 to MAINTENANCE_MODE
I20260709 13:07:24.477618   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 13:07:24.495349 11468 consensus_peers.cc:597] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:24.505149   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:24.506107   431 tablet_replica.cc:333] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9: stopping tablet replica
I20260709 13:07:24.507063   431 raft_consensus.cc:2243] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:24.507192 11658 consensus_queue.cc:1048] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:24.507699   431 pending_rounds.cc:70] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9: Trying to abort 1 pending ops.
I20260709 13:07:24.508025   431 pending_rounds.cc:77] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305635611546873856 op_type: NO_OP noop_request { }
I20260709 13:07:24.508361   431 raft_consensus.cc:2889] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:24.508725   431 raft_consensus.cc:2272] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:24.511402   431 tablet_replica.cc:333] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9: stopping tablet replica
I20260709 13:07:24.512212   431 raft_consensus.cc:2243] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:24.512851   431 raft_consensus.cc:2272] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:24.517014   431 tablet_replica.cc:333] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9: stopping tablet replica
I20260709 13:07:24.517400 11625 consensus_queue.cc:1048] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:24.519609 11624 consensus_queue.cc:1048] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:24.520351   431 raft_consensus.cc:2243] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:24.533998   431 raft_consensus.cc:2272] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:24.536834   431 tablet_replica.cc:333] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9: stopping tablet replica
I20260709 13:07:24.537672   431 raft_consensus.cc:2243] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:24.538164   431 raft_consensus.cc:2272] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:24.540406   431 tablet_replica.cc:333] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9: stopping tablet replica
I20260709 13:07:24.541121   431 raft_consensus.cc:2243] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:24.541613   431 raft_consensus.cc:2272] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:24.544135   431 tablet_replica.cc:333] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9: stopping tablet replica
I20260709 13:07:24.544708   431 raft_consensus.cc:2243] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:24.545154   431 raft_consensus.cc:2272] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:24.547246   431 tablet_replica.cc:333] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9: stopping tablet replica
I20260709 13:07:24.547806   431 raft_consensus.cc:2243] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:24.548386   431 pending_rounds.cc:70] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9: Trying to abort 1 pending ops.
I20260709 13:07:24.548623   431 pending_rounds.cc:77] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305635611151958016 op_type: NO_OP noop_request { }
I20260709 13:07:24.548913   431 raft_consensus.cc:2889] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:24.549197   431 raft_consensus.cc:2272] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:24.551326   431 tablet_replica.cc:333] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9: stopping tablet replica
I20260709 13:07:24.551867   431 raft_consensus.cc:2243] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:24.552309   431 raft_consensus.cc:2272] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:24.554697   431 tablet_replica.cc:333] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9: stopping tablet replica
I20260709 13:07:24.555243   431 raft_consensus.cc:2243] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:24.555632   431 raft_consensus.cc:2272] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:24.557565   431 tablet_replica.cc:333] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9: stopping tablet replica
I20260709 13:07:24.558405   431 raft_consensus.cc:2243] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:24.559266   431 raft_consensus.cc:2272] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:24.590693   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:07:24.611114 11376 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:07:24.722339 11468 consensus_peers.cc:597] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:24.826732 11542 consensus_peers.cc:597] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:24.941007 11468 consensus_peers.cc:597] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:24.959565 11696 consensus_queue.cc:1048] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:24.961336 11468 consensus_peers.cc:597] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:24.985217 11542 consensus_peers.cc:597] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:25.612205 11376 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:25.793597 11625 raft_consensus.cc:493] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a94a4038cc649babfe75a46223a8bf9)
I20260709 13:07:25.794593 11625 raft_consensus.cc:515] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:25.796484 11625 leader_election.cc:290] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
W20260709 13:07:25.798270 11468 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111) [suppressed 121 similar messages]
I20260709 13:07:25.798600 11574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72a07b3addad47cdad5e02a033cdd7cf" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8" is_pre_election: true
W20260709 13:07:25.801312 11468 leader_election.cc:336] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111)
I20260709 13:07:25.801789 11468 leader_election.cc:304] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970; no voters: 1115d86484ef4339933c2ba1911812f8, 7a94a4038cc649babfe75a46223a8bf9
I20260709 13:07:25.802537 11625 raft_consensus.cc:2749] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:25.808028 11696 raft_consensus.cc:493] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a94a4038cc649babfe75a46223a8bf9)
I20260709 13:07:25.808652 11696 raft_consensus.cc:515] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:25.810734 11696 leader_election.cc:290] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:25.812325 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72a07b3addad47cdad5e02a033cdd7cf" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970" is_pre_election: true
I20260709 13:07:25.813201 11505 raft_consensus.cc:2468] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 1.
I20260709 13:07:25.814886 11544 leader_election.cc:304] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 41c59a1982d3480d87bc6a0baa625970; no voters: 
I20260709 13:07:25.816054 11696 raft_consensus.cc:2804] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:07:25.816401 11542 leader_election.cc:336] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111)
I20260709 13:07:25.816515 11696 raft_consensus.cc:493] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a94a4038cc649babfe75a46223a8bf9)
I20260709 13:07:25.817368 11696 raft_consensus.cc:3060] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:25.823719 11696 raft_consensus.cc:515] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:25.825469 11696 leader_election.cc:290] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 2 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:25.826785 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72a07b3addad47cdad5e02a033cdd7cf" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
I20260709 13:07:25.827570 11505 raft_consensus.cc:3060] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:25.829696 11542 leader_election.cc:336] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111)
I20260709 13:07:25.834465 11505 raft_consensus.cc:2468] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 2.
I20260709 13:07:25.835685 11544 leader_election.cc:304] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 41c59a1982d3480d87bc6a0baa625970; no voters: 7a94a4038cc649babfe75a46223a8bf9
I20260709 13:07:25.836937 11696 raft_consensus.cc:2804] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:25.837368 11696 raft_consensus.cc:697] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 2 LEADER]: Becoming Leader. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Running, Role: LEADER
I20260709 13:07:25.838243 11696 consensus_queue.cc:237] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:25.846493 11327 catalog_manager.cc:5697] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 reported cstate change: term changed from 1 to 2, leader changed from 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193) to 1115d86484ef4339933c2ba1911812f8 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "1115d86484ef4339933c2ba1911812f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:25.945389 11625 raft_consensus.cc:493] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.946324 11625 raft_consensus.cc:515] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:25.948921 11625 leader_election.cc:290] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:25.951148 11574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a4c406d34924a419654e30483424cbf" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8" is_pre_election: true
I20260709 13:07:25.952359 11574 raft_consensus.cc:2468] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 1.
I20260709 13:07:25.954252 11468 leader_election.cc:304] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 41c59a1982d3480d87bc6a0baa625970; no voters: 
I20260709 13:07:25.955183 11714 raft_consensus.cc:2804] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:07:25.955315 11468 leader_election.cc:336] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111)
I20260709 13:07:25.955582 11714 raft_consensus.cc:493] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.956303 11714 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:25.963588 11714 raft_consensus.cc:515] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:25.965314 11714 leader_election.cc:290] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 2 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:25.967290 11574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a4c406d34924a419654e30483424cbf" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8"
I20260709 13:07:25.968015 11574 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:25.969941 11468 leader_election.cc:336] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111)
I20260709 13:07:25.974288 11574 raft_consensus.cc:2468] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 2.
I20260709 13:07:25.975543 11468 leader_election.cc:304] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 41c59a1982d3480d87bc6a0baa625970; no voters: 7a94a4038cc649babfe75a46223a8bf9
I20260709 13:07:25.976276 11714 raft_consensus.cc:2804] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:25.976796 11714 raft_consensus.cc:697] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 2 LEADER]: Becoming Leader. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Running, Role: LEADER
I20260709 13:07:25.977615 11714 consensus_queue.cc:237] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:25.985245 11327 catalog_manager.cc:5697] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 reported cstate change: term changed from 1 to 2, leader changed from 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193) to 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "41c59a1982d3480d87bc6a0baa625970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:26.019302 11696 raft_consensus.cc:493] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a94a4038cc649babfe75a46223a8bf9)
I20260709 13:07:26.019932 11696 raft_consensus.cc:515] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:26.021909 11696 leader_election.cc:290] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:26.023214 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f668526011204a8e8370e268ae661324" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970" is_pre_election: true
I20260709 13:07:26.023897 11505 raft_consensus.cc:2468] T f668526011204a8e8370e268ae661324 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 1.
I20260709 13:07:26.025041 11544 leader_election.cc:304] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 41c59a1982d3480d87bc6a0baa625970; no voters: 
I20260709 13:07:26.025908 11696 raft_consensus.cc:2804] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:26.026242 11696 raft_consensus.cc:493] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a94a4038cc649babfe75a46223a8bf9)
I20260709 13:07:26.026597 11696 raft_consensus.cc:3060] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:26.026535 11542 leader_election.cc:336] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111)
I20260709 13:07:26.033043 11696 raft_consensus.cc:515] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:26.035293 11696 leader_election.cc:290] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 2 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:26.036680 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f668526011204a8e8370e268ae661324" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
I20260709 13:07:26.037371 11505 raft_consensus.cc:3060] T f668526011204a8e8370e268ae661324 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:26.039455 11542 leader_election.cc:336] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111)
I20260709 13:07:26.044139 11505 raft_consensus.cc:2468] T f668526011204a8e8370e268ae661324 P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 2.
I20260709 13:07:26.045893 11544 leader_election.cc:304] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 41c59a1982d3480d87bc6a0baa625970; no voters: 7a94a4038cc649babfe75a46223a8bf9
I20260709 13:07:26.046640 11696 raft_consensus.cc:2804] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:26.047171 11696 raft_consensus.cc:697] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [term 2 LEADER]: Becoming Leader. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Running, Role: LEADER
I20260709 13:07:26.047988 11696 consensus_queue.cc:237] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:26.055907 11327 catalog_manager.cc:5697] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 reported cstate change: term changed from 1 to 2, leader changed from 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193) to 1115d86484ef4339933c2ba1911812f8 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "1115d86484ef4339933c2ba1911812f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:26.247469 11714 raft_consensus.cc:493] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.248024 11714 raft_consensus.cc:515] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:26.250029 11714 leader_election.cc:290] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:26.251982 11574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5f7a919c42640ea984e8ae25b9b2338" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8" is_pre_election: true
I20260709 13:07:26.252787 11574 raft_consensus.cc:2468] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 1.
I20260709 13:07:26.253859 11468 leader_election.cc:304] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 41c59a1982d3480d87bc6a0baa625970; no voters: 
I20260709 13:07:26.254734 11714 raft_consensus.cc:2804] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:07:26.254801 11468 leader_election.cc:336] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111)
I20260709 13:07:26.255313 11714 raft_consensus.cc:493] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.255750 11714 raft_consensus.cc:3060] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:26.262547 11714 raft_consensus.cc:515] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:26.264256 11714 leader_election.cc:290] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 2 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:26.266301 11574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5f7a919c42640ea984e8ae25b9b2338" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1115d86484ef4339933c2ba1911812f8"
I20260709 13:07:26.267019 11574 raft_consensus.cc:3060] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:26.268644 11468 leader_election.cc:336] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111)
I20260709 13:07:26.273376 11574 raft_consensus.cc:2468] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 2.
I20260709 13:07:26.274888 11468 leader_election.cc:304] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 41c59a1982d3480d87bc6a0baa625970; no voters: 7a94a4038cc649babfe75a46223a8bf9
I20260709 13:07:26.275589 11625 raft_consensus.cc:2804] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:26.276209 11625 raft_consensus.cc:697] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 2 LEADER]: Becoming Leader. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Running, Role: LEADER
I20260709 13:07:26.277136 11625 consensus_queue.cc:237] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:26.286136 11327 catalog_manager.cc:5697] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 reported cstate change: term changed from 1 to 2, leader changed from 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193) to 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "41c59a1982d3480d87bc6a0baa625970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:26.297032 11542 consensus_peers.cc:597] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:26.348284 11505 raft_consensus.cc:1275] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Refusing update from remote peer 1115d86484ef4339933c2ba1911812f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:26.349795 11696 consensus_queue.cc:1048] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:26.522292 11505 raft_consensus.cc:1275] T f668526011204a8e8370e268ae661324 P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Refusing update from remote peer 1115d86484ef4339933c2ba1911812f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:26.524165 11696 consensus_queue.cc:1048] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:26.541415 11542 consensus_peers.cc:597] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:26.542316 11574 raft_consensus.cc:1275] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Refusing update from remote peer 41c59a1982d3480d87bc6a0baa625970: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:07:26.543984 11625 consensus_queue.cc:1048] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:26.559196 11468 consensus_peers.cc:597] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:26.613541 11376 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:26.787192 11574 raft_consensus.cc:1275] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Refusing update from remote peer 41c59a1982d3480d87bc6a0baa625970: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:07:26.788749 11725 consensus_queue.cc:1048] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:26.809415 11468 consensus_peers.cc:597] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:26.854565 11468 consensus_peers.cc:597] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:27.141175 11542 consensus_peers.cc:597] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:27.306057 11468 consensus_peers.cc:597] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:27.383455 11468 consensus_peers.cc:597] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:27.483521 11542 consensus_peers.cc:597] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:27.615358 11376 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:07:27.711141 11468 consensus_peers.cc:597] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:28.616637 11376 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:07:28.791649 11542 consensus_peers.cc:597] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:28.824541 11542 consensus_peers.cc:597] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:29.087576 11468 consensus_peers.cc:597] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:29.090098 11468 consensus_peers.cc:597] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:29.380852 11468 consensus_peers.cc:597] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:29.558784 11542 consensus_peers.cc:597] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:29.609695   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:29.617304 11729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:29.617772 11376 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:07:29.619014 11730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:29.621131 11732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:29.622498   431 server_base.cc:1061] running on GCE node
I20260709 13:07:29.623505   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:29.623752   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:29.623955   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602449623937 us; error 0 us; skew 500 ppm
I20260709 13:07:29.624557   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:29.627528   431 webserver.cc:533] Webserver started at http://127.0.107.193:43437/ using document root <none> and password file <none>
I20260709 13:07:29.628093   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:29.628250   431 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:07:29.632325   431 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:07:29.636920 11737 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:29.638199   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:07:29.638770   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/ts-0-root
uuid: "7a94a4038cc649babfe75a46223a8bf9"
format_stamp: "Formatted at 2026-07-09 13:07:22 on dist-test-slave-mxd3"
I20260709 13:07:29.639235   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602054858684-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:29.661965   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:29.663296   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:29.664978   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:29.669296 11744 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 13:07:29.708657 11468 consensus_peers.cc:597] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:29.724954   431 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:07:29.725409   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.057s	user 0.003s	sys 0.000s
I20260709 13:07:29.725739   431 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:07:29.731019 11744 tablet_bootstrap.cc:492] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap starting.
I20260709 13:07:29.748567 11744 tablet_bootstrap.cc:492] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:29.749612 11744 tablet_bootstrap.cc:492] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap complete.
I20260709 13:07:29.750362 11744 ts_tablet_manager.cc:1403] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.008s
I20260709 13:07:29.753686 11744 raft_consensus.cc:359] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:29.754547 11744 raft_consensus.cc:740] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.755406 11744 consensus_queue.cc:260] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:29.758491 11744 ts_tablet_manager.cc:1434] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:07:29.759457 11744 tablet_bootstrap.cc:492] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap starting.
I20260709 13:07:29.774269 11744 tablet_bootstrap.cc:492] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:29.775190 11744 tablet_bootstrap.cc:492] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap complete.
I20260709 13:07:29.775925 11744 ts_tablet_manager.cc:1403] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 13:07:29.778617 11744 raft_consensus.cc:359] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:29.779397 11744 raft_consensus.cc:740] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.780154 11744 consensus_queue.cc:260] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:29.782393 11744 ts_tablet_manager.cc:1434] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:07:29.783277 11744 tablet_bootstrap.cc:492] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap starting.
I20260709 13:07:29.791252   431 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:07:29.791692   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.066s	user 0.065s	sys 0.000s
I20260709 13:07:29.799655 11744 tablet_bootstrap.cc:492] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:07:29.801275 11744 tablet_bootstrap.cc:492] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap complete.
I20260709 13:07:29.802430 11744 ts_tablet_manager.cc:1403] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.004s
I20260709 13:07:29.806152 11744 raft_consensus.cc:359] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:29.807399 11744 raft_consensus.cc:740] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.808662 11744 consensus_queue.cc:260] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:29.811756 11744 ts_tablet_manager.cc:1434] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:07:29.813104 11744 tablet_bootstrap.cc:492] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap starting.
I20260709 13:07:29.830490 11744 tablet_bootstrap.cc:492] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:29.831301 11744 tablet_bootstrap.cc:492] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap complete.
I20260709 13:07:29.831817 11744 ts_tablet_manager.cc:1403] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.002s
I20260709 13:07:29.833835 11744 raft_consensus.cc:359] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:29.834355 11744 raft_consensus.cc:740] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.835021 11744 consensus_queue.cc:260] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:29.836499 11744 ts_tablet_manager.cc:1434] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.001s
I20260709 13:07:29.837126 11744 tablet_bootstrap.cc:492] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap starting.
I20260709 13:07:29.851732 11744 tablet_bootstrap.cc:492] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:29.852468 11744 tablet_bootstrap.cc:492] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap complete.
I20260709 13:07:29.852983 11744 ts_tablet_manager.cc:1403] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 13:07:29.854763 11744 raft_consensus.cc:359] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } } peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } }
I20260709 13:07:29.855278 11744 raft_consensus.cc:740] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.855940 11744 consensus_queue.cc:260] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } } peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } }
I20260709 13:07:29.857779 11744 ts_tablet_manager.cc:1434] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:07:29.858417 11744 tablet_bootstrap.cc:492] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap starting.
I20260709 13:07:29.866451   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:40225
I20260709 13:07:29.866572 11807 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:40225 every 8 connection(s)
I20260709 13:07:29.867486 11744 tablet_bootstrap.cc:492] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:29.868181 11744 tablet_bootstrap.cc:492] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap complete.
I20260709 13:07:29.868945 11744 ts_tablet_manager.cc:1403] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9: Time spent bootstrapping tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 13:07:29.871155 11744 raft_consensus.cc:359] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:29.871451   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:29.871861 11744 raft_consensus.cc:740] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.872601 11744 consensus_queue.cc:260] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:29.874184 11744 ts_tablet_manager.cc:1434] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:07:29.875201 11744 tablet_bootstrap.cc:492] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap starting.
I20260709 13:07:29.904297 11808 heartbeater.cc:344] Connected to a master server at 127.0.107.254:37289
I20260709 13:07:29.904927 11808 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:29.905939 11808 heartbeater.cc:507] Master 127.0.107.254:37289 requested a full tablet report, sending...
I20260709 13:07:29.908521 11744 tablet_bootstrap.cc:492] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:07:29.909796 11744 tablet_bootstrap.cc:492] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap complete.
I20260709 13:07:29.910579 11744 ts_tablet_manager.cc:1403] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9: Time spent bootstrapping tablet: real 0.036s	user 0.021s	sys 0.009s
I20260709 13:07:29.914445 11744 raft_consensus.cc:359] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:29.915364 11328 ts_manager.cc:194] Re-registered known tserver with Master: 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225)
I20260709 13:07:29.915429 11744 raft_consensus.cc:740] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.916554 11744 consensus_queue.cc:260] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:29.917845 11505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34fd2daee75140e8ad3c8617bc3b5592"
dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
mode: GRACEFUL
new_leader_uuid: "1115d86484ef4339933c2ba1911812f8"
 from {username='slave'} at 127.0.0.1:39584
I20260709 13:07:29.918618 11505 raft_consensus.cc:606] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 1 LEADER]: Received request to transfer leadership to TS 1115d86484ef4339933c2ba1911812f8
I20260709 13:07:29.919109 11744 ts_tablet_manager.cc:1434] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:07:29.920076 11744 tablet_bootstrap.cc:492] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap starting.
I20260709 13:07:29.924014 11505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a4c406d34924a419654e30483424cbf"
dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
mode: GRACEFUL
new_leader_uuid: "1115d86484ef4339933c2ba1911812f8"
 from {username='slave'} at 127.0.0.1:39584
I20260709 13:07:29.924701 11505 raft_consensus.cc:606] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 2 LEADER]: Received request to transfer leadership to TS 1115d86484ef4339933c2ba1911812f8
I20260709 13:07:29.926225   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:07:29.926997   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:29.927753   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:29.928120 11328 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50222
I20260709 13:07:29.931731 11808 heartbeater.cc:499] Master 127.0.107.254:37289 was elected leader, sending a full tablet report...
I20260709 13:07:29.934810 11744 tablet_bootstrap.cc:492] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:29.935899 11744 tablet_bootstrap.cc:492] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap complete.
I20260709 13:07:29.936617 11744 ts_tablet_manager.cc:1403] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 13:07:29.939167 11744 raft_consensus.cc:359] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:29.939931 11744 raft_consensus.cc:740] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.940742 11744 consensus_queue.cc:260] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:29.943122 11744 ts_tablet_manager.cc:1434] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:07:29.944021 11744 tablet_bootstrap.cc:492] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap starting.
I20260709 13:07:29.958772 11744 tablet_bootstrap.cc:492] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:29.959589 11744 tablet_bootstrap.cc:492] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap complete.
I20260709 13:07:29.960160 11744 ts_tablet_manager.cc:1403] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 13:07:29.962092 11744 raft_consensus.cc:359] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:29.962622 11744 raft_consensus.cc:740] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.963274 11744 consensus_queue.cc:260] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:29.964867 11744 ts_tablet_manager.cc:1434] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:07:29.966019 11744 tablet_bootstrap.cc:492] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap starting.
I20260709 13:07:29.976408 11728 raft_consensus.cc:993] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970: : Instructing follower 1115d86484ef4339933c2ba1911812f8 to start an election
I20260709 13:07:29.977047 11728 raft_consensus.cc:1081] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 1 LEADER]: Signalling peer 1115d86484ef4339933c2ba1911812f8 to start an election
I20260709 13:07:29.978622 11575 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34fd2daee75140e8ad3c8617bc3b5592"
dest_uuid: "1115d86484ef4339933c2ba1911812f8"
 from {username='slave'} at 127.0.0.1:33062
I20260709 13:07:29.979110 11575 raft_consensus.cc:493] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:29.979425 11575 raft_consensus.cc:3060] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:29.981464 11744 tablet_bootstrap.cc:492] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:29.982568 11744 tablet_bootstrap.cc:492] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9: Bootstrap complete.
I20260709 13:07:29.983271 11744 ts_tablet_manager.cc:1403] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 13:07:29.985587 11744 raft_consensus.cc:359] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:29.986220 11744 raft_consensus.cc:740] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94a4038cc649babfe75a46223a8bf9, State: Initialized, Role: FOLLOWER
I20260709 13:07:29.986696 11575 raft_consensus.cc:515] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:29.986829 11744 consensus_queue.cc:260] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:29.989173 11744 ts_tablet_manager.cc:1434] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:07:29.989768 11781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34fd2daee75140e8ad3c8617bc3b5592" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a94a4038cc649babfe75a46223a8bf9"
I20260709 13:07:29.989892 11575 leader_election.cc:290] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 2 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:29.990427 11781 raft_consensus.cc:3060] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:29.991007 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34fd2daee75140e8ad3c8617bc3b5592" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
I20260709 13:07:29.991544 11505 raft_consensus.cc:3055] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:29.991870 11505 raft_consensus.cc:740] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Running, Role: LEADER
I20260709 13:07:29.992679 11505 consensus_queue.cc:260] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:29.993597 11505 raft_consensus.cc:3060] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.001226 11505 raft_consensus.cc:2468] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 2.
I20260709 13:07:30.002712 11544 leader_election.cc:304] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 41c59a1982d3480d87bc6a0baa625970; no voters: 
I20260709 13:07:30.003696 11722 raft_consensus.cc:2804] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:30.004560 11781 raft_consensus.cc:2468] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 2.
I20260709 13:07:30.012914 11722 raft_consensus.cc:697] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 2 LEADER]: Becoming Leader. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Running, Role: LEADER
I20260709 13:07:30.014094 11722 consensus_queue.cc:237] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:30.065531 11328 catalog_manager.cc:5697] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 reported cstate change: term changed from 1 to 2, leader changed from 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194) to 1115d86484ef4339933c2ba1911812f8 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "1115d86484ef4339933c2ba1911812f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:30.145588 11781 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.155084 11781 pending_rounds.cc:85] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9: Aborting all ops after (but not including) 0
I20260709 13:07:30.155383 11781 pending_rounds.cc:107] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:07:30.155840 11781 raft_consensus.cc:2889] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:30.156066 11820 raft_consensus.cc:993] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970: : Instructing follower 1115d86484ef4339933c2ba1911812f8 to start an election
I20260709 13:07:30.157022 11728 raft_consensus.cc:1081] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 2 LEADER]: Signalling peer 1115d86484ef4339933c2ba1911812f8 to start an election
I20260709 13:07:30.159303 11575 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a4c406d34924a419654e30483424cbf"
dest_uuid: "1115d86484ef4339933c2ba1911812f8"
 from {username='slave'} at 127.0.0.1:33062
I20260709 13:07:30.160234 11575 raft_consensus.cc:493] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:30.160692 11575 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:30.168385 11575 raft_consensus.cc:515] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:30.170511 11575 leader_election.cc:290] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 3 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:30.171662 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a4c406d34924a419654e30483424cbf" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
I20260709 13:07:30.172323 11781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a4c406d34924a419654e30483424cbf" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "7a94a4038cc649babfe75a46223a8bf9"
I20260709 13:07:30.172344 11504 raft_consensus.cc:3055] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:30.172931 11781 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:30.172945 11504 raft_consensus.cc:740] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Running, Role: LEADER
I20260709 13:07:30.174340 11504 consensus_queue.cc:260] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:30.175673 11504 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:30.178982 11781 raft_consensus.cc:2468] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 3.
I20260709 13:07:30.180498 11542 leader_election.cc:304] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:30.181618 11823 raft_consensus.cc:2804] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:30.182246 11823 raft_consensus.cc:697] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 3 LEADER]: Becoming Leader. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Running, Role: LEADER
I20260709 13:07:30.183491 11823 consensus_queue.cc:237] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:30.183933 11504 raft_consensus.cc:2468] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 3.
I20260709 13:07:30.192890 11328 catalog_manager.cc:5697] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 reported cstate change: term changed from 2 to 3, leader changed from 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194) to 1115d86484ef4339933c2ba1911812f8 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "1115d86484ef4339933c2ba1911812f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:30.233489 11781 raft_consensus.cc:3060] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.238622 11781 pending_rounds.cc:85] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9: Aborting all ops after (but not including) 0
I20260709 13:07:30.238833 11781 pending_rounds.cc:107] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:07:30.239104 11781 raft_consensus.cc:2889] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:30.270725 11781 raft_consensus.cc:3060] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.356020 11781 raft_consensus.cc:3060] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.474826 11504 raft_consensus.cc:1275] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Refusing update from remote peer 1115d86484ef4339933c2ba1911812f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:30.476770 11834 consensus_queue.cc:1048] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:30.494181 11781 raft_consensus.cc:1275] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: Refusing update from remote peer 1115d86484ef4339933c2ba1911812f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:30.496182 11834 consensus_queue.cc:1048] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:30.619282 11376 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:30.758833 11780 raft_consensus.cc:1275] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 3 FOLLOWER]: Refusing update from remote peer 1115d86484ef4339933c2ba1911812f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:30.760528 11834 consensus_queue.cc:1048] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:30.765825 11504 raft_consensus.cc:1275] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 3 FOLLOWER]: Refusing update from remote peer 1115d86484ef4339933c2ba1911812f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:30.768812 11823 consensus_queue.cc:1048] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:30.936609   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:30.958491 11575 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c510e78737545ebb0aa0366f008c5bd"
dest_uuid: "1115d86484ef4339933c2ba1911812f8"
mode: GRACEFUL
new_leader_uuid: "41c59a1982d3480d87bc6a0baa625970"
 from {username='slave'} at 127.0.0.1:43730
I20260709 13:07:30.959111 11575 raft_consensus.cc:606] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [term 1 LEADER]: Received request to transfer leadership to TS 41c59a1982d3480d87bc6a0baa625970
I20260709 13:07:30.961892 11575 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a4c406d34924a419654e30483424cbf"
dest_uuid: "1115d86484ef4339933c2ba1911812f8"
mode: GRACEFUL
new_leader_uuid: "41c59a1982d3480d87bc6a0baa625970"
 from {username='slave'} at 127.0.0.1:43730
I20260709 13:07:30.962395 11575 raft_consensus.cc:606] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 3 LEADER]: Received request to transfer leadership to TS 41c59a1982d3480d87bc6a0baa625970
I20260709 13:07:30.964912 11505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ddc5f5fecb0403386dd61f18d051b97"
dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
mode: GRACEFUL
new_leader_uuid: "1115d86484ef4339933c2ba1911812f8"
 from {username='slave'} at 127.0.0.1:39584
I20260709 13:07:30.965312 11505 raft_consensus.cc:606] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 1 LEADER]: Received request to transfer leadership to TS 1115d86484ef4339933c2ba1911812f8
I20260709 13:07:30.966952   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:07:30.967643   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:30.968397   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:31.018802 11839 raft_consensus.cc:993] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970: : Instructing follower 1115d86484ef4339933c2ba1911812f8 to start an election
I20260709 13:07:31.019435 11820 raft_consensus.cc:1081] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 1 LEADER]: Signalling peer 1115d86484ef4339933c2ba1911812f8 to start an election
I20260709 13:07:31.020920 11575 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4ddc5f5fecb0403386dd61f18d051b97"
dest_uuid: "1115d86484ef4339933c2ba1911812f8"
 from {username='slave'} at 127.0.0.1:33062
I20260709 13:07:31.021405 11575 raft_consensus.cc:493] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:31.021720 11575 raft_consensus.cc:3060] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:31.027532 11575 raft_consensus.cc:515] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:31.029327 11575 leader_election.cc:290] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 2 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:31.030263 11781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ddc5f5fecb0403386dd61f18d051b97" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a94a4038cc649babfe75a46223a8bf9"
I20260709 13:07:31.030534 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ddc5f5fecb0403386dd61f18d051b97" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
I20260709 13:07:31.030874 11781 raft_consensus.cc:3060] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:31.031106 11505 raft_consensus.cc:3055] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:31.031469 11505 raft_consensus.cc:740] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Running, Role: LEADER
I20260709 13:07:31.032212 11505 consensus_queue.cc:260] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:31.033152 11505 raft_consensus.cc:3060] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:31.036921 11781 raft_consensus.cc:2468] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 2.
I20260709 13:07:31.038132 11542 leader_election.cc:304] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:31.039072 11722 raft_consensus.cc:2804] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:31.039497 11722 raft_consensus.cc:697] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 2 LEADER]: Becoming Leader. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Running, Role: LEADER
I20260709 13:07:31.039527 11505 raft_consensus.cc:2468] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 2.
I20260709 13:07:31.040692 11722 consensus_queue.cc:237] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:31.048231 11327 catalog_manager.cc:5697] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 reported cstate change: term changed from 1 to 2, leader changed from 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194) to 1115d86484ef4339933c2ba1911812f8 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "1115d86484ef4339933c2ba1911812f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:31.164232 11823 raft_consensus.cc:993] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8: : Instructing follower 41c59a1982d3480d87bc6a0baa625970 to start an election
I20260709 13:07:31.164765 11722 raft_consensus.cc:1081] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 3 LEADER]: Signalling peer 41c59a1982d3480d87bc6a0baa625970 to start an election
I20260709 13:07:31.166424 11505 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a4c406d34924a419654e30483424cbf"
dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
 from {username='slave'} at 127.0.0.1:51190
I20260709 13:07:31.166917 11505 raft_consensus.cc:493] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:31.167273 11505 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:31.172684 11505 raft_consensus.cc:515] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:31.174089 11505 leader_election.cc:290] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 4 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:31.175109 11781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a4c406d34924a419654e30483424cbf" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "7a94a4038cc649babfe75a46223a8bf9"
I20260709 13:07:31.175321 11575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a4c406d34924a419654e30483424cbf" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "1115d86484ef4339933c2ba1911812f8"
I20260709 13:07:31.175681 11781 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:31.175972 11575 raft_consensus.cc:3055] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:31.176283 11575 raft_consensus.cc:740] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Running, Role: LEADER
I20260709 13:07:31.176939 11575 consensus_queue.cc:260] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:31.177901 11575 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:31.180780 11781 raft_consensus.cc:2468] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 4.
I20260709 13:07:31.182040 11468 leader_election.cc:304] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:31.182749 11820 raft_consensus.cc:2804] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:31.182937 11575 raft_consensus.cc:2468] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 4.
I20260709 13:07:31.183259 11820 raft_consensus.cc:697] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 4 LEADER]: Becoming Leader. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Running, Role: LEADER
I20260709 13:07:31.184161 11820 consensus_queue.cc:237] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:31.190376 11327 catalog_manager.cc:5697] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 reported cstate change: term changed from 3 to 4, leader changed from 1115d86484ef4339933c2ba1911812f8 (127.0.107.195) to 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "41c59a1982d3480d87bc6a0baa625970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:31.389477 11823 raft_consensus.cc:993] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8: : Instructing follower 41c59a1982d3480d87bc6a0baa625970 to start an election
I20260709 13:07:31.390046 11722 raft_consensus.cc:1081] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [term 1 LEADER]: Signalling peer 41c59a1982d3480d87bc6a0baa625970 to start an election
I20260709 13:07:31.391589 11505 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3c510e78737545ebb0aa0366f008c5bd"
dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
 from {username='slave'} at 127.0.0.1:51190
I20260709 13:07:31.392158 11505 raft_consensus.cc:493] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:31.392482 11505 raft_consensus.cc:3060] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:31.397934 11505 raft_consensus.cc:515] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } } peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } }
I20260709 13:07:31.399453 11505 leader_election.cc:290] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 2 election: Requested vote from peers 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537), 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225)
I20260709 13:07:31.400444 11575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c510e78737545ebb0aa0366f008c5bd" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1115d86484ef4339933c2ba1911812f8"
I20260709 13:07:31.400671 11780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c510e78737545ebb0aa0366f008c5bd" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a94a4038cc649babfe75a46223a8bf9"
I20260709 13:07:31.401086 11575 raft_consensus.cc:3055] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:31.401316 11780 raft_consensus.cc:3060] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:31.401532 11575 raft_consensus.cc:740] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Running, Role: LEADER
I20260709 13:07:31.402697 11575 consensus_queue.cc:260] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } } peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } }
I20260709 13:07:31.403854 11575 raft_consensus.cc:3060] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:31.408358 11780 raft_consensus.cc:2468] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 2.
I20260709 13:07:31.409674 11468 leader_election.cc:304] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:31.410246 11575 raft_consensus.cc:2468] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 2.
I20260709 13:07:31.410424 11820 raft_consensus.cc:2804] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:31.411013 11820 raft_consensus.cc:697] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [term 2 LEADER]: Becoming Leader. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Running, Role: LEADER
I20260709 13:07:31.412236 11820 consensus_queue.cc:237] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } } peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } }
I20260709 13:07:31.420114 11327 catalog_manager.cc:5697] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 reported cstate change: term changed from 1 to 2, leader changed from 1115d86484ef4339933c2ba1911812f8 (127.0.107.195) to 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "41c59a1982d3480d87bc6a0baa625970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:31.504128 11780 raft_consensus.cc:1275] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: Refusing update from remote peer 1115d86484ef4339933c2ba1911812f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:31.505551 11722 consensus_queue.cc:1048] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:31.516857 11505 raft_consensus.cc:1275] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Refusing update from remote peer 1115d86484ef4339933c2ba1911812f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:31.518364 11722 consensus_queue.cc:1048] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:31.620461 11376 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:31.671351 11575 raft_consensus.cc:1275] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 4 FOLLOWER]: Refusing update from remote peer 41c59a1982d3480d87bc6a0baa625970: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:31.673312 11820 consensus_queue.cc:1048] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:31.682044 11780 raft_consensus.cc:1275] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 4 FOLLOWER]: Refusing update from remote peer 41c59a1982d3480d87bc6a0baa625970: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:31.684202 11820 consensus_queue.cc:1048] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:31.808679 11575 raft_consensus.cc:1275] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Refusing update from remote peer 41c59a1982d3480d87bc6a0baa625970: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:31.810233 11820 consensus_queue.cc:1048] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:31.823184 11780 raft_consensus.cc:1275] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: Refusing update from remote peer 41c59a1982d3480d87bc6a0baa625970: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:31.825378 11856 consensus_queue.cc:1048] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:31.975234   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:31.984287 11505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "01c031d7def6441ea735da3573686b3e"
dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
mode: GRACEFUL
new_leader_uuid: "1115d86484ef4339933c2ba1911812f8"
 from {username='slave'} at 127.0.0.1:39584
I20260709 13:07:31.985025 11505 raft_consensus.cc:606] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [term 1 LEADER]: Received request to transfer leadership to TS 1115d86484ef4339933c2ba1911812f8
I20260709 13:07:31.988242 11505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a4c406d34924a419654e30483424cbf"
dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
mode: GRACEFUL
new_leader_uuid: "1115d86484ef4339933c2ba1911812f8"
 from {username='slave'} at 127.0.0.1:39584
I20260709 13:07:31.988857 11505 raft_consensus.cc:606] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 4 LEADER]: Received request to transfer leadership to TS 1115d86484ef4339933c2ba1911812f8
I20260709 13:07:31.992033 11575 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72a07b3addad47cdad5e02a033cdd7cf"
dest_uuid: "1115d86484ef4339933c2ba1911812f8"
mode: GRACEFUL
new_leader_uuid: "41c59a1982d3480d87bc6a0baa625970"
 from {username='slave'} at 127.0.0.1:43730
I20260709 13:07:31.992746 11575 raft_consensus.cc:606] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 2 LEADER]: Received request to transfer leadership to TS 41c59a1982d3480d87bc6a0baa625970
I20260709 13:07:31.994352   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:07:31.994976   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:31.995757   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:32.212309 11820 raft_consensus.cc:993] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970: : Instructing follower 1115d86484ef4339933c2ba1911812f8 to start an election
I20260709 13:07:32.212862 11856 raft_consensus.cc:1081] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 4 LEADER]: Signalling peer 1115d86484ef4339933c2ba1911812f8 to start an election
I20260709 13:07:32.214705 11575 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a4c406d34924a419654e30483424cbf"
dest_uuid: "1115d86484ef4339933c2ba1911812f8"
 from {username='slave'} at 127.0.0.1:33062
I20260709 13:07:32.215238 11575 raft_consensus.cc:493] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:32.215536 11575 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:32.222388 11575 raft_consensus.cc:515] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:32.224318 11575 leader_election.cc:290] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 5 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:32.225662 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a4c406d34924a419654e30483424cbf" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
I20260709 13:07:32.225668 11780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a4c406d34924a419654e30483424cbf" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "7a94a4038cc649babfe75a46223a8bf9"
I20260709 13:07:32.226684 11505 raft_consensus.cc:3055] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:32.226733 11780 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:32.227195 11505 raft_consensus.cc:740] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Running, Role: LEADER
I20260709 13:07:32.228142 11505 consensus_queue.cc:260] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:32.229199 11505 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:32.233546 11780 raft_consensus.cc:2468] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 5.
I20260709 13:07:32.235252 11505 raft_consensus.cc:2468] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 5.
I20260709 13:07:32.235318 11542 leader_election.cc:304] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:32.236563 11859 raft_consensus.cc:2804] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:32.237114 11859 raft_consensus.cc:697] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 5 LEADER]: Becoming Leader. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Running, Role: LEADER
I20260709 13:07:32.237977 11859 consensus_queue.cc:237] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:32.245329 11326 catalog_manager.cc:5697] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 reported cstate change: term changed from 4 to 5, leader changed from 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194) to 1115d86484ef4339933c2ba1911812f8 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "1115d86484ef4339933c2ba1911812f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.301322 11820 raft_consensus.cc:993] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970: : Instructing follower 1115d86484ef4339933c2ba1911812f8 to start an election
I20260709 13:07:32.301981 11856 raft_consensus.cc:1081] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [term 1 LEADER]: Signalling peer 1115d86484ef4339933c2ba1911812f8 to start an election
I20260709 13:07:32.303931 11575 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "01c031d7def6441ea735da3573686b3e"
dest_uuid: "1115d86484ef4339933c2ba1911812f8"
 from {username='slave'} at 127.0.0.1:33062
I20260709 13:07:32.304843 11575 raft_consensus.cc:493] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:32.305215 11575 raft_consensus.cc:3060] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:32.311425 11575 raft_consensus.cc:515] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:32.313514 11575 leader_election.cc:290] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 2 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:32.314652 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c031d7def6441ea735da3573686b3e" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
I20260709 13:07:32.314633 11780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c031d7def6441ea735da3573686b3e" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a94a4038cc649babfe75a46223a8bf9"
I20260709 13:07:32.315274 11505 raft_consensus.cc:3055] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:32.315488 11780 raft_consensus.cc:3060] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:32.315605 11505 raft_consensus.cc:740] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Running, Role: LEADER
I20260709 13:07:32.317087 11505 consensus_queue.cc:260] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:32.318447 11505 raft_consensus.cc:3060] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:32.323308 11780 raft_consensus.cc:2468] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 2.
I20260709 13:07:32.324617 11542 leader_election.cc:304] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:32.325212 11505 raft_consensus.cc:2468] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 2.
I20260709 13:07:32.325704 11859 raft_consensus.cc:2804] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:32.326256 11859 raft_consensus.cc:697] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8 [term 2 LEADER]: Becoming Leader. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Running, Role: LEADER
I20260709 13:07:32.327464 11859 consensus_queue.cc:237] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:32.336125 11326 catalog_manager.cc:5697] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8 reported cstate change: term changed from 1 to 2, leader changed from 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194) to 1115d86484ef4339933c2ba1911812f8 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "1115d86484ef4339933c2ba1911812f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.337919 11834 raft_consensus.cc:993] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8: : Instructing follower 41c59a1982d3480d87bc6a0baa625970 to start an election
I20260709 13:07:32.338564 11859 raft_consensus.cc:1081] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 2 LEADER]: Signalling peer 41c59a1982d3480d87bc6a0baa625970 to start an election
I20260709 13:07:32.340602 11505 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "72a07b3addad47cdad5e02a033cdd7cf"
dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
 from {username='slave'} at 127.0.0.1:51190
I20260709 13:07:32.341259 11505 raft_consensus.cc:493] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:32.341588 11505 raft_consensus.cc:3060] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:32.350445 11505 raft_consensus.cc:515] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:32.353461 11505 leader_election.cc:290] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 3 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:32.355363 11780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72a07b3addad47cdad5e02a033cdd7cf" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7a94a4038cc649babfe75a46223a8bf9"
I20260709 13:07:32.356124 11780 raft_consensus.cc:3060] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:32.356161 11575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72a07b3addad47cdad5e02a033cdd7cf" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1115d86484ef4339933c2ba1911812f8"
I20260709 13:07:32.356828 11575 raft_consensus.cc:3055] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:32.357154 11575 raft_consensus.cc:740] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Running, Role: LEADER
I20260709 13:07:32.358065 11575 consensus_queue.cc:260] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:32.359790 11575 raft_consensus.cc:3060] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:32.364535 11780 raft_consensus.cc:2468] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 3.
I20260709 13:07:32.365996 11468 leader_election.cc:304] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:32.366750 11820 raft_consensus.cc:2804] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:32.367198 11820 raft_consensus.cc:697] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 3 LEADER]: Becoming Leader. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Running, Role: LEADER
I20260709 13:07:32.367439 11575 raft_consensus.cc:2468] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 3.
I20260709 13:07:32.368270 11820 consensus_queue.cc:237] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:32.376312 11326 catalog_manager.cc:5697] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 reported cstate change: term changed from 2 to 3, leader changed from 1115d86484ef4339933c2ba1911812f8 (127.0.107.195) to 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "41c59a1982d3480d87bc6a0baa625970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:32.621668 11376 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:32.727278 11505 raft_consensus.cc:1275] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 5 FOLLOWER]: Refusing update from remote peer 1115d86484ef4339933c2ba1911812f8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:07:32.728565 11859 consensus_queue.cc:1048] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:32.742163 11780 raft_consensus.cc:1275] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 5 FOLLOWER]: Refusing update from remote peer 1115d86484ef4339933c2ba1911812f8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:07:32.743592 11868 consensus_queue.cc:1048] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:32.762750 11780 raft_consensus.cc:1275] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: Refusing update from remote peer 1115d86484ef4339933c2ba1911812f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:32.764220 11868 consensus_queue.cc:1048] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:32.776237 11505 raft_consensus.cc:1275] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Refusing update from remote peer 1115d86484ef4339933c2ba1911812f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:32.777999 11868 consensus_queue.cc:1048] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:32.785567 11575 raft_consensus.cc:1275] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 3 FOLLOWER]: Refusing update from remote peer 41c59a1982d3480d87bc6a0baa625970: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:32.787521 11820 consensus_queue.cc:1048] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:32.800266 11780 raft_consensus.cc:1275] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 3 FOLLOWER]: Refusing update from remote peer 41c59a1982d3480d87bc6a0baa625970: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:32.802386 11856 consensus_queue.cc:1048] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:33.004443   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:33.011814 11575 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34fd2daee75140e8ad3c8617bc3b5592"
dest_uuid: "1115d86484ef4339933c2ba1911812f8"
mode: GRACEFUL
new_leader_uuid: "41c59a1982d3480d87bc6a0baa625970"
 from {username='slave'} at 127.0.0.1:43730
I20260709 13:07:33.012267 11575 raft_consensus.cc:606] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 2 LEADER]: Received request to transfer leadership to TS 41c59a1982d3480d87bc6a0baa625970
I20260709 13:07:33.014770 11575 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a4c406d34924a419654e30483424cbf"
dest_uuid: "1115d86484ef4339933c2ba1911812f8"
mode: GRACEFUL
new_leader_uuid: "41c59a1982d3480d87bc6a0baa625970"
 from {username='slave'} at 127.0.0.1:43730
I20260709 13:07:33.015239 11575 raft_consensus.cc:606] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 5 LEADER]: Received request to transfer leadership to TS 41c59a1982d3480d87bc6a0baa625970
I20260709 13:07:33.017783 11505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5f7a919c42640ea984e8ae25b9b2338"
dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
mode: GRACEFUL
new_leader_uuid: "1115d86484ef4339933c2ba1911812f8"
 from {username='slave'} at 127.0.0.1:39584
I20260709 13:07:33.018244 11505 raft_consensus.cc:606] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 2 LEADER]: Received request to transfer leadership to TS 1115d86484ef4339933c2ba1911812f8
I20260709 13:07:33.019444   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:07:33.019894   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:33.020430   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:33.221290 11834 raft_consensus.cc:993] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8: : Instructing follower 41c59a1982d3480d87bc6a0baa625970 to start an election
I20260709 13:07:33.221873 11868 raft_consensus.cc:1081] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 2 LEADER]: Signalling peer 41c59a1982d3480d87bc6a0baa625970 to start an election
I20260709 13:07:33.223400 11505 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34fd2daee75140e8ad3c8617bc3b5592"
dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
 from {username='slave'} at 127.0.0.1:51190
I20260709 13:07:33.223914 11505 raft_consensus.cc:493] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:33.224313 11505 raft_consensus.cc:3060] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:33.227425 11834 raft_consensus.cc:993] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8: : Instructing follower 41c59a1982d3480d87bc6a0baa625970 to start an election
I20260709 13:07:33.227946 11834 raft_consensus.cc:1081] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 5 LEADER]: Signalling peer 41c59a1982d3480d87bc6a0baa625970 to start an election
I20260709 13:07:33.229468 11504 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a4c406d34924a419654e30483424cbf"
dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
 from {username='slave'} at 127.0.0.1:51190
I20260709 13:07:33.230014 11504 raft_consensus.cc:493] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:33.230369 11504 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:33.230623 11505 raft_consensus.cc:515] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:33.232692 11505 leader_election.cc:290] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 3 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:33.233377 11780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34fd2daee75140e8ad3c8617bc3b5592" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7a94a4038cc649babfe75a46223a8bf9"
I20260709 13:07:33.233985 11780 raft_consensus.cc:3060] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:33.234198 11575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34fd2daee75140e8ad3c8617bc3b5592" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1115d86484ef4339933c2ba1911812f8"
I20260709 13:07:33.234781 11575 raft_consensus.cc:3055] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:33.235062 11575 raft_consensus.cc:740] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Running, Role: LEADER
I20260709 13:07:33.235900 11575 consensus_queue.cc:260] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:33.237078 11575 raft_consensus.cc:3060] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:33.237697 11504 raft_consensus.cc:515] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:33.239547 11780 raft_consensus.cc:2468] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 3.
I20260709 13:07:33.239758 11504 leader_election.cc:290] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 6 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:33.240516 11781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a4c406d34924a419654e30483424cbf" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "7a94a4038cc649babfe75a46223a8bf9"
I20260709 13:07:33.241117 11781 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:33.241359 11468 leader_election.cc:304] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:33.241770 11574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a4c406d34924a419654e30483424cbf" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "1115d86484ef4339933c2ba1911812f8"
I20260709 13:07:33.242687 11574 raft_consensus.cc:3055] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:33.243021 11574 raft_consensus.cc:740] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Running, Role: LEADER
I20260709 13:07:33.242705 11856 raft_consensus.cc:2804] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:33.243629 11856 raft_consensus.cc:697] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 3 LEADER]: Becoming Leader. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Running, Role: LEADER
I20260709 13:07:33.243888 11574 consensus_queue.cc:260] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:33.244436 11856 consensus_queue.cc:237] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:33.244930 11575 raft_consensus.cc:2468] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 3.
I20260709 13:07:33.245306 11574 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:33.248622 11781 raft_consensus.cc:2468] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 6.
I20260709 13:07:33.249892 11468 leader_election.cc:304] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:33.250584 11856 raft_consensus.cc:2804] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:33.251308 11856 raft_consensus.cc:697] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 6 LEADER]: Becoming Leader. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Running, Role: LEADER
I20260709 13:07:33.252130 11856 consensus_queue.cc:237] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:33.252485 11327 catalog_manager.cc:5697] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 reported cstate change: term changed from 2 to 3, leader changed from 1115d86484ef4339933c2ba1911812f8 (127.0.107.195) to 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "41c59a1982d3480d87bc6a0baa625970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:33.253145 11574 raft_consensus.cc:2468] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 6.
I20260709 13:07:33.266337 11327 catalog_manager.cc:5697] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 reported cstate change: term changed from 5 to 6, leader changed from 1115d86484ef4339933c2ba1911812f8 (127.0.107.195) to 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194). New cstate: current_term: 6 leader_uuid: "41c59a1982d3480d87bc6a0baa625970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:33.282763 11820 raft_consensus.cc:993] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970: : Instructing follower 1115d86484ef4339933c2ba1911812f8 to start an election
I20260709 13:07:33.283284 11856 raft_consensus.cc:1081] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 2 LEADER]: Signalling peer 1115d86484ef4339933c2ba1911812f8 to start an election
I20260709 13:07:33.284701 11574 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5f7a919c42640ea984e8ae25b9b2338"
dest_uuid: "1115d86484ef4339933c2ba1911812f8"
 from {username='slave'} at 127.0.0.1:33062
I20260709 13:07:33.285142 11574 raft_consensus.cc:493] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:33.285386 11574 raft_consensus.cc:3060] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:33.290269 11574 raft_consensus.cc:515] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:33.291819 11574 leader_election.cc:290] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 3 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:33.292783 11781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5f7a919c42640ea984e8ae25b9b2338" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "7a94a4038cc649babfe75a46223a8bf9"
I20260709 13:07:33.292799 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5f7a919c42640ea984e8ae25b9b2338" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
I20260709 13:07:33.293483 11781 raft_consensus.cc:3060] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:33.293761 11504 raft_consensus.cc:3055] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:33.294142 11504 raft_consensus.cc:740] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Running, Role: LEADER
I20260709 13:07:33.294848 11504 consensus_queue.cc:260] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:33.295809 11504 raft_consensus.cc:3060] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:33.299031 11781 raft_consensus.cc:2468] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 3.
I20260709 13:07:33.300140 11542 leader_election.cc:304] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:33.300807 11834 raft_consensus.cc:2804] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:33.301307 11834 raft_consensus.cc:697] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [term 3 LEADER]: Becoming Leader. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Running, Role: LEADER
I20260709 13:07:33.302237 11504 raft_consensus.cc:2468] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 3.
I20260709 13:07:33.302335 11834 consensus_queue.cc:237] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:33.310194 11327 catalog_manager.cc:5697] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 reported cstate change: term changed from 2 to 3, leader changed from 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194) to 1115d86484ef4339933c2ba1911812f8 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "1115d86484ef4339933c2ba1911812f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:33.622931 11376 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:33.674861 11574 raft_consensus.cc:1275] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 3 FOLLOWER]: Refusing update from remote peer 41c59a1982d3480d87bc6a0baa625970: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:33.676247 11856 consensus_queue.cc:1048] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:33.687839 11780 raft_consensus.cc:1275] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 6 FOLLOWER]: Refusing update from remote peer 41c59a1982d3480d87bc6a0baa625970: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:07:33.689316 11856 consensus_queue.cc:1048] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:33.690306 11781 raft_consensus.cc:1275] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 3 FOLLOWER]: Refusing update from remote peer 41c59a1982d3480d87bc6a0baa625970: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:33.692099 11856 consensus_queue.cc:1048] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:33.701979 11574 raft_consensus.cc:1275] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 6 FOLLOWER]: Refusing update from remote peer 41c59a1982d3480d87bc6a0baa625970: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:07:33.706405 11881 consensus_queue.cc:1048] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:33.802196 11504 raft_consensus.cc:1275] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 3 FOLLOWER]: Refusing update from remote peer 1115d86484ef4339933c2ba1911812f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:33.803623 11834 consensus_queue.cc:1048] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:33.816226 11780 raft_consensus.cc:1275] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 3 FOLLOWER]: Refusing update from remote peer 1115d86484ef4339933c2ba1911812f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:33.820032 11890 consensus_queue.cc:1048] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:34.026736   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:34.035120 11504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34fd2daee75140e8ad3c8617bc3b5592"
dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
mode: GRACEFUL
new_leader_uuid: "1115d86484ef4339933c2ba1911812f8"
 from {username='slave'} at 127.0.0.1:39584
I20260709 13:07:34.035660 11504 raft_consensus.cc:606] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 3 LEADER]: Received request to transfer leadership to TS 1115d86484ef4339933c2ba1911812f8
I20260709 13:07:34.038007 11504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a4c406d34924a419654e30483424cbf"
dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
mode: GRACEFUL
new_leader_uuid: "1115d86484ef4339933c2ba1911812f8"
 from {username='slave'} at 127.0.0.1:39584
I20260709 13:07:34.038488 11504 raft_consensus.cc:606] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 6 LEADER]: Received request to transfer leadership to TS 1115d86484ef4339933c2ba1911812f8
I20260709 13:07:34.040889 11574 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ddc5f5fecb0403386dd61f18d051b97"
dest_uuid: "1115d86484ef4339933c2ba1911812f8"
mode: GRACEFUL
new_leader_uuid: "41c59a1982d3480d87bc6a0baa625970"
 from {username='slave'} at 127.0.0.1:43730
I20260709 13:07:34.041422 11574 raft_consensus.cc:606] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 2 LEADER]: Received request to transfer leadership to TS 41c59a1982d3480d87bc6a0baa625970
I20260709 13:07:34.042551   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:07:34.043109   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:34.043702   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:34.088967 11820 raft_consensus.cc:993] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970: : Instructing follower 1115d86484ef4339933c2ba1911812f8 to start an election
I20260709 13:07:34.089522 11881 raft_consensus.cc:1081] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 6 LEADER]: Signalling peer 1115d86484ef4339933c2ba1911812f8 to start an election
I20260709 13:07:34.090984 11574 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a4c406d34924a419654e30483424cbf"
dest_uuid: "1115d86484ef4339933c2ba1911812f8"
 from {username='slave'} at 127.0.0.1:33062
I20260709 13:07:34.091426 11574 raft_consensus.cc:493] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:34.091699 11574 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:34.096665 11574 raft_consensus.cc:515] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:34.098258 11574 leader_election.cc:290] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 7 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:34.099120 11504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a4c406d34924a419654e30483424cbf" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
I20260709 13:07:34.099093 11780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a4c406d34924a419654e30483424cbf" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "7a94a4038cc649babfe75a46223a8bf9"
I20260709 13:07:34.099946 11504 raft_consensus.cc:3055] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:34.100124 11780 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:34.100313 11504 raft_consensus.cc:740] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Running, Role: LEADER
I20260709 13:07:34.101104 11504 consensus_queue.cc:260] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:34.102216 11504 raft_consensus.cc:3060] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:34.106179 11780 raft_consensus.cc:2468] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 7.
I20260709 13:07:34.107278 11542 leader_election.cc:304] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:34.107458 11504 raft_consensus.cc:2468] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 7.
I20260709 13:07:34.108129 11889 raft_consensus.cc:2804] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:34.108719 11889 raft_consensus.cc:697] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 7 LEADER]: Becoming Leader. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Running, Role: LEADER
I20260709 13:07:34.109491 11889 consensus_queue.cc:237] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:34.115393 11326 catalog_manager.cc:5697] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 reported cstate change: term changed from 6 to 7, leader changed from 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194) to 1115d86484ef4339933c2ba1911812f8 (127.0.107.195). New cstate: current_term: 7 leader_uuid: "1115d86484ef4339933c2ba1911812f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:34.180436 11820 raft_consensus.cc:993] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970: : Instructing follower 1115d86484ef4339933c2ba1911812f8 to start an election
I20260709 13:07:34.180982 11820 raft_consensus.cc:1081] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 3 LEADER]: Signalling peer 1115d86484ef4339933c2ba1911812f8 to start an election
I20260709 13:07:34.182411 11574 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34fd2daee75140e8ad3c8617bc3b5592"
dest_uuid: "1115d86484ef4339933c2ba1911812f8"
 from {username='slave'} at 127.0.0.1:33062
I20260709 13:07:34.182858 11574 raft_consensus.cc:493] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:34.183121 11574 raft_consensus.cc:3060] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:34.187597 11574 raft_consensus.cc:515] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:34.189285 11574 leader_election.cc:290] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 4 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293)
I20260709 13:07:34.189930 11780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34fd2daee75140e8ad3c8617bc3b5592" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7a94a4038cc649babfe75a46223a8bf9"
I20260709 13:07:34.190269 11505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34fd2daee75140e8ad3c8617bc3b5592" candidate_uuid: "1115d86484ef4339933c2ba1911812f8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
I20260709 13:07:34.190558 11780 raft_consensus.cc:3060] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:34.190784 11505 raft_consensus.cc:3055] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:34.191107 11505 raft_consensus.cc:740] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Running, Role: LEADER
I20260709 13:07:34.191836 11505 consensus_queue.cc:260] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:34.192983 11505 raft_consensus.cc:3060] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:34.197508 11780 raft_consensus.cc:2468] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 4.
I20260709 13:07:34.198684 11542 leader_election.cc:304] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1115d86484ef4339933c2ba1911812f8, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:34.199353 11505 raft_consensus.cc:2468] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1115d86484ef4339933c2ba1911812f8 in term 4.
I20260709 13:07:34.199416 11889 raft_consensus.cc:2804] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:34.200165 11889 raft_consensus.cc:697] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 4 LEADER]: Becoming Leader. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Running, Role: LEADER
I20260709 13:07:34.201007 11889 consensus_queue.cc:237] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:34.207367 11326 catalog_manager.cc:5697] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 reported cstate change: term changed from 3 to 4, leader changed from 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194) to 1115d86484ef4339933c2ba1911812f8 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "1115d86484ef4339933c2ba1911812f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:34.403103 11895 raft_consensus.cc:993] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8: : Instructing follower 41c59a1982d3480d87bc6a0baa625970 to start an election
I20260709 13:07:34.403563 11834 raft_consensus.cc:1081] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 2 LEADER]: Signalling peer 41c59a1982d3480d87bc6a0baa625970 to start an election
I20260709 13:07:34.404989 11505 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4ddc5f5fecb0403386dd61f18d051b97"
dest_uuid: "41c59a1982d3480d87bc6a0baa625970"
 from {username='slave'} at 127.0.0.1:51190
I20260709 13:07:34.405382 11505 raft_consensus.cc:493] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:34.405606 11505 raft_consensus.cc:3060] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:34.410008 11505 raft_consensus.cc:515] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:34.411317 11505 leader_election.cc:290] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 3 election: Requested vote from peers 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225), 1115d86484ef4339933c2ba1911812f8 (127.0.107.195:42537)
I20260709 13:07:34.412192 11780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ddc5f5fecb0403386dd61f18d051b97" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7a94a4038cc649babfe75a46223a8bf9"
I20260709 13:07:34.412362 11574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ddc5f5fecb0403386dd61f18d051b97" candidate_uuid: "41c59a1982d3480d87bc6a0baa625970" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1115d86484ef4339933c2ba1911812f8"
I20260709 13:07:34.412860 11780 raft_consensus.cc:3060] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:34.412952 11574 raft_consensus.cc:3055] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:34.413342 11574 raft_consensus.cc:740] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1115d86484ef4339933c2ba1911812f8, State: Running, Role: LEADER
I20260709 13:07:34.414054 11574 consensus_queue.cc:260] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:34.414911 11574 raft_consensus.cc:3060] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:34.418501 11780 raft_consensus.cc:2468] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 3.
I20260709 13:07:34.419121 11574 raft_consensus.cc:2468] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c59a1982d3480d87bc6a0baa625970 in term 3.
I20260709 13:07:34.419767 11468 leader_election.cc:304] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c59a1982d3480d87bc6a0baa625970, 7a94a4038cc649babfe75a46223a8bf9; no voters: 
I20260709 13:07:34.420418 11820 raft_consensus.cc:2804] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:34.420923 11820 raft_consensus.cc:697] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 3 LEADER]: Becoming Leader. State: Replica: 41c59a1982d3480d87bc6a0baa625970, State: Running, Role: LEADER
I20260709 13:07:34.421778 11820 consensus_queue.cc:237] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } }
I20260709 13:07:34.428201 11326 catalog_manager.cc:5697] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 reported cstate change: term changed from 2 to 3, leader changed from 1115d86484ef4339933c2ba1911812f8 (127.0.107.195) to 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "41c59a1982d3480d87bc6a0baa625970" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:34.541553 11505 raft_consensus.cc:1275] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 7 FOLLOWER]: Refusing update from remote peer 1115d86484ef4339933c2ba1911812f8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:07:34.542706 11834 consensus_queue.cc:1048] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:34.551740 11780 raft_consensus.cc:1275] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 7 FOLLOWER]: Refusing update from remote peer 1115d86484ef4339933c2ba1911812f8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:07:34.553045 11834 consensus_queue.cc:1048] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:34.585791 11780 raft_consensus.cc:1275] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 4 FOLLOWER]: Refusing update from remote peer 1115d86484ef4339933c2ba1911812f8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:34.586992 11834 consensus_queue.cc:1048] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:34.594259 11505 raft_consensus.cc:1275] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 4 FOLLOWER]: Refusing update from remote peer 1115d86484ef4339933c2ba1911812f8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:34.595966 11889 consensus_queue.cc:1048] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c59a1982d3480d87bc6a0baa625970" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:07:34.623850 11376 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:34.860013 11574 raft_consensus.cc:1275] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 3 FOLLOWER]: Refusing update from remote peer 41c59a1982d3480d87bc6a0baa625970: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:34.861246 11820 consensus_queue.cc:1048] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1115d86484ef4339933c2ba1911812f8" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:34.869726 11780 raft_consensus.cc:1275] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [term 3 FOLLOWER]: Refusing update from remote peer 41c59a1982d3480d87bc6a0baa625970: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:34.871554 11820 consensus_queue.cc:1048] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a94a4038cc649babfe75a46223a8bf9" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:35.050596   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:35.085621   431 tablet_server.cc:179] TabletServer@127.0.107.193:40225 shutting down...
W20260709 13:07:35.101404 11542 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 87 similar messages]
W20260709 13:07:35.116998 11542 consensus_peers.cc:597] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:35.125110 11542 consensus_peers.cc:597] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:35.125901 11542 consensus_peers.cc:597] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:35.126953   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:35.127960   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:35.128711   431 raft_consensus.cc:2243] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.129220   431 raft_consensus.cc:2272] T 01c031d7def6441ea735da3573686b3e P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.131855   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:35.132459   431 raft_consensus.cc:2243] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.133087   431 raft_consensus.cc:2272] T f668526011204a8e8370e268ae661324 P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.136209   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:35.137087   431 raft_consensus.cc:2243] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.137707   431 raft_consensus.cc:2272] T 34fd2daee75140e8ad3c8617bc3b5592 P 7a94a4038cc649babfe75a46223a8bf9 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.141453   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:35.142251   431 raft_consensus.cc:2243] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.142738   431 raft_consensus.cc:2272] T 4ddc5f5fecb0403386dd61f18d051b97 P 7a94a4038cc649babfe75a46223a8bf9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.146505   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:35.147161   431 raft_consensus.cc:2243] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.147586   431 raft_consensus.cc:2272] T 3c510e78737545ebb0aa0366f008c5bd P 7a94a4038cc649babfe75a46223a8bf9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.150211   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:35.150758   431 raft_consensus.cc:2243] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.151157   431 raft_consensus.cc:2272] T 72a07b3addad47cdad5e02a033cdd7cf P 7a94a4038cc649babfe75a46223a8bf9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.153237   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:35.153811   431 raft_consensus.cc:2243] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.154481   431 raft_consensus.cc:2272] T c8dad3e10536436f9dd13720971d29b7 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.157872   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:35.158536   431 raft_consensus.cc:2243] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.159284   431 raft_consensus.cc:2272] T c5f7a919c42640ea984e8ae25b9b2338 P 7a94a4038cc649babfe75a46223a8bf9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.162163   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:35.162665   431 raft_consensus.cc:2243] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 7 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.163154   431 raft_consensus.cc:2272] T 4a4c406d34924a419654e30483424cbf P 7a94a4038cc649babfe75a46223a8bf9 [term 7 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.166181   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:35.166749   431 raft_consensus.cc:2243] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.167198   431 raft_consensus.cc:2272] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 7a94a4038cc649babfe75a46223a8bf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.203974   431 tablet_server.cc:196] TabletServer@127.0.107.193:40225 shutdown complete.
W20260709 13:07:35.213613 11468 consensus_peers.cc:597] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 -> Peer 7a94a4038cc649babfe75a46223a8bf9 (127.0.107.193:40225): Couldn't send request to peer 7a94a4038cc649babfe75a46223a8bf9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40225: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:35.238260   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
I20260709 13:07:35.266296   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:35.267483   431 tablet_replica.cc:333] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970: stopping tablet replica
I20260709 13:07:35.268591   431 raft_consensus.cc:2243] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.269323   431 raft_consensus.cc:2272] T c5f7a919c42640ea984e8ae25b9b2338 P 41c59a1982d3480d87bc6a0baa625970 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.272926   431 tablet_replica.cc:333] T f668526011204a8e8370e268ae661324 P 41c59a1982d3480d87bc6a0baa625970: stopping tablet replica
I20260709 13:07:35.273743   431 raft_consensus.cc:2243] T f668526011204a8e8370e268ae661324 P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.274334   431 raft_consensus.cc:2272] T f668526011204a8e8370e268ae661324 P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.277029   431 tablet_replica.cc:333] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970: stopping tablet replica
I20260709 13:07:35.277761   431 raft_consensus.cc:2243] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.278211   431 raft_consensus.cc:2272] T 01c031d7def6441ea735da3573686b3e P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.280529   431 tablet_replica.cc:333] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970: stopping tablet replica
I20260709 13:07:35.281190   431 raft_consensus.cc:2243] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 7 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.281621   431 raft_consensus.cc:2272] T 4a4c406d34924a419654e30483424cbf P 41c59a1982d3480d87bc6a0baa625970 [term 7 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.284646   431 tablet_replica.cc:333] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970: stopping tablet replica
I20260709 13:07:35.285354   431 raft_consensus.cc:2243] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.286363   431 raft_consensus.cc:2272] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.289530   431 tablet_replica.cc:333] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970: stopping tablet replica
I20260709 13:07:35.290249   431 raft_consensus.cc:2243] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.291175   431 raft_consensus.cc:2272] T 3c510e78737545ebb0aa0366f008c5bd P 41c59a1982d3480d87bc6a0baa625970 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.293785   431 tablet_replica.cc:333] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970: stopping tablet replica
I20260709 13:07:35.294553   431 raft_consensus.cc:2243] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.295678   431 raft_consensus.cc:2272] T 4ddc5f5fecb0403386dd61f18d051b97 P 41c59a1982d3480d87bc6a0baa625970 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.298713   431 tablet_replica.cc:333] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970: stopping tablet replica
I20260709 13:07:35.299350   431 raft_consensus.cc:2243] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.299841   431 raft_consensus.cc:2272] T 34fd2daee75140e8ad3c8617bc3b5592 P 41c59a1982d3480d87bc6a0baa625970 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.303205   431 tablet_replica.cc:333] T c8dad3e10536436f9dd13720971d29b7 P 41c59a1982d3480d87bc6a0baa625970: stopping tablet replica
I20260709 13:07:35.303833   431 raft_consensus.cc:2243] T c8dad3e10536436f9dd13720971d29b7 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.304569   431 raft_consensus.cc:2272] T c8dad3e10536436f9dd13720971d29b7 P 41c59a1982d3480d87bc6a0baa625970 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.307655   431 tablet_replica.cc:333] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970: stopping tablet replica
I20260709 13:07:35.308218   431 raft_consensus.cc:2243] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.309139   431 raft_consensus.cc:2272] T 72a07b3addad47cdad5e02a033cdd7cf P 41c59a1982d3480d87bc6a0baa625970 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.349454   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:07:35.349754 11544 consensus_peers.cc:597] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 -> Peer 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293): Couldn't send request to peer 41c59a1982d3480d87bc6a0baa625970. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43293: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:35.369192 11544 consensus_peers.cc:597] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 -> Peer 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293): Couldn't send request to peer 41c59a1982d3480d87bc6a0baa625970. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43293: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:35.379405   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:07:35.402921 11544 consensus_peers.cc:597] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 -> Peer 41c59a1982d3480d87bc6a0baa625970 (127.0.107.194:43293): Couldn't send request to peer 41c59a1982d3480d87bc6a0baa625970. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43293: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:35.420171   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:35.421216   431 tablet_replica.cc:333] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8: stopping tablet replica
I20260709 13:07:35.422115   431 raft_consensus.cc:2243] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.423048   431 raft_consensus.cc:2272] T c8dad3e10536436f9dd13720971d29b7 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.425618   431 tablet_replica.cc:333] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8: stopping tablet replica
I20260709 13:07:35.426406   431 raft_consensus.cc:2243] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.426959   431 raft_consensus.cc:2272] T 72a07b3addad47cdad5e02a033cdd7cf P 1115d86484ef4339933c2ba1911812f8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.429194   431 tablet_replica.cc:333] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8: stopping tablet replica
I20260709 13:07:35.429833   431 raft_consensus.cc:2243] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.430629   431 raft_consensus.cc:2272] T c5f7a919c42640ea984e8ae25b9b2338 P 1115d86484ef4339933c2ba1911812f8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.432952   431 tablet_replica.cc:333] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 1115d86484ef4339933c2ba1911812f8: stopping tablet replica
I20260709 13:07:35.433574   431 raft_consensus.cc:2243] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.434144   431 raft_consensus.cc:2272] T 1e0204688f55492b8bf3ec0ed3df4cf3 P 1115d86484ef4339933c2ba1911812f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.437057   431 tablet_replica.cc:333] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8: stopping tablet replica
I20260709 13:07:35.437709   431 raft_consensus.cc:2243] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 7 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.438757   431 raft_consensus.cc:2272] T 4a4c406d34924a419654e30483424cbf P 1115d86484ef4339933c2ba1911812f8 [term 7 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.442404   431 tablet_replica.cc:333] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8: stopping tablet replica
I20260709 13:07:35.442972   431 raft_consensus.cc:2243] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.443401   431 raft_consensus.cc:2272] T 4ddc5f5fecb0403386dd61f18d051b97 P 1115d86484ef4339933c2ba1911812f8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.446282   431 tablet_replica.cc:333] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8: stopping tablet replica
I20260709 13:07:35.447026   431 raft_consensus.cc:2243] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.448083   431 raft_consensus.cc:2272] T 34fd2daee75140e8ad3c8617bc3b5592 P 1115d86484ef4339933c2ba1911812f8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.451262   431 tablet_replica.cc:333] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8: stopping tablet replica
I20260709 13:07:35.451830   431 raft_consensus.cc:2243] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.452239   431 raft_consensus.cc:2272] T 3c510e78737545ebb0aa0366f008c5bd P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.455324   431 tablet_replica.cc:333] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8: stopping tablet replica
I20260709 13:07:35.456673   431 raft_consensus.cc:2243] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.458482   431 raft_consensus.cc:2272] T 01c031d7def6441ea735da3573686b3e P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.461567   431 tablet_replica.cc:333] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8: stopping tablet replica
I20260709 13:07:35.462335   431 raft_consensus.cc:2243] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.463505   431 raft_consensus.cc:2272] T f668526011204a8e8370e268ae661324 P 1115d86484ef4339933c2ba1911812f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.491091   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:07:35.520265   431 master.cc:562] Master@127.0.107.254:37289 shutting down...
I20260709 13:07:35.555559   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.556460   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.556933   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P f1fd10dba22f4381b09f218ff21d8ea8: stopping tablet replica
I20260709 13:07:35.582406   431 master.cc:584] Master@127.0.107.254:37289 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13156 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:07:35.635028   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:41709
I20260709 13:07:35.636528   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:35.643534 11911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:35.643572 11910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:35.644104 11913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:35.646415   431 server_base.cc:1061] running on GCE node
I20260709 13:07:35.647672   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:35.648022   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:35.648265   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602455648242 us; error 0 us; skew 500 ppm
I20260709 13:07:35.648916   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:35.651796   431 webserver.cc:533] Webserver started at http://127.0.107.254:36757/ using document root <none> and password file <none>
I20260709 13:07:35.652534   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:35.652709   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:35.652940   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:35.654282   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602054858684-431-0/minicluster-data/master-0-root/instance:
uuid: "9766fb1c78d3432ba61d5be429507fd7"
format_stamp: "Formatted at 2026-07-09 13:07:35 on dist-test-slave-mxd3"
I20260709 13:07:35.658962   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:07:35.662777 11918 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:35.663638   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:07:35.663964   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602054858684-431-0/minicluster-data/master-0-root
uuid: "9766fb1c78d3432ba61d5be429507fd7"
format_stamp: "Formatted at 2026-07-09 13:07:35 on dist-test-slave-mxd3"
I20260709 13:07:35.664268   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602054858684-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602054858684-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602054858684-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:35.684342   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:35.686367   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:35.733332   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:41709
I20260709 13:07:35.733475 11969 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:41709 every 8 connection(s)
I20260709 13:07:35.737972 11970 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:35.752136 11970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7: Bootstrap starting.
I20260709 13:07:35.758927 11970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.764922 11970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7: No bootstrap required, opened a new log
I20260709 13:07:35.767746 11970 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9766fb1c78d3432ba61d5be429507fd7" member_type: VOTER }
I20260709 13:07:35.768193 11970 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.768488 11970 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9766fb1c78d3432ba61d5be429507fd7, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.769189 11970 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9766fb1c78d3432ba61d5be429507fd7" member_type: VOTER }
I20260709 13:07:35.769737 11970 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:07:35.769969 11970 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:07:35.770316 11970 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.775964 11970 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9766fb1c78d3432ba61d5be429507fd7" member_type: VOTER }
I20260709 13:07:35.776582 11970 leader_election.cc:304] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9766fb1c78d3432ba61d5be429507fd7; no voters: 
I20260709 13:07:35.778131 11970 leader_election.cc:290] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:07:35.778590 11973 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:35.780448 11973 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7 [term 1 LEADER]: Becoming Leader. State: Replica: 9766fb1c78d3432ba61d5be429507fd7, State: Running, Role: LEADER
I20260709 13:07:35.781236 11973 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9766fb1c78d3432ba61d5be429507fd7" member_type: VOTER }
I20260709 13:07:35.781819 11970 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:07:35.786173 11974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9766fb1c78d3432ba61d5be429507fd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9766fb1c78d3432ba61d5be429507fd7" member_type: VOTER } }
I20260709 13:07:35.786988 11974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:35.786185 11975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9766fb1c78d3432ba61d5be429507fd7. Latest consensus state: current_term: 1 leader_uuid: "9766fb1c78d3432ba61d5be429507fd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9766fb1c78d3432ba61d5be429507fd7" member_type: VOTER } }
I20260709 13:07:35.787597 11975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:35.791149 11979 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:07:35.800625 11979 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:07:35.802188   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:07:35.811776 11979 catalog_manager.cc:1383] Generated new cluster ID: 2ca912fb03fa44098b3e29bd2737f536
I20260709 13:07:35.812261 11979 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:07:35.837277 11979 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:07:35.838876 11979 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:07:35.853953 11979 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7: Generated new TSK 0
I20260709 13:07:35.854703 11979 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:07:35.869560   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:35.877022 11991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:35.877772 11992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:35.881918   431 server_base.cc:1061] running on GCE node
W20260709 13:07:35.881608 11994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:35.885186   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:35.885416   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:35.885627   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602455885608 us; error 0 us; skew 500 ppm
I20260709 13:07:35.886646   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:35.890137   431 webserver.cc:533] Webserver started at http://127.0.107.193:41505/ using document root <none> and password file <none>
I20260709 13:07:35.890722   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:35.891090   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:35.891425   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:35.893095   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602054858684-431-0/minicluster-data/ts-0-root/instance:
uuid: "f1ddc31045d447d595b0475540d17542"
format_stamp: "Formatted at 2026-07-09 13:07:35 on dist-test-slave-mxd3"
I20260709 13:07:35.898934   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:07:35.902798 11999 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:35.903630   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260709 13:07:35.903952   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602054858684-431-0/minicluster-data/ts-0-root
uuid: "f1ddc31045d447d595b0475540d17542"
format_stamp: "Formatted at 2026-07-09 13:07:35 on dist-test-slave-mxd3"
I20260709 13:07:35.904253   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602054858684-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602054858684-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602054858684-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:35.923704   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:35.924835   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:35.926367   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:35.928651   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:35.928889   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:35.929123   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:35.929313   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:35.972491   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:46825
I20260709 13:07:35.972672 12061 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:46825 every 8 connection(s)
I20260709 13:07:35.977659   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:35.984372 12066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:35.985448 12067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:35.988831   431 server_base.cc:1061] running on GCE node
W20260709 13:07:35.989133 12069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:35.990159   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:35.990346   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:35.990360 12062 heartbeater.cc:344] Connected to a master server at 127.0.107.254:41709
I20260709 13:07:35.990479   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602455990468 us; error 0 us; skew 500 ppm
I20260709 13:07:35.990833 12062 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:35.991196   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:35.991483 12062 heartbeater.cc:507] Master 127.0.107.254:41709 requested a full tablet report, sending...
I20260709 13:07:35.993698 11935 ts_manager.cc:194] Registered new tserver with Master: f1ddc31045d447d595b0475540d17542 (127.0.107.193:46825)
I20260709 13:07:35.994031   431 webserver.cc:533] Webserver started at http://127.0.107.194:36379/ using document root <none> and password file <none>
I20260709 13:07:35.994624   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:35.994824   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:35.995083   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:35.995764 11935 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60084
I20260709 13:07:35.996301   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602054858684-431-0/minicluster-data/ts-1-root/instance:
uuid: "76ac35c9ce3d464aa161722c20409dab"
format_stamp: "Formatted at 2026-07-09 13:07:35 on dist-test-slave-mxd3"
I20260709 13:07:36.001168   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 13:07:36.004597 12074 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:36.005487   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:07:36.005825   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602054858684-431-0/minicluster-data/ts-1-root
uuid: "76ac35c9ce3d464aa161722c20409dab"
format_stamp: "Formatted at 2026-07-09 13:07:35 on dist-test-slave-mxd3"
I20260709 13:07:36.006134   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602054858684-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602054858684-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602054858684-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:36.031134   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:36.032275   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:36.033973   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:36.036147   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:36.036365   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:36.036614   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:36.036796   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:36.078685   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:38543
I20260709 13:07:36.078866 12136 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:38543 every 8 connection(s)
I20260709 13:07:36.083593   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:36.090937 12140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:36.093325 12141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:36.096382   431 server_base.cc:1061] running on GCE node
W20260709 13:07:36.097119 12143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:36.097803 12137 heartbeater.cc:344] Connected to a master server at 127.0.107.254:41709
I20260709 13:07:36.098261 12137 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:36.098292   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:36.098690   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:36.098891   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602456098872 us; error 0 us; skew 500 ppm
I20260709 13:07:36.099261 12137 heartbeater.cc:507] Master 127.0.107.254:41709 requested a full tablet report, sending...
I20260709 13:07:36.099722   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:36.101975 11935 ts_manager.cc:194] Registered new tserver with Master: 76ac35c9ce3d464aa161722c20409dab (127.0.107.194:38543)
I20260709 13:07:36.102978   431 webserver.cc:533] Webserver started at http://127.0.107.195:45827/ using document root <none> and password file <none>
I20260709 13:07:36.103696   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:36.103930   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:36.103967 11935 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60088
I20260709 13:07:36.104362   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:36.105870   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602054858684-431-0/minicluster-data/ts-2-root/instance:
uuid: "02d39eaf7c2340ad91fd4b2715ce9e34"
format_stamp: "Formatted at 2026-07-09 13:07:36 on dist-test-slave-mxd3"
I20260709 13:07:36.110460   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:07:36.114135 12148 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:36.115072   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:07:36.115413   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602054858684-431-0/minicluster-data/ts-2-root
uuid: "02d39eaf7c2340ad91fd4b2715ce9e34"
format_stamp: "Formatted at 2026-07-09 13:07:36 on dist-test-slave-mxd3"
I20260709 13:07:36.115795   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602054858684-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602054858684-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602054858684-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:36.130790   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:36.131986   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:36.133540   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:36.135877   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:36.136106   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:36.136354   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:36.136528   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:36.178617   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:35405
I20260709 13:07:36.178773 12210 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:35405 every 8 connection(s)
I20260709 13:07:36.192943 12211 heartbeater.cc:344] Connected to a master server at 127.0.107.254:41709
I20260709 13:07:36.193435 12211 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:36.194167 12211 heartbeater.cc:507] Master 127.0.107.254:41709 requested a full tablet report, sending...
I20260709 13:07:36.196027 11935 ts_manager.cc:194] Registered new tserver with Master: 02d39eaf7c2340ad91fd4b2715ce9e34 (127.0.107.195:35405)
I20260709 13:07:36.196519   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014619626s
I20260709 13:07:36.197392 11935 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60092
I20260709 13:07:36.222453 11935 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60094:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:36.224678 11935 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:36.268625 12027 tablet_service.cc:1511] Processing CreateTablet for tablet cf364215b8ce465283566e05ad4cd9c5 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=0082fc50241b47839c44675cde160f8c]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:36.270174 12027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf364215b8ce465283566e05ad4cd9c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.271900 12026 tablet_service.cc:1511] Processing CreateTablet for tablet a4c46b81357444b59b786dddc6394c6e (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=0082fc50241b47839c44675cde160f8c]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:36.273381 12026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4c46b81357444b59b786dddc6394c6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.273016 12102 tablet_service.cc:1511] Processing CreateTablet for tablet cf364215b8ce465283566e05ad4cd9c5 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=0082fc50241b47839c44675cde160f8c]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:36.275029 12102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf364215b8ce465283566e05ad4cd9c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.274740 12101 tablet_service.cc:1511] Processing CreateTablet for tablet a4c46b81357444b59b786dddc6394c6e (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=0082fc50241b47839c44675cde160f8c]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:36.276113 12101 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4c46b81357444b59b786dddc6394c6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.281872 12176 tablet_service.cc:1511] Processing CreateTablet for tablet cf364215b8ce465283566e05ad4cd9c5 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=0082fc50241b47839c44675cde160f8c]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:36.283300 12176 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf364215b8ce465283566e05ad4cd9c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.287240 12175 tablet_service.cc:1511] Processing CreateTablet for tablet a4c46b81357444b59b786dddc6394c6e (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=0082fc50241b47839c44675cde160f8c]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:36.289806 12175 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4c46b81357444b59b786dddc6394c6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.305558 12226 tablet_bootstrap.cc:492] T a4c46b81357444b59b786dddc6394c6e P 76ac35c9ce3d464aa161722c20409dab: Bootstrap starting.
I20260709 13:07:36.315678 12227 tablet_bootstrap.cc:492] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34: Bootstrap starting.
I20260709 13:07:36.315960 12226 tablet_bootstrap.cc:654] T a4c46b81357444b59b786dddc6394c6e P 76ac35c9ce3d464aa161722c20409dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.320369 12228 tablet_bootstrap.cc:492] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542: Bootstrap starting.
I20260709 13:07:36.328333 12227 tablet_bootstrap.cc:654] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.329378 12226 tablet_bootstrap.cc:492] T a4c46b81357444b59b786dddc6394c6e P 76ac35c9ce3d464aa161722c20409dab: No bootstrap required, opened a new log
I20260709 13:07:36.329999 12226 ts_tablet_manager.cc:1403] T a4c46b81357444b59b786dddc6394c6e P 76ac35c9ce3d464aa161722c20409dab: Time spent bootstrapping tablet: real 0.025s	user 0.006s	sys 0.007s
I20260709 13:07:36.333007 12228 tablet_bootstrap.cc:654] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.333396 12226 raft_consensus.cc:359] T a4c46b81357444b59b786dddc6394c6e P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.334293 12226 raft_consensus.cc:385] T a4c46b81357444b59b786dddc6394c6e P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.334470 12227 tablet_bootstrap.cc:492] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34: No bootstrap required, opened a new log
I20260709 13:07:36.334694 12226 raft_consensus.cc:740] T a4c46b81357444b59b786dddc6394c6e P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76ac35c9ce3d464aa161722c20409dab, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.335004 12227 ts_tablet_manager.cc:1403] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34: Time spent bootstrapping tablet: real 0.020s	user 0.005s	sys 0.007s
I20260709 13:07:36.335572 12226 consensus_queue.cc:260] T a4c46b81357444b59b786dddc6394c6e P 76ac35c9ce3d464aa161722c20409dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.338275 12137 heartbeater.cc:499] Master 127.0.107.254:41709 was elected leader, sending a full tablet report...
I20260709 13:07:36.338042 12227 raft_consensus.cc:359] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.338699 12226 ts_tablet_manager.cc:1434] T a4c46b81357444b59b786dddc6394c6e P 76ac35c9ce3d464aa161722c20409dab: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.006s
I20260709 13:07:36.338909 12227 raft_consensus.cc:385] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.339332 12227 raft_consensus.cc:740] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02d39eaf7c2340ad91fd4b2715ce9e34, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.339663 12228 tablet_bootstrap.cc:492] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542: No bootstrap required, opened a new log
I20260709 13:07:36.339784 12226 tablet_bootstrap.cc:492] T cf364215b8ce465283566e05ad4cd9c5 P 76ac35c9ce3d464aa161722c20409dab: Bootstrap starting.
I20260709 13:07:36.340230 12228 ts_tablet_manager.cc:1403] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.002s
I20260709 13:07:36.340291 12227 consensus_queue.cc:260] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.343533 12228 raft_consensus.cc:359] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.344400 12228 raft_consensus.cc:385] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.344736 12228 raft_consensus.cc:740] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ddc31045d447d595b0475540d17542, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.345568 12228 consensus_queue.cc:260] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.347317 12226 tablet_bootstrap.cc:654] T cf364215b8ce465283566e05ad4cd9c5 P 76ac35c9ce3d464aa161722c20409dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.350777 12227 ts_tablet_manager.cc:1434] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.003s
I20260709 13:07:36.351572 12211 heartbeater.cc:499] Master 127.0.107.254:41709 was elected leader, sending a full tablet report...
I20260709 13:07:36.354396 12227 tablet_bootstrap.cc:492] T cf364215b8ce465283566e05ad4cd9c5 P 02d39eaf7c2340ad91fd4b2715ce9e34: Bootstrap starting.
I20260709 13:07:36.354967 12228 ts_tablet_manager.cc:1434] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.005s
I20260709 13:07:36.355084 12062 heartbeater.cc:499] Master 127.0.107.254:41709 was elected leader, sending a full tablet report...
I20260709 13:07:36.356097 12228 tablet_bootstrap.cc:492] T a4c46b81357444b59b786dddc6394c6e P f1ddc31045d447d595b0475540d17542: Bootstrap starting.
I20260709 13:07:36.361436 12227 tablet_bootstrap.cc:654] T cf364215b8ce465283566e05ad4cd9c5 P 02d39eaf7c2340ad91fd4b2715ce9e34: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.363374 12228 tablet_bootstrap.cc:654] T a4c46b81357444b59b786dddc6394c6e P f1ddc31045d447d595b0475540d17542: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.366505 12226 tablet_bootstrap.cc:492] T cf364215b8ce465283566e05ad4cd9c5 P 76ac35c9ce3d464aa161722c20409dab: No bootstrap required, opened a new log
I20260709 13:07:36.367348 12226 ts_tablet_manager.cc:1403] T cf364215b8ce465283566e05ad4cd9c5 P 76ac35c9ce3d464aa161722c20409dab: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.004s
I20260709 13:07:36.371614 12226 raft_consensus.cc:359] T cf364215b8ce465283566e05ad4cd9c5 P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.374194 12227 tablet_bootstrap.cc:492] T cf364215b8ce465283566e05ad4cd9c5 P 02d39eaf7c2340ad91fd4b2715ce9e34: No bootstrap required, opened a new log
I20260709 13:07:36.374775 12227 ts_tablet_manager.cc:1403] T cf364215b8ce465283566e05ad4cd9c5 P 02d39eaf7c2340ad91fd4b2715ce9e34: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.010s
I20260709 13:07:36.377772 12227 raft_consensus.cc:359] T cf364215b8ce465283566e05ad4cd9c5 P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.378685 12227 raft_consensus.cc:385] T cf364215b8ce465283566e05ad4cd9c5 P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.378995 12227 raft_consensus.cc:740] T cf364215b8ce465283566e05ad4cd9c5 P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02d39eaf7c2340ad91fd4b2715ce9e34, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.379681 12227 consensus_queue.cc:260] T cf364215b8ce465283566e05ad4cd9c5 P 02d39eaf7c2340ad91fd4b2715ce9e34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.381544 12227 ts_tablet_manager.cc:1434] T cf364215b8ce465283566e05ad4cd9c5 P 02d39eaf7c2340ad91fd4b2715ce9e34: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:07:36.372715 12226 raft_consensus.cc:385] T cf364215b8ce465283566e05ad4cd9c5 P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.382572 12226 raft_consensus.cc:740] T cf364215b8ce465283566e05ad4cd9c5 P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76ac35c9ce3d464aa161722c20409dab, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.383651 12226 consensus_queue.cc:260] T cf364215b8ce465283566e05ad4cd9c5 P 76ac35c9ce3d464aa161722c20409dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.386477 12226 ts_tablet_manager.cc:1434] T cf364215b8ce465283566e05ad4cd9c5 P 76ac35c9ce3d464aa161722c20409dab: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:07:36.388616 12228 tablet_bootstrap.cc:492] T a4c46b81357444b59b786dddc6394c6e P f1ddc31045d447d595b0475540d17542: No bootstrap required, opened a new log
I20260709 13:07:36.389164 12228 ts_tablet_manager.cc:1403] T a4c46b81357444b59b786dddc6394c6e P f1ddc31045d447d595b0475540d17542: Time spent bootstrapping tablet: real 0.033s	user 0.019s	sys 0.011s
I20260709 13:07:36.391511 12228 raft_consensus.cc:359] T a4c46b81357444b59b786dddc6394c6e P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.392189 12228 raft_consensus.cc:385] T a4c46b81357444b59b786dddc6394c6e P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.392488 12228 raft_consensus.cc:740] T a4c46b81357444b59b786dddc6394c6e P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ddc31045d447d595b0475540d17542, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.393205 12228 consensus_queue.cc:260] T a4c46b81357444b59b786dddc6394c6e P f1ddc31045d447d595b0475540d17542 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.395434 12228 ts_tablet_manager.cc:1434] T a4c46b81357444b59b786dddc6394c6e P f1ddc31045d447d595b0475540d17542: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.004s
I20260709 13:07:36.517836 12233 raft_consensus.cc:493] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.518488 12233 raft_consensus.cc:515] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.520723 12233 leader_election.cc:290] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1ddc31045d447d595b0475540d17542 (127.0.107.193:46825), 76ac35c9ce3d464aa161722c20409dab (127.0.107.194:38543)
I20260709 13:07:36.529935 12037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c46b81357444b59b786dddc6394c6e" candidate_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1ddc31045d447d595b0475540d17542" is_pre_election: true
I20260709 13:07:36.530712 12037 raft_consensus.cc:2468] T a4c46b81357444b59b786dddc6394c6e P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02d39eaf7c2340ad91fd4b2715ce9e34 in term 0.
I20260709 13:07:36.531957 12151 leader_election.cc:304] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02d39eaf7c2340ad91fd4b2715ce9e34, f1ddc31045d447d595b0475540d17542; no voters: 
I20260709 13:07:36.532083 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c46b81357444b59b786dddc6394c6e" candidate_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76ac35c9ce3d464aa161722c20409dab" is_pre_election: true
I20260709 13:07:36.532740 12112 raft_consensus.cc:2468] T a4c46b81357444b59b786dddc6394c6e P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02d39eaf7c2340ad91fd4b2715ce9e34 in term 0.
I20260709 13:07:36.532819 12233 raft_consensus.cc:2804] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:36.533303 12233 raft_consensus.cc:493] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.533625 12233 raft_consensus.cc:3060] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.539254 12233 raft_consensus.cc:515] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.541308 12233 leader_election.cc:290] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [CANDIDATE]: Term 1 election: Requested vote from peers f1ddc31045d447d595b0475540d17542 (127.0.107.193:46825), 76ac35c9ce3d464aa161722c20409dab (127.0.107.194:38543)
I20260709 13:07:36.542174 12037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c46b81357444b59b786dddc6394c6e" candidate_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1ddc31045d447d595b0475540d17542"
I20260709 13:07:36.542387 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c46b81357444b59b786dddc6394c6e" candidate_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76ac35c9ce3d464aa161722c20409dab"
I20260709 13:07:36.542745 12037 raft_consensus.cc:3060] T a4c46b81357444b59b786dddc6394c6e P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.543095 12112 raft_consensus.cc:3060] T a4c46b81357444b59b786dddc6394c6e P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.548033 12037 raft_consensus.cc:2468] T a4c46b81357444b59b786dddc6394c6e P f1ddc31045d447d595b0475540d17542 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02d39eaf7c2340ad91fd4b2715ce9e34 in term 1.
I20260709 13:07:36.548249 12112 raft_consensus.cc:2468] T a4c46b81357444b59b786dddc6394c6e P 76ac35c9ce3d464aa161722c20409dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02d39eaf7c2340ad91fd4b2715ce9e34 in term 1.
I20260709 13:07:36.549279 12151 leader_election.cc:304] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02d39eaf7c2340ad91fd4b2715ce9e34, f1ddc31045d447d595b0475540d17542; no voters: 
I20260709 13:07:36.550069 12233 raft_consensus.cc:2804] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:36.550961 12233 raft_consensus.cc:697] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 1 LEADER]: Becoming Leader. State: Replica: 02d39eaf7c2340ad91fd4b2715ce9e34, State: Running, Role: LEADER
I20260709 13:07:36.551628 12233 consensus_queue.cc:237] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.559094 11934 catalog_manager.cc:5697] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 02d39eaf7c2340ad91fd4b2715ce9e34 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:36.678277 12234 raft_consensus.cc:493] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.678789 12234 raft_consensus.cc:515] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.680686 12234 leader_election.cc:290] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76ac35c9ce3d464aa161722c20409dab (127.0.107.194:38543), 02d39eaf7c2340ad91fd4b2715ce9e34 (127.0.107.195:35405)
I20260709 13:07:36.691187 12186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf364215b8ce465283566e05ad4cd9c5" candidate_uuid: "f1ddc31045d447d595b0475540d17542" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" is_pre_election: true
I20260709 13:07:36.691942 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf364215b8ce465283566e05ad4cd9c5" candidate_uuid: "f1ddc31045d447d595b0475540d17542" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76ac35c9ce3d464aa161722c20409dab" is_pre_election: true
I20260709 13:07:36.692037 12186 raft_consensus.cc:2468] T cf364215b8ce465283566e05ad4cd9c5 P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1ddc31045d447d595b0475540d17542 in term 0.
I20260709 13:07:36.692679 12112 raft_consensus.cc:2468] T cf364215b8ce465283566e05ad4cd9c5 P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1ddc31045d447d595b0475540d17542 in term 0.
I20260709 13:07:36.693539 12000 leader_election.cc:304] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02d39eaf7c2340ad91fd4b2715ce9e34, f1ddc31045d447d595b0475540d17542; no voters: 
I20260709 13:07:36.694478 12234 raft_consensus.cc:2804] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:36.694839 12234 raft_consensus.cc:493] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.695206 12234 raft_consensus.cc:3060] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.700686 12234 raft_consensus.cc:515] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.702406 12234 leader_election.cc:290] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [CANDIDATE]: Term 1 election: Requested vote from peers 76ac35c9ce3d464aa161722c20409dab (127.0.107.194:38543), 02d39eaf7c2340ad91fd4b2715ce9e34 (127.0.107.195:35405)
I20260709 13:07:36.703298 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf364215b8ce465283566e05ad4cd9c5" candidate_uuid: "f1ddc31045d447d595b0475540d17542" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76ac35c9ce3d464aa161722c20409dab"
I20260709 13:07:36.703537 12186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf364215b8ce465283566e05ad4cd9c5" candidate_uuid: "f1ddc31045d447d595b0475540d17542" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34"
I20260709 13:07:36.704015 12112 raft_consensus.cc:3060] T cf364215b8ce465283566e05ad4cd9c5 P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.704023 12186 raft_consensus.cc:3060] T cf364215b8ce465283566e05ad4cd9c5 P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.711401 12186 raft_consensus.cc:2468] T cf364215b8ce465283566e05ad4cd9c5 P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1ddc31045d447d595b0475540d17542 in term 1.
I20260709 13:07:36.711401 12112 raft_consensus.cc:2468] T cf364215b8ce465283566e05ad4cd9c5 P 76ac35c9ce3d464aa161722c20409dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1ddc31045d447d595b0475540d17542 in term 1.
I20260709 13:07:36.712678 12000 leader_election.cc:304] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02d39eaf7c2340ad91fd4b2715ce9e34, f1ddc31045d447d595b0475540d17542; no voters: 
I20260709 13:07:36.713378 12234 raft_consensus.cc:2804] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:36.714658 12234 raft_consensus.cc:697] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [term 1 LEADER]: Becoming Leader. State: Replica: f1ddc31045d447d595b0475540d17542, State: Running, Role: LEADER
I20260709 13:07:36.715847 12234 consensus_queue.cc:237] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.723002 11934 catalog_manager.cc:5697] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1ddc31045d447d595b0475540d17542 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "f1ddc31045d447d595b0475540d17542" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:36.786744 11934 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60110:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:36.788722 11934 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:36.814471 12026 tablet_service.cc:1511] Processing CreateTablet for tablet 41cf2842993e40ea9d5a6487ce869ef7 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=71c0a4e3f2aa46d2b5f43726f2a8a486]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:36.815910 12026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41cf2842993e40ea9d5a6487ce869ef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.815555 12102 tablet_service.cc:1511] Processing CreateTablet for tablet 41cf2842993e40ea9d5a6487ce869ef7 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=71c0a4e3f2aa46d2b5f43726f2a8a486]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:36.817178 12102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41cf2842993e40ea9d5a6487ce869ef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.818702 12027 tablet_service.cc:1511] Processing CreateTablet for tablet 103b8f10ade14b63b20a93c2fc4502ae (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=71c0a4e3f2aa46d2b5f43726f2a8a486]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:36.820197 12027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 103b8f10ade14b63b20a93c2fc4502ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.822394 12101 tablet_service.cc:1511] Processing CreateTablet for tablet 103b8f10ade14b63b20a93c2fc4502ae (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=71c0a4e3f2aa46d2b5f43726f2a8a486]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:36.818702 12176 tablet_service.cc:1511] Processing CreateTablet for tablet 41cf2842993e40ea9d5a6487ce869ef7 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=71c0a4e3f2aa46d2b5f43726f2a8a486]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:36.823894 12101 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 103b8f10ade14b63b20a93c2fc4502ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.824030 12175 tablet_service.cc:1511] Processing CreateTablet for tablet 103b8f10ade14b63b20a93c2fc4502ae (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=71c0a4e3f2aa46d2b5f43726f2a8a486]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:36.823939 12176 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41cf2842993e40ea9d5a6487ce869ef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.830878 12175 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 103b8f10ade14b63b20a93c2fc4502ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.863808 12228 tablet_bootstrap.cc:492] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542: Bootstrap starting.
I20260709 13:07:36.867220 12226 tablet_bootstrap.cc:492] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab: Bootstrap starting.
I20260709 13:07:36.871595 12228 tablet_bootstrap.cc:654] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.875172 12226 tablet_bootstrap.cc:654] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.882591 12227 tablet_bootstrap.cc:492] T 103b8f10ade14b63b20a93c2fc4502ae P 02d39eaf7c2340ad91fd4b2715ce9e34: Bootstrap starting.
I20260709 13:07:36.886734 12228 tablet_bootstrap.cc:492] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542: No bootstrap required, opened a new log
I20260709 13:07:36.887372 12228 ts_tablet_manager.cc:1403] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.000s
I20260709 13:07:36.890937 12228 raft_consensus.cc:359] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.891734 12228 raft_consensus.cc:385] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.892077 12228 raft_consensus.cc:740] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ddc31045d447d595b0475540d17542, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.892870 12228 consensus_queue.cc:260] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.894860 12227 tablet_bootstrap.cc:654] T 103b8f10ade14b63b20a93c2fc4502ae P 02d39eaf7c2340ad91fd4b2715ce9e34: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.894799 12226 tablet_bootstrap.cc:492] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab: No bootstrap required, opened a new log
I20260709 13:07:36.895511 12228 ts_tablet_manager.cc:1434] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:07:36.895720 12226 ts_tablet_manager.cc:1403] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.013s
I20260709 13:07:36.896799 12228 tablet_bootstrap.cc:492] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542: Bootstrap starting.
I20260709 13:07:36.901932 12226 raft_consensus.cc:359] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.903209 12226 raft_consensus.cc:385] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.903707 12226 raft_consensus.cc:740] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76ac35c9ce3d464aa161722c20409dab, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.904239 12228 tablet_bootstrap.cc:654] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.905157 12226 consensus_queue.cc:260] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.910260 12226 ts_tablet_manager.cc:1434] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:07:36.911468 12226 tablet_bootstrap.cc:492] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab: Bootstrap starting.
I20260709 13:07:36.910326 12227 tablet_bootstrap.cc:492] T 103b8f10ade14b63b20a93c2fc4502ae P 02d39eaf7c2340ad91fd4b2715ce9e34: No bootstrap required, opened a new log
I20260709 13:07:36.913712 12227 ts_tablet_manager.cc:1403] T 103b8f10ade14b63b20a93c2fc4502ae P 02d39eaf7c2340ad91fd4b2715ce9e34: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.015s
I20260709 13:07:36.917575 12227 raft_consensus.cc:359] T 103b8f10ade14b63b20a93c2fc4502ae P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.919330 12227 raft_consensus.cc:385] T 103b8f10ade14b63b20a93c2fc4502ae P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.919656 12227 raft_consensus.cc:740] T 103b8f10ade14b63b20a93c2fc4502ae P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02d39eaf7c2340ad91fd4b2715ce9e34, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.920403 12228 tablet_bootstrap.cc:492] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542: No bootstrap required, opened a new log
I20260709 13:07:36.920449 12227 consensus_queue.cc:260] T 103b8f10ade14b63b20a93c2fc4502ae P 02d39eaf7c2340ad91fd4b2715ce9e34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.921150 12228 ts_tablet_manager.cc:1403] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542: Time spent bootstrapping tablet: real 0.025s	user 0.023s	sys 0.000s
I20260709 13:07:36.923362 12227 ts_tablet_manager.cc:1434] T 103b8f10ade14b63b20a93c2fc4502ae P 02d39eaf7c2340ad91fd4b2715ce9e34: Time spent starting tablet: real 0.009s	user 0.000s	sys 0.007s
I20260709 13:07:36.924415 12226 tablet_bootstrap.cc:654] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.924633 12228 raft_consensus.cc:359] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.925473 12228 raft_consensus.cc:385] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.924489 12227 tablet_bootstrap.cc:492] T 41cf2842993e40ea9d5a6487ce869ef7 P 02d39eaf7c2340ad91fd4b2715ce9e34: Bootstrap starting.
I20260709 13:07:36.925899 12228 raft_consensus.cc:740] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ddc31045d447d595b0475540d17542, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.926893 12228 consensus_queue.cc:260] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.932331 12228 ts_tablet_manager.cc:1434] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:07:36.933314 12227 tablet_bootstrap.cc:654] T 41cf2842993e40ea9d5a6487ce869ef7 P 02d39eaf7c2340ad91fd4b2715ce9e34: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.939548 12227 tablet_bootstrap.cc:492] T 41cf2842993e40ea9d5a6487ce869ef7 P 02d39eaf7c2340ad91fd4b2715ce9e34: No bootstrap required, opened a new log
I20260709 13:07:36.940258 12227 ts_tablet_manager.cc:1403] T 41cf2842993e40ea9d5a6487ce869ef7 P 02d39eaf7c2340ad91fd4b2715ce9e34: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 13:07:36.939548 12226 tablet_bootstrap.cc:492] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab: No bootstrap required, opened a new log
I20260709 13:07:36.941022 12226 ts_tablet_manager.cc:1403] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.006s
I20260709 13:07:36.943300 12227 raft_consensus.cc:359] T 41cf2842993e40ea9d5a6487ce869ef7 P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.944010 12227 raft_consensus.cc:385] T 41cf2842993e40ea9d5a6487ce869ef7 P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.944346 12227 raft_consensus.cc:740] T 41cf2842993e40ea9d5a6487ce869ef7 P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02d39eaf7c2340ad91fd4b2715ce9e34, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.944074 12226 raft_consensus.cc:359] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.944955 12226 raft_consensus.cc:385] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.945272 12226 raft_consensus.cc:740] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76ac35c9ce3d464aa161722c20409dab, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.945336 12227 consensus_queue.cc:260] T 41cf2842993e40ea9d5a6487ce869ef7 P 02d39eaf7c2340ad91fd4b2715ce9e34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.946054 12226 consensus_queue.cc:260] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:36.948123 12227 ts_tablet_manager.cc:1434] T 41cf2842993e40ea9d5a6487ce869ef7 P 02d39eaf7c2340ad91fd4b2715ce9e34: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:36.948310 12226 ts_tablet_manager.cc:1434] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:37.083458 12234 raft_consensus.cc:493] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.084180 12234 raft_consensus.cc:515] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:37.086342 12234 leader_election.cc:290] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76ac35c9ce3d464aa161722c20409dab (127.0.107.194:38543), 02d39eaf7c2340ad91fd4b2715ce9e34 (127.0.107.195:35405)
I20260709 13:07:37.087080 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "103b8f10ade14b63b20a93c2fc4502ae" candidate_uuid: "f1ddc31045d447d595b0475540d17542" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76ac35c9ce3d464aa161722c20409dab" is_pre_election: true
I20260709 13:07:37.087466 12186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "103b8f10ade14b63b20a93c2fc4502ae" candidate_uuid: "f1ddc31045d447d595b0475540d17542" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" is_pre_election: true
I20260709 13:07:37.087725 12112 raft_consensus.cc:2468] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1ddc31045d447d595b0475540d17542 in term 0.
I20260709 13:07:37.088223 12186 raft_consensus.cc:2468] T 103b8f10ade14b63b20a93c2fc4502ae P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1ddc31045d447d595b0475540d17542 in term 0.
I20260709 13:07:37.088971 12001 leader_election.cc:304] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76ac35c9ce3d464aa161722c20409dab, f1ddc31045d447d595b0475540d17542; no voters: 
I20260709 13:07:37.089920 12234 raft_consensus.cc:2804] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:37.090272 12234 raft_consensus.cc:493] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.090560 12234 raft_consensus.cc:3060] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.096197 12234 raft_consensus.cc:515] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:37.097884 12234 leader_election.cc:290] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [CANDIDATE]: Term 1 election: Requested vote from peers 76ac35c9ce3d464aa161722c20409dab (127.0.107.194:38543), 02d39eaf7c2340ad91fd4b2715ce9e34 (127.0.107.195:35405)
I20260709 13:07:37.098776 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "103b8f10ade14b63b20a93c2fc4502ae" candidate_uuid: "f1ddc31045d447d595b0475540d17542" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76ac35c9ce3d464aa161722c20409dab"
I20260709 13:07:37.098860 12186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "103b8f10ade14b63b20a93c2fc4502ae" candidate_uuid: "f1ddc31045d447d595b0475540d17542" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34"
I20260709 13:07:37.099347 12112 raft_consensus.cc:3060] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.099490 12186 raft_consensus.cc:3060] T 103b8f10ade14b63b20a93c2fc4502ae P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.104341 12186 raft_consensus.cc:2468] T 103b8f10ade14b63b20a93c2fc4502ae P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1ddc31045d447d595b0475540d17542 in term 1.
I20260709 13:07:37.104341 12112 raft_consensus.cc:2468] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1ddc31045d447d595b0475540d17542 in term 1.
I20260709 13:07:37.105484 12000 leader_election.cc:304] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02d39eaf7c2340ad91fd4b2715ce9e34, f1ddc31045d447d595b0475540d17542; no voters: 
I20260709 13:07:37.106249 12234 raft_consensus.cc:2804] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:37.106720 12234 raft_consensus.cc:697] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [term 1 LEADER]: Becoming Leader. State: Replica: f1ddc31045d447d595b0475540d17542, State: Running, Role: LEADER
I20260709 13:07:37.107501 12234 consensus_queue.cc:237] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:37.114945 11932 catalog_manager.cc:5697] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1ddc31045d447d595b0475540d17542 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "f1ddc31045d447d595b0475540d17542" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.136022 12234 consensus_queue.cc:1048] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.141813 12233 consensus_queue.cc:1048] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.154532 12263 consensus_queue.cc:1048] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:37.169863 12266 consensus_queue.cc:1048] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:37.179857 12254 raft_consensus.cc:493] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.180612 12254 raft_consensus.cc:515] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:37.194267 12254 leader_election.cc:290] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1ddc31045d447d595b0475540d17542 (127.0.107.193:46825), 02d39eaf7c2340ad91fd4b2715ce9e34 (127.0.107.195:35405)
I20260709 13:07:37.216113 12037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41cf2842993e40ea9d5a6487ce869ef7" candidate_uuid: "76ac35c9ce3d464aa161722c20409dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1ddc31045d447d595b0475540d17542" is_pre_election: true
I20260709 13:07:37.216954 12037 raft_consensus.cc:2468] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76ac35c9ce3d464aa161722c20409dab in term 0.
I20260709 13:07:37.219415 12185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41cf2842993e40ea9d5a6487ce869ef7" candidate_uuid: "76ac35c9ce3d464aa161722c20409dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" is_pre_election: true
I20260709 13:07:37.220149 12185 raft_consensus.cc:2468] T 41cf2842993e40ea9d5a6487ce869ef7 P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76ac35c9ce3d464aa161722c20409dab in term 0.
I20260709 13:07:37.220728 12077 leader_election.cc:304] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76ac35c9ce3d464aa161722c20409dab, f1ddc31045d447d595b0475540d17542; no voters: 
I20260709 13:07:37.221661 12254 raft_consensus.cc:2804] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:37.222050 12254 raft_consensus.cc:493] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.222455 12254 raft_consensus.cc:3060] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.228936 12254 raft_consensus.cc:515] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:37.231769 12254 leader_election.cc:290] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [CANDIDATE]: Term 1 election: Requested vote from peers f1ddc31045d447d595b0475540d17542 (127.0.107.193:46825), 02d39eaf7c2340ad91fd4b2715ce9e34 (127.0.107.195:35405)
I20260709 13:07:37.231869 12037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41cf2842993e40ea9d5a6487ce869ef7" candidate_uuid: "76ac35c9ce3d464aa161722c20409dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1ddc31045d447d595b0475540d17542"
I20260709 13:07:37.232579 12186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41cf2842993e40ea9d5a6487ce869ef7" candidate_uuid: "76ac35c9ce3d464aa161722c20409dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34"
I20260709 13:07:37.232697 12037 raft_consensus.cc:3060] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.233170 12186 raft_consensus.cc:3060] T 41cf2842993e40ea9d5a6487ce869ef7 P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.238693 12186 raft_consensus.cc:2468] T 41cf2842993e40ea9d5a6487ce869ef7 P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76ac35c9ce3d464aa161722c20409dab in term 1.
I20260709 13:07:37.239106 12037 raft_consensus.cc:2468] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76ac35c9ce3d464aa161722c20409dab in term 1.
I20260709 13:07:37.239894 12075 leader_election.cc:304] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02d39eaf7c2340ad91fd4b2715ce9e34, 76ac35c9ce3d464aa161722c20409dab; no voters: 
I20260709 13:07:37.240665 12254 raft_consensus.cc:2804] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:37.241623 12254 raft_consensus.cc:697] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [term 1 LEADER]: Becoming Leader. State: Replica: 76ac35c9ce3d464aa161722c20409dab, State: Running, Role: LEADER
I20260709 13:07:37.242470 12254 consensus_queue.cc:237] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:37.249110 11932 catalog_manager.cc:5697] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab reported cstate change: term changed from 0 to 1, leader changed from <none> to 76ac35c9ce3d464aa161722c20409dab (127.0.107.194). New cstate: current_term: 1 leader_uuid: "76ac35c9ce3d464aa161722c20409dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.336530 12112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41cf2842993e40ea9d5a6487ce869ef7"
dest_uuid: "76ac35c9ce3d464aa161722c20409dab"
mode: GRACEFUL
new_leader_uuid: "f1ddc31045d447d595b0475540d17542"
 from {username='slave'} at 127.0.0.1:55134
I20260709 13:07:37.337165 12112 raft_consensus.cc:606] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [term 1 LEADER]: Received request to transfer leadership to TS f1ddc31045d447d595b0475540d17542
I20260709 13:07:37.639276 12243 consensus_queue.cc:1048] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.651115 12243 consensus_queue.cc:1048] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:37.661513 12254 consensus_queue.cc:1048] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:37.673816 12254 consensus_queue.cc:1048] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.684633 12274 raft_consensus.cc:993] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab: : Instructing follower f1ddc31045d447d595b0475540d17542 to start an election
I20260709 13:07:37.685096 12274 raft_consensus.cc:1081] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [term 1 LEADER]: Signalling peer f1ddc31045d447d595b0475540d17542 to start an election
I20260709 13:07:37.686945 12037 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "41cf2842993e40ea9d5a6487ce869ef7"
dest_uuid: "f1ddc31045d447d595b0475540d17542"
 from {username='slave'} at 127.0.0.1:48294
I20260709 13:07:37.687647 12037 raft_consensus.cc:493] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:37.687999 12037 raft_consensus.cc:3060] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.693248 12037 raft_consensus.cc:515] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:37.695940 12037 leader_election.cc:290] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542 [CANDIDATE]: Term 2 election: Requested vote from peers 76ac35c9ce3d464aa161722c20409dab (127.0.107.194:38543), 02d39eaf7c2340ad91fd4b2715ce9e34 (127.0.107.195:35405)
I20260709 13:07:37.696223 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41cf2842993e40ea9d5a6487ce869ef7" candidate_uuid: "f1ddc31045d447d595b0475540d17542" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76ac35c9ce3d464aa161722c20409dab"
I20260709 13:07:37.696511 12186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41cf2842993e40ea9d5a6487ce869ef7" candidate_uuid: "f1ddc31045d447d595b0475540d17542" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34"
I20260709 13:07:37.696945 12112 raft_consensus.cc:3055] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:37.697177 12186 raft_consensus.cc:3060] T 41cf2842993e40ea9d5a6487ce869ef7 P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.697340 12112 raft_consensus.cc:740] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76ac35c9ce3d464aa161722c20409dab, State: Running, Role: LEADER
I20260709 13:07:37.698256 12112 consensus_queue.cc:260] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:37.699332 12112 raft_consensus.cc:3060] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.703279 12186 raft_consensus.cc:2468] T 41cf2842993e40ea9d5a6487ce869ef7 P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1ddc31045d447d595b0475540d17542 in term 2.
I20260709 13:07:37.704615 12000 leader_election.cc:304] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02d39eaf7c2340ad91fd4b2715ce9e34, f1ddc31045d447d595b0475540d17542; no voters: 
I20260709 13:07:37.705470 12279 raft_consensus.cc:2804] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:37.705919 12112 raft_consensus.cc:2468] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1ddc31045d447d595b0475540d17542 in term 2.
I20260709 13:07:37.706108 12279 raft_consensus.cc:697] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542 [term 2 LEADER]: Becoming Leader. State: Replica: f1ddc31045d447d595b0475540d17542, State: Running, Role: LEADER
I20260709 13:07:37.707192 12279 consensus_queue.cc:237] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:37.713781 11932 catalog_manager.cc:5697] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542 reported cstate change: term changed from 1 to 2, leader changed from 76ac35c9ce3d464aa161722c20409dab (127.0.107.194) to f1ddc31045d447d595b0475540d17542 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "f1ddc31045d447d595b0475540d17542" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.779268   431 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:07:37.780886   431 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:07:37.781291   431 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:07:37.793874 12037 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "103b8f10ade14b63b20a93c2fc4502ae"
dest_uuid: "f1ddc31045d447d595b0475540d17542"
mode: GRACEFUL
new_leader_uuid: "76ac35c9ce3d464aa161722c20409dab"
 from {username='slave'} at 127.0.0.1:48296
I20260709 13:07:37.794456 12037 raft_consensus.cc:606] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [term 1 LEADER]: Received request to transfer leadership to TS 76ac35c9ce3d464aa161722c20409dab
I20260709 13:07:37.795718   431 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:07:38.108868 12243 raft_consensus.cc:993] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542: : Instructing follower 76ac35c9ce3d464aa161722c20409dab to start an election
I20260709 13:07:38.109403 12279 raft_consensus.cc:1081] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [term 1 LEADER]: Signalling peer 76ac35c9ce3d464aa161722c20409dab to start an election
I20260709 13:07:38.111178 12112 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "103b8f10ade14b63b20a93c2fc4502ae"
dest_uuid: "76ac35c9ce3d464aa161722c20409dab"
 from {username='slave'} at 127.0.0.1:55118
I20260709 13:07:38.111682 12112 raft_consensus.cc:493] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:38.111958 12112 raft_consensus.cc:3060] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:38.117179 12112 raft_consensus.cc:515] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:38.119160 12112 leader_election.cc:290] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab [CANDIDATE]: Term 2 election: Requested vote from peers f1ddc31045d447d595b0475540d17542 (127.0.107.193:46825), 02d39eaf7c2340ad91fd4b2715ce9e34 (127.0.107.195:35405)
I20260709 13:07:38.120095 12037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "103b8f10ade14b63b20a93c2fc4502ae" candidate_uuid: "76ac35c9ce3d464aa161722c20409dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1ddc31045d447d595b0475540d17542"
I20260709 13:07:38.120221 12186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "103b8f10ade14b63b20a93c2fc4502ae" candidate_uuid: "76ac35c9ce3d464aa161722c20409dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34"
I20260709 13:07:38.120760 12037 raft_consensus.cc:3055] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:38.120842 12186 raft_consensus.cc:3060] T 103b8f10ade14b63b20a93c2fc4502ae P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:38.121104 12037 raft_consensus.cc:740] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f1ddc31045d447d595b0475540d17542, State: Running, Role: LEADER
I20260709 13:07:38.121955 12037 consensus_queue.cc:260] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:38.122947 12037 raft_consensus.cc:3060] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:38.126549 12186 raft_consensus.cc:2468] T 103b8f10ade14b63b20a93c2fc4502ae P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76ac35c9ce3d464aa161722c20409dab in term 2.
I20260709 13:07:38.127688 12075 leader_election.cc:304] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02d39eaf7c2340ad91fd4b2715ce9e34, 76ac35c9ce3d464aa161722c20409dab; no voters: 
I20260709 13:07:38.127991 12037 raft_consensus.cc:2468] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76ac35c9ce3d464aa161722c20409dab in term 2.
I20260709 13:07:38.128415 12274 raft_consensus.cc:2804] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:38.128973 12274 raft_consensus.cc:697] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab [term 2 LEADER]: Becoming Leader. State: Replica: 76ac35c9ce3d464aa161722c20409dab, State: Running, Role: LEADER
I20260709 13:07:38.129844 12274 consensus_queue.cc:237] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } }
I20260709 13:07:38.135537 11932 catalog_manager.cc:5697] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab reported cstate change: term changed from 1 to 2, leader changed from f1ddc31045d447d595b0475540d17542 (127.0.107.193) to 76ac35c9ce3d464aa161722c20409dab (127.0.107.194). New cstate: current_term: 2 leader_uuid: "76ac35c9ce3d464aa161722c20409dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ddc31045d447d595b0475540d17542" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76ac35c9ce3d464aa161722c20409dab" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02d39eaf7c2340ad91fd4b2715ce9e34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35405 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:38.146139   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:07:38.166453   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:38.167129   431 tablet_replica.cc:333] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542: stopping tablet replica
I20260709 13:07:38.167716   431 raft_consensus.cc:2243] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:38.168180   431 raft_consensus.cc:2272] T 103b8f10ade14b63b20a93c2fc4502ae P f1ddc31045d447d595b0475540d17542 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:38.170357   431 tablet_replica.cc:333] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542: stopping tablet replica
I20260709 13:07:38.170994   431 raft_consensus.cc:2243] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:38.171862   431 pending_rounds.cc:70] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542: Trying to abort 1 pending ops.
I20260709 13:07:38.172115   431 pending_rounds.cc:77] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305635666777366528 op_type: NO_OP noop_request { }
I20260709 13:07:38.172441   431 raft_consensus.cc:2889] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:38.172748   431 raft_consensus.cc:2272] T 41cf2842993e40ea9d5a6487ce869ef7 P f1ddc31045d447d595b0475540d17542 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:38.173269 12186 raft_consensus.cc:1275] T 41cf2842993e40ea9d5a6487ce869ef7 P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 2 FOLLOWER]: Refusing update from remote peer f1ddc31045d447d595b0475540d17542: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:38.175470   431 tablet_replica.cc:333] T a4c46b81357444b59b786dddc6394c6e P f1ddc31045d447d595b0475540d17542: stopping tablet replica
I20260709 13:07:38.176153   431 raft_consensus.cc:2243] T a4c46b81357444b59b786dddc6394c6e P f1ddc31045d447d595b0475540d17542 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:38.176622   431 raft_consensus.cc:2272] T a4c46b81357444b59b786dddc6394c6e P f1ddc31045d447d595b0475540d17542 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:38.179319   431 tablet_replica.cc:333] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542: stopping tablet replica
I20260709 13:07:38.179855   431 raft_consensus.cc:2243] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:38.180689   431 raft_consensus.cc:2272] T cf364215b8ce465283566e05ad4cd9c5 P f1ddc31045d447d595b0475540d17542 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:38.204669   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 13:07:38.216553   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
I20260709 13:07:38.237202   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:38.237996   431 tablet_replica.cc:333] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab: stopping tablet replica
I20260709 13:07:38.238514   431 raft_consensus.cc:2243] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:38.238932   431 raft_consensus.cc:2272] T 41cf2842993e40ea9d5a6487ce869ef7 P 76ac35c9ce3d464aa161722c20409dab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:38.241773   431 tablet_replica.cc:333] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab: stopping tablet replica
I20260709 13:07:38.242439   431 raft_consensus.cc:2243] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:38.243062   431 pending_rounds.cc:70] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab: Trying to abort 1 pending ops.
I20260709 13:07:38.243269   431 pending_rounds.cc:77] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305635668506386432 op_type: NO_OP noop_request { }
I20260709 13:07:38.243548   431 raft_consensus.cc:2889] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:38.243848   431 raft_consensus.cc:2272] T 103b8f10ade14b63b20a93c2fc4502ae P 76ac35c9ce3d464aa161722c20409dab [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:38.243906 12150 consensus_peers.cc:597] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 -> Peer 76ac35c9ce3d464aa161722c20409dab (127.0.107.194:38543): Couldn't send request to peer 76ac35c9ce3d464aa161722c20409dab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:38.246523   431 tablet_replica.cc:333] T cf364215b8ce465283566e05ad4cd9c5 P 76ac35c9ce3d464aa161722c20409dab: stopping tablet replica
I20260709 13:07:38.247171   431 raft_consensus.cc:2243] T cf364215b8ce465283566e05ad4cd9c5 P 76ac35c9ce3d464aa161722c20409dab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:38.247587   431 raft_consensus.cc:2272] T cf364215b8ce465283566e05ad4cd9c5 P 76ac35c9ce3d464aa161722c20409dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:38.249358   431 tablet_replica.cc:333] T a4c46b81357444b59b786dddc6394c6e P 76ac35c9ce3d464aa161722c20409dab: stopping tablet replica
I20260709 13:07:38.249935   431 raft_consensus.cc:2243] T a4c46b81357444b59b786dddc6394c6e P 76ac35c9ce3d464aa161722c20409dab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:38.250356   431 raft_consensus.cc:2272] T a4c46b81357444b59b786dddc6394c6e P 76ac35c9ce3d464aa161722c20409dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:38.272233   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 13:07:38.289016   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 13:07:38.314431   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:38.315176   431 tablet_replica.cc:333] T 41cf2842993e40ea9d5a6487ce869ef7 P 02d39eaf7c2340ad91fd4b2715ce9e34: stopping tablet replica
I20260709 13:07:38.315877   431 raft_consensus.cc:2243] T 41cf2842993e40ea9d5a6487ce869ef7 P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:38.316394   431 raft_consensus.cc:2272] T 41cf2842993e40ea9d5a6487ce869ef7 P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:38.319063   431 tablet_replica.cc:333] T 103b8f10ade14b63b20a93c2fc4502ae P 02d39eaf7c2340ad91fd4b2715ce9e34: stopping tablet replica
I20260709 13:07:38.319672   431 raft_consensus.cc:2243] T 103b8f10ade14b63b20a93c2fc4502ae P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:38.320137   431 raft_consensus.cc:2272] T 103b8f10ade14b63b20a93c2fc4502ae P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:38.323285   431 tablet_replica.cc:333] T cf364215b8ce465283566e05ad4cd9c5 P 02d39eaf7c2340ad91fd4b2715ce9e34: stopping tablet replica
I20260709 13:07:38.323869   431 raft_consensus.cc:2243] T cf364215b8ce465283566e05ad4cd9c5 P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:38.324293   431 raft_consensus.cc:2272] T cf364215b8ce465283566e05ad4cd9c5 P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:38.326144   431 tablet_replica.cc:333] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34: stopping tablet replica
I20260709 13:07:38.326596   431 raft_consensus.cc:2243] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:38.327368   431 raft_consensus.cc:2272] T a4c46b81357444b59b786dddc6394c6e P 02d39eaf7c2340ad91fd4b2715ce9e34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:38.349594   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:07:38.362000   431 master.cc:562] Master@127.0.107.254:41709 shutting down...
I20260709 13:07:38.377904   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:38.378412   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:38.378677   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9766fb1c78d3432ba61d5be429507fd7: stopping tablet replica
I20260709 13:07:38.397941   431 master.cc:584] Master@127.0.107.254:41709 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2794 ms)
[----------] 6 tests from LeaderRebalancerTest (403362 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:07:38.428777   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:41623
I20260709 13:07:38.429883   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:38.435135 12287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:38.435360 12288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:38.436595 12290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:38.438021   431 server_base.cc:1061] running on GCE node
I20260709 13:07:38.438828   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:38.438997   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:38.439110   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602458439098 us; error 0 us; skew 500 ppm
I20260709 13:07:38.439558   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:38.441947   431 webserver.cc:533] Webserver started at http://127.0.107.254:44311/ using document root <none> and password file <none>
I20260709 13:07:38.442380   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:38.442526   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:38.442739   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:38.443722   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602054858684-431-0/minicluster-data/master-0-root/instance:
uuid: "35223ddd203348d8a6dc43fe34afdcf5"
format_stamp: "Formatted at 2026-07-09 13:07:38 on dist-test-slave-mxd3"
I20260709 13:07:38.448374   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:07:38.451905 12295 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:38.452714   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 13:07:38.453022   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602054858684-431-0/minicluster-data/master-0-root
uuid: "35223ddd203348d8a6dc43fe34afdcf5"
format_stamp: "Formatted at 2026-07-09 13:07:38 on dist-test-slave-mxd3"
I20260709 13:07:38.453465   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602054858684-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602054858684-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602054858684-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:38.478955   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:38.479964   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:38.515828   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:41623
I20260709 13:07:38.515954 12346 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:41623 every 8 connection(s)
I20260709 13:07:38.519659 12347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:38.530040 12347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5: Bootstrap starting.
I20260709 13:07:38.534417 12347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.538482 12347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5: No bootstrap required, opened a new log
I20260709 13:07:38.540541 12347 raft_consensus.cc:359] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35223ddd203348d8a6dc43fe34afdcf5" member_type: VOTER }
I20260709 13:07:38.540922 12347 raft_consensus.cc:385] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.541199 12347 raft_consensus.cc:740] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35223ddd203348d8a6dc43fe34afdcf5, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.541846 12347 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35223ddd203348d8a6dc43fe34afdcf5" member_type: VOTER }
I20260709 13:07:38.542308 12347 raft_consensus.cc:399] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:07:38.542510 12347 raft_consensus.cc:493] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:07:38.542793 12347 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.547744 12347 raft_consensus.cc:515] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35223ddd203348d8a6dc43fe34afdcf5" member_type: VOTER }
I20260709 13:07:38.548280 12347 leader_election.cc:304] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35223ddd203348d8a6dc43fe34afdcf5; no voters: 
I20260709 13:07:38.549419 12347 leader_election.cc:290] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:07:38.549983 12350 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:38.551419 12350 raft_consensus.cc:697] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5 [term 1 LEADER]: Becoming Leader. State: Replica: 35223ddd203348d8a6dc43fe34afdcf5, State: Running, Role: LEADER
I20260709 13:07:38.552131 12350 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35223ddd203348d8a6dc43fe34afdcf5" member_type: VOTER }
I20260709 13:07:38.552265 12347 sys_catalog.cc:565] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:07:38.554870 12352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35223ddd203348d8a6dc43fe34afdcf5. Latest consensus state: current_term: 1 leader_uuid: "35223ddd203348d8a6dc43fe34afdcf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35223ddd203348d8a6dc43fe34afdcf5" member_type: VOTER } }
I20260709 13:07:38.554837 12351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35223ddd203348d8a6dc43fe34afdcf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35223ddd203348d8a6dc43fe34afdcf5" member_type: VOTER } }
I20260709 13:07:38.555557 12352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:38.555591 12351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:38.558241 12355 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:07:38.564527 12355 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:07:38.567216   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:07:38.574451 12355 catalog_manager.cc:1383] Generated new cluster ID: c43f3d68348045d0affedbdecaf9bd41
I20260709 13:07:38.574787 12355 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:07:38.591805 12355 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:07:38.593495 12355 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:07:38.609520 12355 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5: Generated new TSK 0
I20260709 13:07:38.610214 12355 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:07:38.633920   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:38.640282 12368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:38.641340 12369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:38.643298 12371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:38.643401   431 server_base.cc:1061] running on GCE node
I20260709 13:07:38.644629   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:38.645071   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:38.645289   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602458645263 us; error 0 us; skew 500 ppm
I20260709 13:07:38.646018   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:38.648593   431 webserver.cc:533] Webserver started at http://127.0.107.193:44363/ using document root <none> and password file <none>
I20260709 13:07:38.649101   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:38.649315   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:38.649591   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:38.650784   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602054858684-431-0/minicluster-data/ts-0-root/instance:
uuid: "1bcc393b4836466e9be8845ba140e5e3"
format_stamp: "Formatted at 2026-07-09 13:07:38 on dist-test-slave-mxd3"
I20260709 13:07:38.655838   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:07:38.659565 12376 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:38.660413   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:07:38.660748   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602054858684-431-0/minicluster-data/ts-0-root
uuid: "1bcc393b4836466e9be8845ba140e5e3"
format_stamp: "Formatted at 2026-07-09 13:07:38 on dist-test-slave-mxd3"
I20260709 13:07:38.661068   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602054858684-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602054858684-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602054858684-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:38.679172   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:38.680471   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:38.682164   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:38.684710   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:38.684928   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:38.685277   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:38.685436   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:38.727569   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:45821
I20260709 13:07:38.727685 12438 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:45821 every 8 connection(s)
I20260709 13:07:38.732709   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:38.740593 12443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:38.743408 12444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:38.744552   431 server_base.cc:1061] running on GCE node
W20260709 13:07:38.745342 12446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:38.746672   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:38.746995   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:38.747223   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602458747205 us; error 0 us; skew 500 ppm
I20260709 13:07:38.747956   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:38.748627 12439 heartbeater.cc:344] Connected to a master server at 127.0.107.254:41623
I20260709 13:07:38.748950 12439 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:38.749694 12439 heartbeater.cc:507] Master 127.0.107.254:41623 requested a full tablet report, sending...
I20260709 13:07:38.751919   431 webserver.cc:533] Webserver started at http://127.0.107.194:41119/ using document root <none> and password file <none>
I20260709 13:07:38.752086 12312 ts_manager.cc:194] Registered new tserver with Master: 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821)
I20260709 13:07:38.752535   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:38.752836   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:38.753108   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:38.754305   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602054858684-431-0/minicluster-data/ts-1-root/instance:
uuid: "b186a014c0b745b8a66c39dae4efed4f"
format_stamp: "Formatted at 2026-07-09 13:07:38 on dist-test-slave-mxd3"
I20260709 13:07:38.754480 12312 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45130
I20260709 13:07:38.759588   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:07:38.763386 12451 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:38.764346   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:07:38.764650   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602054858684-431-0/minicluster-data/ts-1-root
uuid: "b186a014c0b745b8a66c39dae4efed4f"
format_stamp: "Formatted at 2026-07-09 13:07:38 on dist-test-slave-mxd3"
I20260709 13:07:38.764930   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602054858684-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602054858684-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602054858684-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:38.786106   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:38.787106   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:38.788519   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:38.790762   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:38.790939   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:38.791251   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:38.791391   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:38.830156   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:38975
I20260709 13:07:38.830271 12513 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:38975 every 8 connection(s)
I20260709 13:07:38.835233   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:38.843209 12517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:38.843566 12518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:38.847824 12520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:38.848404   431 server_base.cc:1061] running on GCE node
I20260709 13:07:38.848587 12514 heartbeater.cc:344] Connected to a master server at 127.0.107.254:41623
I20260709 13:07:38.849308 12514 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:38.849617   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:38.849946   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:38.850133   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602458850115 us; error 0 us; skew 500 ppm
I20260709 13:07:38.850232 12514 heartbeater.cc:507] Master 127.0.107.254:41623 requested a full tablet report, sending...
I20260709 13:07:38.850853   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:38.852583 12312 ts_manager.cc:194] Registered new tserver with Master: b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975)
I20260709 13:07:38.853917   431 webserver.cc:533] Webserver started at http://127.0.107.195:43501/ using document root <none> and password file <none>
I20260709 13:07:38.854668   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:38.854655 12312 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45134
I20260709 13:07:38.855189   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:38.855533   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:38.857156   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602054858684-431-0/minicluster-data/ts-2-root/instance:
uuid: "07b29f69b9a2421db3983613bdc480ea"
format_stamp: "Formatted at 2026-07-09 13:07:38 on dist-test-slave-mxd3"
I20260709 13:07:38.862174   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:07:38.865981 12525 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:38.867008   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:07:38.867372   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602054858684-431-0/minicluster-data/ts-2-root
uuid: "07b29f69b9a2421db3983613bdc480ea"
format_stamp: "Formatted at 2026-07-09 13:07:38 on dist-test-slave-mxd3"
I20260709 13:07:38.867650   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602054858684-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602054858684-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602054858684-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:38.888826   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:38.890007   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:38.891528   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:38.893914   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:38.894160   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:38.894418   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:38.894596   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:38.935451   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:38269
I20260709 13:07:38.935580 12587 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:38269 every 8 connection(s)
I20260709 13:07:38.949525 12588 heartbeater.cc:344] Connected to a master server at 127.0.107.254:41623
I20260709 13:07:38.950120 12588 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:38.950982 12588 heartbeater.cc:507] Master 127.0.107.254:41623 requested a full tablet report, sending...
I20260709 13:07:38.953080 12312 ts_manager.cc:194] Registered new tserver with Master: 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:38.953552   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014895721s
I20260709 13:07:38.954911 12312 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45140
I20260709 13:07:38.978673 12312 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45146:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:38.981238 12312 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:39.066771 12404 tablet_service.cc:1511] Processing CreateTablet for tablet 806d30f73a06482aaef3642acf872c32 (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:39.068317 12404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 806d30f73a06482aaef3642acf872c32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.076805 12403 tablet_service.cc:1511] Processing CreateTablet for tablet cd2f84a632224dcb84c681af4e30708b (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:39.078379 12403 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd2f84a632224dcb84c681af4e30708b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.078733 12552 tablet_service.cc:1511] Processing CreateTablet for tablet cd2f84a632224dcb84c681af4e30708b (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:39.080158 12552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd2f84a632224dcb84c681af4e30708b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.082351 12551 tablet_service.cc:1511] Processing CreateTablet for tablet 01368bb716a44c1a826f04f41b92f7fd (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:39.083330 12398 tablet_service.cc:1511] Processing CreateTablet for tablet f40a25a0fc5d4d0a99ec544061e5fa76 (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:39.083810 12551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01368bb716a44c1a826f04f41b92f7fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.084731 12398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f40a25a0fc5d4d0a99ec544061e5fa76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.085492 12545 tablet_service.cc:1511] Processing CreateTablet for tablet 88b86d1b66e74927ab047790c2b55af7 (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:39.087040 12545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88b86d1b66e74927ab047790c2b55af7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.086824 12399 tablet_service.cc:1511] Processing CreateTablet for tablet a931166102f144028b066b74033e4e96 (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:39.088222 12399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a931166102f144028b066b74033e4e96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.090741 12400 tablet_service.cc:1511] Processing CreateTablet for tablet 6a8a4aaaf35e4f9bac685028409cdb96 (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:39.091002 12396 tablet_service.cc:1511] Processing CreateTablet for tablet 88b86d1b66e74927ab047790c2b55af7 (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:39.091301 12402 tablet_service.cc:1511] Processing CreateTablet for tablet 01368bb716a44c1a826f04f41b92f7fd (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:39.092233 12400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a8a4aaaf35e4f9bac685028409cdb96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.092787 12397 tablet_service.cc:1511] Processing CreateTablet for tablet c2f1bab6aa8e4183808649fdcb0b1bb0 (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:39.093168 12401 tablet_service.cc:1511] Processing CreateTablet for tablet 20938c13e204477d89420fd9b4e62d0b (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:39.094750 12550 tablet_service.cc:1511] Processing CreateTablet for tablet 20938c13e204477d89420fd9b4e62d0b (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:39.096081 12550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20938c13e204477d89420fd9b4e62d0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.098840 12549 tablet_service.cc:1511] Processing CreateTablet for tablet 6a8a4aaaf35e4f9bac685028409cdb96 (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:39.100766 12549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a8a4aaaf35e4f9bac685028409cdb96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.103169 12402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01368bb716a44c1a826f04f41b92f7fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.103127 12548 tablet_service.cc:1511] Processing CreateTablet for tablet a931166102f144028b066b74033e4e96 (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:39.104940 12397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f1bab6aa8e4183808649fdcb0b1bb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.105170 12548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a931166102f144028b066b74033e4e96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.106343 12401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20938c13e204477d89420fd9b4e62d0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.107146 12547 tablet_service.cc:1511] Processing CreateTablet for tablet f40a25a0fc5d4d0a99ec544061e5fa76 (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:39.111726 12546 tablet_service.cc:1511] Processing CreateTablet for tablet c2f1bab6aa8e4183808649fdcb0b1bb0 (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:39.077248 12553 tablet_service.cc:1511] Processing CreateTablet for tablet 806d30f73a06482aaef3642acf872c32 (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:39.107766 12396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88b86d1b66e74927ab047790c2b55af7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.130213 12553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 806d30f73a06482aaef3642acf872c32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.134137 12547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f40a25a0fc5d4d0a99ec544061e5fa76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.138119 12546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f1bab6aa8e4183808649fdcb0b1bb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.156571 12604 tablet_bootstrap.cc:492] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3: Bootstrap starting.
I20260709 13:07:39.164901 12473 tablet_service.cc:1511] Processing CreateTablet for tablet f40a25a0fc5d4d0a99ec544061e5fa76 (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:39.165669 12479 tablet_service.cc:1511] Processing CreateTablet for tablet 806d30f73a06482aaef3642acf872c32 (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:39.166378 12473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f40a25a0fc5d4d0a99ec544061e5fa76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.167552 12479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 806d30f73a06482aaef3642acf872c32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.168514 12472 tablet_service.cc:1511] Processing CreateTablet for tablet c2f1bab6aa8e4183808649fdcb0b1bb0 (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:39.169858 12472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f1bab6aa8e4183808649fdcb0b1bb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.170583 12478 tablet_service.cc:1511] Processing CreateTablet for tablet cd2f84a632224dcb84c681af4e30708b (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:39.172356 12478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd2f84a632224dcb84c681af4e30708b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.175086 12477 tablet_service.cc:1511] Processing CreateTablet for tablet 01368bb716a44c1a826f04f41b92f7fd (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:39.176910 12477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01368bb716a44c1a826f04f41b92f7fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.179141 12476 tablet_service.cc:1511] Processing CreateTablet for tablet 20938c13e204477d89420fd9b4e62d0b (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:39.180974 12476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20938c13e204477d89420fd9b4e62d0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.183204 12475 tablet_service.cc:1511] Processing CreateTablet for tablet 6a8a4aaaf35e4f9bac685028409cdb96 (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:39.184955 12475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a8a4aaaf35e4f9bac685028409cdb96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.187322 12474 tablet_service.cc:1511] Processing CreateTablet for tablet a931166102f144028b066b74033e4e96 (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:39.189236 12474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a931166102f144028b066b74033e4e96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.175094 12471 tablet_service.cc:1511] Processing CreateTablet for tablet 88b86d1b66e74927ab047790c2b55af7 (DEFAULT_TABLE table=test-workload [id=c3cc21c17ad1473083ad621d5fbea66a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:39.195279 12471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88b86d1b66e74927ab047790c2b55af7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.202250 12603 tablet_bootstrap.cc:492] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea: Bootstrap starting.
I20260709 13:07:39.216919 12603 tablet_bootstrap.cc:654] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.233657 12604 tablet_bootstrap.cc:654] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.255831 12604 tablet_bootstrap.cc:492] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3: No bootstrap required, opened a new log
I20260709 13:07:39.256332 12604 ts_tablet_manager.cc:1403] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3: Time spent bootstrapping tablet: real 0.100s	user 0.011s	sys 0.000s
I20260709 13:07:39.259259 12604 raft_consensus.cc:359] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.260011 12604 raft_consensus.cc:385] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.260301 12604 raft_consensus.cc:740] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.261065 12604 consensus_queue.cc:260] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.263772 12605 tablet_bootstrap.cc:492] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f: Bootstrap starting.
I20260709 13:07:39.289091 12605 tablet_bootstrap.cc:654] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.299741 12604 ts_tablet_manager.cc:1434] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3: Time spent starting tablet: real 0.043s	user 0.010s	sys 0.026s
I20260709 13:07:39.300822 12439 heartbeater.cc:499] Master 127.0.107.254:41623 was elected leader, sending a full tablet report...
I20260709 13:07:39.301297 12604 tablet_bootstrap.cc:492] T a931166102f144028b066b74033e4e96 P 1bcc393b4836466e9be8845ba140e5e3: Bootstrap starting.
I20260709 13:07:39.307592 12603 tablet_bootstrap.cc:492] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea: No bootstrap required, opened a new log
I20260709 13:07:39.308385 12603 ts_tablet_manager.cc:1403] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea: Time spent bootstrapping tablet: real 0.107s	user 0.010s	sys 0.024s
I20260709 13:07:39.312383 12603 raft_consensus.cc:359] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.313407 12603 raft_consensus.cc:385] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.314118 12603 raft_consensus.cc:740] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.315321 12603 consensus_queue.cc:260] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.318009 12604 tablet_bootstrap.cc:654] T a931166102f144028b066b74033e4e96 P 1bcc393b4836466e9be8845ba140e5e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.321025 12605 tablet_bootstrap.cc:492] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f: No bootstrap required, opened a new log
I20260709 13:07:39.322199 12605 ts_tablet_manager.cc:1403] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f: Time spent bootstrapping tablet: real 0.060s	user 0.025s	sys 0.015s
I20260709 13:07:39.325692 12603 ts_tablet_manager.cc:1434] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea: Time spent starting tablet: real 0.017s	user 0.011s	sys 0.007s
I20260709 13:07:39.325985 12588 heartbeater.cc:499] Master 127.0.107.254:41623 was elected leader, sending a full tablet report...
I20260709 13:07:39.325860 12605 raft_consensus.cc:359] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.326968 12603 tablet_bootstrap.cc:492] T cd2f84a632224dcb84c681af4e30708b P 07b29f69b9a2421db3983613bdc480ea: Bootstrap starting.
I20260709 13:07:39.327108 12605 raft_consensus.cc:385] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.327673 12605 raft_consensus.cc:740] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.328786 12605 consensus_queue.cc:260] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.333597 12604 tablet_bootstrap.cc:492] T a931166102f144028b066b74033e4e96 P 1bcc393b4836466e9be8845ba140e5e3: No bootstrap required, opened a new log
I20260709 13:07:39.334427 12603 tablet_bootstrap.cc:654] T cd2f84a632224dcb84c681af4e30708b P 07b29f69b9a2421db3983613bdc480ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.334457 12604 ts_tablet_manager.cc:1403] T a931166102f144028b066b74033e4e96 P 1bcc393b4836466e9be8845ba140e5e3: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.013s
I20260709 13:07:39.339329 12605 ts_tablet_manager.cc:1434] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.001s
I20260709 13:07:39.340162 12514 heartbeater.cc:499] Master 127.0.107.254:41623 was elected leader, sending a full tablet report...
I20260709 13:07:39.340878 12604 raft_consensus.cc:359] T a931166102f144028b066b74033e4e96 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.341694 12604 raft_consensus.cc:385] T a931166102f144028b066b74033e4e96 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.342010 12604 raft_consensus.cc:740] T a931166102f144028b066b74033e4e96 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.342801 12604 consensus_queue.cc:260] T a931166102f144028b066b74033e4e96 P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.344476 12605 tablet_bootstrap.cc:492] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f: Bootstrap starting.
I20260709 13:07:39.349474 12604 ts_tablet_manager.cc:1434] T a931166102f144028b066b74033e4e96 P 1bcc393b4836466e9be8845ba140e5e3: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 13:07:39.350741 12604 tablet_bootstrap.cc:492] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3: Bootstrap starting.
I20260709 13:07:39.352056 12605 tablet_bootstrap.cc:654] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.360334 12603 tablet_bootstrap.cc:492] T cd2f84a632224dcb84c681af4e30708b P 07b29f69b9a2421db3983613bdc480ea: No bootstrap required, opened a new log
I20260709 13:07:39.361011 12603 ts_tablet_manager.cc:1403] T cd2f84a632224dcb84c681af4e30708b P 07b29f69b9a2421db3983613bdc480ea: Time spent bootstrapping tablet: real 0.034s	user 0.026s	sys 0.001s
I20260709 13:07:39.363785 12604 tablet_bootstrap.cc:654] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.364055 12603 raft_consensus.cc:359] T cd2f84a632224dcb84c681af4e30708b P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.365008 12603 raft_consensus.cc:385] T cd2f84a632224dcb84c681af4e30708b P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.365402 12603 raft_consensus.cc:740] T cd2f84a632224dcb84c681af4e30708b P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.366268 12603 consensus_queue.cc:260] T cd2f84a632224dcb84c681af4e30708b P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.368516 12603 ts_tablet_manager.cc:1434] T cd2f84a632224dcb84c681af4e30708b P 07b29f69b9a2421db3983613bdc480ea: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:07:39.370642 12605 tablet_bootstrap.cc:492] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f: No bootstrap required, opened a new log
I20260709 13:07:39.370807 12603 tablet_bootstrap.cc:492] T 6a8a4aaaf35e4f9bac685028409cdb96 P 07b29f69b9a2421db3983613bdc480ea: Bootstrap starting.
I20260709 13:07:39.371271 12605 ts_tablet_manager.cc:1403] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.012s
I20260709 13:07:39.374886 12605 raft_consensus.cc:359] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.375722 12605 raft_consensus.cc:385] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.376979 12604 tablet_bootstrap.cc:492] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3: No bootstrap required, opened a new log
I20260709 13:07:39.377130 12605 raft_consensus.cc:740] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.377682 12604 ts_tablet_manager.cc:1403] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.006s
I20260709 13:07:39.378201 12603 tablet_bootstrap.cc:654] T 6a8a4aaaf35e4f9bac685028409cdb96 P 07b29f69b9a2421db3983613bdc480ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.378218 12605 consensus_queue.cc:260] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.380875 12605 ts_tablet_manager.cc:1434] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:07:39.380647 12604 raft_consensus.cc:359] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.381443 12604 raft_consensus.cc:385] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.381771 12604 raft_consensus.cc:740] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.381935 12605 tablet_bootstrap.cc:492] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f: Bootstrap starting.
I20260709 13:07:39.382738 12604 consensus_queue.cc:260] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.388638 12605 tablet_bootstrap.cc:654] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.390327 12604 ts_tablet_manager.cc:1434] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.004s
I20260709 13:07:39.391503 12604 tablet_bootstrap.cc:492] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3: Bootstrap starting.
I20260709 13:07:39.394078 12603 tablet_bootstrap.cc:492] T 6a8a4aaaf35e4f9bac685028409cdb96 P 07b29f69b9a2421db3983613bdc480ea: No bootstrap required, opened a new log
I20260709 13:07:39.394639 12603 ts_tablet_manager.cc:1403] T 6a8a4aaaf35e4f9bac685028409cdb96 P 07b29f69b9a2421db3983613bdc480ea: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.011s
I20260709 13:07:39.397547 12604 tablet_bootstrap.cc:654] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.397545 12603 raft_consensus.cc:359] T 6a8a4aaaf35e4f9bac685028409cdb96 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.398488 12603 raft_consensus.cc:385] T 6a8a4aaaf35e4f9bac685028409cdb96 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.398828 12603 raft_consensus.cc:740] T 6a8a4aaaf35e4f9bac685028409cdb96 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.399704 12603 consensus_queue.cc:260] T 6a8a4aaaf35e4f9bac685028409cdb96 P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.401619 12605 tablet_bootstrap.cc:492] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f: No bootstrap required, opened a new log
I20260709 13:07:39.402200 12605 ts_tablet_manager.cc:1403] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.000s
I20260709 13:07:39.402510 12603 ts_tablet_manager.cc:1434] T 6a8a4aaaf35e4f9bac685028409cdb96 P 07b29f69b9a2421db3983613bdc480ea: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:07:39.403970 12603 tablet_bootstrap.cc:492] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea: Bootstrap starting.
I20260709 13:07:39.404970 12605 raft_consensus.cc:359] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.405805 12605 raft_consensus.cc:385] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.406104 12605 raft_consensus.cc:740] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.406884 12605 consensus_queue.cc:260] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.409178 12605 ts_tablet_manager.cc:1434] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:07:39.410146 12605 tablet_bootstrap.cc:492] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f: Bootstrap starting.
I20260709 13:07:39.413614 12603 tablet_bootstrap.cc:654] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.416337 12605 tablet_bootstrap.cc:654] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.426924 12603 tablet_bootstrap.cc:492] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea: No bootstrap required, opened a new log
I20260709 13:07:39.427340 12604 tablet_bootstrap.cc:492] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3: No bootstrap required, opened a new log
I20260709 13:07:39.427453 12603 ts_tablet_manager.cc:1403] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.004s
I20260709 13:07:39.427990 12604 ts_tablet_manager.cc:1403] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.018s
I20260709 13:07:39.430683 12603 raft_consensus.cc:359] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } }
I20260709 13:07:39.431041 12604 raft_consensus.cc:359] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.431541 12603 raft_consensus.cc:385] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.431821 12604 raft_consensus.cc:385] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.431921 12603 raft_consensus.cc:740] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.432211 12604 raft_consensus.cc:740] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.433075 12605 tablet_bootstrap.cc:492] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f: No bootstrap required, opened a new log
I20260709 13:07:39.432826 12603 consensus_queue.cc:260] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } }
I20260709 13:07:39.432987 12604 consensus_queue.cc:260] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.433676 12605 ts_tablet_manager.cc:1403] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.007s
I20260709 13:07:39.435209 12603 ts_tablet_manager.cc:1434] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:07:39.436316 12603 tablet_bootstrap.cc:492] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea: Bootstrap starting.
I20260709 13:07:39.438437 12604 ts_tablet_manager.cc:1434] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:07:39.439496 12604 tablet_bootstrap.cc:492] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 1bcc393b4836466e9be8845ba140e5e3: Bootstrap starting.
I20260709 13:07:39.442821 12603 tablet_bootstrap.cc:654] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.443734 12605 raft_consensus.cc:359] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } }
I20260709 13:07:39.444543 12605 raft_consensus.cc:385] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.444829 12605 raft_consensus.cc:740] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.445578 12605 consensus_queue.cc:260] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } }
I20260709 13:07:39.447829 12605 ts_tablet_manager.cc:1434] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:07:39.448920 12605 tablet_bootstrap.cc:492] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f: Bootstrap starting.
I20260709 13:07:39.455994 12603 tablet_bootstrap.cc:492] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea: No bootstrap required, opened a new log
I20260709 13:07:39.456774 12603 ts_tablet_manager.cc:1403] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.004s
I20260709 13:07:39.457515 12604 tablet_bootstrap.cc:654] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 1bcc393b4836466e9be8845ba140e5e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.456075 12605 tablet_bootstrap.cc:654] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.463442 12603 raft_consensus.cc:359] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.464468 12603 raft_consensus.cc:385] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.464777 12603 raft_consensus.cc:740] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.465551 12603 consensus_queue.cc:260] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.468503 12603 ts_tablet_manager.cc:1434] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:07:39.469904 12603 tablet_bootstrap.cc:492] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea: Bootstrap starting.
I20260709 13:07:39.473294 12612 raft_consensus.cc:493] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.473919 12612 raft_consensus.cc:515] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.476732 12612 leader_election.cc:290] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:39.477923 12603 tablet_bootstrap.cc:654] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.478305 12605 tablet_bootstrap.cc:492] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f: No bootstrap required, opened a new log
I20260709 13:07:39.478963 12605 ts_tablet_manager.cc:1403] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.004s
I20260709 13:07:39.482470 12605 raft_consensus.cc:359] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.483649 12605 raft_consensus.cc:385] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.484139 12605 raft_consensus.cc:740] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.485266 12605 consensus_queue.cc:260] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.488406 12605 ts_tablet_manager.cc:1434] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:07:39.490063 12605 tablet_bootstrap.cc:492] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f: Bootstrap starting.
I20260709 13:07:39.497629 12604 tablet_bootstrap.cc:492] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 1bcc393b4836466e9be8845ba140e5e3: No bootstrap required, opened a new log
I20260709 13:07:39.498282 12604 ts_tablet_manager.cc:1403] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 1bcc393b4836466e9be8845ba140e5e3: Time spent bootstrapping tablet: real 0.059s	user 0.016s	sys 0.004s
I20260709 13:07:39.499498 12605 tablet_bootstrap.cc:654] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.504192 12608 raft_consensus.cc:493] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.501489 12604 raft_consensus.cc:359] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.504911 12608 raft_consensus.cc:515] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.507336 12604 raft_consensus.cc:385] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.510040 12608 leader_election.cc:290] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:39.512246 12604 raft_consensus.cc:740] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.515319 12604 consensus_queue.cc:260] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.529983 12604 ts_tablet_manager.cc:1434] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 1bcc393b4836466e9be8845ba140e5e3: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.001s
I20260709 13:07:39.531461 12605 tablet_bootstrap.cc:492] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f: No bootstrap required, opened a new log
I20260709 13:07:39.531903 12603 tablet_bootstrap.cc:492] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea: No bootstrap required, opened a new log
I20260709 13:07:39.532255 12605 ts_tablet_manager.cc:1403] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f: Time spent bootstrapping tablet: real 0.042s	user 0.010s	sys 0.020s
I20260709 13:07:39.532409 12603 ts_tablet_manager.cc:1403] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea: Time spent bootstrapping tablet: real 0.063s	user 0.025s	sys 0.030s
I20260709 13:07:39.534853 12604 tablet_bootstrap.cc:492] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3: Bootstrap starting.
I20260709 13:07:39.536582 12603 raft_consensus.cc:359] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.541399 12612 raft_consensus.cc:493] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.542060 12563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f1bab6aa8e4183808649fdcb0b1bb0" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:39.542011 12612 raft_consensus.cc:515] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
W20260709 13:07:39.544234 12453 leader_election.cc:343] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.544350 12612 leader_election.cc:290] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:39.544185 12605 raft_consensus.cc:359] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.545615 12605 raft_consensus.cc:385] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.545974 12605 raft_consensus.cc:740] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.546360 12563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931166102f144028b066b74033e4e96" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:39.546813 12604 tablet_bootstrap.cc:654] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.546666 12605 consensus_queue.cc:260] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
W20260709 13:07:39.547842 12453 leader_election.cc:343] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.544574 12603 raft_consensus.cc:385] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.548506 12603 raft_consensus.cc:740] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.549381 12605 ts_tablet_manager.cc:1434] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:07:39.550799 12605 tablet_bootstrap.cc:492] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f: Bootstrap starting.
I20260709 13:07:39.549396 12603 consensus_queue.cc:260] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.556924 12603 ts_tablet_manager.cc:1434] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:07:39.558481 12603 tablet_bootstrap.cc:492] T 20938c13e204477d89420fd9b4e62d0b P 07b29f69b9a2421db3983613bdc480ea: Bootstrap starting.
W20260709 13:07:39.564723 12360 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:39.568467 12605 tablet_bootstrap.cc:654] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.568660 12603 tablet_bootstrap.cc:654] T 20938c13e204477d89420fd9b4e62d0b P 07b29f69b9a2421db3983613bdc480ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.571041 12563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8a4aaaf35e4f9bac685028409cdb96" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:39.571892 12563 raft_consensus.cc:2468] T 6a8a4aaaf35e4f9bac685028409cdb96 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 0.
I20260709 13:07:39.576802 12604 tablet_bootstrap.cc:492] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3: No bootstrap required, opened a new log
I20260709 13:07:39.577453 12604 ts_tablet_manager.cc:1403] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3: Time spent bootstrapping tablet: real 0.043s	user 0.010s	sys 0.004s
I20260709 13:07:39.577561 12414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f1bab6aa8e4183808649fdcb0b1bb0" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3" is_pre_election: true
I20260709 13:07:39.578384 12414 raft_consensus.cc:2468] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 0.
I20260709 13:07:39.579615 12489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8a4aaaf35e4f9bac685028409cdb96" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f" is_pre_election: true
I20260709 13:07:39.580394 12603 tablet_bootstrap.cc:492] T 20938c13e204477d89420fd9b4e62d0b P 07b29f69b9a2421db3983613bdc480ea: No bootstrap required, opened a new log
I20260709 13:07:39.580955 12604 raft_consensus.cc:359] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } }
I20260709 13:07:39.581135 12413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931166102f144028b066b74033e4e96" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3" is_pre_election: true
I20260709 13:07:39.581964 12604 raft_consensus.cc:385] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.582011 12455 leader_election.cc:304] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f; no voters: 07b29f69b9a2421db3983613bdc480ea
I20260709 13:07:39.582224 12413 raft_consensus.cc:2468] T a931166102f144028b066b74033e4e96 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 0.
I20260709 13:07:39.582479 12604 raft_consensus.cc:740] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.583331 12612 raft_consensus.cc:2804] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.583709 12612 raft_consensus.cc:493] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.583832 12604 consensus_queue.cc:260] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } }
I20260709 13:07:39.584228 12608 raft_consensus.cc:493] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.584105 12455 leader_election.cc:304] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f; no voters: 07b29f69b9a2421db3983613bdc480ea
I20260709 13:07:39.584956 12608 raft_consensus.cc:515] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.584100 12612 raft_consensus.cc:3060] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.587687 12608 leader_election.cc:290] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:39.581116 12603 ts_tablet_manager.cc:1403] T 20938c13e204477d89420fd9b4e62d0b P 07b29f69b9a2421db3983613bdc480ea: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.005s
I20260709 13:07:39.596884 12610 raft_consensus.cc:493] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.597659 12610 raft_consensus.cc:515] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.604377 12603 raft_consensus.cc:359] T 20938c13e204477d89420fd9b4e62d0b P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.600839 12612 raft_consensus.cc:515] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.610388 12610 leader_election.cc:290] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975)
I20260709 13:07:39.613843 12603 raft_consensus.cc:385] T 20938c13e204477d89420fd9b4e62d0b P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.614326 12563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f1bab6aa8e4183808649fdcb0b1bb0" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:39.614444 12603 raft_consensus.cc:740] T 20938c13e204477d89420fd9b4e62d0b P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.615168 12604 ts_tablet_manager.cc:1434] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3: Time spent starting tablet: real 0.037s	user 0.013s	sys 0.021s
I20260709 13:07:39.577623 12378 leader_election.cc:304] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3; no voters: 
I20260709 13:07:39.615732 12603 consensus_queue.cc:260] T 20938c13e204477d89420fd9b4e62d0b P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
W20260709 13:07:39.616963 12378 leader_election.cc:343] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.618029 12608 raft_consensus.cc:2804] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.618476 12608 raft_consensus.cc:493] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.618842 12608 raft_consensus.cc:3060] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.611728 12612 leader_election.cc:290] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:39.622310 12612 raft_consensus.cc:2804] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.622797 12612 raft_consensus.cc:493] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:39.622839 12453 leader_election.cc:343] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.623366 12612 raft_consensus.cc:3060] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.624248 12563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd2f84a632224dcb84c681af4e30708b" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:39.625957 12563 raft_consensus.cc:2468] T cd2f84a632224dcb84c681af4e30708b P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 0.
I20260709 13:07:39.626204 12604 tablet_bootstrap.cc:492] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3: Bootstrap starting.
I20260709 13:07:39.632186 12608 raft_consensus.cc:515] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.633010 12604 tablet_bootstrap.cc:654] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.634657 12378 leader_election.cc:304] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3; no voters: 
I20260709 13:07:39.637796 12603 ts_tablet_manager.cc:1434] T 20938c13e204477d89420fd9b4e62d0b P 07b29f69b9a2421db3983613bdc480ea: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.002s
I20260709 13:07:39.639253 12603 tablet_bootstrap.cc:492] T a931166102f144028b066b74033e4e96 P 07b29f69b9a2421db3983613bdc480ea: Bootstrap starting.
I20260709 13:07:39.646029 12633 raft_consensus.cc:2804] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.646924 12633 raft_consensus.cc:493] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.647437 12633 raft_consensus.cc:3060] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.649935 12563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8a4aaaf35e4f9bac685028409cdb96" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:39.650905 12563 raft_consensus.cc:3060] T 6a8a4aaaf35e4f9bac685028409cdb96 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.653923 12608 leader_election.cc:290] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 election: Requested vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
W20260709 13:07:39.654927 12455 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45821, user_credentials={real_user=slave}} blocked reactor thread for 70964us
I20260709 13:07:39.658557 12413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f1bab6aa8e4183808649fdcb0b1bb0" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
I20260709 13:07:39.659194 12413 raft_consensus.cc:3060] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.665257 12634 raft_consensus.cc:493] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.671298 12612 raft_consensus.cc:515] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.673921 12612 leader_election.cc:290] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:39.674355 12610 raft_consensus.cc:493] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.675117 12489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd2f84a632224dcb84c681af4e30708b" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f" is_pre_election: true
I20260709 13:07:39.675004 12610 raft_consensus.cc:515] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.676374 12605 tablet_bootstrap.cc:492] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f: No bootstrap required, opened a new log
I20260709 13:07:39.676451 12488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8a4aaaf35e4f9bac685028409cdb96" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:39.677201 12610 leader_election.cc:290] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975)
I20260709 13:07:39.677451 12612 raft_consensus.cc:493] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.677870 12414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931166102f144028b066b74033e4e96" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
I20260709 13:07:39.677238 12605 ts_tablet_manager.cc:1403] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f: Time spent bootstrapping tablet: real 0.127s	user 0.026s	sys 0.060s
W20260709 13:07:39.677897 12378 leader_election.cc:343] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.678187 12612 raft_consensus.cc:515] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } }
I20260709 13:07:39.679875 12414 raft_consensus.cc:3060] T a931166102f144028b066b74033e4e96 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.680778 12612 leader_election.cc:290] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:39.681134 12413 raft_consensus.cc:2468] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 1.
I20260709 13:07:39.681377 12605 raft_consensus.cc:359] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.682235 12605 raft_consensus.cc:385] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.682542 12605 raft_consensus.cc:740] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.684499 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88b86d1b66e74927ab047790c2b55af7" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3" is_pre_election: true
W20260709 13:07:39.686394 12378 leader_election.cc:343] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.686546 12411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f40a25a0fc5d4d0a99ec544061e5fa76" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3" is_pre_election: true
I20260709 13:07:39.686815 12563 raft_consensus.cc:2468] T 6a8a4aaaf35e4f9bac685028409cdb96 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 1.
I20260709 13:07:39.688025 12378 leader_election.cc:304] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3; no voters: b186a014c0b745b8a66c39dae4efed4f
I20260709 13:07:39.688956 12608 raft_consensus.cc:2804] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.686058 12634 raft_consensus.cc:515] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
W20260709 13:07:39.690871 12529 leader_election.cc:343] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.694193 12603 tablet_bootstrap.cc:654] T a931166102f144028b066b74033e4e96 P 07b29f69b9a2421db3983613bdc480ea: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:39.694444 12529 leader_election.cc:343] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.690984 12411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d30f73a06482aaef3642acf872c32" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3" is_pre_election: true
I20260709 13:07:39.690589 12605 consensus_queue.cc:260] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.695602 12411 raft_consensus.cc:2468] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 0.
I20260709 13:07:39.691615 12488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88b86d1b66e74927ab047790c2b55af7" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f" is_pre_election: true
I20260709 13:07:39.689287 12489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f40a25a0fc5d4d0a99ec544061e5fa76" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f" is_pre_election: true
I20260709 13:07:39.697552 12605 ts_tablet_manager.cc:1434] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 13:07:39.697938 12489 raft_consensus.cc:2468] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 0.
I20260709 13:07:39.699966 12527 leader_election.cc:304] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, b186a014c0b745b8a66c39dae4efed4f; no voters: 1bcc393b4836466e9be8845ba140e5e3
I20260709 13:07:39.701040 12604 tablet_bootstrap.cc:492] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3: No bootstrap required, opened a new log
I20260709 13:07:39.701598 12604 ts_tablet_manager.cc:1403] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3: Time spent bootstrapping tablet: real 0.076s	user 0.030s	sys 0.018s
I20260709 13:07:39.701793 12610 raft_consensus.cc:2804] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.702315 12610 raft_consensus.cc:493] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.702756 12610 raft_consensus.cc:3060] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.704991 12604 raft_consensus.cc:359] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.705852 12488 raft_consensus.cc:2468] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 0.
I20260709 13:07:39.706172 12604 raft_consensus.cc:385] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.706665 12604 raft_consensus.cc:740] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.707854 12604 consensus_queue.cc:260] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.711658 12563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931166102f144028b066b74033e4e96" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:39.692466 12414 raft_consensus.cc:2468] T a931166102f144028b066b74033e4e96 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 1.
I20260709 13:07:39.692400 12455 leader_election.cc:304] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f; no voters: 07b29f69b9a2421db3983613bdc480ea
W20260709 13:07:39.714418 12453 leader_election.cc:343] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.714653 12562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d30f73a06482aaef3642acf872c32" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:39.715456 12562 raft_consensus.cc:2468] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 0.
I20260709 13:07:39.715826 12604 ts_tablet_manager.cc:1434] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:07:39.715842 12605 tablet_bootstrap.cc:492] T cd2f84a632224dcb84c681af4e30708b P b186a014c0b745b8a66c39dae4efed4f: Bootstrap starting.
I20260709 13:07:39.693620 12633 raft_consensus.cc:515] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.717268 12527 leader_election.cc:304] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, b186a014c0b745b8a66c39dae4efed4f; no voters: 1bcc393b4836466e9be8845ba140e5e3
I20260709 13:07:39.713478 12641 raft_consensus.cc:493] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.718859 12641 raft_consensus.cc:515] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.722535 12604 tablet_bootstrap.cc:492] T f40a25a0fc5d4d0a99ec544061e5fa76 P 1bcc393b4836466e9be8845ba140e5e3: Bootstrap starting.
I20260709 13:07:39.723917 12605 tablet_bootstrap.cc:654] T cd2f84a632224dcb84c681af4e30708b P b186a014c0b745b8a66c39dae4efed4f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.723687 12610 raft_consensus.cc:515] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.715478 12455 leader_election.cc:304] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:39.725914 12642 raft_consensus.cc:493] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.725589 12612 raft_consensus.cc:493] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.726521 12642 raft_consensus.cc:515] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.727295 12612 raft_consensus.cc:515] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.730535 12644 raft_consensus.cc:2804] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.731029 12644 raft_consensus.cc:493] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.731418 12644 raft_consensus.cc:3060] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.728281 12562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd2f84a632224dcb84c681af4e30708b" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:39.728116 12643 raft_consensus.cc:2804] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.732079 12562 raft_consensus.cc:3060] T cd2f84a632224dcb84c681af4e30708b P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.732388 12643 raft_consensus.cc:493] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.733151 12643 raft_consensus.cc:3060] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.733855 12634 leader_election.cc:290] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:39.732451 12455 leader_election.cc:304] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f; no voters: 07b29f69b9a2421db3983613bdc480ea
I20260709 13:07:39.738755 12488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01368bb716a44c1a826f04f41b92f7fd" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f" is_pre_election: true
I20260709 13:07:39.739468 12634 raft_consensus.cc:2804] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.735050 12563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88b86d1b66e74927ab047790c2b55af7" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:39.741401 12411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88b86d1b66e74927ab047790c2b55af7" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3" is_pre_election: true
I20260709 13:07:39.729139 12638 raft_consensus.cc:2804] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.736398 12608 raft_consensus.cc:697] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Becoming Leader. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Running, Role: LEADER
I20260709 13:07:39.736886 12561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01368bb716a44c1a826f04f41b92f7fd" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:39.743265 12489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd2f84a632224dcb84c681af4e30708b" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:39.743801 12561 raft_consensus.cc:2468] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 0.
I20260709 13:07:39.741917 12487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20938c13e204477d89420fd9b4e62d0b" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f" is_pre_election: true
I20260709 13:07:39.744278 12608 consensus_queue.cc:237] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.739867 12414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01368bb716a44c1a826f04f41b92f7fd" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3" is_pre_election: true
I20260709 13:07:39.745175 12453 leader_election.cc:304] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:39.735042 12612 leader_election.cc:290] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:39.745718 12561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20938c13e204477d89420fd9b4e62d0b" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:39.749976 12561 raft_consensus.cc:2468] T 20938c13e204477d89420fd9b4e62d0b P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 0.
I20260709 13:07:39.729000 12633 leader_election.cc:290] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 election: Requested vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:39.739496 12488 raft_consensus.cc:2468] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 0.
I20260709 13:07:39.751165 12378 leader_election.cc:304] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3; no voters: 
I20260709 13:07:39.752219 12527 leader_election.cc:304] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:39.743367 12641 leader_election.cc:290] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975)
I20260709 13:07:39.745775 12414 raft_consensus.cc:2468] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 0.
I20260709 13:07:39.752705 12562 raft_consensus.cc:2468] T cd2f84a632224dcb84c681af4e30708b P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 1.
I20260709 13:07:39.756841 12633 raft_consensus.cc:2804] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.761287 12633 raft_consensus.cc:493] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.761485 12414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f40a25a0fc5d4d0a99ec544061e5fa76" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
W20260709 13:07:39.761749 12378 leader_election.cc:343] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.762615 12378 leader_election.cc:304] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3; no voters: b186a014c0b745b8a66c39dae4efed4f
I20260709 13:07:39.760593 12610 leader_election.cc:290] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975)
W20260709 13:07:39.763067 12529 leader_election.cc:343] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.763850 12639 raft_consensus.cc:2804] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.746827 12413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01368bb716a44c1a826f04f41b92f7fd" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3" is_pre_election: true
I20260709 13:07:39.752244 12612 raft_consensus.cc:2804] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.758352 12641 raft_consensus.cc:2804] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.746804 12642 leader_election.cc:290] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:39.777613 12638 raft_consensus.cc:697] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [term 1 LEADER]: Becoming Leader. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Running, Role: LEADER
I20260709 13:07:39.778517 12648 raft_consensus.cc:493] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.764804 12633 raft_consensus.cc:3060] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.768736 12612 raft_consensus.cc:493] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.808565 12604 tablet_bootstrap.cc:654] T f40a25a0fc5d4d0a99ec544061e5fa76 P 1bcc393b4836466e9be8845ba140e5e3: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:39.783106 12378 leader_election.cc:343] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.813969 12638 consensus_queue.cc:237] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.814818 12605 tablet_bootstrap.cc:492] T cd2f84a632224dcb84c681af4e30708b P b186a014c0b745b8a66c39dae4efed4f: No bootstrap required, opened a new log
I20260709 13:07:39.815287 12605 ts_tablet_manager.cc:1403] T cd2f84a632224dcb84c681af4e30708b P b186a014c0b745b8a66c39dae4efed4f: Time spent bootstrapping tablet: real 0.100s	user 0.010s	sys 0.024s
I20260709 13:07:39.805225 12648 raft_consensus.cc:515] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.818176 12605 raft_consensus.cc:359] T cd2f84a632224dcb84c681af4e30708b P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.818893 12605 raft_consensus.cc:385] T cd2f84a632224dcb84c681af4e30708b P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.819211 12605 raft_consensus.cc:740] T cd2f84a632224dcb84c681af4e30708b P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.819437 12648 leader_election.cc:290] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:39.805958 12612 raft_consensus.cc:3060] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.820134 12605 consensus_queue.cc:260] T cd2f84a632224dcb84c681af4e30708b P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.822515 12605 ts_tablet_manager.cc:1434] T cd2f84a632224dcb84c681af4e30708b P b186a014c0b745b8a66c39dae4efed4f: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.005s
I20260709 13:07:39.823487 12605 tablet_bootstrap.cc:492] T 20938c13e204477d89420fd9b4e62d0b P b186a014c0b745b8a66c39dae4efed4f: Bootstrap starting.
W20260709 13:07:39.804466 12453 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:38269, user_credentials={real_user=slave}} blocked reactor thread for 59394.1us
I20260709 13:07:39.806058 12639 raft_consensus.cc:697] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Becoming Leader. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Running, Role: LEADER
I20260709 13:07:39.830621 12562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8a4aaaf35e4f9bac685028409cdb96" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:39.831779 12562 raft_consensus.cc:2393] T 6a8a4aaaf35e4f9bac685028409cdb96 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b186a014c0b745b8a66c39dae4efed4f in current term 1: Already voted for candidate 1bcc393b4836466e9be8845ba140e5e3 in this term.
I20260709 13:07:39.830847 12639 consensus_queue.cc:237] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.832093 12612 raft_consensus.cc:515] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
W20260709 13:07:39.774242 12455 leader_election.cc:343] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.838719 12633 raft_consensus.cc:515] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.840071 12605 tablet_bootstrap.cc:654] T 20938c13e204477d89420fd9b4e62d0b P b186a014c0b745b8a66c39dae4efed4f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.837121 12603 tablet_bootstrap.cc:492] T a931166102f144028b066b74033e4e96 P 07b29f69b9a2421db3983613bdc480ea: No bootstrap required, opened a new log
I20260709 13:07:39.834969 12634 raft_consensus.cc:697] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [term 1 LEADER]: Becoming Leader. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Running, Role: LEADER
I20260709 13:07:39.841607 12603 ts_tablet_manager.cc:1403] T a931166102f144028b066b74033e4e96 P 07b29f69b9a2421db3983613bdc480ea: Time spent bootstrapping tablet: real 0.203s	user 0.021s	sys 0.003s
I20260709 13:07:39.842602 12634 consensus_queue.cc:237] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.846606 12644 raft_consensus.cc:515] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } }
I20260709 13:07:39.848371 12644 leader_election.cc:290] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:39.849967 12562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d30f73a06482aaef3642acf872c32" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:39.850809 12562 raft_consensus.cc:3060] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.800259 12641 raft_consensus.cc:493] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.783509 12413 raft_consensus.cc:2468] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 0.
W20260709 13:07:39.853729 12455 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45821, user_credentials={real_user=slave}} blocked reactor thread for 84788.3us
I20260709 13:07:39.857272 12414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8a4aaaf35e4f9bac685028409cdb96" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3" is_pre_election: true
I20260709 13:07:39.857092 12488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20938c13e204477d89420fd9b4e62d0b" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:39.858784 12411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d30f73a06482aaef3642acf872c32" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
I20260709 13:07:39.859462 12411 raft_consensus.cc:3060] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.860435 12603 raft_consensus.cc:359] T a931166102f144028b066b74033e4e96 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.861042 12562 raft_consensus.cc:2468] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 1.
I20260709 13:07:39.854465 12641 raft_consensus.cc:3060] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:39.858929 12378 leader_election.cc:343] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.863693 12561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01368bb716a44c1a826f04f41b92f7fd" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:39.864282 12643 raft_consensus.cc:515] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.865119 12455 leader_election.cc:304] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b186a014c0b745b8a66c39dae4efed4f; no voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3
I20260709 13:07:39.865507 12563 raft_consensus.cc:2393] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b186a014c0b745b8a66c39dae4efed4f in current term 1: Already voted for candidate 07b29f69b9a2421db3983613bdc480ea in this term.
I20260709 13:07:39.866277 12646 raft_consensus.cc:3060] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.866849 12413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01368bb716a44c1a826f04f41b92f7fd" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
I20260709 13:07:39.867666 12413 raft_consensus.cc:3060] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.861220 12603 raft_consensus.cc:385] T a931166102f144028b066b74033e4e96 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.870761 12603 raft_consensus.cc:740] T a931166102f144028b066b74033e4e96 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.871851 12603 consensus_queue.cc:260] T a931166102f144028b066b74033e4e96 P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.875274 12643 leader_election.cc:290] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975)
I20260709 13:07:39.876263 12453 leader_election.cc:304] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:39.876811 12411 raft_consensus.cc:2468] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 1.
I20260709 13:07:39.878141 12414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88b86d1b66e74927ab047790c2b55af7" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
I20260709 13:07:39.878813 12604 tablet_bootstrap.cc:492] T f40a25a0fc5d4d0a99ec544061e5fa76 P 1bcc393b4836466e9be8845ba140e5e3: No bootstrap required, opened a new log
W20260709 13:07:39.882457 12529 leader_election.cc:343] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.883338 12604 ts_tablet_manager.cc:1403] T f40a25a0fc5d4d0a99ec544061e5fa76 P 1bcc393b4836466e9be8845ba140e5e3: Time spent bootstrapping tablet: real 0.161s	user 0.005s	sys 0.016s
I20260709 13:07:39.885488 12646 raft_consensus.cc:2749] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:39.875384 12603 ts_tablet_manager.cc:1434] T a931166102f144028b066b74033e4e96 P 07b29f69b9a2421db3983613bdc480ea: Time spent starting tablet: real 0.033s	user 0.003s	sys 0.003s
I20260709 13:07:39.869894 12612 leader_election.cc:290] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:39.888934 12310 catalog_manager.cc:5697] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "1bcc393b4836466e9be8845ba140e5e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:39.889603 12604 raft_consensus.cc:359] T f40a25a0fc5d4d0a99ec544061e5fa76 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.890617 12604 raft_consensus.cc:385] T f40a25a0fc5d4d0a99ec544061e5fa76 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.890980 12604 raft_consensus.cc:740] T f40a25a0fc5d4d0a99ec544061e5fa76 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.891306 12453 leader_election.cc:304] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b186a014c0b745b8a66c39dae4efed4f; no voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3
I20260709 13:07:39.891959 12604 consensus_queue.cc:260] T f40a25a0fc5d4d0a99ec544061e5fa76 P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
W20260709 13:07:39.883668 12527 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:38975, user_credentials={real_user=slave}} blocked reactor thread for 131535us
I20260709 13:07:39.886137 12638 raft_consensus.cc:2804] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.903671 12603 tablet_bootstrap.cc:492] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea: Bootstrap starting.
I20260709 13:07:39.892705 12634 raft_consensus.cc:3060] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.897020 12641 raft_consensus.cc:515] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.905393 12488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f40a25a0fc5d4d0a99ec544061e5fa76" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:39.907893 12638 raft_consensus.cc:697] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 1 LEADER]: Becoming Leader. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Running, Role: LEADER
I20260709 13:07:39.908223 12488 raft_consensus.cc:3060] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.909453 12638 consensus_queue.cc:237] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } }
I20260709 13:07:39.914319 12561 raft_consensus.cc:2393] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b186a014c0b745b8a66c39dae4efed4f in current term 1: Already voted for candidate 07b29f69b9a2421db3983613bdc480ea in this term.
I20260709 13:07:39.915472 12641 leader_election.cc:290] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975)
I20260709 13:07:39.916402 12411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01368bb716a44c1a826f04f41b92f7fd" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
I20260709 13:07:39.917721 12487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88b86d1b66e74927ab047790c2b55af7" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:39.918715 12489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01368bb716a44c1a826f04f41b92f7fd" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:39.919566 12489 raft_consensus.cc:2393] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 07b29f69b9a2421db3983613bdc480ea in current term 1: Already voted for candidate b186a014c0b745b8a66c39dae4efed4f in this term.
I20260709 13:07:39.920941 12527 leader_election.cc:304] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea; no voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f
I20260709 13:07:39.922309 12641 raft_consensus.cc:2749] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:39.924362 12603 tablet_bootstrap.cc:654] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.926424 12604 ts_tablet_manager.cc:1434] T f40a25a0fc5d4d0a99ec544061e5fa76 P 1bcc393b4836466e9be8845ba140e5e3: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.001s
I20260709 13:07:39.927459 12604 tablet_bootstrap.cc:492] T 88b86d1b66e74927ab047790c2b55af7 P 1bcc393b4836466e9be8845ba140e5e3: Bootstrap starting.
I20260709 13:07:39.938745 12309 catalog_manager.cc:5697] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f reported cstate change: term changed from 0 to 1, leader changed from <none> to b186a014c0b745b8a66c39dae4efed4f (127.0.107.194). New cstate: current_term: 1 leader_uuid: "b186a014c0b745b8a66c39dae4efed4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:39.932433 12413 raft_consensus.cc:2468] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 1.
I20260709 13:07:39.958400 12633 leader_election.cc:290] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 election: Requested vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:39.958619 12488 raft_consensus.cc:2468] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 1.
I20260709 13:07:39.959216 12455 leader_election.cc:304] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f; no voters: 07b29f69b9a2421db3983613bdc480ea
I20260709 13:07:39.960430 12638 raft_consensus.cc:2804] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.960306 12527 leader_election.cc:304] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, b186a014c0b745b8a66c39dae4efed4f; no voters: 1bcc393b4836466e9be8845ba140e5e3
I20260709 13:07:39.961004 12638 raft_consensus.cc:697] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 1 LEADER]: Becoming Leader. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Running, Role: LEADER
I20260709 13:07:39.961207 12561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20938c13e204477d89420fd9b4e62d0b" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:39.961593 12641 raft_consensus.cc:2804] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.961913 12561 raft_consensus.cc:3060] T 20938c13e204477d89420fd9b4e62d0b P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.962324 12641 raft_consensus.cc:697] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Becoming Leader. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Running, Role: LEADER
I20260709 13:07:39.961964 12638 consensus_queue.cc:237] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.963277 12641 consensus_queue.cc:237] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.964766 12634 raft_consensus.cc:2749] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:39.968441 12604 tablet_bootstrap.cc:654] T 88b86d1b66e74927ab047790c2b55af7 P 1bcc393b4836466e9be8845ba140e5e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.970782 12561 raft_consensus.cc:2468] T 20938c13e204477d89420fd9b4e62d0b P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 1.
I20260709 13:07:39.972337 12378 leader_election.cc:304] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3; no voters: b186a014c0b745b8a66c39dae4efed4f
I20260709 13:07:39.973853 12633 raft_consensus.cc:2804] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.974471 12633 raft_consensus.cc:697] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Becoming Leader. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Running, Role: LEADER
I20260709 13:07:39.975575 12633 consensus_queue.cc:237] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.983098 12603 tablet_bootstrap.cc:492] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea: No bootstrap required, opened a new log
I20260709 13:07:39.983628 12603 ts_tablet_manager.cc:1403] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea: Time spent bootstrapping tablet: real 0.080s	user 0.018s	sys 0.008s
I20260709 13:07:39.986516 12603 raft_consensus.cc:359] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.987289 12603 raft_consensus.cc:385] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.987561 12603 raft_consensus.cc:740] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.988829 12605 tablet_bootstrap.cc:492] T 20938c13e204477d89420fd9b4e62d0b P b186a014c0b745b8a66c39dae4efed4f: No bootstrap required, opened a new log
I20260709 13:07:39.989449 12605 ts_tablet_manager.cc:1403] T 20938c13e204477d89420fd9b4e62d0b P b186a014c0b745b8a66c39dae4efed4f: Time spent bootstrapping tablet: real 0.166s	user 0.036s	sys 0.025s
I20260709 13:07:39.991726 12487 raft_consensus.cc:2468] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 1.
I20260709 13:07:39.994827 12527 leader_election.cc:304] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, b186a014c0b745b8a66c39dae4efed4f; no voters: 1bcc393b4836466e9be8845ba140e5e3
I20260709 13:07:39.989779 12603 consensus_queue.cc:260] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.998041 12605 raft_consensus.cc:359] T 20938c13e204477d89420fd9b4e62d0b P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:39.998705 12603 ts_tablet_manager.cc:1434] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:07:39.998776 12605 raft_consensus.cc:385] T 20938c13e204477d89420fd9b4e62d0b P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.996781 12641 raft_consensus.cc:2804] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.999444 12605 raft_consensus.cc:740] T 20938c13e204477d89420fd9b4e62d0b P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.000463 12641 raft_consensus.cc:697] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Becoming Leader. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Running, Role: LEADER
I20260709 13:07:40.000398 12605 consensus_queue.cc:260] T 20938c13e204477d89420fd9b4e62d0b P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:40.001834 12641 consensus_queue.cc:237] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:40.002693 12605 ts_tablet_manager.cc:1434] T 20938c13e204477d89420fd9b4e62d0b P b186a014c0b745b8a66c39dae4efed4f: Time spent starting tablet: real 0.013s	user 0.002s	sys 0.003s
I20260709 13:07:40.011392 12604 tablet_bootstrap.cc:492] T 88b86d1b66e74927ab047790c2b55af7 P 1bcc393b4836466e9be8845ba140e5e3: No bootstrap required, opened a new log
I20260709 13:07:40.010761 12311 catalog_manager.cc:5697] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b29f69b9a2421db3983613bdc480ea (127.0.107.195). New cstate: current_term: 1 leader_uuid: "07b29f69b9a2421db3983613bdc480ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:40.012121 12604 ts_tablet_manager.cc:1403] T 88b86d1b66e74927ab047790c2b55af7 P 1bcc393b4836466e9be8845ba140e5e3: Time spent bootstrapping tablet: real 0.085s	user 0.024s	sys 0.017s
I20260709 13:07:40.015368 12604 raft_consensus.cc:359] T 88b86d1b66e74927ab047790c2b55af7 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:40.016067 12604 raft_consensus.cc:385] T 88b86d1b66e74927ab047790c2b55af7 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.016337 12604 raft_consensus.cc:740] T 88b86d1b66e74927ab047790c2b55af7 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.017014 12604 consensus_queue.cc:260] T 88b86d1b66e74927ab047790c2b55af7 P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:40.019169 12604 ts_tablet_manager.cc:1434] T 88b86d1b66e74927ab047790c2b55af7 P 1bcc393b4836466e9be8845ba140e5e3: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:07:40.021246 12310 catalog_manager.cc:5697] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "1bcc393b4836466e9be8845ba140e5e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:40.023299 12310 catalog_manager.cc:5697] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "1bcc393b4836466e9be8845ba140e5e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:40.026731 12309 catalog_manager.cc:5697] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b29f69b9a2421db3983613bdc480ea (127.0.107.195). New cstate: current_term: 1 leader_uuid: "07b29f69b9a2421db3983613bdc480ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:40.036607 12311 catalog_manager.cc:5697] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f reported cstate change: term changed from 0 to 1, leader changed from <none> to b186a014c0b745b8a66c39dae4efed4f (127.0.107.194). New cstate: current_term: 1 leader_uuid: "b186a014c0b745b8a66c39dae4efed4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:40.038677 12311 catalog_manager.cc:5697] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f reported cstate change: term changed from 0 to 1, leader changed from <none> to b186a014c0b745b8a66c39dae4efed4f (127.0.107.194). New cstate: current_term: 1 leader_uuid: "b186a014c0b745b8a66c39dae4efed4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:40.040064 12311 catalog_manager.cc:5697] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f reported cstate change: term changed from 0 to 1, leader changed from <none> to b186a014c0b745b8a66c39dae4efed4f (127.0.107.194). New cstate: current_term: 1 leader_uuid: "b186a014c0b745b8a66c39dae4efed4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:40.099649 12638 raft_consensus.cc:493] T cd2f84a632224dcb84c681af4e30708b P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.100167 12638 raft_consensus.cc:515] T cd2f84a632224dcb84c681af4e30708b P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:40.101975 12638 leader_election.cc:290] T cd2f84a632224dcb84c681af4e30708b P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:40.102849 12413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd2f84a632224dcb84c681af4e30708b" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3" is_pre_election: true
I20260709 13:07:40.103018 12561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd2f84a632224dcb84c681af4e30708b" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:40.103835 12561 raft_consensus.cc:2393] T cd2f84a632224dcb84c681af4e30708b P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b186a014c0b745b8a66c39dae4efed4f in current term 1: Already voted for candidate 1bcc393b4836466e9be8845ba140e5e3 in this term.
I20260709 13:07:40.105024 12453 leader_election.cc:304] T cd2f84a632224dcb84c681af4e30708b P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b186a014c0b745b8a66c39dae4efed4f; no voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3
I20260709 13:07:40.105742 12638 raft_consensus.cc:3060] T cd2f84a632224dcb84c681af4e30708b P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.110682 12638 raft_consensus.cc:2749] T cd2f84a632224dcb84c681af4e30708b P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.116885 12641 raft_consensus.cc:493] T a931166102f144028b066b74033e4e96 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.117588 12641 raft_consensus.cc:515] T a931166102f144028b066b74033e4e96 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:40.119727 12641 leader_election.cc:290] T a931166102f144028b066b74033e4e96 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975)
I20260709 13:07:40.155452 12487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931166102f144028b066b74033e4e96" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f" is_pre_election: true
I20260709 13:07:40.154523 12413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931166102f144028b066b74033e4e96" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3" is_pre_election: true
I20260709 13:07:40.157457 12413 raft_consensus.cc:2393] T a931166102f144028b066b74033e4e96 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07b29f69b9a2421db3983613bdc480ea in current term 1: Already voted for candidate b186a014c0b745b8a66c39dae4efed4f in this term.
I20260709 13:07:40.159235 12529 leader_election.cc:304] T a931166102f144028b066b74033e4e96 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea; no voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f
I20260709 13:07:40.160261 12641 raft_consensus.cc:3060] T a931166102f144028b066b74033e4e96 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.169170 12641 raft_consensus.cc:2749] T a931166102f144028b066b74033e4e96 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.221328 12633 raft_consensus.cc:493] T f40a25a0fc5d4d0a99ec544061e5fa76 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.222368 12633 raft_consensus.cc:515] T f40a25a0fc5d4d0a99ec544061e5fa76 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:40.225274 12633 leader_election.cc:290] T f40a25a0fc5d4d0a99ec544061e5fa76 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:40.226833 12487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f40a25a0fc5d4d0a99ec544061e5fa76" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f" is_pre_election: true
I20260709 13:07:40.227842 12561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f40a25a0fc5d4d0a99ec544061e5fa76" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:40.228166 12487 raft_consensus.cc:2393] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1bcc393b4836466e9be8845ba140e5e3 in current term 1: Already voted for candidate 07b29f69b9a2421db3983613bdc480ea in this term.
I20260709 13:07:40.230126 12378 leader_election.cc:304] T f40a25a0fc5d4d0a99ec544061e5fa76 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3; no voters: 07b29f69b9a2421db3983613bdc480ea, b186a014c0b745b8a66c39dae4efed4f
I20260709 13:07:40.230949 12633 raft_consensus.cc:3060] T f40a25a0fc5d4d0a99ec544061e5fa76 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.241129 12633 raft_consensus.cc:2749] T f40a25a0fc5d4d0a99ec544061e5fa76 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.243379 12633 consensus_queue.cc:1048] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:40.265829 12487 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "806d30f73a06482aaef3642acf872c32"
dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
mode: GRACEFUL
new_leader_uuid: "1bcc393b4836466e9be8845ba140e5e3"
 from {username='slave'} at 127.0.0.1:38580
I20260709 13:07:40.266839 12487 raft_consensus.cc:606] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 1 LEADER]: Received request to transfer leadership to TS 1bcc393b4836466e9be8845ba140e5e3
I20260709 13:07:40.267242 12608 consensus_queue.cc:1048] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.279554 12488 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "01368bb716a44c1a826f04f41b92f7fd"
dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
mode: GRACEFUL
new_leader_uuid: "1bcc393b4836466e9be8845ba140e5e3"
 from {username='slave'} at 127.0.0.1:38580
I20260709 13:07:40.280279 12488 raft_consensus.cc:606] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 1 LEADER]: Received request to transfer leadership to TS 1bcc393b4836466e9be8845ba140e5e3
I20260709 13:07:40.300333 12413 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a8a4aaaf35e4f9bac685028409cdb96"
dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
mode: GRACEFUL
new_leader_uuid: "b186a014c0b745b8a66c39dae4efed4f"
 from {username='slave'} at 127.0.0.1:44378
I20260709 13:07:40.300920 12413 raft_consensus.cc:606] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Received request to transfer leadership to TS b186a014c0b745b8a66c39dae4efed4f
I20260709 13:07:40.318297 12561 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f40a25a0fc5d4d0a99ec544061e5fa76"
dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
mode: GRACEFUL
new_leader_uuid: "b186a014c0b745b8a66c39dae4efed4f"
 from {username='slave'} at 127.0.0.1:38922
I20260709 13:07:40.318833 12561 raft_consensus.cc:606] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Received request to transfer leadership to TS b186a014c0b745b8a66c39dae4efed4f
I20260709 13:07:40.330446 12608 raft_consensus.cc:493] T 88b86d1b66e74927ab047790c2b55af7 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.331146 12608 raft_consensus.cc:515] T 88b86d1b66e74927ab047790c2b55af7 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:40.332973 12608 leader_election.cc:290] T 88b86d1b66e74927ab047790c2b55af7 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:40.334087 12488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88b86d1b66e74927ab047790c2b55af7" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f" is_pre_election: true
I20260709 13:07:40.334347 12561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88b86d1b66e74927ab047790c2b55af7" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:40.335086 12488 raft_consensus.cc:2393] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1bcc393b4836466e9be8845ba140e5e3 in current term 1: Already voted for candidate 07b29f69b9a2421db3983613bdc480ea in this term.
I20260709 13:07:40.336061 12378 leader_election.cc:304] T 88b86d1b66e74927ab047790c2b55af7 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3; no voters: 07b29f69b9a2421db3983613bdc480ea, b186a014c0b745b8a66c39dae4efed4f
I20260709 13:07:40.336772 12608 raft_consensus.cc:3060] T 88b86d1b66e74927ab047790c2b55af7 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.344049 12608 raft_consensus.cc:2749] T 88b86d1b66e74927ab047790c2b55af7 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.354803 12634 consensus_queue.cc:1048] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.356768 12561 raft_consensus.cc:3060] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.359825 12634 raft_consensus.cc:493] T 20938c13e204477d89420fd9b4e62d0b P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.360882 12634 raft_consensus.cc:515] T 20938c13e204477d89420fd9b4e62d0b P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:40.362644 12638 consensus_queue.cc:1048] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.365396 12634 leader_election.cc:290] T 20938c13e204477d89420fd9b4e62d0b P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:40.367274 12562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20938c13e204477d89420fd9b4e62d0b" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:40.368741 12562 raft_consensus.cc:2393] T 20938c13e204477d89420fd9b4e62d0b P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b186a014c0b745b8a66c39dae4efed4f in current term 1: Already voted for candidate 1bcc393b4836466e9be8845ba140e5e3 in this term.
I20260709 13:07:40.365694 12413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20938c13e204477d89420fd9b4e62d0b" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3" is_pre_election: true
I20260709 13:07:40.371112 12453 leader_election.cc:304] T 20938c13e204477d89420fd9b4e62d0b P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b186a014c0b745b8a66c39dae4efed4f; no voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3
I20260709 13:07:40.373121 12646 raft_consensus.cc:3060] T 20938c13e204477d89420fd9b4e62d0b P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.372982 12638 consensus_queue.cc:1048] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:40.381686 12634 consensus_queue.cc:1048] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:40.382169 12633 consensus_queue.cc:1048] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:40.384163 12638 consensus_queue.cc:1048] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:07:40.397868 12608 consensus_queue.cc:1048] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.403903 12679 raft_consensus.cc:993] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f: : Instructing follower 1bcc393b4836466e9be8845ba140e5e3 to start an election
I20260709 13:07:40.404786 12679 raft_consensus.cc:1081] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 1 LEADER]: Signalling peer 1bcc393b4836466e9be8845ba140e5e3 to start an election
I20260709 13:07:40.417280 12608 consensus_queue.cc:1048] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:07:40.418218 12412 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "806d30f73a06482aaef3642acf872c32"
dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
 from {username='slave'} at 127.0.0.1:44362
I20260709 13:07:40.418972 12412 raft_consensus.cc:493] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:40.419346 12412 raft_consensus.cc:3060] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:40.428531 12612 consensus_queue.cc:1048] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.435672 12412 raft_consensus.cc:515] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } }
I20260709 13:07:40.442127 12646 raft_consensus.cc:2749] T 20938c13e204477d89420fd9b4e62d0b P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.449342 12560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d30f73a06482aaef3642acf872c32" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:40.457113 12412 leader_election.cc:290] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 2 election: Requested vote from peers 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269), b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975)
I20260709 13:07:40.458798 12489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d30f73a06482aaef3642acf872c32" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:40.459811 12489 raft_consensus.cc:3055] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:40.460383 12489 raft_consensus.cc:740] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Running, Role: LEADER
I20260709 13:07:40.461927 12489 consensus_queue.cc:260] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } }
I20260709 13:07:40.464178 12608 consensus_queue.cc:1048] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.464991 12489 raft_consensus.cc:3060] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:40.479416 12643 consensus_queue.cc:1048] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.517795 12489 raft_consensus.cc:2468] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 2.
I20260709 13:07:40.518661 12643 consensus_queue.cc:1048] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:40.520473 12378 leader_election.cc:304] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f; no voters: 07b29f69b9a2421db3983613bdc480ea
I20260709 13:07:40.521947 12639 raft_consensus.cc:2804] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:40.522425 12639 raft_consensus.cc:697] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 LEADER]: Becoming Leader. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Running, Role: LEADER
I20260709 13:07:40.523532 12639 consensus_queue.cc:237] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } }
I20260709 13:07:40.526103 12646 consensus_queue.cc:1048] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.536332 12646 consensus_queue.cc:1048] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.541366 12691 consensus_queue.cc:1048] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.569984 12643 raft_consensus.cc:993] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea: : Instructing follower b186a014c0b745b8a66c39dae4efed4f to start an election
I20260709 13:07:40.570637 12643 raft_consensus.cc:1081] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Signalling peer b186a014c0b745b8a66c39dae4efed4f to start an election
I20260709 13:07:40.574294 12489 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f40a25a0fc5d4d0a99ec544061e5fa76"
dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
 from {username='slave'} at 127.0.0.1:38570
I20260709 13:07:40.574925 12489 raft_consensus.cc:493] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:40.575376 12489 raft_consensus.cc:3060] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:40.578297 12612 raft_consensus.cc:993] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f: : Instructing follower 1bcc393b4836466e9be8845ba140e5e3 to start an election
I20260709 13:07:40.578863 12646 raft_consensus.cc:1081] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 1 LEADER]: Signalling peer 1bcc393b4836466e9be8845ba140e5e3 to start an election
I20260709 13:07:40.583030 12641 consensus_queue.cc:1048] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.586177 12489 raft_consensus.cc:515] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:40.587647 12412 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "01368bb716a44c1a826f04f41b92f7fd"
dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
 from {username='slave'} at 127.0.0.1:44362
I20260709 13:07:40.588359 12412 raft_consensus.cc:493] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:40.588546 12489 leader_election.cc:290] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 2 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:40.588819 12412 raft_consensus.cc:3060] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:40.595664 12560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f40a25a0fc5d4d0a99ec544061e5fa76" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:40.596319 12560 raft_consensus.cc:3055] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:40.596652 12560 raft_consensus.cc:740] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Running, Role: LEADER
I20260709 13:07:40.564694 12311 catalog_manager.cc:5697] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 reported cstate change: term changed from 1 to 2, leader changed from b186a014c0b745b8a66c39dae4efed4f (127.0.107.194) to 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "1bcc393b4836466e9be8845ba140e5e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:40.598027 12413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f40a25a0fc5d4d0a99ec544061e5fa76" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
I20260709 13:07:40.600483 12560 consensus_queue.cc:260] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:40.601802 12560 raft_consensus.cc:3060] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:40.606390 12643 consensus_queue.cc:1175] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b186a014c0b745b8a66c39dae4efed4f" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 13:07:40.609097 12412 raft_consensus.cc:515] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:40.619436 12412 leader_election.cc:290] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 2 election: Requested vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:40.620133 12489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01368bb716a44c1a826f04f41b92f7fd" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:40.620740 12563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01368bb716a44c1a826f04f41b92f7fd" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:40.620795 12489 raft_consensus.cc:3055] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:40.621403 12489 raft_consensus.cc:740] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Running, Role: LEADER
I20260709 13:07:40.621413 12563 raft_consensus.cc:3060] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:40.622803 12489 consensus_queue.cc:260] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:40.623473 12560 raft_consensus.cc:2468] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 2.
I20260709 13:07:40.624420 12489 raft_consensus.cc:3060] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:40.624917 12453 leader_election.cc:304] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, b186a014c0b745b8a66c39dae4efed4f; no voters: 1bcc393b4836466e9be8845ba140e5e3
I20260709 13:07:40.625828 12638 raft_consensus.cc:2804] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:40.626320 12638 raft_consensus.cc:697] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f [term 2 LEADER]: Becoming Leader. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Running, Role: LEADER
I20260709 13:07:40.627336 12638 consensus_queue.cc:237] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:40.632630 12563 raft_consensus.cc:2468] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 2.
I20260709 13:07:40.633420 12489 raft_consensus.cc:2468] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 2.
I20260709 13:07:40.634106 12378 leader_election.cc:304] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3; no voters: 
I20260709 13:07:40.635023 12639 raft_consensus.cc:2804] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:40.635545 12639 raft_consensus.cc:697] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 [term 2 LEADER]: Becoming Leader. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Running, Role: LEADER
I20260709 13:07:40.636595 12639 consensus_queue.cc:237] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:40.651772 12311 catalog_manager.cc:5697] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f reported cstate change: term changed from 1 to 2, leader changed from 07b29f69b9a2421db3983613bdc480ea (127.0.107.195) to b186a014c0b745b8a66c39dae4efed4f (127.0.107.194). New cstate: current_term: 2 leader_uuid: "b186a014c0b745b8a66c39dae4efed4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:40.661020 12310 catalog_manager.cc:5697] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 reported cstate change: term changed from 1 to 2, leader changed from b186a014c0b745b8a66c39dae4efed4f (127.0.107.194) to 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "1bcc393b4836466e9be8845ba140e5e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:40.792378 12608 raft_consensus.cc:993] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3: : Instructing follower b186a014c0b745b8a66c39dae4efed4f to start an election
I20260709 13:07:40.792968 12639 raft_consensus.cc:1081] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Signalling peer b186a014c0b745b8a66c39dae4efed4f to start an election
I20260709 13:07:40.794445 12489 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a8a4aaaf35e4f9bac685028409cdb96"
dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
 from {username='slave'} at 127.0.0.1:38566
I20260709 13:07:40.795056 12489 raft_consensus.cc:493] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:40.795415 12489 raft_consensus.cc:3060] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:40.800567 12489 raft_consensus.cc:515] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:40.802191 12489 leader_election.cc:290] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 2 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:40.803069 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8a4aaaf35e4f9bac685028409cdb96" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
I20260709 13:07:40.803236 12563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a8a4aaaf35e4f9bac685028409cdb96" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:40.803730 12412 raft_consensus.cc:3055] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:40.803807 12563 raft_consensus.cc:3060] T 6a8a4aaaf35e4f9bac685028409cdb96 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:40.804059 12412 raft_consensus.cc:740] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Running, Role: LEADER
I20260709 13:07:40.805039 12412 consensus_queue.cc:260] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:40.806058 12412 raft_consensus.cc:3060] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:40.808928 12563 raft_consensus.cc:2468] T 6a8a4aaaf35e4f9bac685028409cdb96 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 2.
I20260709 13:07:40.810107 12453 leader_election.cc:304] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:40.810792 12638 raft_consensus.cc:2804] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:40.811128 12412 raft_consensus.cc:2468] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 2.
I20260709 13:07:40.811306 12638 raft_consensus.cc:697] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [term 2 LEADER]: Becoming Leader. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Running, Role: LEADER
I20260709 13:07:40.812342 12638 consensus_queue.cc:237] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:40.818686 12310 catalog_manager.cc:5697] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f reported cstate change: term changed from 1 to 2, leader changed from 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193) to b186a014c0b745b8a66c39dae4efed4f (127.0.107.194). New cstate: current_term: 2 leader_uuid: "b186a014c0b745b8a66c39dae4efed4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:40.927829 12563 raft_consensus.cc:3060] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:40.933424 12563 raft_consensus.cc:1275] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Refusing update from remote peer 1bcc393b4836466e9be8845ba140e5e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:40.934873 12633 consensus_queue.cc:1048] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:40.943219 12489 raft_consensus.cc:1275] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Refusing update from remote peer 1bcc393b4836466e9be8845ba140e5e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:40.944593 12639 consensus_queue.cc:1048] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:41.046572 12563 raft_consensus.cc:1275] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Refusing update from remote peer 1bcc393b4836466e9be8845ba140e5e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:41.048266 12639 consensus_queue.cc:1048] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:41.060061 12489 raft_consensus.cc:1275] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Refusing update from remote peer 1bcc393b4836466e9be8845ba140e5e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:41.061632 12633 consensus_queue.cc:1048] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:41.111855 12412 raft_consensus.cc:3060] T f40a25a0fc5d4d0a99ec544061e5fa76 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:41.118957 12412 raft_consensus.cc:1275] T f40a25a0fc5d4d0a99ec544061e5fa76 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Refusing update from remote peer b186a014c0b745b8a66c39dae4efed4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:41.120517 12638 consensus_queue.cc:1048] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:41.130270 12563 raft_consensus.cc:1275] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Refusing update from remote peer b186a014c0b745b8a66c39dae4efed4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:41.131783 12646 consensus_queue.cc:1048] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:41.298764 12563 raft_consensus.cc:1275] T 6a8a4aaaf35e4f9bac685028409cdb96 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Refusing update from remote peer b186a014c0b745b8a66c39dae4efed4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:41.298763 12412 raft_consensus.cc:1275] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Refusing update from remote peer b186a014c0b745b8a66c39dae4efed4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:41.300501 12679 consensus_queue.cc:1048] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:41.301229 12646 consensus_queue.cc:1048] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:43.357982 12310 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45158:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:43.360598 12310 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:43.416191 12396 tablet_service.cc:1511] Processing CreateTablet for tablet 5258afcab7984790b025622d73eaf684 (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:43.417510 12396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5258afcab7984790b025622d73eaf684. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.419627 12476 tablet_service.cc:1511] Processing CreateTablet for tablet 5258afcab7984790b025622d73eaf684 (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:43.421254 12476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5258afcab7984790b025622d73eaf684. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.421334 12546 tablet_service.cc:1511] Processing CreateTablet for tablet 5258afcab7984790b025622d73eaf684 (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:43.423142 12546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5258afcab7984790b025622d73eaf684. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.428225 12404 tablet_service.cc:1511] Processing CreateTablet for tablet ab5220719559499aae8b369be60bfcce (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:43.429171 12478 tablet_service.cc:1511] Processing CreateTablet for tablet ab5220719559499aae8b369be60bfcce (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:43.429843 12404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab5220719559499aae8b369be60bfcce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.430119 12548 tablet_service.cc:1511] Processing CreateTablet for tablet ab5220719559499aae8b369be60bfcce (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:43.430807 12478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab5220719559499aae8b369be60bfcce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.431646 12548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab5220719559499aae8b369be60bfcce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.434918 12402 tablet_service.cc:1511] Processing CreateTablet for tablet 0dc35585b6454b7a89990d0f849e8704 (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:43.436302 12402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dc35585b6454b7a89990d0f849e8704. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.443960 12475 tablet_service.cc:1511] Processing CreateTablet for tablet 0dc35585b6454b7a89990d0f849e8704 (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:43.445719 12475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dc35585b6454b7a89990d0f849e8704. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.452768 12550 tablet_service.cc:1511] Processing CreateTablet for tablet 0dc35585b6454b7a89990d0f849e8704 (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:43.455199 12550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dc35585b6454b7a89990d0f849e8704. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.467772 12398 tablet_service.cc:1511] Processing CreateTablet for tablet 71b883cd452e46209d8ad4b02ec3d180 (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:43.470203 12547 tablet_service.cc:1511] Processing CreateTablet for tablet 71b883cd452e46209d8ad4b02ec3d180 (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:43.471159 12474 tablet_service.cc:1511] Processing CreateTablet for tablet 71b883cd452e46209d8ad4b02ec3d180 (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:43.471843 12547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71b883cd452e46209d8ad4b02ec3d180. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.472683 12474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71b883cd452e46209d8ad4b02ec3d180. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.469476 12398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71b883cd452e46209d8ad4b02ec3d180. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.488447 12397 tablet_service.cc:1511] Processing CreateTablet for tablet a0487087198a422f82c31677e12b5f5a (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:43.489995 12397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0487087198a422f82c31677e12b5f5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.501819 12477 tablet_service.cc:1511] Processing CreateTablet for tablet a0487087198a422f82c31677e12b5f5a (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:43.503374 12477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0487087198a422f82c31677e12b5f5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.514444 12399 tablet_service.cc:1511] Processing CreateTablet for tablet bb62b592dc854cd599b705b23ce7dc84 (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:43.515942 12399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb62b592dc854cd599b705b23ce7dc84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.516882 12551 tablet_service.cc:1511] Processing CreateTablet for tablet a0487087198a422f82c31677e12b5f5a (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:43.518507 12551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0487087198a422f82c31677e12b5f5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.524449 12479 tablet_service.cc:1511] Processing CreateTablet for tablet bb62b592dc854cd599b705b23ce7dc84 (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:43.526062 12479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb62b592dc854cd599b705b23ce7dc84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.535012 12717 tablet_bootstrap.cc:492] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f: Bootstrap starting.
I20260709 13:07:43.552318 12553 tablet_service.cc:1511] Processing CreateTablet for tablet bb62b592dc854cd599b705b23ce7dc84 (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:43.555023 12553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb62b592dc854cd599b705b23ce7dc84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.572439 12396 tablet_service.cc:1511] Processing CreateTablet for tablet 38b1e20fd4b04f08bd39ec6cc9792b5c (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:43.574461 12396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38b1e20fd4b04f08bd39ec6cc9792b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.595417 12716 tablet_bootstrap.cc:492] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3: Bootstrap starting.
I20260709 13:07:43.600101 12402 tablet_service.cc:1511] Processing CreateTablet for tablet f05b5fc6b3b248ffb2e34876863341a4 (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:43.602134 12402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f05b5fc6b3b248ffb2e34876863341a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.602252 12478 tablet_service.cc:1511] Processing CreateTablet for tablet 38b1e20fd4b04f08bd39ec6cc9792b5c (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:43.602780 12549 tablet_service.cc:1511] Processing CreateTablet for tablet 38b1e20fd4b04f08bd39ec6cc9792b5c (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:43.612552 12478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38b1e20fd4b04f08bd39ec6cc9792b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.613612 12549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38b1e20fd4b04f08bd39ec6cc9792b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.621186 12552 tablet_service.cc:1511] Processing CreateTablet for tablet f05b5fc6b3b248ffb2e34876863341a4 (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:43.622668 12552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f05b5fc6b3b248ffb2e34876863341a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.625155 12397 tablet_service.cc:1511] Processing CreateTablet for tablet bce27e34734e4e868e207d32b9665e91 (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:43.629958 12476 tablet_service.cc:1511] Processing CreateTablet for tablet bce27e34734e4e868e207d32b9665e91 (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:43.631433 12476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bce27e34734e4e868e207d32b9665e91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.631350 12397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bce27e34734e4e868e207d32b9665e91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.634547 12474 tablet_service.cc:1511] Processing CreateTablet for tablet f05b5fc6b3b248ffb2e34876863341a4 (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:43.613060 12717 tablet_bootstrap.cc:654] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.635540 12545 tablet_service.cc:1511] Processing CreateTablet for tablet bce27e34734e4e868e207d32b9665e91 (DEFAULT_TABLE table=soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:43.636247 12474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f05b5fc6b3b248ffb2e34876863341a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:43.637043 12545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bce27e34734e4e868e207d32b9665e91. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:07:43.639751 12360 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:43.674840 12716 tablet_bootstrap.cc:654] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.706604 12719 tablet_bootstrap.cc:492] T a0487087198a422f82c31677e12b5f5a P 07b29f69b9a2421db3983613bdc480ea: Bootstrap starting.
I20260709 13:07:43.719034 12717 tablet_bootstrap.cc:492] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f: No bootstrap required, opened a new log
I20260709 13:07:43.719614 12717 ts_tablet_manager.cc:1403] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f: Time spent bootstrapping tablet: real 0.185s	user 0.008s	sys 0.030s
I20260709 13:07:43.722506 12717 raft_consensus.cc:359] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.723318 12717 raft_consensus.cc:385] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.723716 12717 raft_consensus.cc:740] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.724695 12717 consensus_queue.cc:260] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.728555 12719 tablet_bootstrap.cc:654] T a0487087198a422f82c31677e12b5f5a P 07b29f69b9a2421db3983613bdc480ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.736213 12716 tablet_bootstrap.cc:492] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3: No bootstrap required, opened a new log
I20260709 13:07:43.736768 12716 ts_tablet_manager.cc:1403] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3: Time spent bootstrapping tablet: real 0.142s	user 0.015s	sys 0.021s
I20260709 13:07:43.737298 12717 ts_tablet_manager.cc:1434] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:07:43.740121 12716 raft_consensus.cc:359] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.740725 12719 tablet_bootstrap.cc:492] T a0487087198a422f82c31677e12b5f5a P 07b29f69b9a2421db3983613bdc480ea: No bootstrap required, opened a new log
I20260709 13:07:43.740972 12716 raft_consensus.cc:385] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.741305 12719 ts_tablet_manager.cc:1403] T a0487087198a422f82c31677e12b5f5a P 07b29f69b9a2421db3983613bdc480ea: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.008s
I20260709 13:07:43.741333 12716 raft_consensus.cc:740] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.742522 12716 consensus_queue.cc:260] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.744578 12719 raft_consensus.cc:359] T a0487087198a422f82c31677e12b5f5a P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.745549 12719 raft_consensus.cc:385] T a0487087198a422f82c31677e12b5f5a P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.745163 12716 ts_tablet_manager.cc:1434] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:43.745959 12719 raft_consensus.cc:740] T a0487087198a422f82c31677e12b5f5a P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.747090 12716 tablet_bootstrap.cc:492] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3: Bootstrap starting.
I20260709 13:07:43.746827 12719 consensus_queue.cc:260] T a0487087198a422f82c31677e12b5f5a P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.755815 12716 tablet_bootstrap.cc:654] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.762359 12717 tablet_bootstrap.cc:492] T ab5220719559499aae8b369be60bfcce P b186a014c0b745b8a66c39dae4efed4f: Bootstrap starting.
I20260709 13:07:43.767113 12719 ts_tablet_manager.cc:1434] T a0487087198a422f82c31677e12b5f5a P 07b29f69b9a2421db3983613bdc480ea: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 13:07:43.768394 12719 tablet_bootstrap.cc:492] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea: Bootstrap starting.
I20260709 13:07:43.774757 12719 tablet_bootstrap.cc:654] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.780884 12717 tablet_bootstrap.cc:654] T ab5220719559499aae8b369be60bfcce P b186a014c0b745b8a66c39dae4efed4f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.787714 12716 tablet_bootstrap.cc:492] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3: No bootstrap required, opened a new log
I20260709 13:07:43.788302 12716 ts_tablet_manager.cc:1403] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3: Time spent bootstrapping tablet: real 0.041s	user 0.017s	sys 0.017s
I20260709 13:07:43.788751 12717 tablet_bootstrap.cc:492] T ab5220719559499aae8b369be60bfcce P b186a014c0b745b8a66c39dae4efed4f: No bootstrap required, opened a new log
I20260709 13:07:43.789203 12719 tablet_bootstrap.cc:492] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea: No bootstrap required, opened a new log
I20260709 13:07:43.789927 12719 ts_tablet_manager.cc:1403] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.000s
I20260709 13:07:43.791515 12716 raft_consensus.cc:359] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.789217 12717 ts_tablet_manager.cc:1403] T ab5220719559499aae8b369be60bfcce P b186a014c0b745b8a66c39dae4efed4f: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.000s
I20260709 13:07:43.792285 12716 raft_consensus.cc:385] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.792737 12716 raft_consensus.cc:740] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.792928 12719 raft_consensus.cc:359] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.793780 12719 raft_consensus.cc:385] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.793721 12716 consensus_queue.cc:260] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.794206 12719 raft_consensus.cc:740] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.795118 12719 consensus_queue.cc:260] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.796249 12716 ts_tablet_manager.cc:1434] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:07:43.795553 12717 raft_consensus.cc:359] T ab5220719559499aae8b369be60bfcce P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.796798 12717 raft_consensus.cc:385] T ab5220719559499aae8b369be60bfcce P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.797163 12717 raft_consensus.cc:740] T ab5220719559499aae8b369be60bfcce P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.797791 12719 ts_tablet_manager.cc:1434] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:07:43.798868 12719 tablet_bootstrap.cc:492] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea: Bootstrap starting.
I20260709 13:07:43.799137 12716 tablet_bootstrap.cc:492] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3: Bootstrap starting.
I20260709 13:07:43.798128 12717 consensus_queue.cc:260] T ab5220719559499aae8b369be60bfcce P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.806469 12716 tablet_bootstrap.cc:654] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.806694 12717 ts_tablet_manager.cc:1434] T ab5220719559499aae8b369be60bfcce P b186a014c0b745b8a66c39dae4efed4f: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:07:43.807809 12717 tablet_bootstrap.cc:492] T 71b883cd452e46209d8ad4b02ec3d180 P b186a014c0b745b8a66c39dae4efed4f: Bootstrap starting.
I20260709 13:07:43.810974 12719 tablet_bootstrap.cc:654] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.814971 12717 tablet_bootstrap.cc:654] T 71b883cd452e46209d8ad4b02ec3d180 P b186a014c0b745b8a66c39dae4efed4f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.822744 12716 tablet_bootstrap.cc:492] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3: No bootstrap required, opened a new log
I20260709 13:07:43.823529 12716 ts_tablet_manager.cc:1403] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.008s
I20260709 13:07:43.827864 12716 raft_consensus.cc:359] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.828691 12716 raft_consensus.cc:385] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.829038 12716 raft_consensus.cc:740] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.830396 12717 tablet_bootstrap.cc:492] T 71b883cd452e46209d8ad4b02ec3d180 P b186a014c0b745b8a66c39dae4efed4f: No bootstrap required, opened a new log
I20260709 13:07:43.830945 12717 ts_tablet_manager.cc:1403] T 71b883cd452e46209d8ad4b02ec3d180 P b186a014c0b745b8a66c39dae4efed4f: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.000s
I20260709 13:07:43.832211 12719 tablet_bootstrap.cc:492] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea: No bootstrap required, opened a new log
I20260709 13:07:43.832660 12719 ts_tablet_manager.cc:1403] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.011s
I20260709 13:07:43.829998 12716 consensus_queue.cc:260] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.833338 12717 raft_consensus.cc:359] T 71b883cd452e46209d8ad4b02ec3d180 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.834052 12717 raft_consensus.cc:385] T 71b883cd452e46209d8ad4b02ec3d180 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.834408 12717 raft_consensus.cc:740] T 71b883cd452e46209d8ad4b02ec3d180 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.835381 12717 consensus_queue.cc:260] T 71b883cd452e46209d8ad4b02ec3d180 P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.835489 12719 raft_consensus.cc:359] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.836417 12719 raft_consensus.cc:385] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.836675 12716 ts_tablet_manager.cc:1434] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:07:43.836779 12719 raft_consensus.cc:740] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.837918 12716 tablet_bootstrap.cc:492] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3: Bootstrap starting.
I20260709 13:07:43.838340 12717 ts_tablet_manager.cc:1434] T 71b883cd452e46209d8ad4b02ec3d180 P b186a014c0b745b8a66c39dae4efed4f: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:43.837957 12719 consensus_queue.cc:260] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.839684 12717 tablet_bootstrap.cc:492] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f: Bootstrap starting.
I20260709 13:07:43.846720 12716 tablet_bootstrap.cc:654] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.850682 12719 ts_tablet_manager.cc:1434] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.003s
I20260709 13:07:43.852485 12719 tablet_bootstrap.cc:492] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea: Bootstrap starting.
I20260709 13:07:43.850862 12717 tablet_bootstrap.cc:654] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.865691 12719 tablet_bootstrap.cc:654] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.865988 12641 raft_consensus.cc:493] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.866586 12641 raft_consensus.cc:515] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.869041 12641 leader_election.cc:290] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975)
I20260709 13:07:43.870285 12489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5220719559499aae8b369be60bfcce" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f" is_pre_election: true
I20260709 13:07:43.871006 12489 raft_consensus.cc:2468] T ab5220719559499aae8b369be60bfcce P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 0.
I20260709 13:07:43.872481 12527 leader_election.cc:304] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:43.872905 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5220719559499aae8b369be60bfcce" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3" is_pre_election: true
W20260709 13:07:43.878252 12529 leader_election.cc:343] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:43.882479 12641 raft_consensus.cc:2804] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:43.883049 12641 raft_consensus.cc:493] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:43.883463 12641 raft_consensus.cc:3060] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.884545 12718 raft_consensus.cc:493] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.885103 12718 raft_consensus.cc:515] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.885898 12639 raft_consensus.cc:493] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.886915 12716 tablet_bootstrap.cc:492] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3: No bootstrap required, opened a new log
I20260709 13:07:43.886559 12639 raft_consensus.cc:515] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.887125 12719 tablet_bootstrap.cc:492] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea: No bootstrap required, opened a new log
I20260709 13:07:43.887519 12716 ts_tablet_manager.cc:1403] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3: Time spent bootstrapping tablet: real 0.050s	user 0.012s	sys 0.032s
I20260709 13:07:43.887706 12719 ts_tablet_manager.cc:1403] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea: Time spent bootstrapping tablet: real 0.036s	user 0.022s	sys 0.008s
I20260709 13:07:43.889628 12489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5258afcab7984790b025622d73eaf684" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f" is_pre_election: true
I20260709 13:07:43.890339 12489 raft_consensus.cc:2468] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 0.
I20260709 13:07:43.891085 12719 raft_consensus.cc:359] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.891441 12716 raft_consensus.cc:359] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.891867 12719 raft_consensus.cc:385] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.892374 12716 raft_consensus.cc:385] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.892473 12719 raft_consensus.cc:740] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.892884 12716 raft_consensus.cc:740] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.893030 12378 leader_election.cc:304] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:43.893447 12719 consensus_queue.cc:260] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.891906 12563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5258afcab7984790b025622d73eaf684" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:43.893836 12716 consensus_queue.cc:260] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
W20260709 13:07:43.896199 12453 leader_election.cc:343] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:43.897831 12639 leader_election.cc:290] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:43.898764 12639 raft_consensus.cc:2804] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:43.899269 12639 raft_consensus.cc:493] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:43.899552 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5258afcab7984790b025622d73eaf684" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3" is_pre_election: true
I20260709 13:07:43.896874 12641 raft_consensus.cc:515] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.902959 12717 tablet_bootstrap.cc:492] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f: No bootstrap required, opened a new log
I20260709 13:07:43.903554 12717 ts_tablet_manager.cc:1403] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f: Time spent bootstrapping tablet: real 0.064s	user 0.013s	sys 0.002s
I20260709 13:07:43.906267 12641 leader_election.cc:290] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975)
I20260709 13:07:43.907063 12717 raft_consensus.cc:359] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.908628 12717 raft_consensus.cc:385] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.909024 12717 raft_consensus.cc:740] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.910091 12489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5220719559499aae8b369be60bfcce" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:43.909895 12717 consensus_queue.cc:260] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.910763 12489 raft_consensus.cc:3060] T ab5220719559499aae8b369be60bfcce P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.911731 12717 ts_tablet_manager.cc:1434] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:07:43.911780 12716 ts_tablet_manager.cc:1434] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.008s
I20260709 13:07:43.912916 12717 tablet_bootstrap.cc:492] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f: Bootstrap starting.
I20260709 13:07:43.913002 12716 tablet_bootstrap.cc:492] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3: Bootstrap starting.
I20260709 13:07:43.916445 12414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5220719559499aae8b369be60bfcce" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
I20260709 13:07:43.920962 12717 tablet_bootstrap.cc:654] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.921528 12734 raft_consensus.cc:493] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:43.921823 12529 leader_election.cc:343] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:43.922122 12734 raft_consensus.cc:515] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.924425 12734 leader_election.cc:290] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:43.926367 12716 tablet_bootstrap.cc:654] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.926918 12719 ts_tablet_manager.cc:1434] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.014s
I20260709 13:07:43.922524 12718 leader_election.cc:290] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:43.928426 12734 raft_consensus.cc:493] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.929211 12734 raft_consensus.cc:515] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.931447 12719 tablet_bootstrap.cc:492] T 5258afcab7984790b025622d73eaf684 P 07b29f69b9a2421db3983613bdc480ea: Bootstrap starting.
I20260709 13:07:43.931838 12734 leader_election.cc:290] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
W20260709 13:07:43.938175 12378 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:38975, user_credentials={real_user=slave}} blocked reactor thread for 45234.2us
I20260709 13:07:43.938832 12719 tablet_bootstrap.cc:654] T 5258afcab7984790b025622d73eaf684 P 07b29f69b9a2421db3983613bdc480ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.942206 12560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0487087198a422f82c31677e12b5f5a" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:43.942379 12561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc35585b6454b7a89990d0f849e8704" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:43.943092 12560 raft_consensus.cc:2468] T a0487087198a422f82c31677e12b5f5a P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 0.
I20260709 13:07:43.943351 12561 raft_consensus.cc:2468] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 0.
I20260709 13:07:43.944114 12716 tablet_bootstrap.cc:492] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3: No bootstrap required, opened a new log
I20260709 13:07:43.944643 12716 ts_tablet_manager.cc:1403] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.001s
I20260709 13:07:43.945343 12487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc35585b6454b7a89990d0f849e8704" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f" is_pre_election: true
I20260709 13:07:43.945421 12488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0487087198a422f82c31677e12b5f5a" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f" is_pre_election: true
I20260709 13:07:43.946365 12563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5258afcab7984790b025622d73eaf684" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:43.946797 12488 raft_consensus.cc:2468] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 0.
I20260709 13:07:43.947679 12716 raft_consensus.cc:359] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
W20260709 13:07:43.948037 12378 leader_election.cc:343] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:43.899695 12639 raft_consensus.cc:3060] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.949267 12378 leader_election.cc:304] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:43.950758 12734 raft_consensus.cc:2804] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:43.951272 12734 raft_consensus.cc:493] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:43.948431 12716 raft_consensus.cc:385] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.951730 12734 raft_consensus.cc:3060] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.951457 12378 leader_election.cc:304] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3; no voters: b186a014c0b745b8a66c39dae4efed4f
I20260709 13:07:43.951963 12716 raft_consensus.cc:740] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.952723 12732 raft_consensus.cc:2804] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:43.953109 12732 raft_consensus.cc:493] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:43.953491 12719 tablet_bootstrap.cc:492] T 5258afcab7984790b025622d73eaf684 P 07b29f69b9a2421db3983613bdc480ea: No bootstrap required, opened a new log
I20260709 13:07:43.953259 12716 consensus_queue.cc:260] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.953447 12732 raft_consensus.cc:3060] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.954236 12719 ts_tablet_manager.cc:1403] T 5258afcab7984790b025622d73eaf684 P 07b29f69b9a2421db3983613bdc480ea: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.008s
I20260709 13:07:43.958668 12719 raft_consensus.cc:359] T 5258afcab7984790b025622d73eaf684 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.959540 12719 raft_consensus.cc:385] T 5258afcab7984790b025622d73eaf684 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.959893 12719 raft_consensus.cc:740] T 5258afcab7984790b025622d73eaf684 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.960727 12719 consensus_queue.cc:260] T 5258afcab7984790b025622d73eaf684 P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.963299 12719 ts_tablet_manager.cc:1434] T 5258afcab7984790b025622d73eaf684 P 07b29f69b9a2421db3983613bdc480ea: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:07:43.964483 12489 raft_consensus.cc:2468] T ab5220719559499aae8b369be60bfcce P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 1.
I20260709 13:07:43.965008 12719 tablet_bootstrap.cc:492] T bb62b592dc854cd599b705b23ce7dc84 P 07b29f69b9a2421db3983613bdc480ea: Bootstrap starting.
I20260709 13:07:43.966380 12527 leader_election.cc:304] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, b186a014c0b745b8a66c39dae4efed4f; no voters: 1bcc393b4836466e9be8845ba140e5e3
I20260709 13:07:43.968050 12733 raft_consensus.cc:2804] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:43.968672 12733 raft_consensus.cc:697] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Becoming Leader. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Running, Role: LEADER
I20260709 13:07:43.969772 12733 consensus_queue.cc:237] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.972473 12719 tablet_bootstrap.cc:654] T bb62b592dc854cd599b705b23ce7dc84 P 07b29f69b9a2421db3983613bdc480ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.980055 12734 raft_consensus.cc:515] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
W20260709 13:07:43.984579 12378 leader_election.cc:343] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:43.986045 12732 raft_consensus.cc:515] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.987008 12719 tablet_bootstrap.cc:492] T bb62b592dc854cd599b705b23ce7dc84 P 07b29f69b9a2421db3983613bdc480ea: No bootstrap required, opened a new log
I20260709 13:07:43.987597 12719 ts_tablet_manager.cc:1403] T bb62b592dc854cd599b705b23ce7dc84 P 07b29f69b9a2421db3983613bdc480ea: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.003s
I20260709 13:07:43.990757 12719 raft_consensus.cc:359] T bb62b592dc854cd599b705b23ce7dc84 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.991675 12719 raft_consensus.cc:385] T bb62b592dc854cd599b705b23ce7dc84 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.992031 12719 raft_consensus.cc:740] T bb62b592dc854cd599b705b23ce7dc84 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.992853 12719 consensus_queue.cc:260] T bb62b592dc854cd599b705b23ce7dc84 P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:43.995388 12719 ts_tablet_manager.cc:1434] T bb62b592dc854cd599b705b23ce7dc84 P 07b29f69b9a2421db3983613bdc480ea: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:07:43.996450 12719 tablet_bootstrap.cc:492] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea: Bootstrap starting.
I20260709 13:07:44.003567 12719 tablet_bootstrap.cc:654] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:44.005884 12639 raft_consensus.cc:515] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.012367 12717 tablet_bootstrap.cc:492] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f: No bootstrap required, opened a new log
I20260709 13:07:44.012944 12717 ts_tablet_manager.cc:1403] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f: Time spent bootstrapping tablet: real 0.100s	user 0.022s	sys 0.038s
I20260709 13:07:44.013022 12639 leader_election.cc:290] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 election: Requested vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:44.009459 12734 leader_election.cc:290] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 election: Requested vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:44.016280 12412 raft_consensus.cc:2393] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b186a014c0b745b8a66c39dae4efed4f in current term 1: Already voted for candidate 1bcc393b4836466e9be8845ba140e5e3 in this term.
I20260709 13:07:44.016980 12717 raft_consensus.cc:359] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.018124 12717 raft_consensus.cc:385] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:44.018551 12717 raft_consensus.cc:740] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Initialized, Role: FOLLOWER
I20260709 13:07:44.019742 12717 consensus_queue.cc:260] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.021317 12455 leader_election.cc:304] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b186a014c0b745b8a66c39dae4efed4f; no voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3
I20260709 13:07:44.022594 12718 raft_consensus.cc:3060] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.023237 12717 ts_tablet_manager.cc:1434] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.004s
I20260709 13:07:44.024693 12717 tablet_bootstrap.cc:492] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f: Bootstrap starting.
I20260709 13:07:44.031464 12718 raft_consensus.cc:2749] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:44.033876 12717 tablet_bootstrap.cc:654] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:44.034896 12311 catalog_manager.cc:5697] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b29f69b9a2421db3983613bdc480ea (127.0.107.195). New cstate: current_term: 1 leader_uuid: "07b29f69b9a2421db3983613bdc480ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:44.037297 12733 raft_consensus.cc:493] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.037930 12733 raft_consensus.cc:515] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.038966 12489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0487087198a422f82c31677e12b5f5a" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:44.039628 12489 raft_consensus.cc:3060] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.041328 12733 leader_election.cc:290] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975)
I20260709 13:07:44.041913 12488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b883cd452e46209d8ad4b02ec3d180" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f" is_pre_election: true
I20260709 13:07:44.042601 12488 raft_consensus.cc:2468] T 71b883cd452e46209d8ad4b02ec3d180 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 0.
I20260709 13:07:44.043911 12527 leader_election.cc:304] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:44.045531 12733 raft_consensus.cc:2804] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:44.046053 12733 raft_consensus.cc:493] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.046401 12733 raft_consensus.cc:3060] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.046604 12732 leader_election.cc:290] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 election: Requested vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:44.048765 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b883cd452e46209d8ad4b02ec3d180" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3" is_pre_election: true
I20260709 13:07:44.049455 12412 raft_consensus.cc:2468] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 0.
I20260709 13:07:44.051229 12563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0487087198a422f82c31677e12b5f5a" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:44.051807 12488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5258afcab7984790b025622d73eaf684" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:44.052107 12561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5258afcab7984790b025622d73eaf684" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:44.053220 12561 raft_consensus.cc:3060] T 5258afcab7984790b025622d73eaf684 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.051846 12563 raft_consensus.cc:3060] T a0487087198a422f82c31677e12b5f5a P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.057207 12560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc35585b6454b7a89990d0f849e8704" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:44.057911 12560 raft_consensus.cc:3060] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.060429 12703 raft_consensus.cc:493] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.060957 12703 raft_consensus.cc:515] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.057899 12487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc35585b6454b7a89990d0f849e8704" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:44.062109 12733 raft_consensus.cc:515] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.062999 12488 raft_consensus.cc:2468] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 1.
I20260709 13:07:44.064134 12716 ts_tablet_manager.cc:1434] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3: Time spent starting tablet: real 0.119s	user 0.039s	sys 0.025s
I20260709 13:07:44.064949 12703 leader_election.cc:290] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:44.065295 12716 tablet_bootstrap.cc:492] T bb62b592dc854cd599b705b23ce7dc84 P 1bcc393b4836466e9be8845ba140e5e3: Bootstrap starting.
I20260709 13:07:44.066556 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b883cd452e46209d8ad4b02ec3d180" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
W20260709 13:07:44.066817 12378 leader_election.cc:343] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:44.067201 12412 raft_consensus.cc:3060] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.067298 12733 leader_election.cc:290] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975)
I20260709 13:07:44.067605 12378 leader_election.cc:304] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:44.068324 12488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b883cd452e46209d8ad4b02ec3d180" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:44.069113 12488 raft_consensus.cc:3060] T 71b883cd452e46209d8ad4b02ec3d180 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.071692 12414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb62b592dc854cd599b705b23ce7dc84" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3" is_pre_election: true
I20260709 13:07:44.072255 12716 tablet_bootstrap.cc:654] T bb62b592dc854cd599b705b23ce7dc84 P 1bcc393b4836466e9be8845ba140e5e3: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:44.073256 12455 leader_election.cc:343] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:44.074506 12734 raft_consensus.cc:2804] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:44.075210 12734 raft_consensus.cc:697] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Becoming Leader. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Running, Role: LEADER
I20260709 13:07:44.076435 12559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb62b592dc854cd599b705b23ce7dc84" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:44.077163 12559 raft_consensus.cc:2468] T bb62b592dc854cd599b705b23ce7dc84 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 0.
I20260709 13:07:44.078171 12734 consensus_queue.cc:237] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.079228 12560 raft_consensus.cc:2468] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 1.
I20260709 13:07:44.079495 12453 leader_election.cc:304] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, b186a014c0b745b8a66c39dae4efed4f; no voters: 1bcc393b4836466e9be8845ba140e5e3
I20260709 13:07:44.080068 12738 raft_consensus.cc:493] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.086400 12489 raft_consensus.cc:2468] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 1.
I20260709 13:07:44.086769 12378 leader_election.cc:304] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3; no voters: b186a014c0b745b8a66c39dae4efed4f
I20260709 13:07:44.086891 12703 raft_consensus.cc:2804] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:44.089803 12703 raft_consensus.cc:493] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.087857 12738 raft_consensus.cc:515] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.090418 12703 raft_consensus.cc:3060] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.093132 12487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5220719559499aae8b369be60bfcce" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f" is_pre_election: true
I20260709 13:07:44.095855 12487 raft_consensus.cc:2393] T ab5220719559499aae8b369be60bfcce P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1bcc393b4836466e9be8845ba140e5e3 in current term 1: Already voted for candidate 07b29f69b9a2421db3983613bdc480ea in this term.
I20260709 13:07:44.097301 12717 tablet_bootstrap.cc:492] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f: No bootstrap required, opened a new log
I20260709 13:07:44.097884 12717 ts_tablet_manager.cc:1403] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f: Time spent bootstrapping tablet: real 0.074s	user 0.014s	sys 0.008s
I20260709 13:07:44.103216 12717 raft_consensus.cc:359] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.104225 12563 raft_consensus.cc:2468] T a0487087198a422f82c31677e12b5f5a P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 1.
I20260709 13:07:44.104514 12717 raft_consensus.cc:385] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:44.105242 12717 raft_consensus.cc:740] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Initialized, Role: FOLLOWER
I20260709 13:07:44.090353 12732 raft_consensus.cc:2804] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:44.093519 12560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5220719559499aae8b369be60bfcce" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:44.106863 12378 leader_election.cc:304] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:44.110337 12412 raft_consensus.cc:2468] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 1.
I20260709 13:07:44.111816 12561 raft_consensus.cc:2468] T 5258afcab7984790b025622d73eaf684 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 1.
I20260709 13:07:44.114321 12732 raft_consensus.cc:697] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Becoming Leader. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Running, Role: LEADER
I20260709 13:07:44.116464 12529 leader_election.cc:304] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3; no voters: 
I20260709 13:07:44.117095 12732 consensus_queue.cc:237] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.118503 12719 tablet_bootstrap.cc:492] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea: No bootstrap required, opened a new log
I20260709 13:07:44.119212 12719 ts_tablet_manager.cc:1403] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea: Time spent bootstrapping tablet: real 0.123s	user 0.020s	sys 0.004s
I20260709 13:07:44.114779 12639 raft_consensus.cc:2804] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:44.120565 12733 raft_consensus.cc:2804] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:44.121280 12733 raft_consensus.cc:697] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Becoming Leader. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Running, Role: LEADER
I20260709 13:07:44.122226 12733 consensus_queue.cc:237] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.123747 12719 raft_consensus.cc:359] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.099936 12738 leader_election.cc:290] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:44.114897 12378 leader_election.cc:304] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3; no voters: 07b29f69b9a2421db3983613bdc480ea, b186a014c0b745b8a66c39dae4efed4f
I20260709 13:07:44.126186 12639 raft_consensus.cc:697] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Becoming Leader. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Running, Role: LEADER
I20260709 13:07:44.126441 12488 raft_consensus.cc:2468] T 71b883cd452e46209d8ad4b02ec3d180 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 1.
I20260709 13:07:44.127293 12639 consensus_queue.cc:237] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.114899 12717 consensus_queue.cc:260] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.124557 12719 raft_consensus.cc:385] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:44.132725 12734 raft_consensus.cc:3060] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.132251 12719 raft_consensus.cc:740] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Initialized, Role: FOLLOWER
I20260709 13:07:44.138249 12719 consensus_queue.cc:260] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.141309 12717 ts_tablet_manager.cc:1434] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f: Time spent starting tablet: real 0.043s	user 0.009s	sys 0.000s
I20260709 13:07:44.142825 12717 tablet_bootstrap.cc:492] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f: Bootstrap starting.
I20260709 13:07:44.146884 12719 ts_tablet_manager.cc:1434] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea: Time spent starting tablet: real 0.027s	user 0.012s	sys 0.003s
I20260709 13:07:44.148469 12719 tablet_bootstrap.cc:492] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea: Bootstrap starting.
I20260709 13:07:44.153102 12716 tablet_bootstrap.cc:492] T bb62b592dc854cd599b705b23ce7dc84 P 1bcc393b4836466e9be8845ba140e5e3: No bootstrap required, opened a new log
I20260709 13:07:44.153879 12716 ts_tablet_manager.cc:1403] T bb62b592dc854cd599b705b23ce7dc84 P 1bcc393b4836466e9be8845ba140e5e3: Time spent bootstrapping tablet: real 0.089s	user 0.020s	sys 0.020s
I20260709 13:07:44.157482 12716 raft_consensus.cc:359] T bb62b592dc854cd599b705b23ce7dc84 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.158121 12734 raft_consensus.cc:2749] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:44.158264 12716 raft_consensus.cc:385] T bb62b592dc854cd599b705b23ce7dc84 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:44.152796 12703 raft_consensus.cc:515] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.159076 12716 raft_consensus.cc:740] T bb62b592dc854cd599b705b23ce7dc84 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Initialized, Role: FOLLOWER
I20260709 13:07:44.162767 12716 consensus_queue.cc:260] T bb62b592dc854cd599b705b23ce7dc84 P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.165323 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb62b592dc854cd599b705b23ce7dc84" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
I20260709 13:07:44.166055 12719 tablet_bootstrap.cc:654] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:44.166082 12717 tablet_bootstrap.cc:654] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:44.166142 12412 raft_consensus.cc:3060] T bb62b592dc854cd599b705b23ce7dc84 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.165438 12716 ts_tablet_manager.cc:1434] T bb62b592dc854cd599b705b23ce7dc84 P 1bcc393b4836466e9be8845ba140e5e3: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 13:07:44.170573 12716 tablet_bootstrap.cc:492] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3: Bootstrap starting.
I20260709 13:07:44.176448 12412 raft_consensus.cc:2468] T bb62b592dc854cd599b705b23ce7dc84 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 1.
I20260709 13:07:44.178283 12455 leader_election.cc:304] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:44.179477 12718 raft_consensus.cc:2804] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:44.187439 12561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb62b592dc854cd599b705b23ce7dc84" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:44.188352 12561 raft_consensus.cc:3060] T bb62b592dc854cd599b705b23ce7dc84 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.189680 12716 tablet_bootstrap.cc:654] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:44.189765 12717 tablet_bootstrap.cc:492] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f: No bootstrap required, opened a new log
I20260709 13:07:44.191107 12717 ts_tablet_manager.cc:1403] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f: Time spent bootstrapping tablet: real 0.049s	user 0.015s	sys 0.017s
I20260709 13:07:44.194578 12717 raft_consensus.cc:359] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.195523 12717 raft_consensus.cc:385] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:44.195910 12717 raft_consensus.cc:740] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Initialized, Role: FOLLOWER
I20260709 13:07:44.197835 12703 leader_election.cc:290] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:44.201128 12716 tablet_bootstrap.cc:492] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3: No bootstrap required, opened a new log
I20260709 13:07:44.203351 12716 ts_tablet_manager.cc:1403] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.003s
I20260709 13:07:44.196861 12717 consensus_queue.cc:260] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.204510 12718 raft_consensus.cc:697] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [term 1 LEADER]: Becoming Leader. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Running, Role: LEADER
I20260709 13:07:44.206467 12717 ts_tablet_manager.cc:1434] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.001s
I20260709 13:07:44.206116 12718 consensus_queue.cc:237] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.207862 12717 tablet_bootstrap.cc:492] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f: Bootstrap starting.
I20260709 13:07:44.209512 12716 raft_consensus.cc:359] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.210637 12716 raft_consensus.cc:385] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:44.210991 12716 raft_consensus.cc:740] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Initialized, Role: FOLLOWER
I20260709 13:07:44.211848 12716 consensus_queue.cc:260] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.200970 12561 raft_consensus.cc:2468] T bb62b592dc854cd599b705b23ce7dc84 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 1.
I20260709 13:07:44.214679 12716 ts_tablet_manager.cc:1434] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:07:44.215730 12716 tablet_bootstrap.cc:492] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3: Bootstrap starting.
I20260709 13:07:44.218945 12717 tablet_bootstrap.cc:654] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:44.222703 12716 tablet_bootstrap.cc:654] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:44.227828 12310 catalog_manager.cc:5697] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b29f69b9a2421db3983613bdc480ea (127.0.107.195). New cstate: current_term: 1 leader_uuid: "07b29f69b9a2421db3983613bdc480ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:44.238320 12309 catalog_manager.cc:5697] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "1bcc393b4836466e9be8845ba140e5e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:44.240269 12309 catalog_manager.cc:5697] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "1bcc393b4836466e9be8845ba140e5e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:44.244410 12717 tablet_bootstrap.cc:492] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f: No bootstrap required, opened a new log
I20260709 13:07:44.245162 12717 ts_tablet_manager.cc:1403] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f: Time spent bootstrapping tablet: real 0.038s	user 0.019s	sys 0.007s
I20260709 13:07:44.248927 12717 raft_consensus.cc:359] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.249985 12733 raft_consensus.cc:493] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.250303 12717 raft_consensus.cc:385] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:44.247991 12309 catalog_manager.cc:5697] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "1bcc393b4836466e9be8845ba140e5e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:44.250532 12733 raft_consensus.cc:515] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.250763 12717 raft_consensus.cc:740] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Initialized, Role: FOLLOWER
I20260709 13:07:44.251828 12717 consensus_queue.cc:260] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.252903 12733 leader_election.cc:290] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975)
I20260709 13:07:44.253801 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05b5fc6b3b248ffb2e34876863341a4" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3" is_pre_election: true
I20260709 13:07:44.254465 12488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05b5fc6b3b248ffb2e34876863341a4" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f" is_pre_election: true
I20260709 13:07:44.255270 12717 ts_tablet_manager.cc:1434] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.004s
I20260709 13:07:44.255362 12488 raft_consensus.cc:2468] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 0.
I20260709 13:07:44.257117 12717 tablet_bootstrap.cc:492] T bce27e34734e4e868e207d32b9665e91 P b186a014c0b745b8a66c39dae4efed4f: Bootstrap starting.
W20260709 13:07:44.255366 12529 leader_election.cc:343] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:44.263574 12719 tablet_bootstrap.cc:492] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea: No bootstrap required, opened a new log
I20260709 13:07:44.264147 12719 ts_tablet_manager.cc:1403] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea: Time spent bootstrapping tablet: real 0.116s	user 0.038s	sys 0.032s
I20260709 13:07:44.264855 12717 tablet_bootstrap.cc:654] T bce27e34734e4e868e207d32b9665e91 P b186a014c0b745b8a66c39dae4efed4f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:44.269677 12719 raft_consensus.cc:359] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.270702 12719 raft_consensus.cc:385] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:44.271059 12719 raft_consensus.cc:740] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Initialized, Role: FOLLOWER
I20260709 13:07:44.263799 12527 leader_election.cc:304] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, b186a014c0b745b8a66c39dae4efed4f; no voters: 1bcc393b4836466e9be8845ba140e5e3
I20260709 13:07:44.271965 12733 raft_consensus.cc:2804] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:44.271906 12719 consensus_queue.cc:260] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.272419 12733 raft_consensus.cc:493] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.272799 12733 raft_consensus.cc:3060] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.277805 12719 ts_tablet_manager.cc:1434] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:07:44.278851 12719 tablet_bootstrap.cc:492] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 07b29f69b9a2421db3983613bdc480ea: Bootstrap starting.
I20260709 13:07:44.280529 12716 tablet_bootstrap.cc:492] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3: No bootstrap required, opened a new log
I20260709 13:07:44.281121 12716 ts_tablet_manager.cc:1403] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3: Time spent bootstrapping tablet: real 0.066s	user 0.030s	sys 0.031s
I20260709 13:07:44.282583 12733 raft_consensus.cc:515] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.283910 12716 raft_consensus.cc:359] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.289383 12716 raft_consensus.cc:385] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:44.289705 12716 raft_consensus.cc:740] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Initialized, Role: FOLLOWER
I20260709 13:07:44.289883 12310 catalog_manager.cc:5697] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f reported cstate change: term changed from 0 to 1, leader changed from <none> to b186a014c0b745b8a66c39dae4efed4f (127.0.107.194). New cstate: current_term: 1 leader_uuid: "b186a014c0b745b8a66c39dae4efed4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:44.290659 12733 leader_election.cc:290] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975)
I20260709 13:07:44.291292 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05b5fc6b3b248ffb2e34876863341a4" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
I20260709 13:07:44.284629 12719 tablet_bootstrap.cc:654] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 07b29f69b9a2421db3983613bdc480ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:44.291713 12488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05b5fc6b3b248ffb2e34876863341a4" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:44.291581 12716 consensus_queue.cc:260] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.292310 12488 raft_consensus.cc:3060] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.292523 12412 raft_consensus.cc:3060] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.288831 12717 tablet_bootstrap.cc:492] T bce27e34734e4e868e207d32b9665e91 P b186a014c0b745b8a66c39dae4efed4f: No bootstrap required, opened a new log
I20260709 13:07:44.293742 12716 ts_tablet_manager.cc:1434] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.003s
I20260709 13:07:44.294200 12717 ts_tablet_manager.cc:1403] T bce27e34734e4e868e207d32b9665e91 P b186a014c0b745b8a66c39dae4efed4f: Time spent bootstrapping tablet: real 0.037s	user 0.008s	sys 0.008s
I20260709 13:07:44.294703 12716 tablet_bootstrap.cc:492] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3: Bootstrap starting.
I20260709 13:07:44.299762 12412 raft_consensus.cc:2468] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 1.
I20260709 13:07:44.301189 12716 tablet_bootstrap.cc:654] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:44.301164 12529 leader_election.cc:304] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3; no voters: 
I20260709 13:07:44.312932 12488 raft_consensus.cc:2468] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 1.
I20260709 13:07:44.300945 12717 raft_consensus.cc:359] T bce27e34734e4e868e207d32b9665e91 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.313937 12717 raft_consensus.cc:385] T bce27e34734e4e868e207d32b9665e91 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:44.314218 12717 raft_consensus.cc:740] T bce27e34734e4e868e207d32b9665e91 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Initialized, Role: FOLLOWER
I20260709 13:07:44.314913 12717 consensus_queue.cc:260] T bce27e34734e4e868e207d32b9665e91 P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.317329 12717 ts_tablet_manager.cc:1434] T bce27e34734e4e868e207d32b9665e91 P b186a014c0b745b8a66c39dae4efed4f: Time spent starting tablet: real 0.023s	user 0.002s	sys 0.001s
I20260709 13:07:44.318329 12733 raft_consensus.cc:2804] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:44.319120 12733 raft_consensus.cc:697] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Becoming Leader. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Running, Role: LEADER
I20260709 13:07:44.320271 12733 consensus_queue.cc:237] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.344631 12308 catalog_manager.cc:5697] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b29f69b9a2421db3983613bdc480ea (127.0.107.195). New cstate: current_term: 1 leader_uuid: "07b29f69b9a2421db3983613bdc480ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:44.354380 12716 tablet_bootstrap.cc:492] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3: No bootstrap required, opened a new log
I20260709 13:07:44.355052 12716 ts_tablet_manager.cc:1403] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3: Time spent bootstrapping tablet: real 0.061s	user 0.024s	sys 0.032s
I20260709 13:07:44.355316 12719 tablet_bootstrap.cc:492] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 07b29f69b9a2421db3983613bdc480ea: No bootstrap required, opened a new log
I20260709 13:07:44.356305 12719 ts_tablet_manager.cc:1403] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 07b29f69b9a2421db3983613bdc480ea: Time spent bootstrapping tablet: real 0.078s	user 0.024s	sys 0.021s
I20260709 13:07:44.357874 12716 raft_consensus.cc:359] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.358610 12716 raft_consensus.cc:385] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:44.358397 12733 consensus_queue.cc:1048] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.358932 12716 raft_consensus.cc:740] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Initialized, Role: FOLLOWER
I20260709 13:07:44.359177 12719 raft_consensus.cc:359] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.359957 12719 raft_consensus.cc:385] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:44.359687 12716 consensus_queue.cc:260] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.360239 12719 raft_consensus.cc:740] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Initialized, Role: FOLLOWER
I20260709 13:07:44.360909 12719 consensus_queue.cc:260] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.362085 12716 ts_tablet_manager.cc:1434] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:07:44.362955 12719 ts_tablet_manager.cc:1434] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 07b29f69b9a2421db3983613bdc480ea: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:07:44.365549 12732 raft_consensus.cc:493] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.366081 12732 raft_consensus.cc:515] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.368093 12732 leader_election.cc:290] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:44.372148 12489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce27e34734e4e868e207d32b9665e91" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f" is_pre_election: true
I20260709 13:07:44.372466 12561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce27e34734e4e868e207d32b9665e91" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:44.372860 12489 raft_consensus.cc:2468] T bce27e34734e4e868e207d32b9665e91 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 0.
I20260709 13:07:44.373167 12561 raft_consensus.cc:2468] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 0.
I20260709 13:07:44.374104 12378 leader_election.cc:304] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:44.374941 12732 raft_consensus.cc:2804] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:44.375416 12732 raft_consensus.cc:493] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.375890 12732 raft_consensus.cc:3060] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.386018 12732 raft_consensus.cc:515] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.389328 12732 leader_election.cc:290] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 election: Requested vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:44.390293 12488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce27e34734e4e868e207d32b9665e91" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:44.390571 12641 consensus_queue.cc:1048] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.390987 12488 raft_consensus.cc:3060] T bce27e34734e4e868e207d32b9665e91 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.398118 12561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce27e34734e4e868e207d32b9665e91" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:44.398936 12561 raft_consensus.cc:3060] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.406318 12561 raft_consensus.cc:2468] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 1.
I20260709 13:07:44.407265 12378 leader_election.cc:304] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3; no voters: 
I20260709 13:07:44.407760 12488 raft_consensus.cc:2468] T bce27e34734e4e868e207d32b9665e91 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 1.
I20260709 13:07:44.407953 12639 raft_consensus.cc:2804] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:44.408492 12639 raft_consensus.cc:697] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Becoming Leader. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Running, Role: LEADER
I20260709 13:07:44.409318 12639 consensus_queue.cc:237] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.416146 12310 catalog_manager.cc:5697] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "1bcc393b4836466e9be8845ba140e5e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:44.475575 12639 raft_consensus.cc:493] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.476188 12639 raft_consensus.cc:515] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.478197 12639 leader_election.cc:290] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:44.478858 12488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b1e20fd4b04f08bd39ec6cc9792b5c" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f" is_pre_election: true
I20260709 13:07:44.479276 12561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b1e20fd4b04f08bd39ec6cc9792b5c" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:44.479489 12488 raft_consensus.cc:2468] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 0.
I20260709 13:07:44.479840 12561 raft_consensus.cc:2468] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 0.
I20260709 13:07:44.480508 12378 leader_election.cc:304] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:44.481161 12639 raft_consensus.cc:2804] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:44.481515 12639 raft_consensus.cc:493] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.481874 12639 raft_consensus.cc:3060] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.487749 12639 raft_consensus.cc:515] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.489272 12639 leader_election.cc:290] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 election: Requested vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:44.490047 12488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b1e20fd4b04f08bd39ec6cc9792b5c" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:44.490200 12561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b1e20fd4b04f08bd39ec6cc9792b5c" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:44.490603 12488 raft_consensus.cc:3060] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.490787 12561 raft_consensus.cc:3060] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 07b29f69b9a2421db3983613bdc480ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.495469 12488 raft_consensus.cc:2468] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 1.
I20260709 13:07:44.495922 12561 raft_consensus.cc:2468] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 1.
I20260709 13:07:44.496460 12378 leader_election.cc:304] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:44.497264 12639 raft_consensus.cc:2804] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:44.497786 12639 raft_consensus.cc:697] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Becoming Leader. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Running, Role: LEADER
I20260709 13:07:44.498512 12639 consensus_queue.cc:237] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.505307 12310 catalog_manager.cc:5697] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "1bcc393b4836466e9be8845ba140e5e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:44.532725 12767 consensus_queue.cc:1048] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.543509 12641 consensus_queue.cc:1048] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.544445 12732 consensus_queue.cc:1048] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.548192 12718 raft_consensus.cc:493] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.548951 12718 raft_consensus.cc:515] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.553045 12560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc35585b6454b7a89990d0f849e8704" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b29f69b9a2421db3983613bdc480ea" is_pre_election: true
I20260709 13:07:44.554143 12560 raft_consensus.cc:2393] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b186a014c0b745b8a66c39dae4efed4f in current term 1: Already voted for candidate 1bcc393b4836466e9be8845ba140e5e3 in this term.
I20260709 13:07:44.554279 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc35585b6454b7a89990d0f849e8704" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcc393b4836466e9be8845ba140e5e3" is_pre_election: true
I20260709 13:07:44.556967 12453 leader_election.cc:304] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b186a014c0b745b8a66c39dae4efed4f; no voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3
I20260709 13:07:44.558365 12754 raft_consensus.cc:3060] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.568806 12718 leader_election.cc:290] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:44.571139 12734 consensus_queue.cc:1048] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.574129 12738 consensus_queue.cc:1048] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.574332 12754 raft_consensus.cc:2749] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:44.586198 12639 consensus_queue.cc:1048] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.639592 12738 consensus_queue.cc:1048] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.642642 12703 consensus_queue.cc:1048] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.645210 12718 consensus_queue.cc:1048] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.668790 12732 consensus_queue.cc:1048] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.677579 12560 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab5220719559499aae8b369be60bfcce"
dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
mode: GRACEFUL
new_leader_uuid: "1bcc393b4836466e9be8845ba140e5e3"
 from {username='slave'} at 127.0.0.1:38922
I20260709 13:07:44.678565 12560 raft_consensus.cc:606] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Received request to transfer leadership to TS 1bcc393b4836466e9be8845ba140e5e3
I20260709 13:07:44.699460 12561 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71b883cd452e46209d8ad4b02ec3d180"
dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
mode: GRACEFUL
new_leader_uuid: "1bcc393b4836466e9be8845ba140e5e3"
 from {username='slave'} at 127.0.0.1:38922
I20260709 13:07:44.700111 12561 raft_consensus.cc:606] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Received request to transfer leadership to TS 1bcc393b4836466e9be8845ba140e5e3
I20260709 13:07:44.702663 12641 consensus_queue.cc:1048] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.706118 12412 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0487087198a422f82c31677e12b5f5a"
dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
mode: GRACEFUL
new_leader_uuid: "b186a014c0b745b8a66c39dae4efed4f"
 from {username='slave'} at 127.0.0.1:44378
I20260709 13:07:44.706753 12412 raft_consensus.cc:606] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Received request to transfer leadership to TS b186a014c0b745b8a66c39dae4efed4f
I20260709 13:07:44.715446 12767 consensus_queue.cc:1048] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.721674 12414 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38b1e20fd4b04f08bd39ec6cc9792b5c"
dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
mode: GRACEFUL
new_leader_uuid: "b186a014c0b745b8a66c39dae4efed4f"
 from {username='slave'} at 127.0.0.1:44378
I20260709 13:07:44.722354 12414 raft_consensus.cc:606] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Received request to transfer leadership to TS b186a014c0b745b8a66c39dae4efed4f
I20260709 13:07:44.730177 12561 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f05b5fc6b3b248ffb2e34876863341a4"
dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
mode: GRACEFUL
new_leader_uuid: "b186a014c0b745b8a66c39dae4efed4f"
 from {username='slave'} at 127.0.0.1:38922
I20260709 13:07:44.730787 12561 raft_consensus.cc:606] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Received request to transfer leadership to TS b186a014c0b745b8a66c39dae4efed4f
I20260709 13:07:44.736773 12412 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bce27e34734e4e868e207d32b9665e91"
dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
mode: GRACEFUL
new_leader_uuid: "07b29f69b9a2421db3983613bdc480ea"
 from {username='slave'} at 127.0.0.1:44378
I20260709 13:07:44.737511 12412 raft_consensus.cc:606] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Received request to transfer leadership to TS 07b29f69b9a2421db3983613bdc480ea
I20260709 13:07:44.850328 12641 raft_consensus.cc:993] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea: : Instructing follower 1bcc393b4836466e9be8845ba140e5e3 to start an election
I20260709 13:07:44.850054 12732 consensus_queue.cc:1048] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.850904 12767 raft_consensus.cc:1081] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Signalling peer 1bcc393b4836466e9be8845ba140e5e3 to start an election
I20260709 13:07:44.854234 12412 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab5220719559499aae8b369be60bfcce"
dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
 from {username='slave'} at 127.0.0.1:44368
I20260709 13:07:44.854816 12412 raft_consensus.cc:493] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:44.855219 12412 raft_consensus.cc:3060] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:44.860950 12412 raft_consensus.cc:515] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.864411 12488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5220719559499aae8b369be60bfcce" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:44.865192 12488 raft_consensus.cc:3060] T ab5220719559499aae8b369be60bfcce P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:44.866786 12732 raft_consensus.cc:993] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3: : Instructing follower 07b29f69b9a2421db3983613bdc480ea to start an election
I20260709 13:07:44.867343 12732 raft_consensus.cc:1081] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Signalling peer 07b29f69b9a2421db3983613bdc480ea to start an election
I20260709 13:07:44.863432 12412 leader_election.cc:290] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 2 election: Requested vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:44.871035 12563 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bce27e34734e4e868e207d32b9665e91"
dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
 from {username='slave'} at 127.0.0.1:38908
I20260709 13:07:44.872277 12563 raft_consensus.cc:493] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:44.872815 12563 raft_consensus.cc:3060] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:44.873523 12488 raft_consensus.cc:2468] T ab5220719559499aae8b369be60bfcce P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 2.
I20260709 13:07:44.872148 12561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5220719559499aae8b369be60bfcce" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:44.875325 12561 raft_consensus.cc:3055] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:44.876327 12378 leader_election.cc:304] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:44.878109 12734 raft_consensus.cc:2804] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:44.878633 12734 raft_consensus.cc:697] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [term 2 LEADER]: Becoming Leader. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Running, Role: LEADER
I20260709 13:07:44.878489 12732 consensus_queue.cc:1048] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:44.879645 12734 consensus_queue.cc:237] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.875669 12561 raft_consensus.cc:740] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Running, Role: LEADER
I20260709 13:07:44.881003 12563 raft_consensus.cc:515] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.881574 12561 consensus_queue.cc:260] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.883152 12561 raft_consensus.cc:3060] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:44.884195 12414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce27e34734e4e868e207d32b9665e91" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
I20260709 13:07:44.884902 12414 raft_consensus.cc:3055] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:44.885254 12414 raft_consensus.cc:740] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Running, Role: LEADER
I20260709 13:07:44.886121 12414 consensus_queue.cc:260] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.887184 12414 raft_consensus.cc:3060] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:44.894105 12561 raft_consensus.cc:2468] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 2.
I20260709 13:07:44.898327 12563 leader_election.cc:290] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 2 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975)
I20260709 13:07:44.897866 12308 catalog_manager.cc:5697] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 reported cstate change: term changed from 1 to 2, leader changed from 07b29f69b9a2421db3983613bdc480ea (127.0.107.195) to 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "1bcc393b4836466e9be8845ba140e5e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:44.899942 12489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce27e34734e4e868e207d32b9665e91" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:44.901877 12414 raft_consensus.cc:2468] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 2.
I20260709 13:07:44.903064 12529 leader_election.cc:304] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3; no voters: b186a014c0b745b8a66c39dae4efed4f
I20260709 13:07:44.903923 12767 raft_consensus.cc:2804] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:44.904466 12767 raft_consensus.cc:697] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea [term 2 LEADER]: Becoming Leader. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Running, Role: LEADER
I20260709 13:07:44.905344 12767 consensus_queue.cc:237] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.915844 12308 catalog_manager.cc:5697] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea reported cstate change: term changed from 1 to 2, leader changed from 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193) to 07b29f69b9a2421db3983613bdc480ea (127.0.107.195). New cstate: current_term: 2 leader_uuid: "07b29f69b9a2421db3983613bdc480ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:44.929791 12641 raft_consensus.cc:993] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea: : Instructing follower 1bcc393b4836466e9be8845ba140e5e3 to start an election
I20260709 13:07:44.930321 12767 raft_consensus.cc:1081] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Signalling peer 1bcc393b4836466e9be8845ba140e5e3 to start an election
I20260709 13:07:44.931754 12414 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "71b883cd452e46209d8ad4b02ec3d180"
dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
 from {username='slave'} at 127.0.0.1:44368
I20260709 13:07:44.932215 12414 raft_consensus.cc:493] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:44.932497 12414 raft_consensus.cc:3060] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:44.937768 12414 raft_consensus.cc:515] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.939796 12414 leader_election.cc:290] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 2 election: Requested vote from peers b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:44.940912 12489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b883cd452e46209d8ad4b02ec3d180" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:44.941527 12489 raft_consensus.cc:3060] T 71b883cd452e46209d8ad4b02ec3d180 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:44.942226 12563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b883cd452e46209d8ad4b02ec3d180" candidate_uuid: "1bcc393b4836466e9be8845ba140e5e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:44.942313 12734 consensus_queue.cc:1048] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:44.942961 12563 raft_consensus.cc:3055] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:44.943324 12563 raft_consensus.cc:740] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Running, Role: LEADER
I20260709 13:07:44.944113 12563 consensus_queue.cc:260] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.945219 12563 raft_consensus.cc:3060] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:44.949426 12489 raft_consensus.cc:2468] T 71b883cd452e46209d8ad4b02ec3d180 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 2.
I20260709 13:07:44.951056 12378 leader_election.cc:304] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:44.952255 12734 raft_consensus.cc:2804] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:44.952572 12563 raft_consensus.cc:2468] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcc393b4836466e9be8845ba140e5e3 in term 2.
I20260709 13:07:44.952775 12734 raft_consensus.cc:697] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 LEADER]: Becoming Leader. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Running, Role: LEADER
I20260709 13:07:44.953451 12639 consensus_queue.cc:1048] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:44.953780 12734 consensus_queue.cc:237] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.962982 12738 raft_consensus.cc:993] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3: : Instructing follower b186a014c0b745b8a66c39dae4efed4f to start an election
I20260709 13:07:44.963578 12738 raft_consensus.cc:1081] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Signalling peer b186a014c0b745b8a66c39dae4efed4f to start an election
I20260709 13:07:44.964910 12311 catalog_manager.cc:5697] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3 reported cstate change: term changed from 1 to 2, leader changed from 07b29f69b9a2421db3983613bdc480ea (127.0.107.195) to 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "1bcc393b4836466e9be8845ba140e5e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:44.965777 12488 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38b1e20fd4b04f08bd39ec6cc9792b5c"
dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
 from {username='slave'} at 127.0.0.1:38566
I20260709 13:07:44.967676 12488 raft_consensus.cc:493] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:44.968116 12488 raft_consensus.cc:3060] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:44.976246 12488 raft_consensus.cc:515] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.981009 12414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b1e20fd4b04f08bd39ec6cc9792b5c" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
I20260709 13:07:44.981541 12414 raft_consensus.cc:3055] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:44.981894 12414 raft_consensus.cc:740] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Running, Role: LEADER
I20260709 13:07:44.982280 12563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b1e20fd4b04f08bd39ec6cc9792b5c" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:44.982859 12563 raft_consensus.cc:3060] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:44.982796 12414 consensus_queue.cc:260] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:44.981770 12488 leader_election.cc:290] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 2 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:44.984125 12414 raft_consensus.cc:3060] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:44.990862 12414 raft_consensus.cc:2468] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 2.
I20260709 13:07:44.990862 12563 raft_consensus.cc:2468] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 2.
I20260709 13:07:44.992234 12455 leader_election.cc:304] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:44.992923 12703 raft_consensus.cc:2804] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:44.993408 12703 raft_consensus.cc:697] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f [term 2 LEADER]: Becoming Leader. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Running, Role: LEADER
I20260709 13:07:44.994107 12703 consensus_queue.cc:237] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:45.001165 12310 catalog_manager.cc:5697] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f reported cstate change: term changed from 1 to 2, leader changed from 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193) to b186a014c0b745b8a66c39dae4efed4f (127.0.107.194). New cstate: current_term: 2 leader_uuid: "b186a014c0b745b8a66c39dae4efed4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:45.310160 12732 raft_consensus.cc:993] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3: : Instructing follower b186a014c0b745b8a66c39dae4efed4f to start an election
I20260709 13:07:45.310711 12732 raft_consensus.cc:1081] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Signalling peer b186a014c0b745b8a66c39dae4efed4f to start an election
I20260709 13:07:45.313206 12488 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a0487087198a422f82c31677e12b5f5a"
dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
 from {username='slave'} at 127.0.0.1:38566
I20260709 13:07:45.313442 12733 raft_consensus.cc:993] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea: : Instructing follower b186a014c0b745b8a66c39dae4efed4f to start an election
I20260709 13:07:45.313807 12488 raft_consensus.cc:493] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:45.313922 12733 raft_consensus.cc:1081] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Signalling peer b186a014c0b745b8a66c39dae4efed4f to start an election
I20260709 13:07:45.314205 12488 raft_consensus.cc:3060] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:45.317618 12489 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f05b5fc6b3b248ffb2e34876863341a4"
dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
 from {username='slave'} at 127.0.0.1:38570
I20260709 13:07:45.318326 12489 raft_consensus.cc:493] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:45.318738 12489 raft_consensus.cc:3060] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:45.332954 12488 raft_consensus.cc:515] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:45.335249 12488 leader_election.cc:290] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 2 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:45.336483 12563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0487087198a422f82c31677e12b5f5a" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:45.337226 12563 raft_consensus.cc:3060] T a0487087198a422f82c31677e12b5f5a P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:45.341003 12414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0487087198a422f82c31677e12b5f5a" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
I20260709 13:07:45.341715 12414 raft_consensus.cc:3055] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:45.342185 12414 raft_consensus.cc:740] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Running, Role: LEADER
I20260709 13:07:45.342955 12414 consensus_queue.cc:260] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:45.343801 12414 raft_consensus.cc:3060] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:45.345165 12563 raft_consensus.cc:2468] T a0487087198a422f82c31677e12b5f5a P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 2.
I20260709 13:07:45.346529 12453 leader_election.cc:304] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:45.347484 12703 raft_consensus.cc:2804] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:45.347860 12489 raft_consensus.cc:515] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:45.347975 12703 raft_consensus.cc:697] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f [term 2 LEADER]: Becoming Leader. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Running, Role: LEADER
I20260709 13:07:45.351708 12489 leader_election.cc:290] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 2 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), 07b29f69b9a2421db3983613bdc480ea (127.0.107.195:38269)
I20260709 13:07:45.351346 12703 consensus_queue.cc:237] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:45.351979 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05b5fc6b3b248ffb2e34876863341a4" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
I20260709 13:07:45.352725 12412 raft_consensus.cc:3060] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:45.363324 12412 raft_consensus.cc:2468] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 2.
I20260709 13:07:45.365309 12455 leader_election.cc:304] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bcc393b4836466e9be8845ba140e5e3, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:45.365922 12489 raft_consensus.cc:1275] T ab5220719559499aae8b369be60bfcce P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Refusing update from remote peer 1bcc393b4836466e9be8845ba140e5e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:45.366664 12703 raft_consensus.cc:2804] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:45.367420 12703 raft_consensus.cc:697] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [term 2 LEADER]: Becoming Leader. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Running, Role: LEADER
I20260709 13:07:45.368744 12703 consensus_queue.cc:237] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:45.372857 12639 consensus_queue.cc:1048] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:45.374876 12412 raft_consensus.cc:1275] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Refusing update from remote peer 07b29f69b9a2421db3983613bdc480ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:45.376420 12641 consensus_queue.cc:1048] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:45.377197 12561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05b5fc6b3b248ffb2e34876863341a4" candidate_uuid: "b186a014c0b745b8a66c39dae4efed4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
I20260709 13:07:45.377888 12561 raft_consensus.cc:3055] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:45.378275 12561 raft_consensus.cc:740] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Running, Role: LEADER
I20260709 13:07:45.379050 12561 consensus_queue.cc:260] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:45.380410 12561 raft_consensus.cc:3060] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:45.377216 12414 raft_consensus.cc:2468] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 2.
I20260709 13:07:45.391187 12559 raft_consensus.cc:1275] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Refusing update from remote peer 1bcc393b4836466e9be8845ba140e5e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:45.392426 12563 raft_consensus.cc:1240] T a0487087198a422f82c31677e12b5f5a P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Rejecting Update request from peer 1bcc393b4836466e9be8845ba140e5e3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:45.393966 12557 raft_consensus.cc:1275] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Refusing update from remote peer 1bcc393b4836466e9be8845ba140e5e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:45.398904 12488 raft_consensus.cc:3060] T bce27e34734e4e868e207d32b9665e91 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:45.404403 12639 consensus_queue.cc:1048] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:45.405745 12488 raft_consensus.cc:1275] T bce27e34734e4e868e207d32b9665e91 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Refusing update from remote peer 07b29f69b9a2421db3983613bdc480ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:45.406958 12639 consensus_queue.cc:1048] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:45.407660 12733 consensus_queue.cc:1048] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:45.417259 12309 catalog_manager.cc:5697] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f reported cstate change: term changed from 1 to 2, leader changed from 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193) to b186a014c0b745b8a66c39dae4efed4f (127.0.107.194). New cstate: current_term: 2 leader_uuid: "b186a014c0b745b8a66c39dae4efed4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:45.430807 12486 raft_consensus.cc:1275] T 71b883cd452e46209d8ad4b02ec3d180 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Refusing update from remote peer 1bcc393b4836466e9be8845ba140e5e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:45.434777 12561 raft_consensus.cc:2468] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b186a014c0b745b8a66c39dae4efed4f in term 2.
I20260709 13:07:45.442552 12639 consensus_queue.cc:1048] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:45.444281 12310 catalog_manager.cc:5697] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f reported cstate change: term changed from 1 to 2, leader changed from 07b29f69b9a2421db3983613bdc480ea (127.0.107.195) to b186a014c0b745b8a66c39dae4efed4f (127.0.107.194). New cstate: current_term: 2 leader_uuid: "b186a014c0b745b8a66c39dae4efed4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:45.565059 12557 raft_consensus.cc:1275] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Refusing update from remote peer b186a014c0b745b8a66c39dae4efed4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:45.566381 12754 consensus_queue.cc:1048] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:45.575176 12414 raft_consensus.cc:1275] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Refusing update from remote peer b186a014c0b745b8a66c39dae4efed4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:45.577984 12703 consensus_queue.cc:1048] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:45.869132 12414 raft_consensus.cc:1275] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Refusing update from remote peer b186a014c0b745b8a66c39dae4efed4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:45.870198 12557 raft_consensus.cc:1275] T a0487087198a422f82c31677e12b5f5a P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Refusing update from remote peer b186a014c0b745b8a66c39dae4efed4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:45.870388 12703 consensus_queue.cc:1048] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:45.871624 12754 consensus_queue.cc:1048] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:45.893553 12563 raft_consensus.cc:1275] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Refusing update from remote peer b186a014c0b745b8a66c39dae4efed4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:45.894985 12754 consensus_queue.cc:1048] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:45.904654 12414 raft_consensus.cc:1275] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Refusing update from remote peer b186a014c0b745b8a66c39dae4efed4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:45.907357 12791 consensus_queue.cc:1048] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:46.265724 12510 debug-util.cc:398] Leaking SignalData structure 0x7b08002ae6c0 after lost signal to thread 434
W20260709 13:07:46.267443 12510 debug-util.cc:398] Leaking SignalData structure 0x7b08000dcf00 after lost signal to thread 12346
W20260709 13:07:46.269161 12510 debug-util.cc:398] Leaking SignalData structure 0x7b08000d5ac0 after lost signal to thread 12438
W20260709 13:07:46.270828 12510 debug-util.cc:398] Leaking SignalData structure 0x7b08000eac80 after lost signal to thread 12513
W20260709 13:07:46.272260 12510 debug-util.cc:398] Leaking SignalData structure 0x7b0800266ee0 after lost signal to thread 12587
I20260709 13:07:47.792992 12310 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45158:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:07:47.822351 12310 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45158:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:07:47.849025   431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 13:07:48.058830 12819 log.cc:927] Time spent T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
W20260709 13:07:48.149842 12821 log.cc:927] Time spent T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
W20260709 13:07:48.151456 12814 log.cc:927] Time spent T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:07:48.164470 12827 log.cc:927] Time spent T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 13:07:48.173403 12828 log.cc:927] Time spent T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:07:48.220522 12839 tablet.cc:1724] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.249416 12855 tablet.cc:1724] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.277307 12854 tablet.cc:1724] T bb62b592dc854cd599b705b23ce7dc84 P 1bcc393b4836466e9be8845ba140e5e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.301622 12858 tablet.cc:1724] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:07:48.305516 12515 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:5258afcab7984790b025622d73eaf684)
I20260709 13:07:48.354251 12863 tablet.cc:1724] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:07:48.353284 12440 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:5258afcab7984790b025622d73eaf684)
W20260709 13:07:48.398700 12440 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:0dc35585b6454b7a89990d0f849e8704)
W20260709 13:07:48.422565 12862 log.cc:927] Time spent T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:07:48.437592 12870 tablet.cc:1724] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.448277 12867 tablet.cc:1724] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.450970 12871 tablet.cc:1724] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 07b29f69b9a2421db3983613bdc480ea: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.452384 12866 tablet.cc:1724] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.458050 12861 tablet.cc:1724] T bb62b592dc854cd599b705b23ce7dc84 P 07b29f69b9a2421db3983613bdc480ea: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.516396 12411 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0dc35585b6454b7a89990d0f849e8704"
dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
mode: GRACEFUL
new_leader_uuid: "07b29f69b9a2421db3983613bdc480ea"
 from {username='slave'} at 127.0.0.1:44382
I20260709 13:07:48.517230 12411 raft_consensus.cc:606] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Received request to transfer leadership to TS 07b29f69b9a2421db3983613bdc480ea
I20260709 13:07:48.513988 12854 tablet.cc:1724] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.545586 12855 tablet.cc:1724] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.561491 12863 tablet.cc:1724] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.570595 12839 tablet.cc:1724] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.591574 12873 tablet.cc:1724] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.590137 12867 tablet.cc:1724] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.579330 12866 tablet.cc:1724] T bce27e34734e4e868e207d32b9665e91 P b186a014c0b745b8a66c39dae4efed4f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.576450 12858 tablet.cc:1724] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.607093 12854 tablet.cc:1724] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.609521 12855 tablet.cc:1724] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.612032 12839 tablet.cc:1724] T ab5220719559499aae8b369be60bfcce P b186a014c0b745b8a66c39dae4efed4f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.632530 12866 tablet.cc:1724] T 71b883cd452e46209d8ad4b02ec3d180 P b186a014c0b745b8a66c39dae4efed4f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.619236 12867 tablet.cc:1724] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.605481 12872 tablet.cc:1724] T 5258afcab7984790b025622d73eaf684 P 07b29f69b9a2421db3983613bdc480ea: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.650424 12871 tablet.cc:1724] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.674341 12867 tablet.cc:1724] T a0487087198a422f82c31677e12b5f5a P 07b29f69b9a2421db3983613bdc480ea: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.697208 12861 tablet.cc:1724] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:48.711385 12486 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f05b5fc6b3b248ffb2e34876863341a4"
dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
mode: GRACEFUL
new_leader_uuid: "07b29f69b9a2421db3983613bdc480ea"
 from {username='slave'} at 127.0.0.1:40974
I20260709 13:07:48.712091 12486 raft_consensus.cc:606] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [term 2 LEADER]: Received request to transfer leadership to TS 07b29f69b9a2421db3983613bdc480ea
I20260709 13:07:48.713814   431 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:07:48.714679   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:48.724500 12310 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5: soft_deleted_table [id=5dc195cd96864d3ba3b61760ee1fcae5] alter complete (version 1)
I20260709 13:07:48.728120 12412 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "806d30f73a06482aaef3642acf872c32"
dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
mode: GRACEFUL
new_leader_uuid: "07b29f69b9a2421db3983613bdc480ea"
 from {username='slave'} at 127.0.0.1:44382
I20260709 13:07:48.728632 12412 raft_consensus.cc:606] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 LEADER]: Received request to transfer leadership to TS 07b29f69b9a2421db3983613bdc480ea
I20260709 13:07:48.732008 12875 raft_consensus.cc:993] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3: : Instructing follower 07b29f69b9a2421db3983613bdc480ea to start an election
I20260709 13:07:48.732491 12639 raft_consensus.cc:1081] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 LEADER]: Signalling peer 07b29f69b9a2421db3983613bdc480ea to start an election
I20260709 13:07:48.735074 12563 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "806d30f73a06482aaef3642acf872c32"
dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
 from {username='slave'} at 127.0.0.1:38908
I20260709 13:07:48.735632 12563 raft_consensus.cc:493] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:48.735996 12563 raft_consensus.cc:3060] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:48.736204 12486 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2f1bab6aa8e4183808649fdcb0b1bb0"
dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
mode: GRACEFUL
new_leader_uuid: "07b29f69b9a2421db3983613bdc480ea"
 from {username='slave'} at 127.0.0.1:40974
I20260709 13:07:48.736828 12486 raft_consensus.cc:606] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [term 1 LEADER]: Received request to transfer leadership to TS 07b29f69b9a2421db3983613bdc480ea
I20260709 13:07:48.738636   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:07:48.739198   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:48.743352 12563 raft_consensus.cc:515] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } }
I20260709 13:07:48.745867 12563 leader_election.cc:290] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 3 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975)
I20260709 13:07:48.746642 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d30f73a06482aaef3642acf872c32" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
I20260709 13:07:48.747088 12486 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d30f73a06482aaef3642acf872c32" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:48.747291 12412 raft_consensus.cc:3055] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:48.747651 12412 raft_consensus.cc:740] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Running, Role: LEADER
I20260709 13:07:48.747664 12486 raft_consensus.cc:3060] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:48.748508 12412 consensus_queue.cc:260] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } }
I20260709 13:07:48.749562 12412 raft_consensus.cc:3060] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:48.759120 12486 raft_consensus.cc:2468] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 3.
I20260709 13:07:48.760963 12527 leader_election.cc:304] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:48.761853 12796 raft_consensus.cc:2804] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:48.762311 12796 raft_consensus.cc:697] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [term 3 LEADER]: Becoming Leader. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Running, Role: LEADER
I20260709 13:07:48.763089 12796 consensus_queue.cc:237] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } }
I20260709 13:07:48.764604 12412 raft_consensus.cc:2468] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 3.
I20260709 13:07:48.772048 12310 catalog_manager.cc:5697] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea reported cstate change: term changed from 2 to 3, leader changed from 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193) to 07b29f69b9a2421db3983613bdc480ea (127.0.107.195). New cstate: current_term: 3 leader_uuid: "07b29f69b9a2421db3983613bdc480ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:48.802942 12875 raft_consensus.cc:993] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3: : Instructing follower 07b29f69b9a2421db3983613bdc480ea to start an election
I20260709 13:07:48.803442 12639 raft_consensus.cc:1081] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Signalling peer 07b29f69b9a2421db3983613bdc480ea to start an election
I20260709 13:07:48.804862 12562 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0dc35585b6454b7a89990d0f849e8704"
dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
 from {username='slave'} at 127.0.0.1:38908
I20260709 13:07:48.805464 12562 raft_consensus.cc:493] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:48.805894 12562 raft_consensus.cc:3060] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:48.814213 12562 raft_consensus.cc:515] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:48.816931 12562 leader_election.cc:290] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 2 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975)
I20260709 13:07:48.817085 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc35585b6454b7a89990d0f849e8704" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
I20260709 13:07:48.817567 12486 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc35585b6454b7a89990d0f849e8704" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:48.817780 12412 raft_consensus.cc:3055] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:48.818099 12412 raft_consensus.cc:740] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1bcc393b4836466e9be8845ba140e5e3, State: Running, Role: LEADER
I20260709 13:07:48.818178 12486 raft_consensus.cc:3060] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:48.818876 12412 consensus_queue.cc:260] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:48.819924 12412 raft_consensus.cc:3060] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:48.825397 12486 raft_consensus.cc:2468] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 2.
I20260709 13:07:48.826452 12412 raft_consensus.cc:2468] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 2.
I20260709 13:07:48.827037 12527 leader_election.cc:304] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, b186a014c0b745b8a66c39dae4efed4f; no voters: 
I20260709 13:07:48.828225 12796 raft_consensus.cc:2804] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:48.828696 12796 raft_consensus.cc:697] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea [term 2 LEADER]: Becoming Leader. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Running, Role: LEADER
I20260709 13:07:48.829612 12796 consensus_queue.cc:237] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:48.836932 12310 catalog_manager.cc:5697] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea reported cstate change: term changed from 1 to 2, leader changed from 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193) to 07b29f69b9a2421db3983613bdc480ea (127.0.107.195). New cstate: current_term: 2 leader_uuid: "07b29f69b9a2421db3983613bdc480ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:48.840405 12838 raft_consensus.cc:993] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f: : Instructing follower 07b29f69b9a2421db3983613bdc480ea to start an election
I20260709 13:07:48.841176 12798 raft_consensus.cc:1081] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [term 1 LEADER]: Signalling peer 07b29f69b9a2421db3983613bdc480ea to start an election
I20260709 13:07:48.842875 12562 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2f1bab6aa8e4183808649fdcb0b1bb0"
dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
 from {username='slave'} at 127.0.0.1:38904
I20260709 13:07:48.843474 12562 raft_consensus.cc:493] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:48.843998 12562 raft_consensus.cc:3060] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:48.852249 12562 raft_consensus.cc:515] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:48.855034 12562 leader_election.cc:290] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 2 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975)
I20260709 13:07:48.855409 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f1bab6aa8e4183808649fdcb0b1bb0" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
I20260709 13:07:48.856180 12412 raft_consensus.cc:3060] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:48.856235 12486 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f1bab6aa8e4183808649fdcb0b1bb0" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:48.856940 12486 raft_consensus.cc:3055] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:48.857316 12486 raft_consensus.cc:740] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Running, Role: LEADER
I20260709 13:07:48.858399 12486 consensus_queue.cc:260] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:48.859766 12486 raft_consensus.cc:3060] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:48.863655 12412 raft_consensus.cc:2468] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 2.
I20260709 13:07:48.864904 12529 leader_election.cc:304] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3; no voters: 
I20260709 13:07:48.865772 12796 raft_consensus.cc:2804] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:48.867277 12796 raft_consensus.cc:697] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea [term 2 LEADER]: Becoming Leader. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Running, Role: LEADER
I20260709 13:07:48.868279 12796 consensus_queue.cc:237] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:48.871232 12486 raft_consensus.cc:2468] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 2.
I20260709 13:07:48.876732 12310 catalog_manager.cc:5697] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea reported cstate change: term changed from 1 to 2, leader changed from b186a014c0b745b8a66c39dae4efed4f (127.0.107.194) to 07b29f69b9a2421db3983613bdc480ea (127.0.107.195). New cstate: current_term: 2 leader_uuid: "07b29f69b9a2421db3983613bdc480ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:48.917474 12838 raft_consensus.cc:993] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f: : Instructing follower 07b29f69b9a2421db3983613bdc480ea to start an election
I20260709 13:07:48.917994 12798 raft_consensus.cc:1081] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [term 2 LEADER]: Signalling peer 07b29f69b9a2421db3983613bdc480ea to start an election
I20260709 13:07:48.919534 12562 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f05b5fc6b3b248ffb2e34876863341a4"
dest_uuid: "07b29f69b9a2421db3983613bdc480ea"
 from {username='slave'} at 127.0.0.1:38904
I20260709 13:07:48.920038 12562 raft_consensus.cc:493] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:48.920387 12562 raft_consensus.cc:3060] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:48.925717 12562 raft_consensus.cc:515] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:48.928067 12562 leader_election.cc:290] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 3 election: Requested vote from peers 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821), b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975)
I20260709 13:07:48.928789 12412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05b5fc6b3b248ffb2e34876863341a4" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "1bcc393b4836466e9be8845ba140e5e3"
I20260709 13:07:48.929360 12489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05b5fc6b3b248ffb2e34876863341a4" candidate_uuid: "07b29f69b9a2421db3983613bdc480ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "b186a014c0b745b8a66c39dae4efed4f"
I20260709 13:07:48.930198 12489 raft_consensus.cc:3055] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:48.930627 12489 raft_consensus.cc:740] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b186a014c0b745b8a66c39dae4efed4f, State: Running, Role: LEADER
I20260709 13:07:48.931607 12489 consensus_queue.cc:260] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:48.933166 12489 raft_consensus.cc:3060] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:48.929411 12412 raft_consensus.cc:3060] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:48.940229 12412 raft_consensus.cc:2468] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 3.
I20260709 13:07:48.941582 12529 leader_election.cc:304] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b29f69b9a2421db3983613bdc480ea, 1bcc393b4836466e9be8845ba140e5e3; no voters: 
I20260709 13:07:48.942369 12796 raft_consensus.cc:2804] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:48.942849 12796 raft_consensus.cc:697] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 3 LEADER]: Becoming Leader. State: Replica: 07b29f69b9a2421db3983613bdc480ea, State: Running, Role: LEADER
I20260709 13:07:48.943696 12796 consensus_queue.cc:237] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } }
I20260709 13:07:48.945475 12489 raft_consensus.cc:2468] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b29f69b9a2421db3983613bdc480ea in term 3.
I20260709 13:07:48.950379 12310 catalog_manager.cc:5697] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea reported cstate change: term changed from 2 to 3, leader changed from b186a014c0b745b8a66c39dae4efed4f (127.0.107.194) to 07b29f69b9a2421db3983613bdc480ea (127.0.107.195). New cstate: current_term: 3 leader_uuid: "07b29f69b9a2421db3983613bdc480ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b29f69b9a2421db3983613bdc480ea" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38269 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:49.296172 12489 raft_consensus.cc:1275] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Refusing update from remote peer 07b29f69b9a2421db3983613bdc480ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:49.297770 12796 consensus_queue.cc:1048] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:49.313334 12414 raft_consensus.cc:1275] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Refusing update from remote peer 07b29f69b9a2421db3983613bdc480ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:49.315106 12412 raft_consensus.cc:1275] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [term 3 FOLLOWER]: Refusing update from remote peer 07b29f69b9a2421db3983613bdc480ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:49.317920 12836 consensus_queue.cc:1048] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:49.318531 12796 consensus_queue.cc:1048] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:49.320171 12487 raft_consensus.cc:1275] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 3 FOLLOWER]: Refusing update from remote peer 07b29f69b9a2421db3983613bdc480ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:49.321707 12486 raft_consensus.cc:1275] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Refusing update from remote peer 07b29f69b9a2421db3983613bdc480ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:07:49.326175 12882 consensus_queue.cc:1048] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:49.330945 12411 raft_consensus.cc:1275] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Refusing update from remote peer 07b29f69b9a2421db3983613bdc480ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:07:49.338798 12836 consensus_queue.cc:1048] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:49.339548 12882 consensus_queue.cc:1048] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:49.482364 12414 raft_consensus.cc:1275] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3 [term 3 FOLLOWER]: Refusing update from remote peer 07b29f69b9a2421db3983613bdc480ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:07:49.483776 12882 consensus_queue.cc:1048] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcc393b4836466e9be8845ba140e5e3" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:49.493148 12489 raft_consensus.cc:1275] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [term 3 FOLLOWER]: Refusing update from remote peer 07b29f69b9a2421db3983613bdc480ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:07:49.494777 12836 consensus_queue.cc:1048] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b186a014c0b745b8a66c39dae4efed4f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:49.803714   431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:07:49.808485   431 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:07:49.808915   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:49.816007   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:07:49.816597   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:50.848217   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 13:07:50.855159 12529 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 7 similar messages]
W20260709 13:07:50.860954 12529 consensus_peers.cc:597] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea -> Peer 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821): Couldn't send request to peer 1bcc393b4836466e9be8845ba140e5e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:50.885885   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:50.887094   431 tablet_replica.cc:333] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3: stopping tablet replica
I20260709 13:07:50.888754   431 raft_consensus.cc:2243] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:50.889710   431 raft_consensus.cc:2272] T bce27e34734e4e868e207d32b9665e91 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:50.892586   431 tablet_replica.cc:333] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3: stopping tablet replica
I20260709 13:07:50.893337   431 raft_consensus.cc:2243] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:50.893973   431 raft_consensus.cc:2272] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:50.896520   431 tablet_replica.cc:333] T bb62b592dc854cd599b705b23ce7dc84 P 1bcc393b4836466e9be8845ba140e5e3: stopping tablet replica
I20260709 13:07:50.897102   431 raft_consensus.cc:2243] T bb62b592dc854cd599b705b23ce7dc84 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:50.897734   431 raft_consensus.cc:2272] T bb62b592dc854cd599b705b23ce7dc84 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:50.900259   431 tablet_replica.cc:333] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3: stopping tablet replica
I20260709 13:07:50.900892   431 raft_consensus.cc:2243] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:50.901988   431 raft_consensus.cc:2272] T 71b883cd452e46209d8ad4b02ec3d180 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:50.904368   431 tablet_replica.cc:333] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3: stopping tablet replica
I20260709 13:07:50.904958   431 raft_consensus.cc:2243] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:50.905532   431 raft_consensus.cc:2272] T a0487087198a422f82c31677e12b5f5a P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:50.908245   431 tablet_replica.cc:333] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3: stopping tablet replica
I20260709 13:07:50.909150   431 raft_consensus.cc:2243] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:50.910157   431 raft_consensus.cc:2272] T ab5220719559499aae8b369be60bfcce P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:50.912587   431 tablet_replica.cc:333] T 88b86d1b66e74927ab047790c2b55af7 P 1bcc393b4836466e9be8845ba140e5e3: stopping tablet replica
I20260709 13:07:50.913192   431 raft_consensus.cc:2243] T 88b86d1b66e74927ab047790c2b55af7 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:50.913789   431 raft_consensus.cc:2272] T 88b86d1b66e74927ab047790c2b55af7 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:50.916302   431 tablet_replica.cc:333] T a931166102f144028b066b74033e4e96 P 1bcc393b4836466e9be8845ba140e5e3: stopping tablet replica
I20260709 13:07:50.916944   431 raft_consensus.cc:2243] T a931166102f144028b066b74033e4e96 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:50.917557   431 raft_consensus.cc:2272] T a931166102f144028b066b74033e4e96 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:50.920240   431 tablet_replica.cc:333] T f40a25a0fc5d4d0a99ec544061e5fa76 P 1bcc393b4836466e9be8845ba140e5e3: stopping tablet replica
I20260709 13:07:50.920850   431 raft_consensus.cc:2243] T f40a25a0fc5d4d0a99ec544061e5fa76 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:50.921388   431 raft_consensus.cc:2272] T f40a25a0fc5d4d0a99ec544061e5fa76 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:50.923861   431 tablet_replica.cc:333] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3: stopping tablet replica
I20260709 13:07:50.924466   431 raft_consensus.cc:2243] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:50.925379   431 raft_consensus.cc:2272] T 01368bb716a44c1a826f04f41b92f7fd P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:50.928445   431 tablet_replica.cc:333] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3: stopping tablet replica
I20260709 13:07:50.929467   431 raft_consensus.cc:2243] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:50.930320   431 raft_consensus.cc:2272] T 6a8a4aaaf35e4f9bac685028409cdb96 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:50.930814 12455 consensus_peers.cc:597] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f -> Peer 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821): Couldn't send request to peer 1bcc393b4836466e9be8845ba140e5e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:50.933063   431 tablet_replica.cc:333] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3: stopping tablet replica
I20260709 13:07:50.933786   431 raft_consensus.cc:2243] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:50.934686   431 raft_consensus.cc:2272] T cd2f84a632224dcb84c681af4e30708b P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:50.937134   431 tablet_replica.cc:333] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3: stopping tablet replica
I20260709 13:07:50.937709   431 raft_consensus.cc:2243] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:50.938664   431 raft_consensus.cc:2272] T 5258afcab7984790b025622d73eaf684 P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:50.941156   431 tablet_replica.cc:333] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3: stopping tablet replica
I20260709 13:07:50.941800   431 raft_consensus.cc:2243] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:50.942327   431 raft_consensus.cc:2272] T 806d30f73a06482aaef3642acf872c32 P 1bcc393b4836466e9be8845ba140e5e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:50.944621   431 tablet_replica.cc:333] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3: stopping tablet replica
I20260709 13:07:50.945187   431 raft_consensus.cc:2243] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:50.945825   431 raft_consensus.cc:2272] T f05b5fc6b3b248ffb2e34876863341a4 P 1bcc393b4836466e9be8845ba140e5e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:50.948149   431 tablet_replica.cc:333] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3: stopping tablet replica
I20260709 13:07:50.948730   431 raft_consensus.cc:2243] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:50.949673   431 raft_consensus.cc:2272] T 20938c13e204477d89420fd9b4e62d0b P 1bcc393b4836466e9be8845ba140e5e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:50.952032   431 tablet_replica.cc:333] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3: stopping tablet replica
I20260709 13:07:50.952596   431 raft_consensus.cc:2243] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:50.953112   431 raft_consensus.cc:2272] T 0dc35585b6454b7a89990d0f849e8704 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:50.955608   431 tablet_replica.cc:333] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 1bcc393b4836466e9be8845ba140e5e3: stopping tablet replica
I20260709 13:07:50.956076   431 raft_consensus.cc:2243] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:50.956602   431 raft_consensus.cc:2272] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 1bcc393b4836466e9be8845ba140e5e3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:50.961951 12455 consensus_peers.cc:597] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f -> Peer 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821): Couldn't send request to peer 1bcc393b4836466e9be8845ba140e5e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:50.965180 12529 consensus_peers.cc:597] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea -> Peer 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821): Couldn't send request to peer 1bcc393b4836466e9be8845ba140e5e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:51.007942   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:07:51.026645 12455 consensus_peers.cc:597] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f -> Peer 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821): Couldn't send request to peer 1bcc393b4836466e9be8845ba140e5e3. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:51.034037 12455 consensus_peers.cc:597] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f -> Peer 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821): Couldn't send request to peer 1bcc393b4836466e9be8845ba140e5e3. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:51.060197   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:07:51.073074 12527 consensus_peers.cc:597] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea -> Peer b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975): Couldn't send request to peer b186a014c0b745b8a66c39dae4efed4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:51.095940   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:51.096808   431 tablet_replica.cc:333] T bce27e34734e4e868e207d32b9665e91 P b186a014c0b745b8a66c39dae4efed4f: stopping tablet replica
I20260709 13:07:51.097790   431 raft_consensus.cc:2243] T bce27e34734e4e868e207d32b9665e91 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.098429   431 raft_consensus.cc:2272] T bce27e34734e4e868e207d32b9665e91 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.101007   431 tablet_replica.cc:333] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f: stopping tablet replica
I20260709 13:07:51.101760   431 raft_consensus.cc:2243] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.102341   431 raft_consensus.cc:2272] T 0dc35585b6454b7a89990d0f849e8704 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.104838   431 tablet_replica.cc:333] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f: stopping tablet replica
I20260709 13:07:51.105541   431 raft_consensus.cc:2243] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.106595   431 raft_consensus.cc:2272] T 38b1e20fd4b04f08bd39ec6cc9792b5c P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.108935   431 tablet_replica.cc:333] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f: stopping tablet replica
I20260709 13:07:51.109577   431 raft_consensus.cc:2243] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.110792   431 raft_consensus.cc:2272] T bb62b592dc854cd599b705b23ce7dc84 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.114636   431 tablet_replica.cc:333] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f: stopping tablet replica
I20260709 13:07:51.115682   431 raft_consensus.cc:2243] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.117101   431 raft_consensus.cc:2272] T a0487087198a422f82c31677e12b5f5a P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:51.117239 12529 consensus_peers.cc:597] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea -> Peer 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821): Couldn't send request to peer 1bcc393b4836466e9be8845ba140e5e3. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:51.117851 12455 consensus_peers.cc:597] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f -> Peer 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821): Couldn't send request to peer 1bcc393b4836466e9be8845ba140e5e3. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:51.119923   431 tablet_replica.cc:333] T 71b883cd452e46209d8ad4b02ec3d180 P b186a014c0b745b8a66c39dae4efed4f: stopping tablet replica
I20260709 13:07:51.120677   431 raft_consensus.cc:2243] T 71b883cd452e46209d8ad4b02ec3d180 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.121381   431 raft_consensus.cc:2272] T 71b883cd452e46209d8ad4b02ec3d180 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.123872   431 tablet_replica.cc:333] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f: stopping tablet replica
I20260709 13:07:51.124473   431 raft_consensus.cc:2243] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.124989   431 raft_consensus.cc:2272] T 88b86d1b66e74927ab047790c2b55af7 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.127645   431 tablet_replica.cc:333] T ab5220719559499aae8b369be60bfcce P b186a014c0b745b8a66c39dae4efed4f: stopping tablet replica
I20260709 13:07:51.128253   431 raft_consensus.cc:2243] T ab5220719559499aae8b369be60bfcce P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.128886   431 raft_consensus.cc:2272] T ab5220719559499aae8b369be60bfcce P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.131574   431 tablet_replica.cc:333] T cd2f84a632224dcb84c681af4e30708b P b186a014c0b745b8a66c39dae4efed4f: stopping tablet replica
I20260709 13:07:51.132257   431 raft_consensus.cc:2243] T cd2f84a632224dcb84c681af4e30708b P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.132763   431 raft_consensus.cc:2272] T cd2f84a632224dcb84c681af4e30708b P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.135140   431 tablet_replica.cc:333] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f: stopping tablet replica
I20260709 13:07:51.135739   431 raft_consensus.cc:2243] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.136677   431 raft_consensus.cc:2272] T 6a8a4aaaf35e4f9bac685028409cdb96 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.139469   431 tablet_replica.cc:333] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f: stopping tablet replica
I20260709 13:07:51.140144   431 raft_consensus.cc:2243] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.140758   431 raft_consensus.cc:2272] T 01368bb716a44c1a826f04f41b92f7fd P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.143466   431 tablet_replica.cc:333] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f: stopping tablet replica
I20260709 13:07:51.144119   431 raft_consensus.cc:2243] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.144688   431 raft_consensus.cc:2272] T c2f1bab6aa8e4183808649fdcb0b1bb0 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.146996   431 tablet_replica.cc:333] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f: stopping tablet replica
I20260709 13:07:51.147552   431 raft_consensus.cc:2243] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.148471   431 raft_consensus.cc:2272] T f40a25a0fc5d4d0a99ec544061e5fa76 P b186a014c0b745b8a66c39dae4efed4f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.150707   431 tablet_replica.cc:333] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f: stopping tablet replica
I20260709 13:07:51.151237   431 raft_consensus.cc:2243] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.152107   431 raft_consensus.cc:2272] T a931166102f144028b066b74033e4e96 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.154374   431 tablet_replica.cc:333] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f: stopping tablet replica
I20260709 13:07:51.154883   431 raft_consensus.cc:2243] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.155480   431 raft_consensus.cc:2272] T 5258afcab7984790b025622d73eaf684 P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.157655   431 tablet_replica.cc:333] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f: stopping tablet replica
I20260709 13:07:51.158095   431 raft_consensus.cc:2243] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.158520   431 raft_consensus.cc:2272] T f05b5fc6b3b248ffb2e34876863341a4 P b186a014c0b745b8a66c39dae4efed4f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.160564   431 tablet_replica.cc:333] T 20938c13e204477d89420fd9b4e62d0b P b186a014c0b745b8a66c39dae4efed4f: stopping tablet replica
I20260709 13:07:51.161052   431 raft_consensus.cc:2243] T 20938c13e204477d89420fd9b4e62d0b P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.161459   431 raft_consensus.cc:2272] T 20938c13e204477d89420fd9b4e62d0b P b186a014c0b745b8a66c39dae4efed4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.163630   431 tablet_replica.cc:333] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f: stopping tablet replica
I20260709 13:07:51.164053   431 raft_consensus.cc:2243] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.164530   431 raft_consensus.cc:2272] T 806d30f73a06482aaef3642acf872c32 P b186a014c0b745b8a66c39dae4efed4f [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:51.170186 12527 consensus_peers.cc:597] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea -> Peer b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975): Couldn't send request to peer b186a014c0b745b8a66c39dae4efed4f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:51.213843 12529 consensus_peers.cc:597] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea -> Peer 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821): Couldn't send request to peer 1bcc393b4836466e9be8845ba140e5e3. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:51.228677   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:07:51.247072 12527 consensus_peers.cc:597] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea -> Peer b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975): Couldn't send request to peer b186a014c0b745b8a66c39dae4efed4f. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:38975: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:51.247874 12529 consensus_peers.cc:597] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea -> Peer 1bcc393b4836466e9be8845ba140e5e3 (127.0.107.193:45821): Couldn't send request to peer 1bcc393b4836466e9be8845ba140e5e3. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:51.271183   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 13:07:51.302909   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:51.303854   431 tablet_replica.cc:333] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea: stopping tablet replica
I20260709 13:07:51.304817   431 raft_consensus.cc:2243] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.305914   431 raft_consensus.cc:2272] T bce27e34734e4e868e207d32b9665e91 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.309181   431 tablet_replica.cc:333] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 07b29f69b9a2421db3983613bdc480ea: stopping tablet replica
I20260709 13:07:51.309975   431 raft_consensus.cc:2243] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.310554   431 raft_consensus.cc:2272] T 38b1e20fd4b04f08bd39ec6cc9792b5c P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.312927   431 tablet_replica.cc:333] T bb62b592dc854cd599b705b23ce7dc84 P 07b29f69b9a2421db3983613bdc480ea: stopping tablet replica
I20260709 13:07:51.313517   431 raft_consensus.cc:2243] T bb62b592dc854cd599b705b23ce7dc84 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.314086   431 raft_consensus.cc:2272] T bb62b592dc854cd599b705b23ce7dc84 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.316471   431 tablet_replica.cc:333] T 5258afcab7984790b025622d73eaf684 P 07b29f69b9a2421db3983613bdc480ea: stopping tablet replica
I20260709 13:07:51.317072   431 raft_consensus.cc:2243] T 5258afcab7984790b025622d73eaf684 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.317849   431 raft_consensus.cc:2272] T 5258afcab7984790b025622d73eaf684 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.320854   431 tablet_replica.cc:333] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea: stopping tablet replica
W20260709 13:07:51.321267 12527 consensus_peers.cc:597] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea -> Peer b186a014c0b745b8a66c39dae4efed4f (127.0.107.194:38975): Couldn't send request to peer b186a014c0b745b8a66c39dae4efed4f. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:38975: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:51.321513   431 raft_consensus.cc:2243] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.322579   431 raft_consensus.cc:2272] T 0dc35585b6454b7a89990d0f849e8704 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.324617   431 tablet_replica.cc:333] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea: stopping tablet replica
I20260709 13:07:51.325165   431 raft_consensus.cc:2243] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.325624   431 raft_consensus.cc:2272] T 71b883cd452e46209d8ad4b02ec3d180 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.327737   431 tablet_replica.cc:333] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea: stopping tablet replica
I20260709 13:07:51.328199   431 raft_consensus.cc:2243] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.329141   431 raft_consensus.cc:2272] T c2f1bab6aa8e4183808649fdcb0b1bb0 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.331431   431 tablet_replica.cc:333] T a931166102f144028b066b74033e4e96 P 07b29f69b9a2421db3983613bdc480ea: stopping tablet replica
I20260709 13:07:51.331976   431 raft_consensus.cc:2243] T a931166102f144028b066b74033e4e96 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.332373   431 raft_consensus.cc:2272] T a931166102f144028b066b74033e4e96 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.334906   431 tablet_replica.cc:333] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea: stopping tablet replica
I20260709 13:07:51.335698   431 raft_consensus.cc:2243] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.336301   431 raft_consensus.cc:2272] T f40a25a0fc5d4d0a99ec544061e5fa76 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.338835   431 tablet_replica.cc:333] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea: stopping tablet replica
I20260709 13:07:51.339320   431 raft_consensus.cc:2243] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.340113   431 raft_consensus.cc:2272] T f05b5fc6b3b248ffb2e34876863341a4 P 07b29f69b9a2421db3983613bdc480ea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.342096   431 tablet_replica.cc:333] T 20938c13e204477d89420fd9b4e62d0b P 07b29f69b9a2421db3983613bdc480ea: stopping tablet replica
I20260709 13:07:51.342557   431 raft_consensus.cc:2243] T 20938c13e204477d89420fd9b4e62d0b P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.342991   431 raft_consensus.cc:2272] T 20938c13e204477d89420fd9b4e62d0b P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.345842   431 tablet_replica.cc:333] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea: stopping tablet replica
I20260709 13:07:51.346300   431 raft_consensus.cc:2243] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.347074   431 raft_consensus.cc:2272] T 806d30f73a06482aaef3642acf872c32 P 07b29f69b9a2421db3983613bdc480ea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.349056   431 tablet_replica.cc:333] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea: stopping tablet replica
I20260709 13:07:51.349550   431 raft_consensus.cc:2243] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.350009   431 raft_consensus.cc:2272] T 01368bb716a44c1a826f04f41b92f7fd P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.352034   431 tablet_replica.cc:333] T cd2f84a632224dcb84c681af4e30708b P 07b29f69b9a2421db3983613bdc480ea: stopping tablet replica
I20260709 13:07:51.352545   431 raft_consensus.cc:2243] T cd2f84a632224dcb84c681af4e30708b P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.352998   431 raft_consensus.cc:2272] T cd2f84a632224dcb84c681af4e30708b P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.355145   431 tablet_replica.cc:333] T 6a8a4aaaf35e4f9bac685028409cdb96 P 07b29f69b9a2421db3983613bdc480ea: stopping tablet replica
I20260709 13:07:51.355655   431 raft_consensus.cc:2243] T 6a8a4aaaf35e4f9bac685028409cdb96 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.356230   431 raft_consensus.cc:2272] T 6a8a4aaaf35e4f9bac685028409cdb96 P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.358176   431 tablet_replica.cc:333] T a0487087198a422f82c31677e12b5f5a P 07b29f69b9a2421db3983613bdc480ea: stopping tablet replica
I20260709 13:07:51.358665   431 raft_consensus.cc:2243] T a0487087198a422f82c31677e12b5f5a P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.359171   431 raft_consensus.cc:2272] T a0487087198a422f82c31677e12b5f5a P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.361224   431 tablet_replica.cc:333] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea: stopping tablet replica
I20260709 13:07:51.361739   431 raft_consensus.cc:2243] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.362262   431 raft_consensus.cc:2272] T ab5220719559499aae8b369be60bfcce P 07b29f69b9a2421db3983613bdc480ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.364166   431 tablet_replica.cc:333] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea: stopping tablet replica
I20260709 13:07:51.364581   431 raft_consensus.cc:2243] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.365314   431 raft_consensus.cc:2272] T 88b86d1b66e74927ab047790c2b55af7 P 07b29f69b9a2421db3983613bdc480ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.406385   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:07:51.443856   431 master.cc:562] Master@127.0.107.254:41623 shutting down...
I20260709 13:07:51.478328   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.479182   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.479663   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 35223ddd203348d8a6dc43fe34afdcf5: stopping tablet replica
I20260709 13:07:51.502066   431 master.cc:584] Master@127.0.107.254:41623 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (13127 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:07:51.556623   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:40089
I20260709 13:07:51.557981   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:51.563985 12901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:51.564312 12902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:51.565352 12904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:51.565412   431 server_base.cc:1061] running on GCE node
I20260709 13:07:51.566705   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:51.566881   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:51.567003   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602471566992 us; error 0 us; skew 500 ppm
I20260709 13:07:51.567425   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:51.569965   431 webserver.cc:533] Webserver started at http://127.0.107.254:38875/ using document root <none> and password file <none>
I20260709 13:07:51.570475   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:51.570667   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:51.570964   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:51.572165   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602054858684-431-0/minicluster-data/master-0-root/instance:
uuid: "36a67cb3227c4b49858794f6cda16813"
format_stamp: "Formatted at 2026-07-09 13:07:51 on dist-test-slave-mxd3"
I20260709 13:07:51.576869   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260709 13:07:51.580632 12909 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:51.581565   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:07:51.581873   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602054858684-431-0/minicluster-data/master-0-root
uuid: "36a67cb3227c4b49858794f6cda16813"
format_stamp: "Formatted at 2026-07-09 13:07:51 on dist-test-slave-mxd3"
I20260709 13:07:51.582118   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602054858684-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602054858684-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602054858684-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:51.591710   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:51.592852   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:51.631453   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:40089
I20260709 13:07:51.631773 12960 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:40089 every 8 connection(s)
I20260709 13:07:51.635699 12961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:51.647569 12961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813: Bootstrap starting.
I20260709 13:07:51.652591 12961 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.657013 12961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813: No bootstrap required, opened a new log
I20260709 13:07:51.659363 12961 raft_consensus.cc:359] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a67cb3227c4b49858794f6cda16813" member_type: VOTER }
I20260709 13:07:51.659813 12961 raft_consensus.cc:385] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.660065 12961 raft_consensus.cc:740] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36a67cb3227c4b49858794f6cda16813, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.660627 12961 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a67cb3227c4b49858794f6cda16813" member_type: VOTER }
I20260709 13:07:51.661105 12961 raft_consensus.cc:399] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:07:51.661316 12961 raft_consensus.cc:493] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:07:51.661628 12961 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.666427 12961 raft_consensus.cc:515] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a67cb3227c4b49858794f6cda16813" member_type: VOTER }
I20260709 13:07:51.667002 12961 leader_election.cc:304] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36a67cb3227c4b49858794f6cda16813; no voters: 
I20260709 13:07:51.668304 12961 leader_election.cc:290] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:07:51.668792 12964 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:51.670256 12964 raft_consensus.cc:697] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813 [term 1 LEADER]: Becoming Leader. State: Replica: 36a67cb3227c4b49858794f6cda16813, State: Running, Role: LEADER
I20260709 13:07:51.671056 12964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a67cb3227c4b49858794f6cda16813" member_type: VOTER }
I20260709 13:07:51.671480 12961 sys_catalog.cc:565] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:07:51.675863 12966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36a67cb3227c4b49858794f6cda16813. Latest consensus state: current_term: 1 leader_uuid: "36a67cb3227c4b49858794f6cda16813" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a67cb3227c4b49858794f6cda16813" member_type: VOTER } }
I20260709 13:07:51.675822 12965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36a67cb3227c4b49858794f6cda16813" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a67cb3227c4b49858794f6cda16813" member_type: VOTER } }
I20260709 13:07:51.676604 12966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:51.676751 12965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:51.680662 12971 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:07:51.688900 12971 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:07:51.689987   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:07:51.698061 12971 catalog_manager.cc:1383] Generated new cluster ID: 6a9bc4b78dce4a90a5714f0da2576492
I20260709 13:07:51.698367 12971 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:07:51.715342 12971 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:07:51.716642 12971 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:07:51.732372 12971 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813: Generated new TSK 0
I20260709 13:07:51.733026 12971 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:07:51.757114   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:51.763819 12982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:51.764685 12983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:51.766327 12985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:51.767099   431 server_base.cc:1061] running on GCE node
I20260709 13:07:51.768261   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:51.768476   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:51.768602   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602471768591 us; error 0 us; skew 500 ppm
I20260709 13:07:51.769089   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:51.771500   431 webserver.cc:533] Webserver started at http://127.0.107.193:42637/ using document root <none> and password file <none>
I20260709 13:07:51.772174   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:51.772352   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:51.772567   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:51.773923   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602054858684-431-0/minicluster-data/ts-0-root/instance:
uuid: "8593949d5fb7413ca30c9ef5dfd31411"
format_stamp: "Formatted at 2026-07-09 13:07:51 on dist-test-slave-mxd3"
I20260709 13:07:51.778438   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:07:51.782550 12990 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:51.783527   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:07:51.783915   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602054858684-431-0/minicluster-data/ts-0-root
uuid: "8593949d5fb7413ca30c9ef5dfd31411"
format_stamp: "Formatted at 2026-07-09 13:07:51 on dist-test-slave-mxd3"
I20260709 13:07:51.784209   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602054858684-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602054858684-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602054858684-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:51.810611   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:51.811879   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:51.813378   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:51.815927   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:51.816159   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:51.816413   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:51.816613   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:51.859407   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:44253
I20260709 13:07:51.859575 13052 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:44253 every 8 connection(s)
I20260709 13:07:51.864687   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:51.872313 13057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:51.874287 13058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:51.878294 13060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:51.880074 13053 heartbeater.cc:344] Connected to a master server at 127.0.107.254:40089
I20260709 13:07:51.880525 13053 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:51.880614   431 server_base.cc:1061] running on GCE node
I20260709 13:07:51.881572 13053 heartbeater.cc:507] Master 127.0.107.254:40089 requested a full tablet report, sending...
I20260709 13:07:51.881798   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:51.882110   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:51.882309   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602471882292 us; error 0 us; skew 500 ppm
I20260709 13:07:51.882992   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:51.884290 12926 ts_manager.cc:194] Registered new tserver with Master: 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253)
I20260709 13:07:51.886034   431 webserver.cc:533] Webserver started at http://127.0.107.194:46061/ using document root <none> and password file <none>
I20260709 13:07:51.886732   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:51.886961   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:51.887086 12926 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58342
I20260709 13:07:51.887259   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:51.888761   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602054858684-431-0/minicluster-data/ts-1-root/instance:
uuid: "20f53f29f26f48d6b818f46c9e755f69"
format_stamp: "Formatted at 2026-07-09 13:07:51 on dist-test-slave-mxd3"
I20260709 13:07:51.893356   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 13:07:51.897123 13065 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:51.898375   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:07:51.898763   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602054858684-431-0/minicluster-data/ts-1-root
uuid: "20f53f29f26f48d6b818f46c9e755f69"
format_stamp: "Formatted at 2026-07-09 13:07:51 on dist-test-slave-mxd3"
I20260709 13:07:51.899083   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602054858684-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602054858684-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602054858684-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:51.919019   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:51.920292   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:51.921924   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:51.924285   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:51.924521   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:51.924799   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:51.924953   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:51.969658   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:46507
I20260709 13:07:51.969830 13127 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:46507 every 8 connection(s)
I20260709 13:07:51.974960   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:51.982981 13131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:51.984086 13132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:51.988068   431 server_base.cc:1061] running on GCE node
W20260709 13:07:51.988955 13135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:51.989413 13128 heartbeater.cc:344] Connected to a master server at 127.0.107.254:40089
I20260709 13:07:51.989869 13128 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:51.990167   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:51.990478   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:51.990676 13128 heartbeater.cc:507] Master 127.0.107.254:40089 requested a full tablet report, sending...
I20260709 13:07:51.990682   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602471990668 us; error 0 us; skew 500 ppm
I20260709 13:07:51.991729   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:51.992985 12926 ts_manager.cc:194] Registered new tserver with Master: 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507)
I20260709 13:07:51.994563   431 webserver.cc:533] Webserver started at http://127.0.107.195:38651/ using document root <none> and password file <none>
I20260709 13:07:51.994752 12926 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58352
I20260709 13:07:51.995213   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:51.995468   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:51.995772   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:51.997035   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602054858684-431-0/minicluster-data/ts-2-root/instance:
uuid: "824d687f218b4cb091e5d62fa9d18187"
format_stamp: "Formatted at 2026-07-09 13:07:51 on dist-test-slave-mxd3"
I20260709 13:07:52.001912   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 13:07:52.005548 13139 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.006529   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:07:52.006965   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602054858684-431-0/minicluster-data/ts-2-root
uuid: "824d687f218b4cb091e5d62fa9d18187"
format_stamp: "Formatted at 2026-07-09 13:07:51 on dist-test-slave-mxd3"
I20260709 13:07:52.007292   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602054858684-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602054858684-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskcnQcHX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602054858684-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:52.019624   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:52.020915   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:52.022552   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:52.024863   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:52.025113   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.025379   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:52.025589   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.067729   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:44499
I20260709 13:07:52.067824 13201 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:44499 every 8 connection(s)
I20260709 13:07:52.082547 13202 heartbeater.cc:344] Connected to a master server at 127.0.107.254:40089
I20260709 13:07:52.083034 13202 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:52.083866 13202 heartbeater.cc:507] Master 127.0.107.254:40089 requested a full tablet report, sending...
I20260709 13:07:52.086174 12926 ts_manager.cc:194] Registered new tserver with Master: 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:52.086735   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01569544s
I20260709 13:07:52.087760 12926 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58368
I20260709 13:07:52.113372 12926 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58378:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:52.116235 12926 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:52.195721 13093 tablet_service.cc:1511] Processing CreateTablet for tablet d727f4f4eacb428889625360bb663857 (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:52.197655 13093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d727f4f4eacb428889625360bb663857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.197405 13090 tablet_service.cc:1511] Processing CreateTablet for tablet 68f4b09178c244199a36c43c3989a45b (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:52.198704 13091 tablet_service.cc:1511] Processing CreateTablet for tablet f5fffab9019146fd9cc617253eb19f87 (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:52.200238 13090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68f4b09178c244199a36c43c3989a45b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.200795 13091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5fffab9019146fd9cc617253eb19f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.202947 13092 tablet_service.cc:1511] Processing CreateTablet for tablet e175694e0fa6424f900caf84dbb2945f (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:52.204411 13092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e175694e0fa6424f900caf84dbb2945f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.209834 13089 tablet_service.cc:1511] Processing CreateTablet for tablet d393f71c2afb406098b92973cb2be621 (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:52.212225 13088 tablet_service.cc:1511] Processing CreateTablet for tablet eab0853d0dd94bcf8327c0ef09c6fad4 (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:52.213578 13017 tablet_service.cc:1511] Processing CreateTablet for tablet e175694e0fa6424f900caf84dbb2945f (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:52.214849 13016 tablet_service.cc:1511] Processing CreateTablet for tablet f5fffab9019146fd9cc617253eb19f87 (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:52.220875 13017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e175694e0fa6424f900caf84dbb2945f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.221529 13016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5fffab9019146fd9cc617253eb19f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.214849 13087 tablet_service.cc:1511] Processing CreateTablet for tablet 4448333d61c640e494f7ae000878dbdf (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:52.215396 13089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d393f71c2afb406098b92973cb2be621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.225317 13088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eab0853d0dd94bcf8327c0ef09c6fad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.227692 13010 tablet_service.cc:1511] Processing CreateTablet for tablet a22a6947bab14423aa83c01130c20d77 (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:52.229344 13010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a22a6947bab14423aa83c01130c20d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.218400 13013 tablet_service.cc:1511] Processing CreateTablet for tablet eab0853d0dd94bcf8327c0ef09c6fad4 (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:52.231302 13013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eab0853d0dd94bcf8327c0ef09c6fad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.230962 13011 tablet_service.cc:1511] Processing CreateTablet for tablet 0d8b4f1a658d47cc954ba897df965354 (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:52.232291 13011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d8b4f1a658d47cc954ba897df965354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.232699 13087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4448333d61c640e494f7ae000878dbdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.212225 13018 tablet_service.cc:1511] Processing CreateTablet for tablet d727f4f4eacb428889625360bb663857 (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:52.242782 13086 tablet_service.cc:1511] Processing CreateTablet for tablet 0d8b4f1a658d47cc954ba897df965354 (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:52.244299 13086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d8b4f1a658d47cc954ba897df965354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.216110 13015 tablet_service.cc:1511] Processing CreateTablet for tablet 68f4b09178c244199a36c43c3989a45b (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:52.217306 13014 tablet_service.cc:1511] Processing CreateTablet for tablet d393f71c2afb406098b92973cb2be621 (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:52.217192 13012 tablet_service.cc:1511] Processing CreateTablet for tablet 4448333d61c640e494f7ae000878dbdf (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:52.247572 13085 tablet_service.cc:1511] Processing CreateTablet for tablet a22a6947bab14423aa83c01130c20d77 (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:52.249109 13085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a22a6947bab14423aa83c01130c20d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.249145 13018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d727f4f4eacb428889625360bb663857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.254037 13015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68f4b09178c244199a36c43c3989a45b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.255950 13014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d393f71c2afb406098b92973cb2be621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.262085 13012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4448333d61c640e494f7ae000878dbdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.307158 13166 tablet_service.cc:1511] Processing CreateTablet for tablet e175694e0fa6424f900caf84dbb2945f (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:52.307956 13160 tablet_service.cc:1511] Processing CreateTablet for tablet 0d8b4f1a658d47cc954ba897df965354 (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:52.308848 13166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e175694e0fa6424f900caf84dbb2945f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.310285 13159 tablet_service.cc:1511] Processing CreateTablet for tablet a22a6947bab14423aa83c01130c20d77 (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:52.310813 13164 tablet_service.cc:1511] Processing CreateTablet for tablet 68f4b09178c244199a36c43c3989a45b (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:52.311691 13159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a22a6947bab14423aa83c01130c20d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.314471 13163 tablet_service.cc:1511] Processing CreateTablet for tablet d393f71c2afb406098b92973cb2be621 (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:52.316390 13163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d393f71c2afb406098b92973cb2be621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.319825 13160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d8b4f1a658d47cc954ba897df965354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.323489 13164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68f4b09178c244199a36c43c3989a45b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.324929 13162 tablet_service.cc:1511] Processing CreateTablet for tablet eab0853d0dd94bcf8327c0ef09c6fad4 (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:52.326021 13167 tablet_service.cc:1511] Processing CreateTablet for tablet d727f4f4eacb428889625360bb663857 (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:52.326457 13162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eab0853d0dd94bcf8327c0ef09c6fad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.327349 13167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d727f4f4eacb428889625360bb663857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.327993 13218 tablet_bootstrap.cc:492] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411: Bootstrap starting.
I20260709 13:07:52.330503 13161 tablet_service.cc:1511] Processing CreateTablet for tablet 4448333d61c640e494f7ae000878dbdf (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:52.331943 13161 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4448333d61c640e494f7ae000878dbdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.334257 13165 tablet_service.cc:1511] Processing CreateTablet for tablet f5fffab9019146fd9cc617253eb19f87 (DEFAULT_TABLE table=test-workload [id=fdd98109318946958be57dd0de751cac]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:52.335692 13165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5fffab9019146fd9cc617253eb19f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.354911 13217 tablet_bootstrap.cc:492] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69: Bootstrap starting.
I20260709 13:07:52.366762 13218 tablet_bootstrap.cc:654] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.400674 13219 tablet_bootstrap.cc:492] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187: Bootstrap starting.
I20260709 13:07:52.401548 13217 tablet_bootstrap.cc:654] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.407379 13219 tablet_bootstrap.cc:654] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.412220 13218 tablet_bootstrap.cc:492] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411: No bootstrap required, opened a new log
I20260709 13:07:52.412856 13218 ts_tablet_manager.cc:1403] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411: Time spent bootstrapping tablet: real 0.085s	user 0.021s	sys 0.013s
I20260709 13:07:52.417181 13218 raft_consensus.cc:359] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.418494 13218 raft_consensus.cc:385] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.418992 13218 raft_consensus.cc:740] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.420142 13218 consensus_queue.cc:260] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.429086 13053 heartbeater.cc:499] Master 127.0.107.254:40089 was elected leader, sending a full tablet report...
I20260709 13:07:52.430161 13218 ts_tablet_manager.cc:1434] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411: Time spent starting tablet: real 0.017s	user 0.013s	sys 0.005s
I20260709 13:07:52.431290 13218 tablet_bootstrap.cc:492] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411: Bootstrap starting.
I20260709 13:07:52.439353 13218 tablet_bootstrap.cc:654] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.449623 13217 tablet_bootstrap.cc:492] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69: No bootstrap required, opened a new log
I20260709 13:07:52.451050 13217 ts_tablet_manager.cc:1403] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69: Time spent bootstrapping tablet: real 0.096s	user 0.042s	sys 0.008s
I20260709 13:07:52.451848 13218 tablet_bootstrap.cc:492] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411: No bootstrap required, opened a new log
I20260709 13:07:52.452620 13218 ts_tablet_manager.cc:1403] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.005s
I20260709 13:07:52.451843 13219 tablet_bootstrap.cc:492] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187: No bootstrap required, opened a new log
I20260709 13:07:52.454011 13219 ts_tablet_manager.cc:1403] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187: Time spent bootstrapping tablet: real 0.054s	user 0.015s	sys 0.020s
I20260709 13:07:52.454089 13217 raft_consensus.cc:359] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } } peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } }
I20260709 13:07:52.454952 13217 raft_consensus.cc:385] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.455271 13217 raft_consensus.cc:740] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.455554 13218 raft_consensus.cc:359] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.456379 13218 raft_consensus.cc:385] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.456122 13217 consensus_queue.cc:260] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } } peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } }
I20260709 13:07:52.456720 13218 raft_consensus.cc:740] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.457078 13219 raft_consensus.cc:359] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.457719 13218 consensus_queue.cc:260] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.458073 13219 raft_consensus.cc:385] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.459398 13219 raft_consensus.cc:740] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.459998 13218 ts_tablet_manager.cc:1434] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:07:52.461038 13218 tablet_bootstrap.cc:492] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411: Bootstrap starting.
I20260709 13:07:52.460577 13219 consensus_queue.cc:260] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.466374 13217 ts_tablet_manager.cc:1434] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 13:07:52.467423 13128 heartbeater.cc:499] Master 127.0.107.254:40089 was elected leader, sending a full tablet report...
I20260709 13:07:52.471062 13218 tablet_bootstrap.cc:654] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.471458 13223 raft_consensus.cc:493] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.472046 13223 raft_consensus.cc:515] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.473223 13217 tablet_bootstrap.cc:492] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69: Bootstrap starting.
I20260709 13:07:52.481392 13217 tablet_bootstrap.cc:654] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.481648 13223 leader_election.cc:290] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:52.483176 13202 heartbeater.cc:499] Master 127.0.107.254:40089 was elected leader, sending a full tablet report...
I20260709 13:07:52.491549 13218 tablet_bootstrap.cc:492] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411: No bootstrap required, opened a new log
I20260709 13:07:52.492089 13218 ts_tablet_manager.cc:1403] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.002s
I20260709 13:07:52.492709 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e175694e0fa6424f900caf84dbb2945f" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69" is_pre_election: true
I20260709 13:07:52.495255 13218 raft_consensus.cc:359] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.496053 13218 raft_consensus.cc:385] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.496338 13218 raft_consensus.cc:740] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.497529 13219 ts_tablet_manager.cc:1434] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.008s
I20260709 13:07:52.501910 13219 tablet_bootstrap.cc:492] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187: Bootstrap starting.
W20260709 13:07:52.502496 12992 leader_election.cc:343] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:52.503166 13218 consensus_queue.cc:260] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.505788 13218 ts_tablet_manager.cc:1434] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:07:52.506855 13218 tablet_bootstrap.cc:492] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411: Bootstrap starting.
I20260709 13:07:52.511710 13226 raft_consensus.cc:493] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.512396 13226 raft_consensus.cc:515] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.513300 13218 tablet_bootstrap.cc:654] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.519065 13219 tablet_bootstrap.cc:654] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.524650 13218 tablet_bootstrap.cc:492] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411: No bootstrap required, opened a new log
I20260709 13:07:52.525205 13218 ts_tablet_manager.cc:1403] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.008s
I20260709 13:07:52.527829 13218 raft_consensus.cc:359] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.528419 13218 raft_consensus.cc:385] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.534507 13218 raft_consensus.cc:740] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.535032 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e175694e0fa6424f900caf84dbb2945f" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
I20260709 13:07:52.535378 13218 consensus_queue.cc:260] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
W20260709 13:07:52.536784 12994 leader_election.cc:343] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:52.537272 12994 leader_election.cc:304] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8593949d5fb7413ca30c9ef5dfd31411; no voters: 20f53f29f26f48d6b818f46c9e755f69, 824d687f218b4cb091e5d62fa9d18187
I20260709 13:07:52.538177 13218 ts_tablet_manager.cc:1434] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:07:52.538441 13223 raft_consensus.cc:2749] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:52.538532 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d393f71c2afb406098b92973cb2be621" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
I20260709 13:07:52.539458 13218 tablet_bootstrap.cc:492] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411: Bootstrap starting.
I20260709 13:07:52.539422 13028 raft_consensus.cc:2468] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 0.
I20260709 13:07:52.541196 13140 leader_election.cc:304] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:52.529163 13226 leader_election.cc:290] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507)
I20260709 13:07:52.544165 13226 raft_consensus.cc:2804] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:52.544555 13226 raft_consensus.cc:493] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.544934 13226 raft_consensus.cc:3060] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:52.545562 13225 raft_consensus.cc:493] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.546134 13225 raft_consensus.cc:515] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } } peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } }
I20260709 13:07:52.547341 13218 tablet_bootstrap.cc:654] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.555336 13225 leader_election.cc:290] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499), 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253)
I20260709 13:07:52.560712 13217 tablet_bootstrap.cc:492] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69: No bootstrap required, opened a new log
I20260709 13:07:52.561391 13217 ts_tablet_manager.cc:1403] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69: Time spent bootstrapping tablet: real 0.088s	user 0.020s	sys 0.014s
I20260709 13:07:52.575337 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d727f4f4eacb428889625360bb663857" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
I20260709 13:07:52.576192 13218 tablet_bootstrap.cc:492] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411: No bootstrap required, opened a new log
I20260709 13:07:52.576048 13226 raft_consensus.cc:515] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.576821 13218 ts_tablet_manager.cc:1403] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411: Time spent bootstrapping tablet: real 0.038s	user 0.028s	sys 0.008s
W20260709 13:07:52.577212 13066 leader_election.cc:343] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:52.572794 13217 raft_consensus.cc:359] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.578063 13217 raft_consensus.cc:385] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.578390 13217 raft_consensus.cc:740] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.579643 13226 leader_election.cc:290] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507)
I20260709 13:07:52.580384 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d393f71c2afb406098b92973cb2be621" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:52.580615 13223 raft_consensus.cc:493] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.581035 13028 raft_consensus.cc:3060] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:52.581121 13223 raft_consensus.cc:515] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.581583 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d727f4f4eacb428889625360bb663857" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
I20260709 13:07:52.581955 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d393f71c2afb406098b92973cb2be621" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69" is_pre_election: true
I20260709 13:07:52.583335 13223 leader_election.cc:290] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
W20260709 13:07:52.583629 13069 leader_election.cc:343] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:52.584299 13069 leader_election.cc:304] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69; no voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411
I20260709 13:07:52.585083 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5fffab9019146fd9cc617253eb19f87" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
I20260709 13:07:52.584827 13218 raft_consensus.cc:359] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.585927 13218 raft_consensus.cc:385] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.586321 13218 raft_consensus.cc:740] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.586594 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5fffab9019146fd9cc617253eb19f87" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69" is_pre_election: true
I20260709 13:07:52.587371 13218 consensus_queue.cc:260] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
W20260709 13:07:52.586737 12994 leader_election.cc:343] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:52.588980 12992 leader_election.cc:343] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:52.589190 13028 raft_consensus.cc:2468] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 1.
I20260709 13:07:52.589710 12992 leader_election.cc:304] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8593949d5fb7413ca30c9ef5dfd31411; no voters: 20f53f29f26f48d6b818f46c9e755f69, 824d687f218b4cb091e5d62fa9d18187
I20260709 13:07:52.589922 13225 raft_consensus.cc:2749] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:52.590919 13223 raft_consensus.cc:2749] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:52.580392 13217 consensus_queue.cc:260] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.592192 13218 ts_tablet_manager.cc:1434] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 13:07:52.593458 13218 tablet_bootstrap.cc:492] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411: Bootstrap starting.
I20260709 13:07:52.593801 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d393f71c2afb406098b92973cb2be621" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:07:52.596892 13217 ts_tablet_manager.cc:1434] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69: Time spent starting tablet: real 0.028s	user 0.002s	sys 0.004s
I20260709 13:07:52.597997 13217 tablet_bootstrap.cc:492] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69: Bootstrap starting.
I20260709 13:07:52.598248 13219 tablet_bootstrap.cc:492] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187: No bootstrap required, opened a new log
I20260709 13:07:52.598875 13219 ts_tablet_manager.cc:1403] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187: Time spent bootstrapping tablet: real 0.097s	user 0.016s	sys 0.025s
I20260709 13:07:52.600441 13218 tablet_bootstrap.cc:654] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.602352 13219 raft_consensus.cc:359] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.603245 13219 raft_consensus.cc:385] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.603600 13219 raft_consensus.cc:740] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.604851 13217 tablet_bootstrap.cc:654] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.604421 13219 consensus_queue.cc:260] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.610287 13140 leader_election.cc:304] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
W20260709 13:07:52.611120 13141 leader_election.cc:343] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:52.612254 13141 leader_election.cc:343] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:52.619822 13226 raft_consensus.cc:2804] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:52.620735 13226 raft_consensus.cc:697] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 1 LEADER]: Becoming Leader. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Running, Role: LEADER
I20260709 13:07:52.621963 13226 consensus_queue.cc:237] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.628165 13223 raft_consensus.cc:493] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.671514 13225 raft_consensus.cc:493] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.672333 13225 raft_consensus.cc:515] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.629072 13223 raft_consensus.cc:515] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
W20260709 13:07:52.684091 12974 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:52.684051 13219 ts_tablet_manager.cc:1434] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187: Time spent starting tablet: real 0.085s	user 0.027s	sys 0.036s
I20260709 13:07:52.686708 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a22a6947bab14423aa83c01130c20d77" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69" is_pre_election: true
I20260709 13:07:52.687050 13218 tablet_bootstrap.cc:492] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411: No bootstrap required, opened a new log
I20260709 13:07:52.687336 13225 leader_election.cc:290] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:52.687911 13218 ts_tablet_manager.cc:1403] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411: Time spent bootstrapping tablet: real 0.095s	user 0.032s	sys 0.043s
I20260709 13:07:52.688021 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a22a6947bab14423aa83c01130c20d77" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
W20260709 13:07:52.688364 12992 leader_election.cc:343] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:52.688867 13177 raft_consensus.cc:2468] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 0.
I20260709 13:07:52.685899 13219 tablet_bootstrap.cc:492] T 4448333d61c640e494f7ae000878dbdf P 824d687f218b4cb091e5d62fa9d18187: Bootstrap starting.
I20260709 13:07:52.691434 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68f4b09178c244199a36c43c3989a45b" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
I20260709 13:07:52.691574 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68f4b09178c244199a36c43c3989a45b" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
I20260709 13:07:52.692198 13028 raft_consensus.cc:2468] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 0.
I20260709 13:07:52.691699 13218 raft_consensus.cc:359] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.692836 13218 raft_consensus.cc:385] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.693214 13218 raft_consensus.cc:740] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Initialized, Role: FOLLOWER
W20260709 13:07:52.693399 13069 leader_election.cc:343] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:52.694041 13217 tablet_bootstrap.cc:492] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69: No bootstrap required, opened a new log
I20260709 13:07:52.694602 13217 ts_tablet_manager.cc:1403] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69: Time spent bootstrapping tablet: real 0.097s	user 0.041s	sys 0.045s
I20260709 13:07:52.694401 13066 leader_election.cc:304] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 824d687f218b4cb091e5d62fa9d18187
I20260709 13:07:52.694257 13218 consensus_queue.cc:260] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
W20260709 13:07:52.685813 13140 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:44253, user_credentials={real_user=slave}} blocked reactor thread for 75635.1us
I20260709 13:07:52.691200 12994 leader_election.cc:304] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 20f53f29f26f48d6b818f46c9e755f69
I20260709 13:07:52.696699 13223 leader_election.cc:290] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:52.697614 13223 raft_consensus.cc:2804] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:52.698062 13223 raft_consensus.cc:493] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.698444 13223 raft_consensus.cc:3060] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:52.698482 13217 raft_consensus.cc:359] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.699791 13217 raft_consensus.cc:385] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.700347 13217 raft_consensus.cc:740] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.704062 13217 consensus_queue.cc:260] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.704653 13218 ts_tablet_manager.cc:1434] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.007s
I20260709 13:07:52.706288 13218 tablet_bootstrap.cc:492] T eab0853d0dd94bcf8327c0ef09c6fad4 P 8593949d5fb7413ca30c9ef5dfd31411: Bootstrap starting.
I20260709 13:07:52.712280 13219 tablet_bootstrap.cc:654] T 4448333d61c640e494f7ae000878dbdf P 824d687f218b4cb091e5d62fa9d18187: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.707890 13223 raft_consensus.cc:515] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.717516 13223 leader_election.cc:290] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 election: Requested vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:52.717983 13217 ts_tablet_manager.cc:1434] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.014s
I20260709 13:07:52.718549 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a22a6947bab14423aa83c01130c20d77" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:07:52.719079 13218 tablet_bootstrap.cc:654] T eab0853d0dd94bcf8327c0ef09c6fad4 P 8593949d5fb7413ca30c9ef5dfd31411: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:52.720269 12992 leader_election.cc:343] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:52.721845 13217 tablet_bootstrap.cc:492] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69: Bootstrap starting.
I20260709 13:07:52.716861 13243 raft_consensus.cc:2804] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:52.725051 13243 raft_consensus.cc:493] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.725539 13243 raft_consensus.cc:3060] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:52.728462 12925 catalog_manager.cc:5697] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 reported cstate change: term changed from 0 to 1, leader changed from <none> to 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "824d687f218b4cb091e5d62fa9d18187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:52.748169 13217 tablet_bootstrap.cc:654] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.748764 13243 raft_consensus.cc:515] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.755331 13243 leader_election.cc:290] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:52.762122 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68f4b09178c244199a36c43c3989a45b" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:52.763151 13028 raft_consensus.cc:3060] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:52.767407 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68f4b09178c244199a36c43c3989a45b" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
W20260709 13:07:52.770213 13069 leader_election.cc:343] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:52.781155 13217 tablet_bootstrap.cc:492] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69: No bootstrap required, opened a new log
I20260709 13:07:52.782024 13217 ts_tablet_manager.cc:1403] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69: Time spent bootstrapping tablet: real 0.060s	user 0.024s	sys 0.012s
W20260709 13:07:52.783084 12994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 91990.9us
I20260709 13:07:52.785094 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a22a6947bab14423aa83c01130c20d77" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:52.786002 13177 raft_consensus.cc:3060] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:52.786832 13217 raft_consensus.cc:359] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.787920 13217 raft_consensus.cc:385] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.788282 13217 raft_consensus.cc:740] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.789418 13217 consensus_queue.cc:260] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.793006 13217 ts_tablet_manager.cc:1434] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:07:52.794507 13217 tablet_bootstrap.cc:492] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69: Bootstrap starting.
I20260709 13:07:52.797245 13028 raft_consensus.cc:2468] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 1.
I20260709 13:07:52.800927 13177 raft_consensus.cc:2468] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 1.
I20260709 13:07:52.802457 13066 leader_election.cc:304] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 824d687f218b4cb091e5d62fa9d18187
I20260709 13:07:52.803016 13219 tablet_bootstrap.cc:492] T 4448333d61c640e494f7ae000878dbdf P 824d687f218b4cb091e5d62fa9d18187: No bootstrap required, opened a new log
I20260709 13:07:52.803224 13217 tablet_bootstrap.cc:654] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.802840 12994 leader_election.cc:304] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 20f53f29f26f48d6b818f46c9e755f69
I20260709 13:07:52.803668 13219 ts_tablet_manager.cc:1403] T 4448333d61c640e494f7ae000878dbdf P 824d687f218b4cb091e5d62fa9d18187: Time spent bootstrapping tablet: real 0.118s	user 0.037s	sys 0.053s
I20260709 13:07:52.803710 13243 raft_consensus.cc:2804] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:52.804104 13242 raft_consensus.cc:2804] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:52.804404 13243 raft_consensus.cc:697] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Becoming Leader. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:07:52.804804 13242 raft_consensus.cc:697] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Becoming Leader. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Running, Role: LEADER
I20260709 13:07:52.805341 13243 consensus_queue.cc:237] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.805711 13242 consensus_queue.cc:237] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.807343 13219 raft_consensus.cc:359] T 4448333d61c640e494f7ae000878dbdf P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.808275 13219 raft_consensus.cc:385] T 4448333d61c640e494f7ae000878dbdf P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.808826 13219 raft_consensus.cc:740] T 4448333d61c640e494f7ae000878dbdf P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.800967 13218 tablet_bootstrap.cc:492] T eab0853d0dd94bcf8327c0ef09c6fad4 P 8593949d5fb7413ca30c9ef5dfd31411: No bootstrap required, opened a new log
I20260709 13:07:52.809844 13218 ts_tablet_manager.cc:1403] T eab0853d0dd94bcf8327c0ef09c6fad4 P 8593949d5fb7413ca30c9ef5dfd31411: Time spent bootstrapping tablet: real 0.104s	user 0.038s	sys 0.035s
I20260709 13:07:52.810017 13219 consensus_queue.cc:260] T 4448333d61c640e494f7ae000878dbdf P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.817093 13218 raft_consensus.cc:359] T eab0853d0dd94bcf8327c0ef09c6fad4 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.818034 13217 tablet_bootstrap.cc:492] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69: No bootstrap required, opened a new log
I20260709 13:07:52.818198 13218 raft_consensus.cc:385] T eab0853d0dd94bcf8327c0ef09c6fad4 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.818478 13219 ts_tablet_manager.cc:1434] T 4448333d61c640e494f7ae000878dbdf P 824d687f218b4cb091e5d62fa9d18187: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.003s
I20260709 13:07:52.818743 13218 raft_consensus.cc:740] T eab0853d0dd94bcf8327c0ef09c6fad4 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.819690 13219 tablet_bootstrap.cc:492] T eab0853d0dd94bcf8327c0ef09c6fad4 P 824d687f218b4cb091e5d62fa9d18187: Bootstrap starting.
I20260709 13:07:52.822031 13218 consensus_queue.cc:260] T eab0853d0dd94bcf8327c0ef09c6fad4 P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.824370 13218 ts_tablet_manager.cc:1434] T eab0853d0dd94bcf8327c0ef09c6fad4 P 8593949d5fb7413ca30c9ef5dfd31411: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:07:52.825407 13218 tablet_bootstrap.cc:492] T 4448333d61c640e494f7ae000878dbdf P 8593949d5fb7413ca30c9ef5dfd31411: Bootstrap starting.
I20260709 13:07:52.828311 13219 tablet_bootstrap.cc:654] T eab0853d0dd94bcf8327c0ef09c6fad4 P 824d687f218b4cb091e5d62fa9d18187: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.831784 13218 tablet_bootstrap.cc:654] T 4448333d61c640e494f7ae000878dbdf P 8593949d5fb7413ca30c9ef5dfd31411: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.832625 12923 catalog_manager.cc:5697] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "8593949d5fb7413ca30c9ef5dfd31411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:52.818641 13217 ts_tablet_manager.cc:1403] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.005s
I20260709 13:07:52.840721 13218 tablet_bootstrap.cc:492] T 4448333d61c640e494f7ae000878dbdf P 8593949d5fb7413ca30c9ef5dfd31411: No bootstrap required, opened a new log
I20260709 13:07:52.840847 13217 raft_consensus.cc:359] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.840802 13219 tablet_bootstrap.cc:492] T eab0853d0dd94bcf8327c0ef09c6fad4 P 824d687f218b4cb091e5d62fa9d18187: No bootstrap required, opened a new log
I20260709 13:07:52.841771 13217 raft_consensus.cc:385] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.842116 13219 ts_tablet_manager.cc:1403] T eab0853d0dd94bcf8327c0ef09c6fad4 P 824d687f218b4cb091e5d62fa9d18187: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.003s
I20260709 13:07:52.842252 13218 ts_tablet_manager.cc:1403] T 4448333d61c640e494f7ae000878dbdf P 8593949d5fb7413ca30c9ef5dfd31411: Time spent bootstrapping tablet: real 0.017s	user 0.016s	sys 0.000s
I20260709 13:07:52.845063 13218 raft_consensus.cc:359] T 4448333d61c640e494f7ae000878dbdf P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.845840 13218 raft_consensus.cc:385] T 4448333d61c640e494f7ae000878dbdf P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.845392 13219 raft_consensus.cc:359] T eab0853d0dd94bcf8327c0ef09c6fad4 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.846134 13218 raft_consensus.cc:740] T 4448333d61c640e494f7ae000878dbdf P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.846339 13219 raft_consensus.cc:385] T eab0853d0dd94bcf8327c0ef09c6fad4 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.846921 13219 raft_consensus.cc:740] T eab0853d0dd94bcf8327c0ef09c6fad4 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.846889 13218 consensus_queue.cc:260] T 4448333d61c640e494f7ae000878dbdf P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.848376 13219 consensus_queue.cc:260] T eab0853d0dd94bcf8327c0ef09c6fad4 P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.849118 13218 ts_tablet_manager.cc:1434] T 4448333d61c640e494f7ae000878dbdf P 8593949d5fb7413ca30c9ef5dfd31411: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 13:07:52.848745 12922 catalog_manager.cc:5697] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "20f53f29f26f48d6b818f46c9e755f69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:52.850121 13218 tablet_bootstrap.cc:492] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411: Bootstrap starting.
I20260709 13:07:52.850884 13219 ts_tablet_manager.cc:1434] T eab0853d0dd94bcf8327c0ef09c6fad4 P 824d687f218b4cb091e5d62fa9d18187: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:07:52.853266 13219 tablet_bootstrap.cc:492] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187: Bootstrap starting.
I20260709 13:07:52.842125 13217 raft_consensus.cc:740] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.854573 13217 consensus_queue.cc:260] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.856949 13217 ts_tablet_manager.cc:1434] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.001s
I20260709 13:07:52.858137 13217 tablet_bootstrap.cc:492] T f5fffab9019146fd9cc617253eb19f87 P 20f53f29f26f48d6b818f46c9e755f69: Bootstrap starting.
I20260709 13:07:52.859834 13219 tablet_bootstrap.cc:654] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.865121 13217 tablet_bootstrap.cc:654] T f5fffab9019146fd9cc617253eb19f87 P 20f53f29f26f48d6b818f46c9e755f69: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.871474 13218 tablet_bootstrap.cc:654] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.871774 13217 tablet_bootstrap.cc:492] T f5fffab9019146fd9cc617253eb19f87 P 20f53f29f26f48d6b818f46c9e755f69: No bootstrap required, opened a new log
I20260709 13:07:52.872259 13217 ts_tablet_manager.cc:1403] T f5fffab9019146fd9cc617253eb19f87 P 20f53f29f26f48d6b818f46c9e755f69: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.001s
I20260709 13:07:52.873987 13219 tablet_bootstrap.cc:492] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187: No bootstrap required, opened a new log
I20260709 13:07:52.874601 13219 ts_tablet_manager.cc:1403] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.004s
I20260709 13:07:52.875891 13217 raft_consensus.cc:359] T f5fffab9019146fd9cc617253eb19f87 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.876899 13217 raft_consensus.cc:385] T f5fffab9019146fd9cc617253eb19f87 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.877151 13217 raft_consensus.cc:740] T f5fffab9019146fd9cc617253eb19f87 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.877771 13219 raft_consensus.cc:359] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.878015 13217 consensus_queue.cc:260] T f5fffab9019146fd9cc617253eb19f87 P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.878518 13219 raft_consensus.cc:385] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.878751 13218 tablet_bootstrap.cc:492] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411: No bootstrap required, opened a new log
I20260709 13:07:52.878897 13219 raft_consensus.cc:740] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.879338 13218 ts_tablet_manager.cc:1403] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.000s
I20260709 13:07:52.879798 13219 consensus_queue.cc:260] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.880724 13217 ts_tablet_manager.cc:1434] T f5fffab9019146fd9cc617253eb19f87 P 20f53f29f26f48d6b818f46c9e755f69: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.004s
I20260709 13:07:52.881757 13217 tablet_bootstrap.cc:492] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69: Bootstrap starting.
I20260709 13:07:52.882138 13219 ts_tablet_manager.cc:1434] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:07:52.882089 13218 raft_consensus.cc:359] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } } peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } }
I20260709 13:07:52.882956 13218 raft_consensus.cc:385] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.883229 13218 raft_consensus.cc:740] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.883447 13219 tablet_bootstrap.cc:492] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187: Bootstrap starting.
I20260709 13:07:52.883988 13218 consensus_queue.cc:260] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } } peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } }
I20260709 13:07:52.888825 13217 tablet_bootstrap.cc:654] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.891014 13219 tablet_bootstrap.cc:654] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.900214 13218 ts_tablet_manager.cc:1434] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:07:52.901146 13217 tablet_bootstrap.cc:492] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69: No bootstrap required, opened a new log
I20260709 13:07:52.901746 13217 ts_tablet_manager.cc:1403] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.003s
I20260709 13:07:52.904770 13217 raft_consensus.cc:359] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.905542 13217 raft_consensus.cc:385] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.907150 13217 raft_consensus.cc:740] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.908087 13217 consensus_queue.cc:260] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.908521 13219 tablet_bootstrap.cc:492] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187: No bootstrap required, opened a new log
I20260709 13:07:52.909312 13219 ts_tablet_manager.cc:1403] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.010s
I20260709 13:07:52.910928 13217 ts_tablet_manager.cc:1434] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:07:52.912046 13217 tablet_bootstrap.cc:492] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69: Bootstrap starting.
I20260709 13:07:52.912360 13219 raft_consensus.cc:359] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.913194 13219 raft_consensus.cc:385] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.913484 13219 raft_consensus.cc:740] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.914271 13219 consensus_queue.cc:260] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.915061 13243 raft_consensus.cc:493] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.915592 13243 raft_consensus.cc:515] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.916620 13219 ts_tablet_manager.cc:1434] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:07:52.917557 13243 leader_election.cc:290] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:52.917732 13219 tablet_bootstrap.cc:492] T d727f4f4eacb428889625360bb663857 P 824d687f218b4cb091e5d62fa9d18187: Bootstrap starting.
I20260709 13:07:52.919441 13217 tablet_bootstrap.cc:654] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.919693 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4448333d61c640e494f7ae000878dbdf" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
I20260709 13:07:52.920418 13177 raft_consensus.cc:2468] T 4448333d61c640e494f7ae000878dbdf P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 0.
I20260709 13:07:52.922415 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4448333d61c640e494f7ae000878dbdf" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
I20260709 13:07:52.923189 13028 raft_consensus.cc:2468] T 4448333d61c640e494f7ae000878dbdf P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 0.
I20260709 13:07:52.924664 13069 leader_election.cc:304] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 824d687f218b4cb091e5d62fa9d18187; no voters: 
I20260709 13:07:52.925518 13219 tablet_bootstrap.cc:654] T d727f4f4eacb428889625360bb663857 P 824d687f218b4cb091e5d62fa9d18187: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.926606 13243 raft_consensus.cc:2804] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:52.927080 13243 raft_consensus.cc:493] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.927460 13243 raft_consensus.cc:3060] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:52.935604 13243 raft_consensus.cc:515] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.939857 13219 tablet_bootstrap.cc:492] T d727f4f4eacb428889625360bb663857 P 824d687f218b4cb091e5d62fa9d18187: No bootstrap required, opened a new log
I20260709 13:07:52.940618 13219 ts_tablet_manager.cc:1403] T d727f4f4eacb428889625360bb663857 P 824d687f218b4cb091e5d62fa9d18187: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.004s
I20260709 13:07:52.941982 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4448333d61c640e494f7ae000878dbdf" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:52.942317 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4448333d61c640e494f7ae000878dbdf" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:52.942653 13028 raft_consensus.cc:3060] T 4448333d61c640e494f7ae000878dbdf P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:52.942935 13177 raft_consensus.cc:3060] T 4448333d61c640e494f7ae000878dbdf P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:52.943295 13217 tablet_bootstrap.cc:492] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69: No bootstrap required, opened a new log
I20260709 13:07:52.943900 13217 ts_tablet_manager.cc:1403] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69: Time spent bootstrapping tablet: real 0.032s	user 0.026s	sys 0.004s
I20260709 13:07:52.943480 13219 raft_consensus.cc:359] T d727f4f4eacb428889625360bb663857 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } } peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } }
I20260709 13:07:52.944331 13219 raft_consensus.cc:385] T d727f4f4eacb428889625360bb663857 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.944667 13219 raft_consensus.cc:740] T d727f4f4eacb428889625360bb663857 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.945457 13219 consensus_queue.cc:260] T d727f4f4eacb428889625360bb663857 P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } } peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } }
I20260709 13:07:52.947149 13217 raft_consensus.cc:359] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.947940 13217 raft_consensus.cc:385] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.948276 13217 raft_consensus.cc:740] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.948756 13219 ts_tablet_manager.cc:1434] T d727f4f4eacb428889625360bb663857 P 824d687f218b4cb091e5d62fa9d18187: Time spent starting tablet: real 0.008s	user 0.010s	sys 0.000s
I20260709 13:07:52.949195 13217 consensus_queue.cc:260] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.949811 13219 tablet_bootstrap.cc:492] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187: Bootstrap starting.
I20260709 13:07:52.950084 13028 raft_consensus.cc:2468] T 4448333d61c640e494f7ae000878dbdf P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 1.
I20260709 13:07:52.950696 13243 leader_election.cc:290] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:52.952402 13066 leader_election.cc:304] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:52.953481 13217 ts_tablet_manager.cc:1434] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:07:52.954555 13217 tablet_bootstrap.cc:492] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69: Bootstrap starting.
I20260709 13:07:52.957823 13219 tablet_bootstrap.cc:654] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.958673 13243 raft_consensus.cc:2804] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:52.959422 13243 raft_consensus.cc:697] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Becoming Leader. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:07:52.960582 13217 tablet_bootstrap.cc:654] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.960279 13243 consensus_queue.cc:237] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.966137 13177 raft_consensus.cc:2468] T 4448333d61c640e494f7ae000878dbdf P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 1.
I20260709 13:07:52.971168 13219 tablet_bootstrap.cc:492] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187: No bootstrap required, opened a new log
I20260709 13:07:52.971714 13219 ts_tablet_manager.cc:1403] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.004s
I20260709 13:07:52.972656 13217 tablet_bootstrap.cc:492] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69: No bootstrap required, opened a new log
I20260709 13:07:52.973266 13217 ts_tablet_manager.cc:1403] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.007s
I20260709 13:07:52.974382 13219 raft_consensus.cc:359] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.975122 13219 raft_consensus.cc:385] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.975384 13219 raft_consensus.cc:740] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.976212 13217 raft_consensus.cc:359] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.976131 13219 consensus_queue.cc:260] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.976949 13217 raft_consensus.cc:385] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.977298 13217 raft_consensus.cc:740] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.978199 13225 raft_consensus.cc:493] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.978658 13225 raft_consensus.cc:515] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.980631 13225 leader_election.cc:290] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:52.980684 13219 ts_tablet_manager.cc:1434] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 13:07:52.981752 13219 tablet_bootstrap.cc:492] T 68f4b09178c244199a36c43c3989a45b P 824d687f218b4cb091e5d62fa9d18187: Bootstrap starting.
I20260709 13:07:52.981815 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d393f71c2afb406098b92973cb2be621" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
I20260709 13:07:52.982295 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d393f71c2afb406098b92973cb2be621" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
I20260709 13:07:52.983039 13028 raft_consensus.cc:2393] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20f53f29f26f48d6b818f46c9e755f69 in current term 1: Already voted for candidate 824d687f218b4cb091e5d62fa9d18187 in this term.
I20260709 13:07:52.984079 13066 leader_election.cc:304] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69; no voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411
I20260709 13:07:52.984160 13217 consensus_queue.cc:260] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.985267 13225 raft_consensus.cc:3060] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:52.984992 12922 catalog_manager.cc:5697] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "20f53f29f26f48d6b818f46c9e755f69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:52.986792 13217 ts_tablet_manager.cc:1434] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:07:52.987846 13219 tablet_bootstrap.cc:654] T 68f4b09178c244199a36c43c3989a45b P 824d687f218b4cb091e5d62fa9d18187: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.988229 13226 raft_consensus.cc:493] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.988752 13226 raft_consensus.cc:515] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:52.991169 13226 leader_election.cc:290] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507)
I20260709 13:07:52.993300 13225 raft_consensus.cc:2749] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:52.994135 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5fffab9019146fd9cc617253eb19f87" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
I20260709 13:07:52.995052 13028 raft_consensus.cc:2468] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 0.
I20260709 13:07:52.997519 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5fffab9019146fd9cc617253eb19f87" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69" is_pre_election: true
I20260709 13:07:52.998394 13103 raft_consensus.cc:2468] T f5fffab9019146fd9cc617253eb19f87 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 0.
I20260709 13:07:52.999917 13140 leader_election.cc:304] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:53.001073 13226 raft_consensus.cc:2804] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:53.001452 13226 raft_consensus.cc:493] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.001820 13226 raft_consensus.cc:3060] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.007175 13219 tablet_bootstrap.cc:492] T 68f4b09178c244199a36c43c3989a45b P 824d687f218b4cb091e5d62fa9d18187: No bootstrap required, opened a new log
I20260709 13:07:53.007746 13219 ts_tablet_manager.cc:1403] T 68f4b09178c244199a36c43c3989a45b P 824d687f218b4cb091e5d62fa9d18187: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.008s
I20260709 13:07:53.008642 13226 raft_consensus.cc:515] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.010462 13219 raft_consensus.cc:359] T 68f4b09178c244199a36c43c3989a45b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.011158 13219 raft_consensus.cc:385] T 68f4b09178c244199a36c43c3989a45b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:53.011438 13219 raft_consensus.cc:740] T 68f4b09178c244199a36c43c3989a45b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Initialized, Role: FOLLOWER
I20260709 13:07:53.011713 13226 leader_election.cc:290] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507)
I20260709 13:07:53.011914 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5fffab9019146fd9cc617253eb19f87" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:53.012185 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5fffab9019146fd9cc617253eb19f87" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:07:53.012554 13028 raft_consensus.cc:3060] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.012109 13219 consensus_queue.cc:260] T 68f4b09178c244199a36c43c3989a45b P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.012883 13103 raft_consensus.cc:3060] T f5fffab9019146fd9cc617253eb19f87 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.014439 13219 ts_tablet_manager.cc:1434] T 68f4b09178c244199a36c43c3989a45b P 824d687f218b4cb091e5d62fa9d18187: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:07:53.015614 13243 raft_consensus.cc:493] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.016144 13243 raft_consensus.cc:515] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.018318 13243 leader_election.cc:290] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:53.019450 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e175694e0fa6424f900caf84dbb2945f" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
I20260709 13:07:53.019534 13028 raft_consensus.cc:2468] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 1.
I20260709 13:07:53.020051 13027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e175694e0fa6424f900caf84dbb2945f" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
I20260709 13:07:53.020329 13177 raft_consensus.cc:2468] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 0.
I20260709 13:07:53.020754 13027 raft_consensus.cc:2468] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 0.
I20260709 13:07:53.020998 13140 leader_election.cc:304] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:53.021798 13069 leader_election.cc:304] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 824d687f218b4cb091e5d62fa9d18187; no voters: 
I20260709 13:07:53.022199 13226 raft_consensus.cc:2804] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:53.022717 13243 raft_consensus.cc:2804] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:53.022948 13226 raft_consensus.cc:697] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [term 1 LEADER]: Becoming Leader. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Running, Role: LEADER
I20260709 13:07:53.023144 13243 raft_consensus.cc:493] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.023766 13243 raft_consensus.cc:3060] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.024397 13103 raft_consensus.cc:2468] T f5fffab9019146fd9cc617253eb19f87 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 1.
I20260709 13:07:53.023905 13226 consensus_queue.cc:237] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.027268 13225 raft_consensus.cc:493] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.027788 13225 raft_consensus.cc:515] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.029857 13225 leader_election.cc:290] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:53.031829 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eab0853d0dd94bcf8327c0ef09c6fad4" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
I20260709 13:07:53.031716 13243 raft_consensus.cc:515] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.032565 13177 raft_consensus.cc:2468] T eab0853d0dd94bcf8327c0ef09c6fad4 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 0.
I20260709 13:07:53.033910 13069 leader_election.cc:304] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 824d687f218b4cb091e5d62fa9d18187; no voters: 
I20260709 13:07:53.034130 13243 leader_election.cc:290] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:53.034794 13243 raft_consensus.cc:2804] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:53.035171 13243 raft_consensus.cc:493] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.035604 13243 raft_consensus.cc:3060] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.035759 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eab0853d0dd94bcf8327c0ef09c6fad4" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
I20260709 13:07:53.036276 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e175694e0fa6424f900caf84dbb2945f" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:53.036502 13028 raft_consensus.cc:2468] T eab0853d0dd94bcf8327c0ef09c6fad4 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 0.
I20260709 13:07:53.036873 13177 raft_consensus.cc:3060] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.038583 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e175694e0fa6424f900caf84dbb2945f" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:53.039172 13028 raft_consensus.cc:3060] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.039680 12922 catalog_manager.cc:5697] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 reported cstate change: term changed from 0 to 1, leader changed from <none> to 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "824d687f218b4cb091e5d62fa9d18187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:53.044070 13177 raft_consensus.cc:2468] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 1.
I20260709 13:07:53.043780 13243 raft_consensus.cc:515] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.045702 13242 raft_consensus.cc:493] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.046257 13242 raft_consensus.cc:515] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.046869 13243 leader_election.cc:290] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:53.045336 13069 leader_election.cc:304] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 824d687f218b4cb091e5d62fa9d18187; no voters: 
I20260709 13:07:53.047528 13027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eab0853d0dd94bcf8327c0ef09c6fad4" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:53.048177 13027 raft_consensus.cc:3060] T eab0853d0dd94bcf8327c0ef09c6fad4 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.048362 13242 leader_election.cc:290] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:53.048853 13226 raft_consensus.cc:493] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.049597 13226 raft_consensus.cc:515] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.050271 13176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8b4f1a658d47cc954ba897df965354" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
I20260709 13:07:53.050949 13176 raft_consensus.cc:2468] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 0.
I20260709 13:07:53.051932 13226 leader_election.cc:290] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507)
I20260709 13:07:53.052764 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8b4f1a658d47cc954ba897df965354" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69" is_pre_election: true
I20260709 13:07:53.048626 13243 raft_consensus.cc:2804] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:53.053412 13103 raft_consensus.cc:2468] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 0.
I20260709 13:07:53.053604 13243 raft_consensus.cc:697] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Becoming Leader. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:07:53.054613 13026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8b4f1a658d47cc954ba897df965354" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
I20260709 13:07:53.054684 13243 consensus_queue.cc:237] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.055424 13026 raft_consensus.cc:2468] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 0.
I20260709 13:07:53.055691 13027 raft_consensus.cc:2468] T eab0853d0dd94bcf8327c0ef09c6fad4 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 1.
I20260709 13:07:53.056618 13102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8b4f1a658d47cc954ba897df965354" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69" is_pre_election: true
I20260709 13:07:53.057165 13066 leader_election.cc:304] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:53.057682 13028 raft_consensus.cc:2468] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 1.
I20260709 13:07:53.057976 12994 leader_election.cc:304] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:53.058228 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eab0853d0dd94bcf8327c0ef09c6fad4" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:53.058843 13177 raft_consensus.cc:3060] T eab0853d0dd94bcf8327c0ef09c6fad4 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.060310 13140 leader_election.cc:304] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:53.057284 13102 raft_consensus.cc:2468] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 0.
I20260709 13:07:53.063445 13242 raft_consensus.cc:2804] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:53.063639 13268 raft_consensus.cc:2804] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:53.063962 13242 raft_consensus.cc:493] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.064309 13268 raft_consensus.cc:697] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Becoming Leader. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:07:53.064392 13242 raft_consensus.cc:3060] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.065174 13268 consensus_queue.cc:237] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.066603 13177 raft_consensus.cc:2468] T eab0853d0dd94bcf8327c0ef09c6fad4 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 1.
I20260709 13:07:53.069303 13226 raft_consensus.cc:2804] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:53.069929 13226 raft_consensus.cc:493] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.070340 13226 raft_consensus.cc:3060] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.072399 12922 catalog_manager.cc:5697] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "20f53f29f26f48d6b818f46c9e755f69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:53.074803 13242 raft_consensus.cc:515] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.076911 13242 leader_election.cc:290] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 election: Requested vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:53.077126 13226 raft_consensus.cc:515] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.077719 13102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8b4f1a658d47cc954ba897df965354" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:07:53.078338 13102 raft_consensus.cc:3060] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.078867 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8b4f1a658d47cc954ba897df965354" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:53.079672 13177 raft_consensus.cc:2393] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8593949d5fb7413ca30c9ef5dfd31411 in current term 1: Already voted for candidate 824d687f218b4cb091e5d62fa9d18187 in this term.
I20260709 13:07:53.080921 13226 leader_election.cc:290] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507)
I20260709 13:07:53.082124 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8b4f1a658d47cc954ba897df965354" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:07:53.082861 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8b4f1a658d47cc954ba897df965354" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:53.083671 13028 raft_consensus.cc:2393] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 824d687f218b4cb091e5d62fa9d18187 in current term 1: Already voted for candidate 8593949d5fb7413ca30c9ef5dfd31411 in this term.
I20260709 13:07:53.084721 13140 leader_election.cc:304] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187; no voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411
I20260709 13:07:53.085815 13226 raft_consensus.cc:2749] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:53.086237 13102 raft_consensus.cc:2468] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 1.
I20260709 13:07:53.087348 12992 leader_election.cc:304] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 824d687f218b4cb091e5d62fa9d18187
I20260709 13:07:53.088192 13242 raft_consensus.cc:2804] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:53.089506 13242 raft_consensus.cc:697] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Becoming Leader. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Running, Role: LEADER
I20260709 13:07:53.090364 13242 consensus_queue.cc:237] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.090643 12922 catalog_manager.cc:5697] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "20f53f29f26f48d6b818f46c9e755f69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:53.100044 12926 catalog_manager.cc:5697] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "8593949d5fb7413ca30c9ef5dfd31411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:53.152294 13226 consensus_queue.cc:1048] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:53.165391 13234 consensus_queue.cc:1048] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:53.220734 13242 raft_consensus.cc:493] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.221453 13242 raft_consensus.cc:515] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } } peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } }
I20260709 13:07:53.223385 13242 leader_election.cc:290] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:53.224368 13102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d727f4f4eacb428889625360bb663857" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69" is_pre_election: true
I20260709 13:07:53.224483 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d727f4f4eacb428889625360bb663857" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
I20260709 13:07:53.225047 13102 raft_consensus.cc:2468] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 0.
I20260709 13:07:53.225189 13177 raft_consensus.cc:2468] T d727f4f4eacb428889625360bb663857 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 0.
I20260709 13:07:53.226182 12992 leader_election.cc:304] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:53.226960 13242 raft_consensus.cc:2804] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:53.227276 13242 raft_consensus.cc:493] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.227617 13242 raft_consensus.cc:3060] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.229265 13268 consensus_queue.cc:1048] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:53.234503 13242 raft_consensus.cc:515] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } } peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } }
I20260709 13:07:53.237344 13102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d727f4f4eacb428889625360bb663857" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:07:53.237694 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d727f4f4eacb428889625360bb663857" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:53.237978 13102 raft_consensus.cc:3060] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.238330 13177 raft_consensus.cc:3060] T d727f4f4eacb428889625360bb663857 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.239949 13176 raft_consensus.cc:3060] T 68f4b09178c244199a36c43c3989a45b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.244825 13102 raft_consensus.cc:2468] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 1.
I20260709 13:07:53.248360 13177 raft_consensus.cc:2468] T d727f4f4eacb428889625360bb663857 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 1.
I20260709 13:07:53.251315 12992 leader_election.cc:304] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:53.248872 13242 leader_election.cc:290] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 election: Requested vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:53.252197 13223 raft_consensus.cc:2804] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:53.252701 13223 raft_consensus.cc:697] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Becoming Leader. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Running, Role: LEADER
I20260709 13:07:53.249142 13243 consensus_queue.cc:1048] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:53.253412 13223 consensus_queue.cc:237] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } } peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } }
I20260709 13:07:53.261833 12926 catalog_manager.cc:5697] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "8593949d5fb7413ca30c9ef5dfd31411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:53.324577 13243 raft_consensus.cc:493] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.325186 13243 raft_consensus.cc:515] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.327215 13243 leader_election.cc:290] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:53.328058 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a22a6947bab14423aa83c01130c20d77" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
I20260709 13:07:53.328228 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a22a6947bab14423aa83c01130c20d77" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
I20260709 13:07:53.329180 13177 raft_consensus.cc:2393] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20f53f29f26f48d6b818f46c9e755f69 in current term 1: Already voted for candidate 8593949d5fb7413ca30c9ef5dfd31411 in this term.
I20260709 13:07:53.330111 13069 leader_election.cc:304] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69; no voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411
I20260709 13:07:53.330788 13243 raft_consensus.cc:3060] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.337157 13243 raft_consensus.cc:2749] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:53.340632 13223 consensus_queue.cc:1048] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:53.347261 13223 consensus_queue.cc:1048] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:53.362545 13243 consensus_queue.cc:1048] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:53.381719 13268 consensus_queue.cc:1048] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:53.407088 13226 consensus_queue.cc:1048] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:53.408819 13102 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e175694e0fa6424f900caf84dbb2945f"
dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
mode: GRACEFUL
new_leader_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
 from {username='slave'} at 127.0.0.1:40832
I20260709 13:07:53.409660 13102 raft_consensus.cc:606] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Received request to transfer leadership to TS 8593949d5fb7413ca30c9ef5dfd31411
I20260709 13:07:53.418656 13234 consensus_queue.cc:1048] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:53.429086 13176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f5fffab9019146fd9cc617253eb19f87"
dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
mode: GRACEFUL
new_leader_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
 from {username='slave'} at 127.0.0.1:55890
I20260709 13:07:53.429796 13176 raft_consensus.cc:606] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [term 1 LEADER]: Received request to transfer leadership to TS 8593949d5fb7413ca30c9ef5dfd31411
I20260709 13:07:53.435537 13103 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "68f4b09178c244199a36c43c3989a45b"
dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
mode: GRACEFUL
new_leader_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
 from {username='slave'} at 127.0.0.1:40832
I20260709 13:07:53.436084 13103 raft_consensus.cc:606] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Received request to transfer leadership to TS 8593949d5fb7413ca30c9ef5dfd31411
I20260709 13:07:53.438455 13288 raft_consensus.cc:993] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187: : Instructing follower 8593949d5fb7413ca30c9ef5dfd31411 to start an election
I20260709 13:07:53.439111 13234 raft_consensus.cc:1081] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [term 1 LEADER]: Signalling peer 8593949d5fb7413ca30c9ef5dfd31411 to start an election
I20260709 13:07:53.440665 13028 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f5fffab9019146fd9cc617253eb19f87"
dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
 from {username='slave'} at 127.0.0.1:55244
I20260709 13:07:53.441252 13028 raft_consensus.cc:493] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:53.441691 13028 raft_consensus.cc:3060] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.442289 13176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d393f71c2afb406098b92973cb2be621"
dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
mode: GRACEFUL
new_leader_uuid: "20f53f29f26f48d6b818f46c9e755f69"
 from {username='slave'} at 127.0.0.1:55890
I20260709 13:07:53.442909 13176 raft_consensus.cc:606] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 1 LEADER]: Received request to transfer leadership to TS 20f53f29f26f48d6b818f46c9e755f69
I20260709 13:07:53.448478 13028 raft_consensus.cc:515] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.450845 13028 leader_election.cc:290] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 2 election: Requested vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:53.451051 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5fffab9019146fd9cc617253eb19f87" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:07:53.451664 13103 raft_consensus.cc:3060] T f5fffab9019146fd9cc617253eb19f87 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.452458 13176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5fffab9019146fd9cc617253eb19f87" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:53.453272 13176 raft_consensus.cc:3055] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:53.457775 13176 raft_consensus.cc:740] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Running, Role: LEADER
I20260709 13:07:53.458837 13176 consensus_queue.cc:260] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.459205 13103 raft_consensus.cc:2468] T f5fffab9019146fd9cc617253eb19f87 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 2.
I20260709 13:07:53.460315 13176 raft_consensus.cc:3060] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.461053 12992 leader_election.cc:304] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:53.462457 13242 raft_consensus.cc:2804] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:53.462962 13242 raft_consensus.cc:697] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 LEADER]: Becoming Leader. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Running, Role: LEADER
I20260709 13:07:53.463708 13242 consensus_queue.cc:237] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.468461 13176 raft_consensus.cc:2468] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 2.
I20260709 13:07:53.471443 12925 catalog_manager.cc:5697] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 reported cstate change: term changed from 1 to 2, leader changed from 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195) to 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "8593949d5fb7413ca30c9ef5dfd31411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:53.472673 13028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d8b4f1a658d47cc954ba897df965354"
dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
mode: GRACEFUL
new_leader_uuid: "20f53f29f26f48d6b818f46c9e755f69"
 from {username='slave'} at 127.0.0.1:55266
I20260709 13:07:53.473318 13028 raft_consensus.cc:606] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Received request to transfer leadership to TS 20f53f29f26f48d6b818f46c9e755f69
I20260709 13:07:53.478549 13028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a22a6947bab14423aa83c01130c20d77"
dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
mode: GRACEFUL
new_leader_uuid: "824d687f218b4cb091e5d62fa9d18187"
 from {username='slave'} at 127.0.0.1:55266
I20260709 13:07:53.479142 13028 raft_consensus.cc:606] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Received request to transfer leadership to TS 824d687f218b4cb091e5d62fa9d18187
I20260709 13:07:53.484256 13268 consensus_queue.cc:1048] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:53.495596 13225 consensus_queue.cc:1048] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:53.541874 13225 consensus_queue.cc:1048] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:53.554077 13225 consensus_queue.cc:1048] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:53.561600 13243 raft_consensus.cc:993] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69: : Instructing follower 8593949d5fb7413ca30c9ef5dfd31411 to start an election
I20260709 13:07:53.562114 13225 raft_consensus.cc:1081] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Signalling peer 8593949d5fb7413ca30c9ef5dfd31411 to start an election
I20260709 13:07:53.563617 13028 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e175694e0fa6424f900caf84dbb2945f"
dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
 from {username='slave'} at 127.0.0.1:55258
I20260709 13:07:53.564306 13028 raft_consensus.cc:493] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:53.564728 13028 raft_consensus.cc:3060] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.570585 13028 raft_consensus.cc:515] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.572835 13028 leader_election.cc:290] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 2 election: Requested vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:53.573843 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e175694e0fa6424f900caf84dbb2945f" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:07:53.574112 13176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e175694e0fa6424f900caf84dbb2945f" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:53.574533 13103 raft_consensus.cc:3055] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:53.574859 13176 raft_consensus.cc:3060] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.574985 13103 raft_consensus.cc:740] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:07:53.575894 13103 consensus_queue.cc:260] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.576905 13103 raft_consensus.cc:3060] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.582090 13176 raft_consensus.cc:2468] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 2.
I20260709 13:07:53.582492 13103 raft_consensus.cc:2468] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 2.
I20260709 13:07:53.583647 12994 leader_election.cc:304] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:53.584496 13242 raft_consensus.cc:2804] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:53.585115 13242 raft_consensus.cc:697] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 LEADER]: Becoming Leader. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Running, Role: LEADER
I20260709 13:07:53.586045 13242 consensus_queue.cc:237] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.593009 12925 catalog_manager.cc:5697] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 reported cstate change: term changed from 1 to 2, leader changed from 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194) to 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "8593949d5fb7413ca30c9ef5dfd31411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:53.649724 13242 consensus_queue.cc:1048] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:53.655762 13226 raft_consensus.cc:993] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187: : Instructing follower 20f53f29f26f48d6b818f46c9e755f69 to start an election
I20260709 13:07:53.656286 13234 raft_consensus.cc:1081] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 1 LEADER]: Signalling peer 20f53f29f26f48d6b818f46c9e755f69 to start an election
I20260709 13:07:53.658061 13103 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d393f71c2afb406098b92973cb2be621"
dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
 from {username='slave'} at 127.0.0.1:40816
I20260709 13:07:53.659698 13103 raft_consensus.cc:493] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:53.660099 13103 raft_consensus.cc:3060] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.662150 13282 consensus_queue.cc:1048] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:53.668819 13103 raft_consensus.cc:515] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.673452 13103 leader_election.cc:290] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 2 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:53.673621 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d393f71c2afb406098b92973cb2be621" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:53.674361 13177 raft_consensus.cc:3055] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:53.674726 13177 raft_consensus.cc:740] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Running, Role: LEADER
I20260709 13:07:53.673056 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d393f71c2afb406098b92973cb2be621" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:53.675612 13028 raft_consensus.cc:3060] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.675647 13177 consensus_queue.cc:260] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.677171 13177 raft_consensus.cc:3060] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.683531 13028 raft_consensus.cc:2468] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 2.
I20260709 13:07:53.685181 13066 leader_election.cc:304] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:53.686297 13225 raft_consensus.cc:2804] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:53.686867 13225 raft_consensus.cc:697] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 LEADER]: Becoming Leader. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:07:53.687234 13177 raft_consensus.cc:2468] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 2.
I20260709 13:07:53.687769 13225 consensus_queue.cc:237] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.697309 12922 catalog_manager.cc:5697] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 reported cstate change: term changed from 1 to 2, leader changed from 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195) to 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "20f53f29f26f48d6b818f46c9e755f69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:53.699136 13298 consensus_queue.cc:1048] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:53.702418 13298 consensus_queue.cc:1048] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:53.714970 13298 raft_consensus.cc:993] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411: : Instructing follower 20f53f29f26f48d6b818f46c9e755f69 to start an election
I20260709 13:07:53.715554 13282 raft_consensus.cc:1081] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Signalling peer 20f53f29f26f48d6b818f46c9e755f69 to start an election
I20260709 13:07:53.719377 13102 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0d8b4f1a658d47cc954ba897df965354"
dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
 from {username='slave'} at 127.0.0.1:40804
I20260709 13:07:53.720498 13102 raft_consensus.cc:493] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:53.720960 13102 raft_consensus.cc:3060] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.728086 13102 raft_consensus.cc:515] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.730971 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8b4f1a658d47cc954ba897df965354" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:53.731472 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8b4f1a658d47cc954ba897df965354" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:53.732154 13177 raft_consensus.cc:3060] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.733779 13102 leader_election.cc:290] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 2 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:53.731637 13028 raft_consensus.cc:3055] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:53.736541 13028 raft_consensus.cc:740] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Running, Role: LEADER
I20260709 13:07:53.737546 13028 consensus_queue.cc:260] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.739041 13028 raft_consensus.cc:3060] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.738988 13177 raft_consensus.cc:2468] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 2.
I20260709 13:07:53.740579 13069 leader_election.cc:304] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 824d687f218b4cb091e5d62fa9d18187; no voters: 
I20260709 13:07:53.741704 13225 raft_consensus.cc:2804] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:53.742170 13225 raft_consensus.cc:697] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 LEADER]: Becoming Leader. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:07:53.742933 13225 consensus_queue.cc:237] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.745950 13028 raft_consensus.cc:2468] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 2.
I20260709 13:07:53.750931 12925 catalog_manager.cc:5697] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 reported cstate change: term changed from 1 to 2, leader changed from 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193) to 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "20f53f29f26f48d6b818f46c9e755f69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:53.754413 13242 raft_consensus.cc:993] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411: : Instructing follower 824d687f218b4cb091e5d62fa9d18187 to start an election
I20260709 13:07:53.755128 13223 raft_consensus.cc:1081] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Signalling peer 824d687f218b4cb091e5d62fa9d18187 to start an election
I20260709 13:07:53.756894 13176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a22a6947bab14423aa83c01130c20d77"
dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
 from {username='slave'} at 127.0.0.1:55876
I20260709 13:07:53.757423 13176 raft_consensus.cc:493] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:53.757776 13176 raft_consensus.cc:3060] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.765101 13176 raft_consensus.cc:515] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.767791 13176 leader_election.cc:290] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 2 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507)
I20260709 13:07:53.768234 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a22a6947bab14423aa83c01130c20d77" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:53.769016 13028 raft_consensus.cc:3055] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:53.769069 13102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a22a6947bab14423aa83c01130c20d77" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:07:53.769399 13028 raft_consensus.cc:740] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Running, Role: LEADER
I20260709 13:07:53.769728 13102 raft_consensus.cc:3060] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.770350 13028 consensus_queue.cc:260] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.771317 13028 raft_consensus.cc:3060] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.775679 13102 raft_consensus.cc:2468] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 2.
I20260709 13:07:53.777263 13141 leader_election.cc:304] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 824d687f218b4cb091e5d62fa9d18187; no voters: 
I20260709 13:07:53.777675 13028 raft_consensus.cc:2468] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 2.
I20260709 13:07:53.778124 13234 raft_consensus.cc:2804] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:53.778838 13234 raft_consensus.cc:697] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 [term 2 LEADER]: Becoming Leader. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Running, Role: LEADER
I20260709 13:07:53.779891 13234 consensus_queue.cc:237] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.787072 12925 catalog_manager.cc:5697] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 reported cstate change: term changed from 1 to 2, leader changed from 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193) to 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "824d687f218b4cb091e5d62fa9d18187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:53.855173 13102 raft_consensus.cc:1275] T f5fffab9019146fd9cc617253eb19f87 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Refusing update from remote peer 8593949d5fb7413ca30c9ef5dfd31411: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:53.856642 13223 consensus_queue.cc:1048] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:53.861163 13243 raft_consensus.cc:993] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69: : Instructing follower 8593949d5fb7413ca30c9ef5dfd31411 to start an election
I20260709 13:07:53.861745 13243 raft_consensus.cc:1081] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Signalling peer 8593949d5fb7413ca30c9ef5dfd31411 to start an election
I20260709 13:07:53.863502 13028 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "68f4b09178c244199a36c43c3989a45b"
dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
 from {username='slave'} at 127.0.0.1:55258
I20260709 13:07:53.864085 13028 raft_consensus.cc:493] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:53.864468 13028 raft_consensus.cc:3060] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.866750 13176 raft_consensus.cc:1275] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Refusing update from remote peer 8593949d5fb7413ca30c9ef5dfd31411: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:53.868637 13282 consensus_queue.cc:1048] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:53.878070 13028 raft_consensus.cc:515] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.880414 13028 leader_election.cc:290] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 2 election: Requested vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:53.882246 13102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68f4b09178c244199a36c43c3989a45b" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:07:53.882922 13176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68f4b09178c244199a36c43c3989a45b" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:53.883699 13176 raft_consensus.cc:3060] T 68f4b09178c244199a36c43c3989a45b P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.886650 13102 raft_consensus.cc:3055] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:53.887183 13102 raft_consensus.cc:740] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:07:53.888183 13102 consensus_queue.cc:260] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.889840 13102 raft_consensus.cc:3060] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.893160 13176 raft_consensus.cc:2468] T 68f4b09178c244199a36c43c3989a45b P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 2.
I20260709 13:07:53.894695 12994 leader_election.cc:304] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:53.895550 13298 raft_consensus.cc:2804] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:53.896024 13298 raft_consensus.cc:697] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 LEADER]: Becoming Leader. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Running, Role: LEADER
I20260709 13:07:53.896870 13298 consensus_queue.cc:237] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:53.897122 13102 raft_consensus.cc:2468] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 2.
I20260709 13:07:53.903471 12925 catalog_manager.cc:5697] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411 reported cstate change: term changed from 1 to 2, leader changed from 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194) to 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "8593949d5fb7413ca30c9ef5dfd31411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:53.984728 13102 raft_consensus.cc:1275] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Refusing update from remote peer 8593949d5fb7413ca30c9ef5dfd31411: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:53.986312 13298 consensus_queue.cc:1048] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:53.996791 13176 raft_consensus.cc:1275] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Refusing update from remote peer 8593949d5fb7413ca30c9ef5dfd31411: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:53.998286 13298 consensus_queue.cc:1048] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:54.091702 13028 raft_consensus.cc:1275] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Refusing update from remote peer 20f53f29f26f48d6b818f46c9e755f69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:54.093057 13225 consensus_queue.cc:1048] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:54.103457 13176 raft_consensus.cc:1275] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Refusing update from remote peer 20f53f29f26f48d6b818f46c9e755f69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:54.105254 13295 consensus_queue.cc:1048] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:54.258755 13102 raft_consensus.cc:1275] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Refusing update from remote peer 824d687f218b4cb091e5d62fa9d18187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:54.260228 13234 consensus_queue.cc:1048] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:54.268273 13028 raft_consensus.cc:1275] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Refusing update from remote peer 824d687f218b4cb091e5d62fa9d18187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:54.269799 13302 consensus_queue.cc:1048] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:54.290205 13176 raft_consensus.cc:1275] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Refusing update from remote peer 20f53f29f26f48d6b818f46c9e755f69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:54.291563 13295 consensus_queue.cc:1048] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:54.299060 13028 raft_consensus.cc:1275] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Refusing update from remote peer 20f53f29f26f48d6b818f46c9e755f69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:54.301247 13243 consensus_queue.cc:1048] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:54.337987 13102 raft_consensus.cc:1275] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Refusing update from remote peer 8593949d5fb7413ca30c9ef5dfd31411: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:54.339265 13298 consensus_queue.cc:1048] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:54.349071 13176 raft_consensus.cc:1275] T 68f4b09178c244199a36c43c3989a45b P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Refusing update from remote peer 8593949d5fb7413ca30c9ef5dfd31411: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:54.352559 13242 consensus_queue.cc:1048] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:56.514240 12922 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58384:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:56.516382 12922 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:56.569829 13086 tablet_service.cc:1511] Processing CreateTablet for tablet 32b76fdbda6342bd887202061755151b (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:56.570645 13012 tablet_service.cc:1511] Processing CreateTablet for tablet 32b76fdbda6342bd887202061755151b (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:56.574203 13018 tablet_service.cc:1511] Processing CreateTablet for tablet 790b9908004d4336a3093ecfe11ea707 (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:56.573489 13089 tablet_service.cc:1511] Processing CreateTablet for tablet 790b9908004d4336a3093ecfe11ea707 (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:56.574896 13012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32b76fdbda6342bd887202061755151b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.575798 13018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 790b9908004d4336a3093ecfe11ea707. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.576185 13086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32b76fdbda6342bd887202061755151b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.577704 13164 tablet_service.cc:1511] Processing CreateTablet for tablet 790b9908004d4336a3093ecfe11ea707 (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:56.572862 13166 tablet_service.cc:1511] Processing CreateTablet for tablet 32b76fdbda6342bd887202061755151b (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:56.580803 13091 tablet_service.cc:1511] Processing CreateTablet for tablet 43c7f51f72dc4e92b878a4ed950d4c5b (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:56.582716 13164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 790b9908004d4336a3093ecfe11ea707. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.577847 13089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 790b9908004d4336a3093ecfe11ea707. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.584532 13013 tablet_service.cc:1511] Processing CreateTablet for tablet 43c7f51f72dc4e92b878a4ed950d4c5b (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:56.587288 13165 tablet_service.cc:1511] Processing CreateTablet for tablet 06488c29a51d4bcda1841bdc7fd32b6d (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:56.586158 13092 tablet_service.cc:1511] Processing CreateTablet for tablet 06488c29a51d4bcda1841bdc7fd32b6d (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:56.580803 13167 tablet_service.cc:1511] Processing CreateTablet for tablet 43c7f51f72dc4e92b878a4ed950d4c5b (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:56.590739 13165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06488c29a51d4bcda1841bdc7fd32b6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.591596 13166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32b76fdbda6342bd887202061755151b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.592172 13093 tablet_service.cc:1511] Processing CreateTablet for tablet ed00ffda0221409fbcadb48892fcbadc (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:56.592824 13160 tablet_service.cc:1511] Processing CreateTablet for tablet ed00ffda0221409fbcadb48892fcbadc (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:56.594476 13167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43c7f51f72dc4e92b878a4ed950d4c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.595847 13160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed00ffda0221409fbcadb48892fcbadc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.599211 13010 tablet_service.cc:1511] Processing CreateTablet for tablet 06488c29a51d4bcda1841bdc7fd32b6d (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:56.603554 13014 tablet_service.cc:1511] Processing CreateTablet for tablet ed00ffda0221409fbcadb48892fcbadc (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:56.606001 13085 tablet_service.cc:1511] Processing CreateTablet for tablet c506771cdafd4db99eaf39cc5d9015ac (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:56.608690 13162 tablet_service.cc:1511] Processing CreateTablet for tablet c506771cdafd4db99eaf39cc5d9015ac (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:56.609259 13011 tablet_service.cc:1511] Processing CreateTablet for tablet c506771cdafd4db99eaf39cc5d9015ac (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:56.610220 13162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c506771cdafd4db99eaf39cc5d9015ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.611879 13092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06488c29a51d4bcda1841bdc7fd32b6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.613036 13087 tablet_service.cc:1511] Processing CreateTablet for tablet f4f2dbd8b1bd424e95b9a5ee46d2ea5b (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:56.613454 13163 tablet_service.cc:1511] Processing CreateTablet for tablet f4f2dbd8b1bd424e95b9a5ee46d2ea5b (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:56.614960 13163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4f2dbd8b1bd424e95b9a5ee46d2ea5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.615877 13088 tablet_service.cc:1511] Processing CreateTablet for tablet daf0cc5885a14c65b1bd2fdab90c9b26 (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:56.618984 13161 tablet_service.cc:1511] Processing CreateTablet for tablet daf0cc5885a14c65b1bd2fdab90c9b26 (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:56.619768 13093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed00ffda0221409fbcadb48892fcbadc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.620880 13161 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daf0cc5885a14c65b1bd2fdab90c9b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.622687 13013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43c7f51f72dc4e92b878a4ed950d4c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.622934 13088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daf0cc5885a14c65b1bd2fdab90c9b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.625221 13085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c506771cdafd4db99eaf39cc5d9015ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.628880 13091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43c7f51f72dc4e92b878a4ed950d4c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.630475 13087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4f2dbd8b1bd424e95b9a5ee46d2ea5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.634503 13014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed00ffda0221409fbcadb48892fcbadc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.636591 13090 tablet_service.cc:1511] Processing CreateTablet for tablet 8ffaa4a08cf24ed3ab65b5aecd4c3721 (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:56.638038 13090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ffaa4a08cf24ed3ab65b5aecd4c3721. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.639304 13015 tablet_service.cc:1511] Processing CreateTablet for tablet f4f2dbd8b1bd424e95b9a5ee46d2ea5b (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:56.639657 13017 tablet_service.cc:1511] Processing CreateTablet for tablet daf0cc5885a14c65b1bd2fdab90c9b26 (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:56.640098 13016 tablet_service.cc:1511] Processing CreateTablet for tablet 8ffaa4a08cf24ed3ab65b5aecd4c3721 (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:56.640733 13015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4f2dbd8b1bd424e95b9a5ee46d2ea5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.641629 13016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ffaa4a08cf24ed3ab65b5aecd4c3721. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.643258 13011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c506771cdafd4db99eaf39cc5d9015ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.644932 13017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daf0cc5885a14c65b1bd2fdab90c9b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.645900 13010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06488c29a51d4bcda1841bdc7fd32b6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.645563 13159 tablet_service.cc:1511] Processing CreateTablet for tablet 8ffaa4a08cf24ed3ab65b5aecd4c3721 (DEFAULT_TABLE table=soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:56.647032 13159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ffaa4a08cf24ed3ab65b5aecd4c3721. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.681028 13317 tablet_bootstrap.cc:492] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69: Bootstrap starting.
I20260709 13:07:56.716636 13317 tablet_bootstrap.cc:654] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:56.733335 12974 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:56.762235 13318 tablet_bootstrap.cc:492] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411: Bootstrap starting.
I20260709 13:07:56.769686 13319 tablet_bootstrap.cc:492] T daf0cc5885a14c65b1bd2fdab90c9b26 P 824d687f218b4cb091e5d62fa9d18187: Bootstrap starting.
I20260709 13:07:56.807101 13317 tablet_bootstrap.cc:492] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69: No bootstrap required, opened a new log
I20260709 13:07:56.807725 13317 ts_tablet_manager.cc:1403] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69: Time spent bootstrapping tablet: real 0.127s	user 0.009s	sys 0.005s
I20260709 13:07:56.832948 13319 tablet_bootstrap.cc:654] T daf0cc5885a14c65b1bd2fdab90c9b26 P 824d687f218b4cb091e5d62fa9d18187: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.835702 13317 raft_consensus.cc:359] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:56.836724 13317 raft_consensus.cc:385] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.837028 13317 raft_consensus.cc:740] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.807377 13318 tablet_bootstrap.cc:654] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.837860 13317 consensus_queue.cc:260] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:56.840786 13317 ts_tablet_manager.cc:1434] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:07:56.841846 13317 tablet_bootstrap.cc:492] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69: Bootstrap starting.
I20260709 13:07:56.849273 13317 tablet_bootstrap.cc:654] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.871840 13318 tablet_bootstrap.cc:492] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411: No bootstrap required, opened a new log
I20260709 13:07:56.871901 13319 tablet_bootstrap.cc:492] T daf0cc5885a14c65b1bd2fdab90c9b26 P 824d687f218b4cb091e5d62fa9d18187: No bootstrap required, opened a new log
I20260709 13:07:56.872001 13317 tablet_bootstrap.cc:492] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69: No bootstrap required, opened a new log
I20260709 13:07:56.872576 13318 ts_tablet_manager.cc:1403] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411: Time spent bootstrapping tablet: real 0.111s	user 0.024s	sys 0.000s
I20260709 13:07:56.872666 13319 ts_tablet_manager.cc:1403] T daf0cc5885a14c65b1bd2fdab90c9b26 P 824d687f218b4cb091e5d62fa9d18187: Time spent bootstrapping tablet: real 0.103s	user 0.017s	sys 0.027s
I20260709 13:07:56.872769 13317 ts_tablet_manager.cc:1403] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69: Time spent bootstrapping tablet: real 0.031s	user 0.017s	sys 0.012s
I20260709 13:07:56.875742 13318 raft_consensus.cc:359] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:56.876410 13318 raft_consensus.cc:385] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.876706 13318 raft_consensus.cc:740] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.876502 13319 raft_consensus.cc:359] T daf0cc5885a14c65b1bd2fdab90c9b26 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:56.876888 13317 raft_consensus.cc:359] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:56.877698 13319 raft_consensus.cc:385] T daf0cc5885a14c65b1bd2fdab90c9b26 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.878015 13317 raft_consensus.cc:385] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.878077 13319 raft_consensus.cc:740] T daf0cc5885a14c65b1bd2fdab90c9b26 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.877900 13318 consensus_queue.cc:260] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:56.878865 13317 raft_consensus.cc:740] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.879362 13319 consensus_queue.cc:260] T daf0cc5885a14c65b1bd2fdab90c9b26 P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:56.881134 13318 ts_tablet_manager.cc:1434] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:56.881886 13319 ts_tablet_manager.cc:1434] T daf0cc5885a14c65b1bd2fdab90c9b26 P 824d687f218b4cb091e5d62fa9d18187: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:07:56.882321 13318 tablet_bootstrap.cc:492] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411: Bootstrap starting.
I20260709 13:07:56.883085 13319 tablet_bootstrap.cc:492] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187: Bootstrap starting.
I20260709 13:07:56.879900 13317 consensus_queue.cc:260] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:56.886027 13317 ts_tablet_manager.cc:1434] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:07:56.887650 13317 tablet_bootstrap.cc:492] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69: Bootstrap starting.
I20260709 13:07:56.890884 13319 tablet_bootstrap.cc:654] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.891148 13318 tablet_bootstrap.cc:654] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:56.891957 13203 tablet.cc:2406] T daf0cc5885a14c65b1bd2fdab90c9b26 P 824d687f218b4cb091e5d62fa9d18187: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:07:56.895161 13317 tablet_bootstrap.cc:654] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.906929 13318 tablet_bootstrap.cc:492] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411: No bootstrap required, opened a new log
I20260709 13:07:56.907557 13318 ts_tablet_manager.cc:1403] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.005s
I20260709 13:07:56.911356 13318 raft_consensus.cc:359] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:56.912350 13318 raft_consensus.cc:385] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.913093 13318 raft_consensus.cc:740] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.917794 13317 tablet_bootstrap.cc:492] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69: No bootstrap required, opened a new log
I20260709 13:07:56.918728 13317 ts_tablet_manager.cc:1403] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.014s
I20260709 13:07:56.922169 13317 raft_consensus.cc:359] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:56.922962 13317 raft_consensus.cc:385] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.923260 13317 raft_consensus.cc:740] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.924232 13317 consensus_queue.cc:260] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:56.926034 13318 consensus_queue.cc:260] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:56.928049 13318 ts_tablet_manager.cc:1434] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.004s
I20260709 13:07:56.929394 13318 tablet_bootstrap.cc:492] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 8593949d5fb7413ca30c9ef5dfd31411: Bootstrap starting.
I20260709 13:07:56.937309 13318 tablet_bootstrap.cc:654] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 8593949d5fb7413ca30c9ef5dfd31411: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.938865 13317 ts_tablet_manager.cc:1434] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 13:07:56.940151 13317 tablet_bootstrap.cc:492] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69: Bootstrap starting.
I20260709 13:07:56.943401 13319 tablet_bootstrap.cc:492] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187: No bootstrap required, opened a new log
I20260709 13:07:56.944042 13319 ts_tablet_manager.cc:1403] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187: Time spent bootstrapping tablet: real 0.061s	user 0.026s	sys 0.027s
I20260709 13:07:56.946998 13317 tablet_bootstrap.cc:654] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.947149 13319 raft_consensus.cc:359] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:56.948058 13319 raft_consensus.cc:385] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.948338 13319 raft_consensus.cc:740] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.949026 13319 consensus_queue.cc:260] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:56.951097 13319 ts_tablet_manager.cc:1434] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:07:56.952226 13319 tablet_bootstrap.cc:492] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187: Bootstrap starting.
I20260709 13:07:56.955610 13318 tablet_bootstrap.cc:492] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 8593949d5fb7413ca30c9ef5dfd31411: No bootstrap required, opened a new log
I20260709 13:07:56.956367 13318 ts_tablet_manager.cc:1403] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 8593949d5fb7413ca30c9ef5dfd31411: Time spent bootstrapping tablet: real 0.027s	user 0.005s	sys 0.009s
I20260709 13:07:56.958912 13319 tablet_bootstrap.cc:654] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.960189 13318 raft_consensus.cc:359] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:56.961055 13318 raft_consensus.cc:385] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.961419 13318 raft_consensus.cc:740] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.962289 13318 consensus_queue.cc:260] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:56.965189 13317 tablet_bootstrap.cc:492] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69: No bootstrap required, opened a new log
I20260709 13:07:56.965737 13318 ts_tablet_manager.cc:1434] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 8593949d5fb7413ca30c9ef5dfd31411: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:07:56.965903 13317 ts_tablet_manager.cc:1403] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.006s
I20260709 13:07:56.968748 13319 tablet_bootstrap.cc:492] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187: No bootstrap required, opened a new log
I20260709 13:07:56.969314 13319 ts_tablet_manager.cc:1403] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.003s
I20260709 13:07:56.969700 13317 raft_consensus.cc:359] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:56.970502 13317 raft_consensus.cc:385] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.970778 13317 raft_consensus.cc:740] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.971690 13317 consensus_queue.cc:260] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:56.972417 13319 raft_consensus.cc:359] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:56.973043 13318 tablet_bootstrap.cc:492] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411: Bootstrap starting.
I20260709 13:07:56.973412 13319 raft_consensus.cc:385] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.973781 13319 raft_consensus.cc:740] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.974282 13317 ts_tablet_manager.cc:1434] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:56.974642 13319 consensus_queue.cc:260] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:56.975648 13317 tablet_bootstrap.cc:492] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69: Bootstrap starting.
I20260709 13:07:56.977401 13319 ts_tablet_manager.cc:1434] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:07:56.980541 13319 tablet_bootstrap.cc:492] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187: Bootstrap starting.
I20260709 13:07:56.985199 13318 tablet_bootstrap.cc:654] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.990406 13317 tablet_bootstrap.cc:654] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.995152 13319 tablet_bootstrap.cc:654] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.004428 13317 tablet_bootstrap.cc:492] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69: No bootstrap required, opened a new log
I20260709 13:07:57.005263 13317 ts_tablet_manager.cc:1403] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.000s
I20260709 13:07:57.012719 13318 tablet_bootstrap.cc:492] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411: No bootstrap required, opened a new log
I20260709 13:07:57.013715 13318 ts_tablet_manager.cc:1403] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.005s
I20260709 13:07:57.017392 13319 tablet_bootstrap.cc:492] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187: No bootstrap required, opened a new log
I20260709 13:07:57.018108 13319 ts_tablet_manager.cc:1403] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187: Time spent bootstrapping tablet: real 0.038s	user 0.017s	sys 0.007s
I20260709 13:07:57.018101 13318 raft_consensus.cc:359] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.019129 13318 raft_consensus.cc:385] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.019433 13318 raft_consensus.cc:740] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.020478 13317 raft_consensus.cc:359] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.020512 13318 consensus_queue.cc:260] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.021463 13317 raft_consensus.cc:385] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.021811 13319 raft_consensus.cc:359] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.022939 13319 raft_consensus.cc:385] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.023300 13319 raft_consensus.cc:740] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.024205 13318 ts_tablet_manager.cc:1434] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:07:57.024325 13319 consensus_queue.cc:260] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.025849 13318 tablet_bootstrap.cc:492] T c506771cdafd4db99eaf39cc5d9015ac P 8593949d5fb7413ca30c9ef5dfd31411: Bootstrap starting.
I20260709 13:07:57.026683 13319 ts_tablet_manager.cc:1434] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:07:57.022221 13317 raft_consensus.cc:740] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.027755 13317 consensus_queue.cc:260] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.030308 13317 ts_tablet_manager.cc:1434] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.002s
I20260709 13:07:57.031582 13317 tablet_bootstrap.cc:492] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69: Bootstrap starting.
I20260709 13:07:57.034376 13321 raft_consensus.cc:493] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.035095 13318 tablet_bootstrap.cc:654] T c506771cdafd4db99eaf39cc5d9015ac P 8593949d5fb7413ca30c9ef5dfd31411: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.035372 13321 raft_consensus.cc:515] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.038523 13319 tablet_bootstrap.cc:492] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 824d687f218b4cb091e5d62fa9d18187: Bootstrap starting.
I20260709 13:07:57.041837 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b76fdbda6342bd887202061755151b" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
I20260709 13:07:57.042855 13028 raft_consensus.cc:2468] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 0.
I20260709 13:07:57.043576 13317 tablet_bootstrap.cc:654] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.048126 13066 leader_election.cc:304] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:57.049744 13319 tablet_bootstrap.cc:654] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 824d687f218b4cb091e5d62fa9d18187: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.050634 13225 raft_consensus.cc:2804] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:57.051456 13225 raft_consensus.cc:493] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.052070 13225 raft_consensus.cc:3060] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.053336 13302 raft_consensus.cc:493] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.054024 13302 raft_consensus.cc:515] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.061762 13302 leader_election.cc:290] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507)
I20260709 13:07:57.063892 13102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "790b9908004d4336a3093ecfe11ea707" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69" is_pre_election: true
I20260709 13:07:57.065017 13331 raft_consensus.cc:493] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.065846 13331 raft_consensus.cc:515] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.068374 13302 raft_consensus.cc:493] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:57.068518 13141 leader_election.cc:343] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.068954 13302 raft_consensus.cc:515] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.071164 13176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daf0cc5885a14c65b1bd2fdab90c9b26" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
I20260709 13:07:57.071475 13321 leader_election.cc:290] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:57.071079 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daf0cc5885a14c65b1bd2fdab90c9b26" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69" is_pre_election: true
I20260709 13:07:57.073766 13027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c506771cdafd4db99eaf39cc5d9015ac" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
I20260709 13:07:57.071201 13302 leader_election.cc:290] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507)
I20260709 13:07:57.081257 13103 raft_consensus.cc:2468] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 0.
I20260709 13:07:57.083086 12992 leader_election.cc:304] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:57.073951 13101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c506771cdafd4db99eaf39cc5d9015ac" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69" is_pre_election: true
I20260709 13:07:57.084146 13332 raft_consensus.cc:2804] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:57.084404 13101 raft_consensus.cc:2468] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 0.
I20260709 13:07:57.076591 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b76fdbda6342bd887202061755151b" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
I20260709 13:07:57.077504 13331 leader_election.cc:290] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:57.071940 13176 raft_consensus.cc:2468] T daf0cc5885a14c65b1bd2fdab90c9b26 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 0.
I20260709 13:07:57.086732 13141 leader_election.cc:304] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 824d687f218b4cb091e5d62fa9d18187; no voters: 
I20260709 13:07:57.087751 13302 raft_consensus.cc:2804] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:07:57.087883 13140 leader_election.cc:343] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.084569 13332 raft_consensus.cc:493] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.089540 13332 raft_consensus.cc:3060] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.093963 13302 raft_consensus.cc:493] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.094645 13302 raft_consensus.cc:3060] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:57.097250 13069 leader_election.cc:343] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.071040 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "790b9908004d4336a3093ecfe11ea707" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
W20260709 13:07:57.133663 13140 leader_election.cc:343] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.134239 13140 leader_election.cc:304] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187; no voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411
I20260709 13:07:57.161075 13340 raft_consensus.cc:2749] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:57.170835 13321 raft_consensus.cc:493] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.171702 13225 raft_consensus.cc:515] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.171445 13321 raft_consensus.cc:515] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.180845 13321 leader_election.cc:290] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:57.182634 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ffaa4a08cf24ed3ab65b5aecd4c3721" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
I20260709 13:07:57.183779 13028 raft_consensus.cc:2468] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 0.
I20260709 13:07:57.188297 13340 raft_consensus.cc:493] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.189118 13340 raft_consensus.cc:515] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.190057 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ffaa4a08cf24ed3ab65b5aecd4c3721" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
I20260709 13:07:57.187180 13066 leader_election.cc:304] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
W20260709 13:07:57.192615 13069 leader_election.cc:343] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.193080 13340 leader_election.cc:290] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507)
I20260709 13:07:57.194317 13321 raft_consensus.cc:2804] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:57.195477 13321 raft_consensus.cc:493] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.196045 13321 raft_consensus.cc:3060] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.206477 13101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c7f51f72dc4e92b878a4ed950d4c5b" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69" is_pre_election: true
W20260709 13:07:57.209223 13141 leader_election.cc:343] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.211334 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c7f51f72dc4e92b878a4ed950d4c5b" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
W20260709 13:07:57.213075 13140 leader_election.cc:343] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.213559 13140 leader_election.cc:304] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187; no voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411
I20260709 13:07:57.215678 13340 raft_consensus.cc:2749] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:57.171168 13332 raft_consensus.cc:515] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.226878 13101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daf0cc5885a14c65b1bd2fdab90c9b26" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:07:57.227674 13101 raft_consensus.cc:3060] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.227918 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daf0cc5885a14c65b1bd2fdab90c9b26" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:57.228647 13177 raft_consensus.cc:3060] T daf0cc5885a14c65b1bd2fdab90c9b26 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.237416 13177 raft_consensus.cc:2468] T daf0cc5885a14c65b1bd2fdab90c9b26 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 1.
I20260709 13:07:57.237263 13101 raft_consensus.cc:2468] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 1.
I20260709 13:07:57.239542 12994 leader_election.cc:304] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:57.240557 13331 raft_consensus.cc:2804] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:57.241178 13331 raft_consensus.cc:697] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Becoming Leader. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Running, Role: LEADER
I20260709 13:07:57.242973 13331 consensus_queue.cc:237] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.254799 13319 tablet_bootstrap.cc:492] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 824d687f218b4cb091e5d62fa9d18187: No bootstrap required, opened a new log
I20260709 13:07:57.255324 13225 leader_election.cc:290] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:57.255450 13319 ts_tablet_manager.cc:1403] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 824d687f218b4cb091e5d62fa9d18187: Time spent bootstrapping tablet: real 0.217s	user 0.078s	sys 0.110s
I20260709 13:07:57.255481 13318 tablet_bootstrap.cc:492] T c506771cdafd4db99eaf39cc5d9015ac P 8593949d5fb7413ca30c9ef5dfd31411: No bootstrap required, opened a new log
I20260709 13:07:57.256304 13318 ts_tablet_manager.cc:1403] T c506771cdafd4db99eaf39cc5d9015ac P 8593949d5fb7413ca30c9ef5dfd31411: Time spent bootstrapping tablet: real 0.231s	user 0.084s	sys 0.087s
I20260709 13:07:57.256757 13176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b76fdbda6342bd887202061755151b" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:57.258913 13319 raft_consensus.cc:359] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.177995 13302 raft_consensus.cc:515] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.259845 13318 raft_consensus.cc:359] T c506771cdafd4db99eaf39cc5d9015ac P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.260257 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b76fdbda6342bd887202061755151b" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:57.261044 13318 raft_consensus.cc:385] T c506771cdafd4db99eaf39cc5d9015ac P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:07:57.259346 13069 leader_election.cc:343] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.261399 13318 raft_consensus.cc:740] T c506771cdafd4db99eaf39cc5d9015ac P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.263705 13318 consensus_queue.cc:260] T c506771cdafd4db99eaf39cc5d9015ac P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.264472 13321 raft_consensus.cc:515] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.259946 13319 raft_consensus.cc:385] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.259888 13317 tablet_bootstrap.cc:492] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69: No bootstrap required, opened a new log
I20260709 13:07:57.268308 13319 raft_consensus.cc:740] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.269172 13317 ts_tablet_manager.cc:1403] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69: Time spent bootstrapping tablet: real 0.238s	user 0.058s	sys 0.113s
I20260709 13:07:57.267724 12923 catalog_manager.cc:5697] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "8593949d5fb7413ca30c9ef5dfd31411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:57.269426 13319 consensus_queue.cc:260] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.271988 13319 ts_tablet_manager.cc:1434] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 824d687f218b4cb091e5d62fa9d18187: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.001s
I20260709 13:07:57.273162 13319 tablet_bootstrap.cc:492] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187: Bootstrap starting.
I20260709 13:07:57.273205 13317 raft_consensus.cc:359] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.273854 13101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c506771cdafd4db99eaf39cc5d9015ac" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:07:57.274290 13317 raft_consensus.cc:385] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.274549 13101 raft_consensus.cc:3060] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.274696 13317 raft_consensus.cc:740] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.275568 13317 consensus_queue.cc:260] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.278573 13317 ts_tablet_manager.cc:1434] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:07:57.279719 13317 tablet_bootstrap.cc:492] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69: Bootstrap starting.
I20260709 13:07:57.266793 13318 ts_tablet_manager.cc:1434] T c506771cdafd4db99eaf39cc5d9015ac P 8593949d5fb7413ca30c9ef5dfd31411: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.002s
I20260709 13:07:57.266935 13027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c506771cdafd4db99eaf39cc5d9015ac" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:57.282245 13027 raft_consensus.cc:3060] T c506771cdafd4db99eaf39cc5d9015ac P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.282716 13318 tablet_bootstrap.cc:492] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411: Bootstrap starting.
I20260709 13:07:57.259990 13332 leader_election.cc:290] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 election: Requested vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:57.284292 13319 tablet_bootstrap.cc:654] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.287264 13101 raft_consensus.cc:2468] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 1.
I20260709 13:07:57.288760 13141 leader_election.cc:304] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 824d687f218b4cb091e5d62fa9d18187; no voters: 
I20260709 13:07:57.290354 13340 raft_consensus.cc:2804] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:57.291745 13318 tablet_bootstrap.cc:654] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.303376 13317 tablet_bootstrap.cc:654] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.308935 13332 raft_consensus.cc:493] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.310056 13332 raft_consensus.cc:515] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.314088 13101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f2dbd8b1bd424e95b9a5ee46d2ea5b" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69" is_pre_election: true
W20260709 13:07:57.315943 12992 leader_election.cc:343] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.261262 13028 raft_consensus.cc:3060] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.318931 13176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f2dbd8b1bd424e95b9a5ee46d2ea5b" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
I20260709 13:07:57.319831 13176 raft_consensus.cc:2468] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 0.
I20260709 13:07:57.322369 12994 leader_election.cc:304] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 20f53f29f26f48d6b818f46c9e755f69
I20260709 13:07:57.323797 13331 raft_consensus.cc:2804] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:57.324397 13331 raft_consensus.cc:493] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.324882 13331 raft_consensus.cc:3060] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.324544 13332 leader_election.cc:290] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:57.328264 13027 raft_consensus.cc:2468] T c506771cdafd4db99eaf39cc5d9015ac P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 1.
I20260709 13:07:57.336050 13317 tablet_bootstrap.cc:492] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69: No bootstrap required, opened a new log
I20260709 13:07:57.337345 13317 ts_tablet_manager.cc:1403] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69: Time spent bootstrapping tablet: real 0.058s	user 0.019s	sys 0.005s
I20260709 13:07:57.267849 13302 leader_election.cc:290] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507)
I20260709 13:07:57.349076 13317 raft_consensus.cc:359] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.350128 13317 raft_consensus.cc:385] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.350463 13317 raft_consensus.cc:740] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.351341 13317 consensus_queue.cc:260] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.352748 13026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ffaa4a08cf24ed3ab65b5aecd4c3721" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:57.353484 13026 raft_consensus.cc:3060] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.354641 13317 ts_tablet_manager.cc:1434] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.001s
I20260709 13:07:57.356401 13317 tablet_bootstrap.cc:492] T 790b9908004d4336a3093ecfe11ea707 P 20f53f29f26f48d6b818f46c9e755f69: Bootstrap starting.
I20260709 13:07:57.372982 13028 raft_consensus.cc:2468] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 1.
I20260709 13:07:57.380369 13339 raft_consensus.cc:493] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.382436 13339 raft_consensus.cc:515] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.384752 13339 leader_election.cc:290] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:57.386319 13318 tablet_bootstrap.cc:492] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411: No bootstrap required, opened a new log
I20260709 13:07:57.386535 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed00ffda0221409fbcadb48892fcbadc" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
W20260709 13:07:57.388403 13069 leader_election.cc:343] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.390015 13321 leader_election.cc:290] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:57.391798 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ffaa4a08cf24ed3ab65b5aecd4c3721" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
W20260709 13:07:57.394335 13069 leader_election.cc:343] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.393347 13318 ts_tablet_manager.cc:1403] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411: Time spent bootstrapping tablet: real 0.105s	user 0.033s	sys 0.018s
I20260709 13:07:57.380097 13066 leader_election.cc:304] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 824d687f218b4cb091e5d62fa9d18187
I20260709 13:07:57.380489 13340 raft_consensus.cc:697] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [term 1 LEADER]: Becoming Leader. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Running, Role: LEADER
I20260709 13:07:57.399623 13026 raft_consensus.cc:2468] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 1.
I20260709 13:07:57.403756 13066 leader_election.cc:304] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 824d687f218b4cb091e5d62fa9d18187
I20260709 13:07:57.404716 13225 raft_consensus.cc:2804] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:57.405248 13225 raft_consensus.cc:697] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Becoming Leader. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:07:57.399251 13331 raft_consensus.cc:515] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.405303 13026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed00ffda0221409fbcadb48892fcbadc" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
I20260709 13:07:57.406111 13319 tablet_bootstrap.cc:492] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187: No bootstrap required, opened a new log
I20260709 13:07:57.406759 13319 ts_tablet_manager.cc:1403] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187: Time spent bootstrapping tablet: real 0.134s	user 0.040s	sys 0.057s
I20260709 13:07:57.406200 13225 consensus_queue.cc:237] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
W20260709 13:07:57.406883 13066 leader_election.cc:343] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.407469 13066 leader_election.cc:304] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69; no voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411
I20260709 13:07:57.407850 13101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f2dbd8b1bd424e95b9a5ee46d2ea5b" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:07:57.399027 13321 raft_consensus.cc:2804] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:57.408701 13318 raft_consensus.cc:359] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
W20260709 13:07:57.409349 12992 leader_election.cc:343] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.411321 13331 leader_election.cc:290] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 election: Requested vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:57.411621 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f2dbd8b1bd424e95b9a5ee46d2ea5b" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:57.412451 13177 raft_consensus.cc:3060] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.412241 13340 consensus_queue.cc:237] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.412640 13319 raft_consensus.cc:359] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.413388 13319 raft_consensus.cc:385] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.413718 13319 raft_consensus.cc:740] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.409531 13318 raft_consensus.cc:385] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.414019 13341 raft_consensus.cc:2749] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:57.414326 13318 raft_consensus.cc:740] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.415112 13318 consensus_queue.cc:260] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.416245 13317 tablet_bootstrap.cc:654] T 790b9908004d4336a3093ecfe11ea707 P 20f53f29f26f48d6b818f46c9e755f69: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.417479 13318 ts_tablet_manager.cc:1434] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.000s
I20260709 13:07:57.418778 13318 tablet_bootstrap.cc:492] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411: Bootstrap starting.
I20260709 13:07:57.426156 13319 consensus_queue.cc:260] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.428968 13319 ts_tablet_manager.cc:1434] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.001s
I20260709 13:07:57.430326 13319 tablet_bootstrap.cc:492] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187: Bootstrap starting.
I20260709 13:07:57.438853 13318 tablet_bootstrap.cc:654] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.442966 13319 tablet_bootstrap.cc:654] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.444171 13177 raft_consensus.cc:2468] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 1.
I20260709 13:07:57.447212 12994 leader_election.cc:304] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 20f53f29f26f48d6b818f46c9e755f69
I20260709 13:07:57.448820 13331 raft_consensus.cc:2804] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:57.449424 13331 raft_consensus.cc:697] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Becoming Leader. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Running, Role: LEADER
I20260709 13:07:57.450521 13331 consensus_queue.cc:237] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.461957 13321 raft_consensus.cc:697] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Becoming Leader. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:07:57.465145 13321 consensus_queue.cc:237] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.467545 13317 tablet_bootstrap.cc:492] T 790b9908004d4336a3093ecfe11ea707 P 20f53f29f26f48d6b818f46c9e755f69: No bootstrap required, opened a new log
I20260709 13:07:57.468315 13317 ts_tablet_manager.cc:1403] T 790b9908004d4336a3093ecfe11ea707 P 20f53f29f26f48d6b818f46c9e755f69: Time spent bootstrapping tablet: real 0.112s	user 0.020s	sys 0.004s
I20260709 13:07:57.471802 13317 raft_consensus.cc:359] T 790b9908004d4336a3093ecfe11ea707 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.472635 13318 tablet_bootstrap.cc:492] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411: No bootstrap required, opened a new log
I20260709 13:07:57.473197 13318 ts_tablet_manager.cc:1403] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411: Time spent bootstrapping tablet: real 0.055s	user 0.014s	sys 0.006s
I20260709 13:07:57.474960 13319 tablet_bootstrap.cc:492] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187: No bootstrap required, opened a new log
I20260709 13:07:57.475487 13319 ts_tablet_manager.cc:1403] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187: Time spent bootstrapping tablet: real 0.045s	user 0.015s	sys 0.006s
I20260709 13:07:57.476171 13318 raft_consensus.cc:359] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.476933 13318 raft_consensus.cc:385] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.477245 13318 raft_consensus.cc:740] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.477962 13318 consensus_queue.cc:260] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.472476 13317 raft_consensus.cc:385] T 790b9908004d4336a3093ecfe11ea707 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.479485 13317 raft_consensus.cc:740] T 790b9908004d4336a3093ecfe11ea707 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.480199 13317 consensus_queue.cc:260] T 790b9908004d4336a3093ecfe11ea707 P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.478470 12924 catalog_manager.cc:5697] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "8593949d5fb7413ca30c9ef5dfd31411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:57.482549 13317 ts_tablet_manager.cc:1434] T 790b9908004d4336a3093ecfe11ea707 P 20f53f29f26f48d6b818f46c9e755f69: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:07:57.478572 13319 raft_consensus.cc:359] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.483686 13319 raft_consensus.cc:385] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.484157 13319 raft_consensus.cc:740] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.485175 13319 consensus_queue.cc:260] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.488463 13319 ts_tablet_manager.cc:1434] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:07:57.489761 13319 tablet_bootstrap.cc:492] T 06488c29a51d4bcda1841bdc7fd32b6d P 824d687f218b4cb091e5d62fa9d18187: Bootstrap starting.
I20260709 13:07:57.493991 13317 tablet_bootstrap.cc:492] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69: Bootstrap starting.
I20260709 13:07:57.495407 12923 catalog_manager.cc:5697] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 reported cstate change: term changed from 0 to 1, leader changed from <none> to 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "824d687f218b4cb091e5d62fa9d18187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:57.500337 13317 tablet_bootstrap.cc:654] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.481032 13318 ts_tablet_manager.cc:1434] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:57.502475 13318 tablet_bootstrap.cc:492] T ed00ffda0221409fbcadb48892fcbadc P 8593949d5fb7413ca30c9ef5dfd31411: Bootstrap starting.
I20260709 13:07:57.508371 13319 tablet_bootstrap.cc:654] T 06488c29a51d4bcda1841bdc7fd32b6d P 824d687f218b4cb091e5d62fa9d18187: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.509127 13318 tablet_bootstrap.cc:654] T ed00ffda0221409fbcadb48892fcbadc P 8593949d5fb7413ca30c9ef5dfd31411: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.509204 12922 catalog_manager.cc:5697] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "20f53f29f26f48d6b818f46c9e755f69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:57.512490 12922 catalog_manager.cc:5697] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "20f53f29f26f48d6b818f46c9e755f69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:57.516345 13317 tablet_bootstrap.cc:492] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69: No bootstrap required, opened a new log
I20260709 13:07:57.516867 13317 ts_tablet_manager.cc:1403] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.000s
I20260709 13:07:57.519620 13317 raft_consensus.cc:359] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.520256 13317 raft_consensus.cc:385] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.520545 13317 raft_consensus.cc:740] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.521167 13317 consensus_queue.cc:260] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.522060 13351 raft_consensus.cc:493] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.522495 13351 raft_consensus.cc:515] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.523633 13317 ts_tablet_manager.cc:1434] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:07:57.524690 13351 leader_election.cc:290] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:57.526657 13319 tablet_bootstrap.cc:492] T 06488c29a51d4bcda1841bdc7fd32b6d P 824d687f218b4cb091e5d62fa9d18187: No bootstrap required, opened a new log
I20260709 13:07:57.526518 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c7f51f72dc4e92b878a4ed950d4c5b" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
I20260709 13:07:57.527153 13319 ts_tablet_manager.cc:1403] T 06488c29a51d4bcda1841bdc7fd32b6d P 824d687f218b4cb091e5d62fa9d18187: Time spent bootstrapping tablet: real 0.038s	user 0.014s	sys 0.006s
I20260709 13:07:57.527200 13177 raft_consensus.cc:2468] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 0.
I20260709 13:07:57.528389 13026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c7f51f72dc4e92b878a4ed950d4c5b" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
I20260709 13:07:57.529392 13318 tablet_bootstrap.cc:492] T ed00ffda0221409fbcadb48892fcbadc P 8593949d5fb7413ca30c9ef5dfd31411: No bootstrap required, opened a new log
W20260709 13:07:57.529794 13066 leader_election.cc:343] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.529922 13318 ts_tablet_manager.cc:1403] T ed00ffda0221409fbcadb48892fcbadc P 8593949d5fb7413ca30c9ef5dfd31411: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.002s
I20260709 13:07:57.530089 13319 raft_consensus.cc:359] T 06488c29a51d4bcda1841bdc7fd32b6d P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.530826 13319 raft_consensus.cc:385] T 06488c29a51d4bcda1841bdc7fd32b6d P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.531096 13319 raft_consensus.cc:740] T 06488c29a51d4bcda1841bdc7fd32b6d P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.531322 13069 leader_election.cc:304] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 824d687f218b4cb091e5d62fa9d18187; no voters: 8593949d5fb7413ca30c9ef5dfd31411
I20260709 13:07:57.531780 13319 consensus_queue.cc:260] T 06488c29a51d4bcda1841bdc7fd32b6d P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.532861 13351 raft_consensus.cc:2804] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:57.533264 13351 raft_consensus.cc:493] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.533953 13319 ts_tablet_manager.cc:1434] T 06488c29a51d4bcda1841bdc7fd32b6d P 824d687f218b4cb091e5d62fa9d18187: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:07:57.533450 13318 raft_consensus.cc:359] T ed00ffda0221409fbcadb48892fcbadc P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.534507 13318 raft_consensus.cc:385] T ed00ffda0221409fbcadb48892fcbadc P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.534868 13318 raft_consensus.cc:740] T ed00ffda0221409fbcadb48892fcbadc P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.535065 13319 tablet_bootstrap.cc:492] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187: Bootstrap starting.
I20260709 13:07:57.535632 13318 consensus_queue.cc:260] T ed00ffda0221409fbcadb48892fcbadc P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.536144 13351 raft_consensus.cc:3060] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.544680 13318 ts_tablet_manager.cc:1434] T ed00ffda0221409fbcadb48892fcbadc P 8593949d5fb7413ca30c9ef5dfd31411: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.002s
I20260709 13:07:57.545782 13319 tablet_bootstrap.cc:654] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.548293 13318 tablet_bootstrap.cc:492] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411: Bootstrap starting.
I20260709 13:07:57.548671 13331 raft_consensus.cc:493] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.549180 13331 raft_consensus.cc:515] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.551324 13331 leader_election.cc:290] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:57.552110 13101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "790b9908004d4336a3093ecfe11ea707" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69" is_pre_election: true
I20260709 13:07:57.552076 13351 raft_consensus.cc:515] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.553009 13101 raft_consensus.cc:2468] T 790b9908004d4336a3093ecfe11ea707 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 0.
I20260709 13:07:57.553848 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "790b9908004d4336a3093ecfe11ea707" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
I20260709 13:07:57.554507 13177 raft_consensus.cc:2468] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 0.
I20260709 13:07:57.555675 12992 leader_election.cc:304] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:57.557118 13305 raft_consensus.cc:2804] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:57.557595 13305 raft_consensus.cc:493] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.558059 13305 raft_consensus.cc:3060] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.558398 13351 leader_election.cc:290] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:57.559095 13026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c7f51f72dc4e92b878a4ed950d4c5b" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:57.560060 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c7f51f72dc4e92b878a4ed950d4c5b" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
W20260709 13:07:57.560515 13066 leader_election.cc:343] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.560631 13177 raft_consensus.cc:3060] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.567889 13177 raft_consensus.cc:2468] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 1.
I20260709 13:07:57.569314 13069 leader_election.cc:304] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 824d687f218b4cb091e5d62fa9d18187; no voters: 8593949d5fb7413ca30c9ef5dfd31411
I20260709 13:07:57.570508 13351 raft_consensus.cc:2804] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:57.571012 13351 raft_consensus.cc:697] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Becoming Leader. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:07:57.571934 13351 consensus_queue.cc:237] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.585388 13305 raft_consensus.cc:515] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.590207 13101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "790b9908004d4336a3093ecfe11ea707" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:07:57.590500 13176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "790b9908004d4336a3093ecfe11ea707" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:57.591120 13101 raft_consensus.cc:3060] T 790b9908004d4336a3093ecfe11ea707 P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.591488 13176 raft_consensus.cc:3060] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.601660 13318 tablet_bootstrap.cc:654] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.602169 13176 raft_consensus.cc:2468] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 1.
I20260709 13:07:57.604378 12994 leader_election.cc:304] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:57.605746 13305 leader_election.cc:290] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 election: Requested vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:57.605850 13331 raft_consensus.cc:2804] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:57.612574 13331 raft_consensus.cc:697] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Becoming Leader. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Running, Role: LEADER
I20260709 13:07:57.611881 12922 catalog_manager.cc:5697] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "20f53f29f26f48d6b818f46c9e755f69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:57.612995 13319 tablet_bootstrap.cc:492] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187: No bootstrap required, opened a new log
I20260709 13:07:57.613471 13319 ts_tablet_manager.cc:1403] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187: Time spent bootstrapping tablet: real 0.079s	user 0.032s	sys 0.038s
I20260709 13:07:57.613466 13331 consensus_queue.cc:237] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.616516 13319 raft_consensus.cc:359] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.602458 13101 raft_consensus.cc:2468] T 790b9908004d4336a3093ecfe11ea707 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 1.
I20260709 13:07:57.618052 13319 raft_consensus.cc:385] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.618417 13319 raft_consensus.cc:740] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.619174 13319 consensus_queue.cc:260] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.621256 13319 ts_tablet_manager.cc:1434] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.003s
I20260709 13:07:57.632239 13318 tablet_bootstrap.cc:492] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411: No bootstrap required, opened a new log
I20260709 13:07:57.632831 13318 ts_tablet_manager.cc:1403] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411: Time spent bootstrapping tablet: real 0.085s	user 0.011s	sys 0.017s
I20260709 13:07:57.634951 13318 raft_consensus.cc:359] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.635478 13318 raft_consensus.cc:385] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.635705 13318 raft_consensus.cc:740] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.636226 13318 consensus_queue.cc:260] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.636387 12925 catalog_manager.cc:5697] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "8593949d5fb7413ca30c9ef5dfd31411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:57.638083 13318 ts_tablet_manager.cc:1434] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.003s
I20260709 13:07:57.640195 13331 raft_consensus.cc:493] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.640671 13331 raft_consensus.cc:515] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.642676 13331 leader_election.cc:290] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:57.643796 13176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c7f51f72dc4e92b878a4ed950d4c5b" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
I20260709 13:07:57.644595 13176 raft_consensus.cc:2393] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8593949d5fb7413ca30c9ef5dfd31411 in current term 1: Already voted for candidate 20f53f29f26f48d6b818f46c9e755f69 in this term.
I20260709 13:07:57.645694 13340 raft_consensus.cc:493] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.646221 13101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c7f51f72dc4e92b878a4ed950d4c5b" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69" is_pre_election: true
I20260709 13:07:57.646189 13340 raft_consensus.cc:515] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.647570 12992 leader_election.cc:304] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8593949d5fb7413ca30c9ef5dfd31411; no voters: 20f53f29f26f48d6b818f46c9e755f69, 824d687f218b4cb091e5d62fa9d18187
I20260709 13:07:57.648264 13340 leader_election.cc:290] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507)
I20260709 13:07:57.649256 13331 raft_consensus.cc:3060] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.649377 13026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ffaa4a08cf24ed3ab65b5aecd4c3721" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
I20260709 13:07:57.650224 13026 raft_consensus.cc:2393] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 824d687f218b4cb091e5d62fa9d18187 in current term 1: Already voted for candidate 20f53f29f26f48d6b818f46c9e755f69 in this term.
I20260709 13:07:57.650707 13101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ffaa4a08cf24ed3ab65b5aecd4c3721" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69" is_pre_election: true
I20260709 13:07:57.651981 13141 leader_election.cc:304] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187; no voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411
I20260709 13:07:57.652788 13340 raft_consensus.cc:3060] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.656337 13331 raft_consensus.cc:2749] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:57.658887 13340 raft_consensus.cc:2749] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:57.659884 13340 raft_consensus.cc:493] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.660277 13340 raft_consensus.cc:515] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.661962 13340 leader_election.cc:290] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507)
I20260709 13:07:57.662937 13026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed00ffda0221409fbcadb48892fcbadc" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
I20260709 13:07:57.663089 13101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed00ffda0221409fbcadb48892fcbadc" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69" is_pre_election: true
I20260709 13:07:57.663550 13026 raft_consensus.cc:2468] T ed00ffda0221409fbcadb48892fcbadc P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 0.
I20260709 13:07:57.663836 13101 raft_consensus.cc:2468] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 0.
I20260709 13:07:57.664649 13140 leader_election.cc:304] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:57.665863 13340 raft_consensus.cc:2804] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:57.666221 13340 raft_consensus.cc:493] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.666507 13340 raft_consensus.cc:3060] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.671881 13340 raft_consensus.cc:515] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.673749 13340 leader_election.cc:290] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507)
I20260709 13:07:57.674882 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed00ffda0221409fbcadb48892fcbadc" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:57.674882 13101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed00ffda0221409fbcadb48892fcbadc" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:07:57.675833 13028 raft_consensus.cc:3060] T ed00ffda0221409fbcadb48892fcbadc P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.675860 13101 raft_consensus.cc:3060] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.683162 13028 raft_consensus.cc:2468] T ed00ffda0221409fbcadb48892fcbadc P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 1.
I20260709 13:07:57.683794 13351 raft_consensus.cc:493] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.684072 13101 raft_consensus.cc:2468] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 1.
I20260709 13:07:57.684547 13140 leader_election.cc:304] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:57.684408 13351 raft_consensus.cc:515] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.685607 13340 raft_consensus.cc:2804] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:57.686304 13340 raft_consensus.cc:697] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [term 1 LEADER]: Becoming Leader. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Running, Role: LEADER
I20260709 13:07:57.687496 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f2dbd8b1bd424e95b9a5ee46d2ea5b" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
I20260709 13:07:57.687677 13340 consensus_queue.cc:237] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.688236 13176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f2dbd8b1bd424e95b9a5ee46d2ea5b" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
I20260709 13:07:57.689042 13176 raft_consensus.cc:2393] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20f53f29f26f48d6b818f46c9e755f69 in current term 1: Already voted for candidate 8593949d5fb7413ca30c9ef5dfd31411 in this term.
I20260709 13:07:57.689409 13351 leader_election.cc:290] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:57.689976 13069 leader_election.cc:304] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69; no voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411
I20260709 13:07:57.690802 13351 raft_consensus.cc:3060] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.697793 13351 raft_consensus.cc:2749] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:57.697417 12925 catalog_manager.cc:5697] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 reported cstate change: term changed from 0 to 1, leader changed from <none> to 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "824d687f218b4cb091e5d62fa9d18187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:57.737164 12974 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:57.751312 13351 raft_consensus.cc:493] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.752072 13351 raft_consensus.cc:515] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.754048 13351 leader_election.cc:290] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:57.755043 13176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06488c29a51d4bcda1841bdc7fd32b6d" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187" is_pre_election: true
I20260709 13:07:57.754921 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06488c29a51d4bcda1841bdc7fd32b6d" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
I20260709 13:07:57.755846 13176 raft_consensus.cc:2468] T 06488c29a51d4bcda1841bdc7fd32b6d P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 0.
I20260709 13:07:57.755882 13028 raft_consensus.cc:2468] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 0.
I20260709 13:07:57.757123 13069 leader_election.cc:304] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 824d687f218b4cb091e5d62fa9d18187; no voters: 
I20260709 13:07:57.757880 13351 raft_consensus.cc:2804] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:57.758170 13351 raft_consensus.cc:493] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.758509 13351 raft_consensus.cc:3060] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.763644 13351 raft_consensus.cc:515] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.765251 13351 leader_election.cc:290] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:57.766258 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06488c29a51d4bcda1841bdc7fd32b6d" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:57.766520 13176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06488c29a51d4bcda1841bdc7fd32b6d" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:57.766870 13028 raft_consensus.cc:3060] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.767128 13176 raft_consensus.cc:3060] T 06488c29a51d4bcda1841bdc7fd32b6d P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.771991 13028 raft_consensus.cc:2468] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 1.
I20260709 13:07:57.772111 13176 raft_consensus.cc:2468] T 06488c29a51d4bcda1841bdc7fd32b6d P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 1.
I20260709 13:07:57.773339 13069 leader_election.cc:304] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 824d687f218b4cb091e5d62fa9d18187; no voters: 
I20260709 13:07:57.774210 13351 raft_consensus.cc:2804] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:57.774745 13351 raft_consensus.cc:697] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Becoming Leader. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:07:57.775456 13351 consensus_queue.cc:237] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.783360 12925 catalog_manager.cc:5697] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "20f53f29f26f48d6b818f46c9e755f69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:57.788703 13331 consensus_queue.cc:1048] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:57.799232 13340 raft_consensus.cc:493] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.799904 13331 consensus_queue.cc:1048] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:57.799813 13340 raft_consensus.cc:515] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:57.803715 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b76fdbda6342bd887202061755151b" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411" is_pre_election: true
I20260709 13:07:57.804525 13028 raft_consensus.cc:2393] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 824d687f218b4cb091e5d62fa9d18187 in current term 1: Already voted for candidate 20f53f29f26f48d6b818f46c9e755f69 in this term.
I20260709 13:07:57.806200 13340 leader_election.cc:290] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507)
I20260709 13:07:57.806603 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b76fdbda6342bd887202061755151b" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f53f29f26f48d6b818f46c9e755f69" is_pre_election: true
I20260709 13:07:57.808584 13141 leader_election.cc:304] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187; no voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411
I20260709 13:07:57.810093 13340 raft_consensus.cc:3060] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.816488 13340 raft_consensus.cc:2749] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:57.825758 13340 consensus_queue.cc:1048] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:57.836926 13340 consensus_queue.cc:1048] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:57.880455 13305 consensus_queue.cc:1048] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:57.890969 13305 consensus_queue.cc:1048] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:57.896659 13351 consensus_queue.cc:1048] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:57.909729 13321 consensus_queue.cc:1048] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:57.963446 13321 consensus_queue.cc:1048] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:57.974431 13339 consensus_queue.cc:1048] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:57.974684 13101 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32b76fdbda6342bd887202061755151b"
dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
mode: GRACEFUL
new_leader_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
 from {username='slave'} at 127.0.0.1:40832
I20260709 13:07:57.975349 13101 raft_consensus.cc:606] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Received request to transfer leadership to TS 8593949d5fb7413ca30c9ef5dfd31411
I20260709 13:07:57.986142 13339 consensus_queue.cc:1048] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:57.986770 13101 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43c7f51f72dc4e92b878a4ed950d4c5b"
dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
mode: GRACEFUL
new_leader_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
 from {username='slave'} at 127.0.0.1:40832
I20260709 13:07:57.987320 13101 raft_consensus.cc:606] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Received request to transfer leadership to TS 8593949d5fb7413ca30c9ef5dfd31411
I20260709 13:07:57.992769 13101 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06488c29a51d4bcda1841bdc7fd32b6d"
dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
mode: GRACEFUL
new_leader_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
 from {username='slave'} at 127.0.0.1:40832
I20260709 13:07:57.993203 13101 raft_consensus.cc:606] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Received request to transfer leadership to TS 8593949d5fb7413ca30c9ef5dfd31411
I20260709 13:07:57.999352 13321 consensus_queue.cc:1048] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:58.001047 13176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed00ffda0221409fbcadb48892fcbadc"
dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
mode: GRACEFUL
new_leader_uuid: "20f53f29f26f48d6b818f46c9e755f69"
 from {username='slave'} at 127.0.0.1:55890
I20260709 13:07:58.001739 13176 raft_consensus.cc:606] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [term 1 LEADER]: Received request to transfer leadership to TS 20f53f29f26f48d6b818f46c9e755f69
I20260709 13:07:58.012281 13177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c506771cdafd4db99eaf39cc5d9015ac"
dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
mode: GRACEFUL
new_leader_uuid: "20f53f29f26f48d6b818f46c9e755f69"
 from {username='slave'} at 127.0.0.1:55890
I20260709 13:07:58.012818 13177 raft_consensus.cc:606] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [term 1 LEADER]: Received request to transfer leadership to TS 20f53f29f26f48d6b818f46c9e755f69
I20260709 13:07:58.018064 13026 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4f2dbd8b1bd424e95b9a5ee46d2ea5b"
dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
mode: GRACEFUL
new_leader_uuid: "20f53f29f26f48d6b818f46c9e755f69"
 from {username='slave'} at 127.0.0.1:55266
I20260709 13:07:58.018623 13026 raft_consensus.cc:606] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Received request to transfer leadership to TS 20f53f29f26f48d6b818f46c9e755f69
I20260709 13:07:58.023720 13026 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "daf0cc5885a14c65b1bd2fdab90c9b26"
dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
mode: GRACEFUL
new_leader_uuid: "20f53f29f26f48d6b818f46c9e755f69"
 from {username='slave'} at 127.0.0.1:55266
I20260709 13:07:58.024168 13026 raft_consensus.cc:606] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Received request to transfer leadership to TS 20f53f29f26f48d6b818f46c9e755f69
I20260709 13:07:58.029187 13101 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ffaa4a08cf24ed3ab65b5aecd4c3721"
dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
mode: GRACEFUL
new_leader_uuid: "824d687f218b4cb091e5d62fa9d18187"
 from {username='slave'} at 127.0.0.1:40832
I20260709 13:07:58.029814 13101 raft_consensus.cc:606] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Received request to transfer leadership to TS 824d687f218b4cb091e5d62fa9d18187
I20260709 13:07:58.156366 13305 consensus_queue.cc:1048] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:58.168134 13332 consensus_queue.cc:1048] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:58.197016 13321 consensus_queue.cc:1048] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:58.201275 13340 consensus_queue.cc:1048] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:58.204052 13351 raft_consensus.cc:993] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69: : Instructing follower 8593949d5fb7413ca30c9ef5dfd31411 to start an election
I20260709 13:07:58.204567 13225 raft_consensus.cc:1081] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Signalling peer 8593949d5fb7413ca30c9ef5dfd31411 to start an election
I20260709 13:07:58.208211 13026 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06488c29a51d4bcda1841bdc7fd32b6d"
dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
 from {username='slave'} at 127.0.0.1:55258
I20260709 13:07:58.208806 13026 raft_consensus.cc:493] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:58.209189 13026 raft_consensus.cc:3060] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.210625 13339 consensus_queue.cc:1048] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:58.222833 13026 raft_consensus.cc:515] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.227180 13302 consensus_queue.cc:1048] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:58.229368 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06488c29a51d4bcda1841bdc7fd32b6d" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:58.230013 13026 leader_election.cc:290] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 2 election: Requested vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:58.230170 13177 raft_consensus.cc:3060] T 06488c29a51d4bcda1841bdc7fd32b6d P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.229378 13101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06488c29a51d4bcda1841bdc7fd32b6d" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:07:58.231300 13101 raft_consensus.cc:3055] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:58.231637 13101 raft_consensus.cc:740] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:07:58.232460 13101 consensus_queue.cc:260] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.233587 13101 raft_consensus.cc:3060] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.241514 13177 raft_consensus.cc:2468] T 06488c29a51d4bcda1841bdc7fd32b6d P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 2.
I20260709 13:07:58.242971 12994 leader_election.cc:304] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:58.243456 13101 raft_consensus.cc:2468] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 2.
I20260709 13:07:58.244558 13331 raft_consensus.cc:2804] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:58.245072 13331 raft_consensus.cc:697] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 LEADER]: Becoming Leader. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Running, Role: LEADER
I20260709 13:07:58.245962 13331 consensus_queue.cc:237] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.251238 13340 raft_consensus.cc:993] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187: : Instructing follower 20f53f29f26f48d6b818f46c9e755f69 to start an election
I20260709 13:07:58.251778 13340 raft_consensus.cc:1081] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [term 1 LEADER]: Signalling peer 20f53f29f26f48d6b818f46c9e755f69 to start an election
I20260709 13:07:58.253791 12925 catalog_manager.cc:5697] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411 reported cstate change: term changed from 1 to 2, leader changed from 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194) to 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "8593949d5fb7413ca30c9ef5dfd31411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:58.255264 13103 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed00ffda0221409fbcadb48892fcbadc"
dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
 from {username='slave'} at 127.0.0.1:40816
I20260709 13:07:58.255822 13103 raft_consensus.cc:493] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:58.256268 13103 raft_consensus.cc:3060] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.262734 13103 raft_consensus.cc:515] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.264462 13103 leader_election.cc:290] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 2 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:58.265262 13026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed00ffda0221409fbcadb48892fcbadc" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:58.265511 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed00ffda0221409fbcadb48892fcbadc" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:58.265987 13026 raft_consensus.cc:3060] T ed00ffda0221409fbcadb48892fcbadc P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.266093 13177 raft_consensus.cc:3055] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:58.266512 13177 raft_consensus.cc:740] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Running, Role: LEADER
I20260709 13:07:58.267341 13177 consensus_queue.cc:260] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.268615 13177 raft_consensus.cc:3060] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.273419 13026 raft_consensus.cc:2468] T ed00ffda0221409fbcadb48892fcbadc P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 2.
I20260709 13:07:58.274854 13066 leader_election.cc:304] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:58.275324 13177 raft_consensus.cc:2468] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 2.
I20260709 13:07:58.275554 13339 raft_consensus.cc:2804] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:58.276124 13339 raft_consensus.cc:697] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [term 2 LEADER]: Becoming Leader. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:07:58.276894 13339 consensus_queue.cc:237] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.283601 12925 catalog_manager.cc:5697] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 reported cstate change: term changed from 1 to 2, leader changed from 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195) to 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "20f53f29f26f48d6b818f46c9e755f69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:58.294903 13302 raft_consensus.cc:993] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187: : Instructing follower 20f53f29f26f48d6b818f46c9e755f69 to start an election
I20260709 13:07:58.295323 13302 raft_consensus.cc:1081] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [term 1 LEADER]: Signalling peer 20f53f29f26f48d6b818f46c9e755f69 to start an election
I20260709 13:07:58.296728 13103 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c506771cdafd4db99eaf39cc5d9015ac"
dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
 from {username='slave'} at 127.0.0.1:40816
I20260709 13:07:58.297257 13103 raft_consensus.cc:493] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:58.297626 13103 raft_consensus.cc:3060] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.299685 13332 raft_consensus.cc:993] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411: : Instructing follower 20f53f29f26f48d6b818f46c9e755f69 to start an election
I20260709 13:07:58.300251 13331 raft_consensus.cc:1081] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Signalling peer 20f53f29f26f48d6b818f46c9e755f69 to start an election
I20260709 13:07:58.302469 13101 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "daf0cc5885a14c65b1bd2fdab90c9b26"
dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
 from {username='slave'} at 127.0.0.1:40804
I20260709 13:07:58.303017 13101 raft_consensus.cc:493] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:58.303416 13101 raft_consensus.cc:3060] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.304708 13103 raft_consensus.cc:515] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.306507 13103 leader_election.cc:290] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 2 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:58.307287 13026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c506771cdafd4db99eaf39cc5d9015ac" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:58.307788 13175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c506771cdafd4db99eaf39cc5d9015ac" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:58.307920 13026 raft_consensus.cc:3060] T c506771cdafd4db99eaf39cc5d9015ac P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.308460 13175 raft_consensus.cc:3055] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:58.308842 13175 raft_consensus.cc:740] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Running, Role: LEADER
I20260709 13:07:58.309564 13175 consensus_queue.cc:260] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.310582 13175 raft_consensus.cc:3060] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.311105 13101 raft_consensus.cc:515] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.314888 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daf0cc5885a14c65b1bd2fdab90c9b26" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:58.315027 13026 raft_consensus.cc:2468] T c506771cdafd4db99eaf39cc5d9015ac P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 2.
I20260709 13:07:58.315454 13028 raft_consensus.cc:3055] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:58.315766 13028 raft_consensus.cc:740] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Running, Role: LEADER
I20260709 13:07:58.315909 13101 leader_election.cc:290] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 2 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:58.316206 13066 leader_election.cc:304] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:58.316507 13028 consensus_queue.cc:260] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.317124 13339 raft_consensus.cc:2804] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:58.317070 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daf0cc5885a14c65b1bd2fdab90c9b26" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:58.317799 13339 raft_consensus.cc:697] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69 [term 2 LEADER]: Becoming Leader. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:07:58.317032 13175 raft_consensus.cc:2468] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 2.
I20260709 13:07:58.317927 13177 raft_consensus.cc:3060] T daf0cc5885a14c65b1bd2fdab90c9b26 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.318099 13028 raft_consensus.cc:3060] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.318763 13339 consensus_queue.cc:237] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.323789 13177 raft_consensus.cc:2468] T daf0cc5885a14c65b1bd2fdab90c9b26 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 2.
I20260709 13:07:58.325044 13069 leader_election.cc:304] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 824d687f218b4cb091e5d62fa9d18187; no voters: 
I20260709 13:07:58.325831 13351 raft_consensus.cc:2804] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:58.326332 13351 raft_consensus.cc:697] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 LEADER]: Becoming Leader. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:07:58.326651 12925 catalog_manager.cc:5697] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69 reported cstate change: term changed from 1 to 2, leader changed from 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195) to 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "20f53f29f26f48d6b818f46c9e755f69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:58.327126 13351 consensus_queue.cc:237] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.330541 13028 raft_consensus.cc:2468] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 2.
I20260709 13:07:58.334262 13351 raft_consensus.cc:993] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69: : Instructing follower 8593949d5fb7413ca30c9ef5dfd31411 to start an election
I20260709 13:07:58.334734 13225 raft_consensus.cc:1081] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Signalling peer 8593949d5fb7413ca30c9ef5dfd31411 to start an election
I20260709 13:07:58.336375 13028 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32b76fdbda6342bd887202061755151b"
dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
 from {username='slave'} at 127.0.0.1:55258
I20260709 13:07:58.336855 13028 raft_consensus.cc:493] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:58.337175 13028 raft_consensus.cc:3060] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.341372 12925 catalog_manager.cc:5697] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 reported cstate change: term changed from 1 to 2, leader changed from 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193) to 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "20f53f29f26f48d6b818f46c9e755f69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:58.343505 13028 raft_consensus.cc:515] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.345036 13028 leader_election.cc:290] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 2 election: Requested vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:58.345930 13101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b76fdbda6342bd887202061755151b" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:07:58.346117 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b76fdbda6342bd887202061755151b" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:58.346643 13177 raft_consensus.cc:3060] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.346592 13101 raft_consensus.cc:3055] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:58.347133 13101 raft_consensus.cc:740] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:07:58.347921 13101 consensus_queue.cc:260] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.348879 13101 raft_consensus.cc:3060] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.354125 13177 raft_consensus.cc:2468] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 2.
I20260709 13:07:58.354362 13101 raft_consensus.cc:2468] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 2.
I20260709 13:07:58.355111 12994 leader_election.cc:304] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:58.356027 13332 raft_consensus.cc:2804] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:58.356487 13332 raft_consensus.cc:697] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 LEADER]: Becoming Leader. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Running, Role: LEADER
I20260709 13:07:58.357247 13332 consensus_queue.cc:237] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.363631 12925 catalog_manager.cc:5697] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 reported cstate change: term changed from 1 to 2, leader changed from 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194) to 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "8593949d5fb7413ca30c9ef5dfd31411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:58.408280 13305 raft_consensus.cc:993] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411: : Instructing follower 20f53f29f26f48d6b818f46c9e755f69 to start an election
I20260709 13:07:58.408783 13331 raft_consensus.cc:1081] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Signalling peer 20f53f29f26f48d6b818f46c9e755f69 to start an election
I20260709 13:07:58.410400 13103 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4f2dbd8b1bd424e95b9a5ee46d2ea5b"
dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
 from {username='slave'} at 127.0.0.1:40804
I20260709 13:07:58.410907 13103 raft_consensus.cc:493] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:58.411237 13103 raft_consensus.cc:3060] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.416172 13103 raft_consensus.cc:515] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.417737 13103 leader_election.cc:290] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 2 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:58.418573 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f2dbd8b1bd424e95b9a5ee46d2ea5b" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:58.418706 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f2dbd8b1bd424e95b9a5ee46d2ea5b" candidate_uuid: "20f53f29f26f48d6b818f46c9e755f69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:58.419224 13028 raft_consensus.cc:3055] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:58.419284 13177 raft_consensus.cc:3060] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.419499 13028 raft_consensus.cc:740] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Running, Role: LEADER
I20260709 13:07:58.420240 13028 consensus_queue.cc:260] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.421094 13028 raft_consensus.cc:3060] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.423815 13177 raft_consensus.cc:2468] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 2.
I20260709 13:07:58.424819 13069 leader_election.cc:304] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f53f29f26f48d6b818f46c9e755f69, 824d687f218b4cb091e5d62fa9d18187; no voters: 
I20260709 13:07:58.425525 13225 raft_consensus.cc:2804] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:58.425983 13225 raft_consensus.cc:697] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [term 2 LEADER]: Becoming Leader. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:07:58.426414 13028 raft_consensus.cc:2468] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f53f29f26f48d6b818f46c9e755f69 in term 2.
I20260709 13:07:58.426688 13225 consensus_queue.cc:237] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.433092 12925 catalog_manager.cc:5697] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 reported cstate change: term changed from 1 to 2, leader changed from 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193) to 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "20f53f29f26f48d6b818f46c9e755f69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:58.476884 13225 raft_consensus.cc:993] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69: : Instructing follower 8593949d5fb7413ca30c9ef5dfd31411 to start an election
I20260709 13:07:58.477384 13321 raft_consensus.cc:1081] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Signalling peer 8593949d5fb7413ca30c9ef5dfd31411 to start an election
I20260709 13:07:58.479274 13026 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43c7f51f72dc4e92b878a4ed950d4c5b"
dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
 from {username='slave'} at 127.0.0.1:55258
I20260709 13:07:58.479692 13026 raft_consensus.cc:493] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:58.479933 13026 raft_consensus.cc:3060] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.486178 13026 raft_consensus.cc:515] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.487913 13026 leader_election.cc:290] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 2 election: Requested vote from peers 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507), 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195:44499)
I20260709 13:07:58.488754 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c7f51f72dc4e92b878a4ed950d4c5b" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:07:58.488891 13177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c7f51f72dc4e92b878a4ed950d4c5b" candidate_uuid: "8593949d5fb7413ca30c9ef5dfd31411" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
I20260709 13:07:58.489349 13103 raft_consensus.cc:3055] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:58.489439 13177 raft_consensus.cc:3060] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.489697 13103 raft_consensus.cc:740] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:07:58.490449 13103 consensus_queue.cc:260] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.491245 13103 raft_consensus.cc:3060] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.494773 13177 raft_consensus.cc:2468] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 2.
I20260709 13:07:58.496256 13103 raft_consensus.cc:2468] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8593949d5fb7413ca30c9ef5dfd31411 in term 2.
I20260709 13:07:58.496904 12994 leader_election.cc:304] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:58.497974 13331 raft_consensus.cc:2804] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:58.498442 13331 raft_consensus.cc:697] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 LEADER]: Becoming Leader. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Running, Role: LEADER
I20260709 13:07:58.499207 13331 consensus_queue.cc:237] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.508412 12925 catalog_manager.cc:5697] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 reported cstate change: term changed from 1 to 2, leader changed from 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194) to 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "8593949d5fb7413ca30c9ef5dfd31411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:58.609190 13225 raft_consensus.cc:993] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69: : Instructing follower 824d687f218b4cb091e5d62fa9d18187 to start an election
I20260709 13:07:58.609673 13321 raft_consensus.cc:1081] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Signalling peer 824d687f218b4cb091e5d62fa9d18187 to start an election
I20260709 13:07:58.611286 13177 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ffaa4a08cf24ed3ab65b5aecd4c3721"
dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
 from {username='slave'} at 127.0.0.1:55882
I20260709 13:07:58.611826 13177 raft_consensus.cc:493] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:58.612164 13177 raft_consensus.cc:3060] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.616736 13177 raft_consensus.cc:515] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.618127 13177 leader_election.cc:290] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 2 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507)
I20260709 13:07:58.619004 13026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ffaa4a08cf24ed3ab65b5aecd4c3721" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:07:58.619237 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ffaa4a08cf24ed3ab65b5aecd4c3721" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:07:58.619683 13026 raft_consensus.cc:3060] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.619771 13103 raft_consensus.cc:3055] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:58.620175 13103 raft_consensus.cc:740] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:07:58.621031 13103 consensus_queue.cc:260] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.622299 13103 raft_consensus.cc:3060] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.627278 13026 raft_consensus.cc:2468] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 2.
I20260709 13:07:58.628486 13140 leader_election.cc:304] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:07:58.629302 13302 raft_consensus.cc:2804] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:58.629346 13103 raft_consensus.cc:2468] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 2.
I20260709 13:07:58.629859 13302 raft_consensus.cc:697] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [term 2 LEADER]: Becoming Leader. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Running, Role: LEADER
I20260709 13:07:58.630699 13302 consensus_queue.cc:237] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:07:58.637898 12925 catalog_manager.cc:5697] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 reported cstate change: term changed from 1 to 2, leader changed from 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194) to 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "824d687f218b4cb091e5d62fa9d18187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:58.711954 13177 raft_consensus.cc:1275] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Refusing update from remote peer 20f53f29f26f48d6b818f46c9e755f69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:58.713297 13321 consensus_queue.cc:1048] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:58.718262 13175 raft_consensus.cc:1275] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Refusing update from remote peer 20f53f29f26f48d6b818f46c9e755f69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:58.719954 13339 consensus_queue.cc:1048] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:58.724005 13026 raft_consensus.cc:1275] T ed00ffda0221409fbcadb48892fcbadc P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Refusing update from remote peer 20f53f29f26f48d6b818f46c9e755f69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:58.725731 13351 consensus_queue.cc:1048] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:58.737865 13026 raft_consensus.cc:1275] T c506771cdafd4db99eaf39cc5d9015ac P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Refusing update from remote peer 20f53f29f26f48d6b818f46c9e755f69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:58.739789 13339 consensus_queue.cc:1048] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:58.762645 13103 raft_consensus.cc:1275] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Refusing update from remote peer 8593949d5fb7413ca30c9ef5dfd31411: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:58.763911 13331 consensus_queue.cc:1048] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:58.772238 13175 raft_consensus.cc:1275] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Refusing update from remote peer 8593949d5fb7413ca30c9ef5dfd31411: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:58.772724 13026 raft_consensus.cc:1275] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Refusing update from remote peer 20f53f29f26f48d6b818f46c9e755f69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:58.774011 13305 consensus_queue.cc:1048] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:58.774396 13339 consensus_queue.cc:1048] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:58.783972 13175 raft_consensus.cc:1275] T daf0cc5885a14c65b1bd2fdab90c9b26 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Refusing update from remote peer 20f53f29f26f48d6b818f46c9e755f69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:58.785579 13225 consensus_queue.cc:1048] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:58.827781 13175 raft_consensus.cc:1275] T 06488c29a51d4bcda1841bdc7fd32b6d P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Refusing update from remote peer 8593949d5fb7413ca30c9ef5dfd31411: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:58.829020 13332 consensus_queue.cc:1048] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:58.836623 13103 raft_consensus.cc:1275] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Refusing update from remote peer 8593949d5fb7413ca30c9ef5dfd31411: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:58.838584 13391 consensus_queue.cc:1048] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:58.944021 13026 raft_consensus.cc:1275] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Refusing update from remote peer 20f53f29f26f48d6b818f46c9e755f69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:58.945508 13225 consensus_queue.cc:1048] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:58.954217 13175 raft_consensus.cc:1275] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Refusing update from remote peer 20f53f29f26f48d6b818f46c9e755f69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:58.955981 13225 consensus_queue.cc:1048] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:58.967253 13103 raft_consensus.cc:1275] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Refusing update from remote peer 8593949d5fb7413ca30c9ef5dfd31411: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:58.968657 13391 consensus_queue.cc:1048] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:58.979189 13175 raft_consensus.cc:1275] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Refusing update from remote peer 8593949d5fb7413ca30c9ef5dfd31411: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:58.980930 13331 consensus_queue.cc:1048] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:59.042666 13103 raft_consensus.cc:1275] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Refusing update from remote peer 824d687f218b4cb091e5d62fa9d18187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:59.044428 13302 consensus_queue.cc:1048] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:59.055531 13026 raft_consensus.cc:1275] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Refusing update from remote peer 824d687f218b4cb091e5d62fa9d18187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:59.056996 13302 consensus_queue.cc:1048] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:01.033109 12924 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58384:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:08:01.057219 12924 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58384:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:08:01.137259   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:08:01.232092 13413 log.cc:927] Time spent T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69: Append to log took a long time: real 0.063s	user 0.002s	sys 0.000s
W20260709 13:08:01.276206 13410 log.cc:927] Time spent T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:08:01.341296 13439 tablet.cc:1724] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.379870 13449 tablet.cc:1724] T c506771cdafd4db99eaf39cc5d9015ac P 8593949d5fb7413ca30c9ef5dfd31411: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:08:01.424539 13437 log.cc:927] Time spent T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187: Append to log took a long time: real 0.112s	user 0.002s	sys 0.000s
I20260709 13:08:01.436156 13458 tablet.cc:1724] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.468457 13455 tablet.cc:1724] T 790b9908004d4336a3093ecfe11ea707 P 20f53f29f26f48d6b818f46c9e755f69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.465761 13460 tablet.cc:1724] T ed00ffda0221409fbcadb48892fcbadc P 8593949d5fb7413ca30c9ef5dfd31411: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.493957 13463 tablet.cc:1724] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.496682 13101 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d393f71c2afb406098b92973cb2be621"
dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
mode: GRACEFUL
new_leader_uuid: "824d687f218b4cb091e5d62fa9d18187"
 from {username='slave'} at 127.0.0.1:35430
I20260709 13:08:01.497789 13101 raft_consensus.cc:606] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 LEADER]: Received request to transfer leadership to TS 824d687f218b4cb091e5d62fa9d18187
I20260709 13:08:01.500427 13465 tablet.cc:1724] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.506632 13461 tablet.cc:1724] T 06488c29a51d4bcda1841bdc7fd32b6d P 824d687f218b4cb091e5d62fa9d18187: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.452872 13451 tablet.cc:1724] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:08:01.527244 13464 log.cc:927] Time spent T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411: Append to log took a long time: real 0.078s	user 0.002s	sys 0.000s
I20260709 13:08:01.545256 13459 tablet.cc:1724] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:08:01.486294 13141 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:46507, user_credentials={real_user=slave}} blocked reactor thread for 97339.2us
I20260709 13:08:01.555630 13470 tablet.cc:1724] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.571743 13470 tablet.cc:1724] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:08:01.558678 12994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 69867.9us
I20260709 13:08:01.576906 13439 tablet.cc:1724] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.554183 13468 tablet.cc:1724] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.606392 13455 tablet.cc:1724] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.611689 13459 tablet.cc:1724] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.585762 13449 tablet.cc:1724] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.605408 13463 tablet.cc:1724] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.632267 13460 tablet.cc:1724] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 8593949d5fb7413ca30c9ef5dfd31411: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.622830 13451 tablet.cc:1724] T daf0cc5885a14c65b1bd2fdab90c9b26 P 824d687f218b4cb091e5d62fa9d18187: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.624778 13465 tablet.cc:1724] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 824d687f218b4cb091e5d62fa9d18187: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.641880 13468 tablet.cc:1724] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.658660 13470 tablet.cc:1724] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.640810 13458 tablet.cc:1724] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.674181 13461 tablet.cc:1724] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.683877 13028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e175694e0fa6424f900caf84dbb2945f"
dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
mode: GRACEFUL
new_leader_uuid: "824d687f218b4cb091e5d62fa9d18187"
 from {username='slave'} at 127.0.0.1:58832
I20260709 13:08:01.693419 13028 raft_consensus.cc:606] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 LEADER]: Received request to transfer leadership to TS 824d687f218b4cb091e5d62fa9d18187
I20260709 13:08:01.685767 13449 tablet.cc:1724] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.695760   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:08:01.696460   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:01.690471 13465 tablet.cc:1724] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:01.728897 12923 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813: soft_deleted_table [id=734a02cae49e49a0aafd16335e92a3d8] alter complete (version 1)
I20260709 13:08:01.895011 13419 raft_consensus.cc:993] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69: : Instructing follower 824d687f218b4cb091e5d62fa9d18187 to start an election
I20260709 13:08:01.895556 13419 raft_consensus.cc:1081] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 LEADER]: Signalling peer 824d687f218b4cb091e5d62fa9d18187 to start an election
I20260709 13:08:01.897425 13175 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d393f71c2afb406098b92973cb2be621"
dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
 from {username='slave'} at 127.0.0.1:55882
I20260709 13:08:01.898051 13175 raft_consensus.cc:493] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:01.898360 13175 raft_consensus.cc:3060] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:01.904371 13175 raft_consensus.cc:515] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:08:01.906351 13175 leader_election.cc:290] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 3 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507)
I20260709 13:08:01.907563 13102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d393f71c2afb406098b92973cb2be621" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:08:01.907258 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d393f71c2afb406098b92973cb2be621" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:08:01.908288 13102 raft_consensus.cc:3055] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:01.908335 13028 raft_consensus.cc:3060] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:01.908815 13102 raft_consensus.cc:740] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20f53f29f26f48d6b818f46c9e755f69, State: Running, Role: LEADER
I20260709 13:08:01.909878 13102 consensus_queue.cc:260] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:08:01.911311 13102 raft_consensus.cc:3060] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:01.916692 13028 raft_consensus.cc:2468] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 3.
I20260709 13:08:01.918277 13140 leader_election.cc:304] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:08:01.919011 13102 raft_consensus.cc:2468] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 3.
I20260709 13:08:01.919312 13457 raft_consensus.cc:2804] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:01.919873 13457 raft_consensus.cc:697] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 3 LEADER]: Becoming Leader. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Running, Role: LEADER
I20260709 13:08:01.920848 13457 consensus_queue.cc:237] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:08:01.928602 13471 raft_consensus.cc:993] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411: : Instructing follower 824d687f218b4cb091e5d62fa9d18187 to start an election
I20260709 13:08:01.929231 13469 raft_consensus.cc:1081] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 LEADER]: Signalling peer 824d687f218b4cb091e5d62fa9d18187 to start an election
I20260709 13:08:01.930794 13175 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e175694e0fa6424f900caf84dbb2945f"
dest_uuid: "824d687f218b4cb091e5d62fa9d18187"
 from {username='slave'} at 127.0.0.1:55876
I20260709 13:08:01.931259 13175 raft_consensus.cc:493] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:01.931640 13175 raft_consensus.cc:3060] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:01.931424 12923 catalog_manager.cc:5697] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 reported cstate change: term changed from 2 to 3, leader changed from 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194) to 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "824d687f218b4cb091e5d62fa9d18187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:01.938144 13175 raft_consensus.cc:515] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:08:01.941072 13175 leader_election.cc:290] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 3 election: Requested vote from peers 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253), 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507)
I20260709 13:08:01.941958 13028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e175694e0fa6424f900caf84dbb2945f" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8593949d5fb7413ca30c9ef5dfd31411"
I20260709 13:08:01.942570 13028 raft_consensus.cc:3055] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:01.942876 13028 raft_consensus.cc:740] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8593949d5fb7413ca30c9ef5dfd31411, State: Running, Role: LEADER
I20260709 13:08:01.943674 13028 consensus_queue.cc:260] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:08:01.944548 13102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e175694e0fa6424f900caf84dbb2945f" candidate_uuid: "824d687f218b4cb091e5d62fa9d18187" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "20f53f29f26f48d6b818f46c9e755f69"
I20260709 13:08:01.944811 13028 raft_consensus.cc:3060] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:01.945191 13102 raft_consensus.cc:3060] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:01.949810 13028 raft_consensus.cc:2468] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 3.
I20260709 13:08:01.950667 13102 raft_consensus.cc:2468] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 824d687f218b4cb091e5d62fa9d18187 in term 3.
I20260709 13:08:01.951076 13140 leader_election.cc:304] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 824d687f218b4cb091e5d62fa9d18187, 8593949d5fb7413ca30c9ef5dfd31411; no voters: 
I20260709 13:08:01.952054 13457 raft_consensus.cc:2804] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:01.952540 13457 raft_consensus.cc:697] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [term 3 LEADER]: Becoming Leader. State: Replica: 824d687f218b4cb091e5d62fa9d18187, State: Running, Role: LEADER
I20260709 13:08:01.953429 13457 consensus_queue.cc:237] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } }
I20260709 13:08:01.960383 12923 catalog_manager.cc:5697] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 reported cstate change: term changed from 2 to 3, leader changed from 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193) to 824d687f218b4cb091e5d62fa9d18187 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "824d687f218b4cb091e5d62fa9d18187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "824d687f218b4cb091e5d62fa9d18187" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44499 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:02.337898 13027 raft_consensus.cc:1275] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411 [term 3 FOLLOWER]: Refusing update from remote peer 824d687f218b4cb091e5d62fa9d18187: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:02.339185 13457 consensus_queue.cc:1048] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:02.351293 13102 raft_consensus.cc:1275] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 3 FOLLOWER]: Refusing update from remote peer 824d687f218b4cb091e5d62fa9d18187: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:02.353021 13480 consensus_queue.cc:1048] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:02.483441 13027 raft_consensus.cc:1275] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 3 FOLLOWER]: Refusing update from remote peer 824d687f218b4cb091e5d62fa9d18187: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:02.484787 13480 consensus_queue.cc:1048] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593949d5fb7413ca30c9ef5dfd31411" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:02.497611 13102 raft_consensus.cc:1275] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 3 FOLLOWER]: Refusing update from remote peer 824d687f218b4cb091e5d62fa9d18187: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:02.499336 13485 consensus_queue.cc:1048] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f53f29f26f48d6b818f46c9e755f69" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:02.775851   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:08:02.822088   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:02.823300   431 tablet_replica.cc:333] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411: stopping tablet replica
I20260709 13:08:02.824474   431 raft_consensus.cc:2243] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:02.825690   431 raft_consensus.cc:2272] T 43c7f51f72dc4e92b878a4ed950d4c5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:02.828429   431 tablet_replica.cc:333] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411: stopping tablet replica
I20260709 13:08:02.829193   431 raft_consensus.cc:2243] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:02.830109   431 raft_consensus.cc:2272] T 06488c29a51d4bcda1841bdc7fd32b6d P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:02.833216   431 tablet_replica.cc:333] T ed00ffda0221409fbcadb48892fcbadc P 8593949d5fb7413ca30c9ef5dfd31411: stopping tablet replica
I20260709 13:08:02.833966   431 raft_consensus.cc:2243] T ed00ffda0221409fbcadb48892fcbadc P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:02.834481   431 raft_consensus.cc:2272] T ed00ffda0221409fbcadb48892fcbadc P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:02.838405   431 tablet_replica.cc:333] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 8593949d5fb7413ca30c9ef5dfd31411: stopping tablet replica
I20260709 13:08:02.839737   431 raft_consensus.cc:2243] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:02.840425   431 raft_consensus.cc:2272] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:02.843839   431 tablet_replica.cc:333] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411: stopping tablet replica
I20260709 13:08:02.844918   431 raft_consensus.cc:2243] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:02.845582   431 raft_consensus.cc:2272] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:02.849793   431 tablet_replica.cc:333] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411: stopping tablet replica
I20260709 13:08:02.851287   431 raft_consensus.cc:2243] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:02.852033   431 raft_consensus.cc:2272] T d393f71c2afb406098b92973cb2be621 P 8593949d5fb7413ca30c9ef5dfd31411 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:02.856218   431 tablet_replica.cc:333] T eab0853d0dd94bcf8327c0ef09c6fad4 P 8593949d5fb7413ca30c9ef5dfd31411: stopping tablet replica
I20260709 13:08:02.857177   431 raft_consensus.cc:2243] T eab0853d0dd94bcf8327c0ef09c6fad4 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:02.857918   431 raft_consensus.cc:2272] T eab0853d0dd94bcf8327c0ef09c6fad4 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:02.861706   431 tablet_replica.cc:333] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411: stopping tablet replica
I20260709 13:08:02.862602   431 raft_consensus.cc:2243] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:02.863952   431 raft_consensus.cc:2272] T d727f4f4eacb428889625360bb663857 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:02.867332   431 tablet_replica.cc:333] T 4448333d61c640e494f7ae000878dbdf P 8593949d5fb7413ca30c9ef5dfd31411: stopping tablet replica
I20260709 13:08:02.868057   431 raft_consensus.cc:2243] T 4448333d61c640e494f7ae000878dbdf P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:02.868719   431 raft_consensus.cc:2272] T 4448333d61c640e494f7ae000878dbdf P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:02.871308   431 tablet_replica.cc:333] T c506771cdafd4db99eaf39cc5d9015ac P 8593949d5fb7413ca30c9ef5dfd31411: stopping tablet replica
I20260709 13:08:02.872027   431 raft_consensus.cc:2243] T c506771cdafd4db99eaf39cc5d9015ac P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:02.872514   431 raft_consensus.cc:2272] T c506771cdafd4db99eaf39cc5d9015ac P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:02.875352   431 tablet_replica.cc:333] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411: stopping tablet replica
I20260709 13:08:02.875988   431 raft_consensus.cc:2243] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:02.876856   431 raft_consensus.cc:2272] T 32b76fdbda6342bd887202061755151b P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:02.879482   431 tablet_replica.cc:333] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411: stopping tablet replica
I20260709 13:08:02.880035   431 raft_consensus.cc:2243] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:02.880474   431 raft_consensus.cc:2272] T daf0cc5885a14c65b1bd2fdab90c9b26 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:02.883193   431 tablet_replica.cc:333] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411: stopping tablet replica
I20260709 13:08:02.884186   431 raft_consensus.cc:2243] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:02.884730   431 raft_consensus.cc:2272] T 0d8b4f1a658d47cc954ba897df965354 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:02.887218   431 tablet_replica.cc:333] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411: stopping tablet replica
I20260709 13:08:02.888186   431 raft_consensus.cc:2243] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:02.889305   431 raft_consensus.cc:2272] T 68f4b09178c244199a36c43c3989a45b P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:02.891822   431 tablet_replica.cc:333] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411: stopping tablet replica
I20260709 13:08:02.892572   431 raft_consensus.cc:2243] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:02.893276   431 raft_consensus.cc:2272] T e175694e0fa6424f900caf84dbb2945f P 8593949d5fb7413ca30c9ef5dfd31411 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:02.896860 13066 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 13 similar messages]
I20260709 13:08:02.897215   431 tablet_replica.cc:333] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411: stopping tablet replica
I20260709 13:08:02.898332   431 raft_consensus.cc:2243] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:02.899080   431 raft_consensus.cc:2272] T a22a6947bab14423aa83c01130c20d77 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:02.901043 13066 consensus_peers.cc:597] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 -> Peer 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253): Couldn't send request to peer 8593949d5fb7413ca30c9ef5dfd31411. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:02.902235   431 tablet_replica.cc:333] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411: stopping tablet replica
I20260709 13:08:02.902822   431 raft_consensus.cc:2243] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:02.903669   431 raft_consensus.cc:2272] T 790b9908004d4336a3093ecfe11ea707 P 8593949d5fb7413ca30c9ef5dfd31411 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:02.906100   431 tablet_replica.cc:333] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411: stopping tablet replica
I20260709 13:08:02.906576   431 raft_consensus.cc:2243] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:02.907382   431 raft_consensus.cc:2272] T f5fffab9019146fd9cc617253eb19f87 P 8593949d5fb7413ca30c9ef5dfd31411 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:02.956542   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:08:02.960615 13140 consensus_peers.cc:597] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 -> Peer 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253): Couldn't send request to peer 8593949d5fb7413ca30c9ef5dfd31411. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:02.971832 13140 consensus_peers.cc:597] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 -> Peer 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253): Couldn't send request to peer 8593949d5fb7413ca30c9ef5dfd31411. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:03.007627   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
I20260709 13:08:03.050191   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:03.051429   431 tablet_replica.cc:333] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69: stopping tablet replica
I20260709 13:08:03.052881   431 raft_consensus.cc:2243] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.054193   431 raft_consensus.cc:2272] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.056998   431 tablet_replica.cc:333] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69: stopping tablet replica
I20260709 13:08:03.057861   431 raft_consensus.cc:2243] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.058415   431 raft_consensus.cc:2272] T 06488c29a51d4bcda1841bdc7fd32b6d P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.060931   431 tablet_replica.cc:333] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69: stopping tablet replica
I20260709 13:08:03.061623   431 raft_consensus.cc:2243] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.062242   431 raft_consensus.cc:2272] T 43c7f51f72dc4e92b878a4ed950d4c5b P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.065549   431 tablet_replica.cc:333] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69: stopping tablet replica
I20260709 13:08:03.066720   431 raft_consensus.cc:2243] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.068154   431 raft_consensus.cc:2272] T ed00ffda0221409fbcadb48892fcbadc P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:03.068181 13066 consensus_peers.cc:597] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 -> Peer 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253): Couldn't send request to peer 8593949d5fb7413ca30c9ef5dfd31411. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:03.072393   431 tablet_replica.cc:333] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69: stopping tablet replica
W20260709 13:08:03.072511 13141 consensus_peers.cc:597] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 -> Peer 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507): Couldn't send request to peer 20f53f29f26f48d6b818f46c9e755f69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:03.073567   431 raft_consensus.cc:2243] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.074241   431 raft_consensus.cc:2272] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.078083   431 tablet_replica.cc:333] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69: stopping tablet replica
I20260709 13:08:03.078899   431 raft_consensus.cc:2243] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:08:03.079126 13140 consensus_peers.cc:597] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 -> Peer 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253): Couldn't send request to peer 8593949d5fb7413ca30c9ef5dfd31411. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:03.079869   431 raft_consensus.cc:2272] T c506771cdafd4db99eaf39cc5d9015ac P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.082595   431 tablet_replica.cc:333] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69: stopping tablet replica
I20260709 13:08:03.083325   431 raft_consensus.cc:2243] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.084515   431 raft_consensus.cc:2272] T daf0cc5885a14c65b1bd2fdab90c9b26 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.087076   431 tablet_replica.cc:333] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69: stopping tablet replica
I20260709 13:08:03.087749   431 raft_consensus.cc:2243] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.088234   431 raft_consensus.cc:2272] T 32b76fdbda6342bd887202061755151b P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.090642   431 tablet_replica.cc:333] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69: stopping tablet replica
I20260709 13:08:03.091224   431 raft_consensus.cc:2243] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.092095   431 raft_consensus.cc:2272] T 0d8b4f1a658d47cc954ba897df965354 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.094496   431 tablet_replica.cc:333] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69: stopping tablet replica
I20260709 13:08:03.095096   431 raft_consensus.cc:2243] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.095556   431 raft_consensus.cc:2272] T d727f4f4eacb428889625360bb663857 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.097795   431 tablet_replica.cc:333] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69: stopping tablet replica
I20260709 13:08:03.098356   431 raft_consensus.cc:2243] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.099354   431 raft_consensus.cc:2272] T eab0853d0dd94bcf8327c0ef09c6fad4 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.101699   431 tablet_replica.cc:333] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69: stopping tablet replica
I20260709 13:08:03.102294   431 raft_consensus.cc:2243] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.102720   431 raft_consensus.cc:2272] T d393f71c2afb406098b92973cb2be621 P 20f53f29f26f48d6b818f46c9e755f69 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.105499   431 tablet_replica.cc:333] T 790b9908004d4336a3093ecfe11ea707 P 20f53f29f26f48d6b818f46c9e755f69: stopping tablet replica
I20260709 13:08:03.106151   431 raft_consensus.cc:2243] T 790b9908004d4336a3093ecfe11ea707 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.106626   431 raft_consensus.cc:2272] T 790b9908004d4336a3093ecfe11ea707 P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.108853   431 tablet_replica.cc:333] T f5fffab9019146fd9cc617253eb19f87 P 20f53f29f26f48d6b818f46c9e755f69: stopping tablet replica
I20260709 13:08:03.109407   431 raft_consensus.cc:2243] T f5fffab9019146fd9cc617253eb19f87 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.109962   431 raft_consensus.cc:2272] T f5fffab9019146fd9cc617253eb19f87 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.112210   431 tablet_replica.cc:333] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69: stopping tablet replica
I20260709 13:08:03.112762   431 raft_consensus.cc:2243] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.113288   431 raft_consensus.cc:2272] T e175694e0fa6424f900caf84dbb2945f P 20f53f29f26f48d6b818f46c9e755f69 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.116881   431 tablet_replica.cc:333] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69: stopping tablet replica
I20260709 13:08:03.117478   431 raft_consensus.cc:2243] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.118000   431 raft_consensus.cc:2272] T a22a6947bab14423aa83c01130c20d77 P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.120599   431 tablet_replica.cc:333] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69: stopping tablet replica
I20260709 13:08:03.121109   431 raft_consensus.cc:2243] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.122074   431 raft_consensus.cc:2272] T 4448333d61c640e494f7ae000878dbdf P 20f53f29f26f48d6b818f46c9e755f69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.124421   431 tablet_replica.cc:333] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69: stopping tablet replica
I20260709 13:08:03.124876   431 raft_consensus.cc:2243] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.125293   431 raft_consensus.cc:2272] T 68f4b09178c244199a36c43c3989a45b P 20f53f29f26f48d6b818f46c9e755f69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.169366   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:08:03.176187 13141 consensus_peers.cc:597] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 -> Peer 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507): Couldn't send request to peer 20f53f29f26f48d6b818f46c9e755f69. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:46507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:03.189772 13140 consensus_peers.cc:597] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 -> Peer 8593949d5fb7413ca30c9ef5dfd31411 (127.0.107.193:44253): Couldn't send request to peer 8593949d5fb7413ca30c9ef5dfd31411. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44253: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:03.210319 13141 consensus_peers.cc:597] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 -> Peer 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507): Couldn't send request to peer 20f53f29f26f48d6b818f46c9e755f69. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:46507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:03.221328   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:08:03.241477 13141 consensus_peers.cc:597] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 -> Peer 20f53f29f26f48d6b818f46c9e755f69 (127.0.107.194:46507): Couldn't send request to peer 20f53f29f26f48d6b818f46c9e755f69. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:46507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:03.255831   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:03.256811   431 tablet_replica.cc:333] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187: stopping tablet replica
I20260709 13:08:03.257619   431 raft_consensus.cc:2243] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.258638   431 raft_consensus.cc:2272] T 8ffaa4a08cf24ed3ab65b5aecd4c3721 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.261248   431 tablet_replica.cc:333] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187: stopping tablet replica
I20260709 13:08:03.262163   431 raft_consensus.cc:2243] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.262698   431 raft_consensus.cc:2272] T ed00ffda0221409fbcadb48892fcbadc P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.265013   431 tablet_replica.cc:333] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 824d687f218b4cb091e5d62fa9d18187: stopping tablet replica
I20260709 13:08:03.265620   431 raft_consensus.cc:2243] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.266212   431 raft_consensus.cc:2272] T f4f2dbd8b1bd424e95b9a5ee46d2ea5b P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.268543   431 tablet_replica.cc:333] T 06488c29a51d4bcda1841bdc7fd32b6d P 824d687f218b4cb091e5d62fa9d18187: stopping tablet replica
I20260709 13:08:03.269205   431 raft_consensus.cc:2243] T 06488c29a51d4bcda1841bdc7fd32b6d P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.269769   431 raft_consensus.cc:2272] T 06488c29a51d4bcda1841bdc7fd32b6d P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.272418   431 tablet_replica.cc:333] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187: stopping tablet replica
I20260709 13:08:03.273097   431 raft_consensus.cc:2243] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.273810   431 raft_consensus.cc:2272] T 43c7f51f72dc4e92b878a4ed950d4c5b P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.275954   431 tablet_replica.cc:333] T 68f4b09178c244199a36c43c3989a45b P 824d687f218b4cb091e5d62fa9d18187: stopping tablet replica
I20260709 13:08:03.276539   431 raft_consensus.cc:2243] T 68f4b09178c244199a36c43c3989a45b P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.276998   431 raft_consensus.cc:2272] T 68f4b09178c244199a36c43c3989a45b P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.279323   431 tablet_replica.cc:333] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187: stopping tablet replica
I20260709 13:08:03.279963   431 raft_consensus.cc:2243] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.280814   431 raft_consensus.cc:2272] T d393f71c2afb406098b92973cb2be621 P 824d687f218b4cb091e5d62fa9d18187 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.285189   431 tablet_replica.cc:333] T eab0853d0dd94bcf8327c0ef09c6fad4 P 824d687f218b4cb091e5d62fa9d18187: stopping tablet replica
I20260709 13:08:03.285889   431 raft_consensus.cc:2243] T eab0853d0dd94bcf8327c0ef09c6fad4 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.286397   431 raft_consensus.cc:2272] T eab0853d0dd94bcf8327c0ef09c6fad4 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.289037   431 tablet_replica.cc:333] T d727f4f4eacb428889625360bb663857 P 824d687f218b4cb091e5d62fa9d18187: stopping tablet replica
I20260709 13:08:03.290059   431 raft_consensus.cc:2243] T d727f4f4eacb428889625360bb663857 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.290864   431 raft_consensus.cc:2272] T d727f4f4eacb428889625360bb663857 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.293723   431 tablet_replica.cc:333] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187: stopping tablet replica
I20260709 13:08:03.294497   431 raft_consensus.cc:2243] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.295154   431 raft_consensus.cc:2272] T 32b76fdbda6342bd887202061755151b P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.297499   431 tablet_replica.cc:333] T daf0cc5885a14c65b1bd2fdab90c9b26 P 824d687f218b4cb091e5d62fa9d18187: stopping tablet replica
I20260709 13:08:03.298192   431 raft_consensus.cc:2243] T daf0cc5885a14c65b1bd2fdab90c9b26 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.298710   431 raft_consensus.cc:2272] T daf0cc5885a14c65b1bd2fdab90c9b26 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.301028   431 tablet_replica.cc:333] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187: stopping tablet replica
I20260709 13:08:03.301591   431 raft_consensus.cc:2243] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.302114   431 raft_consensus.cc:2272] T c506771cdafd4db99eaf39cc5d9015ac P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.304512   431 tablet_replica.cc:333] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187: stopping tablet replica
I20260709 13:08:03.305166   431 raft_consensus.cc:2243] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.305751   431 raft_consensus.cc:2272] T 790b9908004d4336a3093ecfe11ea707 P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.308272   431 tablet_replica.cc:333] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187: stopping tablet replica
I20260709 13:08:03.309019   431 raft_consensus.cc:2243] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.309559   431 raft_consensus.cc:2272] T f5fffab9019146fd9cc617253eb19f87 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.312827   431 tablet_replica.cc:333] T 4448333d61c640e494f7ae000878dbdf P 824d687f218b4cb091e5d62fa9d18187: stopping tablet replica
I20260709 13:08:03.313416   431 raft_consensus.cc:2243] T 4448333d61c640e494f7ae000878dbdf P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.313920   431 raft_consensus.cc:2272] T 4448333d61c640e494f7ae000878dbdf P 824d687f218b4cb091e5d62fa9d18187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.316265   431 tablet_replica.cc:333] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187: stopping tablet replica
I20260709 13:08:03.316825   431 raft_consensus.cc:2243] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.317276   431 raft_consensus.cc:2272] T 0d8b4f1a658d47cc954ba897df965354 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.319443   431 tablet_replica.cc:333] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187: stopping tablet replica
I20260709 13:08:03.319943   431 raft_consensus.cc:2243] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.320806   431 raft_consensus.cc:2272] T e175694e0fa6424f900caf84dbb2945f P 824d687f218b4cb091e5d62fa9d18187 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.324157   431 tablet_replica.cc:333] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187: stopping tablet replica
I20260709 13:08:03.324720   431 raft_consensus.cc:2243] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.325593   431 raft_consensus.cc:2272] T a22a6947bab14423aa83c01130c20d77 P 824d687f218b4cb091e5d62fa9d18187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.372759   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:08:03.416858   431 master.cc:562] Master@127.0.107.254:40089 shutting down...
I20260709 13:08:03.445257   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.446278   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.446710   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 36a67cb3227c4b49858794f6cda16813: stopping tablet replica
I20260709 13:08:03.469488   431 master.cc:584] Master@127.0.107.254:40089 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (11975 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (25102 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (428465 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:08:03.535652   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 422 messages since previous log ~15 seconds ago
I20260709 13:08:03.535943   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 421 messages since previous log ~15 seconds ago
I20260709 13:08:03.536139   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 420 messages since previous log ~15 seconds ago
I20260709 13:08:03.536352   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~33 seconds ago
I20260709 13:08:03.536554   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~33 seconds ago
I20260709 13:08:03.536779   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/master/catalog_manager.cc:4755: suppressed but not reported on 1 messages since previous log ~79 seconds ago
I20260709 13:08:03.537014   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 10 messages since previous log ~190 seconds ago
I20260709 13:08:03.537213   431 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 6 messages since previous log ~244 seconds ago
I20260709 13:08:03.537418   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/service_pool.cc:136: suppressed but not reported on 6 messages since previous log ~245 seconds ago
I20260709 13:08:03.537710   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 9 messages since previous log ~0 seconds ago
I20260709 13:08:03.538036   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 26 messages since previous log ~6 seconds ago
